Skip to content

Conversation

alyst
Copy link
Contributor

@alyst alyst commented Nov 2, 2024

This PR is another subset of #193 changes and adds ParamsArrays type to simplify RAM specification.

ParamsArray{T,N} object defines N-dimensional array, whose elements are either constants of type T or parameters (specified as indices in parameters array).
materialize(pararr, parvals), materialize!(arr, parrarr, parvals) and sparse_materialize(pararr, parvals) are used to create/update existing arrays with the parvals parameter values according to the pararr::ParamsArray specification.

It also supports fast sparse gradients by the parameters vector via sparse_gradient!().

For lower level iteration over parameter references in an array, one can use the param_occurences(pararr, i) method that returns the linear indices of i-th param occurences.

The PR switches RAMMatrices to use the ParamsArray for A and S matrices and M vector.

Copy link

codecov bot commented Nov 2, 2024

Codecov Report

Attention: Patch coverage is 81.96721% with 55 lines in your changes missing coverage. Please review.

Project coverage is 74.14%. Comparing base (0cecaa8) to head (8c26c35).
Report is 26 commits behind head on devel.

Files with missing lines Patch % Lines
src/additional_functions/params_array.jl 67.36% 31 Missing ⚠️
src/frontend/specification/RAMMatrices.jl 88.07% 13 Missing ⚠️
...c/additional_functions/start_val/start_partable.jl 0.00% 9 Missing ⚠️
src/additional_functions/start_val/start_fabin3.jl 96.66% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##            devel     #219      +/-   ##
==========================================
+ Coverage   71.69%   74.14%   +2.44%     
==========================================
  Files          52       52              
  Lines        2120     2197      +77     
==========================================
+ Hits         1520     1629     +109     
+ Misses        600      568      -32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

a.param_ptr == b.param_ptr &&
a.linear_indices == b.linear_indices

Base.hash(a::ParamsArray, h::UInt) = hash(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we need that for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added == and hash() so that ParamsArray could be efficiently used as a key in AbstractDict (particularly for matrices large dimensions).
The reason why I need it -- when I construct my regularized multi-group SEM, some terms might be duplicate and could therefore be merged, but it's hard to detect in advance (e.g. I may have series of replicate experiments that have either 100% identical design or each of the series may have some unique samples not present in the other series). So with Dict{ParamsArray, SEMTerm}() I can automatically detect duplicate terms.

SEM.jl does not need/use it, so in principle I can move it to my code. But that's relatively little code, and by keeping it in SEM.jl we avoid "type piracy".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, seems reasonable - let's keep it here then.

@Maximilian-Stefan-Ernst
Copy link
Collaborator

Thanks a lot, great PR - I just had some small comments, otherwise, i would merge it.

Alexey Stukalov and others added 2 commits November 22, 2024 09:21
alyst and others added 10 commits November 22, 2024 11:36
replaces RAMMatrices indices and constants vectors with dedicated class
that incapsulate this logic,
resulting in overall cleaner interface

A_ind, S_ind, M_ind become ParamsArray
F_ind becomes SparseMatrixCSC

parameters.jl is not longer required and is removed
@alyst
Copy link
Contributor Author

alyst commented Nov 22, 2024

Thanks a lot, great PR - I just had some small comments, otherwise, i would merge it.

Thank you for the review! I've addressed your comments.
Also, I have rebased the PR against the current devel branch to keep the commit history clean.
I have locally rechecked for formatting of the code in the PR -- that should be fine, the CI check complains about the other files/line.

@Maximilian-Stefan-Ernst Maximilian-Stefan-Ernst merged commit 5b9327e into StructuralEquationModels:devel Nov 23, 2024
4 of 5 checks passed
@alyst alyst deleted the params_array branch November 23, 2024 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants