![]() Registered Member ![]()
|
Eigen::SparseMatrix<double,Eigen::RowMajor, long long int>
storing such a matrix with the SaveMarket function works fine because of data type independent implementation. But a restore of such saved files will fail with LoadMarked function. ![]() I noticed that currently hard coded data types of type "int" ![]() Beyond that in "Eigen/src/SparseCore/SparseMatrix.h" the function set_from_triplets allocates internally a sparse matrix of type "int". As far as I noticed till now this make loading of big matrices (as in the example) impossible. ![]() My sparse matrices use up to 280GB of RAM and in the text file version up to 1.2TB of disk space hence an instantiation as presented is necessary. Is there anybody who had some comparable experience? Should this be reported as a bug? Maybe there is a more general way to implement the above reported hard coded data types (like often used the Index typedef). Best Martin |
![]() Moderator ![]()
|
patch welcome. It's just a matter of replacing int by SparseMatrixType::Index.
|
![]() Registered Member ![]()
|
Seems to work.
![]() But I noticed another thing: Currently a vector is used for temporary storing the triplets, this sometimes leads to "bad memory allocation" errors when used with big matrices. For troubleshooting I replaced it by a std::list. This is a little slower for insert but more flexible with memory management and the API is equivalent. I'll also include it in the patch. |
![]() Moderator ![]()
|
hm, that's very strange hat a list work since it requires more memory and about twice same amount of memory required by the std::vector are needed to assemble the matrix.
|
![]() Registered Member ![]()
|
From my perspective the advantage of a std::list is that non-contiguous memory is used instead by a vector were in general contiguous memory is required.
This is especially the case when a big amount of non-zero values in a large sparse matrix exist. For sure a std::list requires more memory but allocation is performed in a more flexible way. Maybe std::forward_list which is taken from C++11 bridges the gap between std:vector and std::list. As in some other forums reported std:vector is insufficient for application with "big data". This is equates with my experience from a hard learned lesson ![]() ![]() |
Registered users: Bing [Bot], claydoh, Google [Bot], rblackwell, Yahoo [Bot]