Community
cancel
Showing results for 
Search instead for 
Did you mean: 
S__MPay
Beginner
132 Views

MKL Packed Storage scheme for matrix with many zoroes

Jump to solution

I have a very large matrix but many of its members are zero however it does not have a specific shape like being banded, etc.

Is there any possibility that I can reduce memory usage for storing this matrix for using MKL to solve marix operations?

Currently for a not very large model I need more than 32GB of RAM.

0 Kudos
1 Solution
mecej4
Black Belt
132 Views

If you visit the elements in some arbitrary order and compute node-node coefficients, you may need to use the coordinate list format ("COO" format) and then convert to CSR format. See this thread: https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/777804 .

View solution in original post

4 Replies
jirina
Novice
132 Views

Have you considered the CSR (Compressed Sparse Row) format? See e.g. here.

S__MPay
Beginner
132 Views

jirina wrote:

Have you considered the CSR (Compressed Sparse Row) format? See e.g. here.

Dear Jirina, Thanks that is what I am looking for but the only problem is that I assemble the matrix element by element and don't have values arranged by rows and columns. Do you know if there is a standard way to assemble the matrix using CSR format with minimum calculation cost?

mecej4
Black Belt
133 Views

If you visit the elements in some arbitrary order and compute node-node coefficients, you may need to use the coordinate list format ("COO" format) and then convert to CSR format. See this thread: https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/777804 .

View solution in original post

Kirill_V_Intel
Employee
132 Views

mecej4 wrote:

If you visit the elements in some arbitrary order and compute node-node coefficients, you may need to use the coordinate list format ("COO" format) and then convert to CSR format. See this thread: https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/... .

Or you can construct CSR with a hand-written routine without overhead of creating an intermediate COO representation. For this, you basically need to perform two passes through the data. On the first pass you create rowIndex (or pointerB and pointerE for 4-array CSR) by just looping over your tuples and counting number of nonzeros in each row. After the second pass you allocate colIndx and values and fill them during the second pass. During the second pass you can then use the offsets into colIndx and values from the pre-computed rowIndex.

This can be in many cases faster than creating a coordinate format and then converting it into CSR.

Best,
Kirill

Reply