Using PARDISO OOC with very large Matrices

[dntal]

Hello to all!
I am trying to use PARDISO in OOC mode to solve large systems. As I can see from the function definitions, I have to pass an array containing the non-zero elements of A (along with matrices ia and ja) as arguments to the PARDISO function.

However, the non zero elements of my problem are written to the disk, and I cannot read them to memory, because they don't fit to it. Is there any way to call the PARDISO solver and instruct it to read the values directly from the disk? Is there any other way around my problem.

I will appreciate any help. Thank you for your time.
Dimitris.

[Gennady_F_Intel]

Quoting - dntal

Hello to all!
I am trying to use PARDISO in OOC mode to solve large systems. As I can see from the function definitions, I have to pass an array containing the non-zero elements of A (along with matrices ia and ja) as arguments to the PARDISO function.

However, the non zero elements of my problem are written to the disk, and I cannot read them to memory, because they don't fit to it. Is there any way to call the PARDISO solver and instruct it to read the values directly from the disk? Is there any other way around my problem.

I will appreciate any help. Thank you for your time.
Dimitris.

Dimitris,
Could you provide more details, please? What is the original sizes of nnz, ia and ja? What is the system RAM size?
OS?
--Gennady

[dntal]

Quoting - Gennady Fedorov (Intel)

Dimitris,
Could you provide more details, please? What is the original sizes of nnz, ia and ja? What is the system RAM size?
OS?
--Gennady

The original system can have a size of 16Gb or more, and it is stored in the disk. Depending on the thresholding criteria I use in my application, the number of non-zero elements of this matrix can vary from 20% up to 80% of the original size.

In the average case, where about 50% of the original elements of the matirx is cut, the size of the corresponding sparse matrix exceeds by far the size of the RAM I have in my system (~4Gb).

Thank you very much for your answer,
Dimitris.

[Gennady_F_Intel · Accepted solution]

Quoting - dntal

The original system can have a size of 16Gb or more, and it is stored in the disk. Depending on the thresholding criteria I use in my application, the number of non-zero elements of this matrix can vary from 20% up to 80% of the original size.

In the average case, where about 50% of the original elements of the matirx is cut, the size of the corresponding sparse matrix exceeds by far the size of the RAM I have in my system (~4Gb).

Thank you very much for your answer,
Dimitris.

Hi Dimitris,

There are no opportunity in the current versionto use PARDISO with similar very big matrixes.
I submitted the Feature Request to the our internal database for the future development.

One comment for this issue:

If you have 8 Gb of original matrix size ( it will be smth around 2*10^9 nnz ), then after reordering phase the number of non-zeroes in L+U might be around 10^10 ... 10^11 ( it will depends of many factors but in the average case this is an expected numbers of nnz in L+U)

That's mean it will require (10^10 ... 10^11)*sizeof(double precision) ~0.1 ... 1.0TB memory in the disk for storing data on the disk..
I think ( especially if we will take into account the RAM == 4 GB, therefore available RAM will be ~ 3Gb ) that this task will take a very long time to execute this task...

but for today, from my point of view, there is only one way to split the task on smaller task to fit accordingly your systems size.

-- Gennady