Memory requirements for Pardiso solver

[alan_ritchie1]

Hi

Can anyone tell me roughly how much memory the Pardiso solver should need to solve a real positive definite matrix with around 500,000 unknowns (25 million non-zero values), and one how well this would scale up to one with a matrix with one million unknowns.

Alternatively, what is the largest matrix size that can be solved on a 32bit windows machine?

Thanks
Alan

[basel]

Hi Alan,

I tried a 2D Laplace matrix (real positive definite matrix with 1250000 unknowns, 5 nonzeros/row) and the solver consumes 1.9 GB on a 32-bit system.

Regards,
Olaf

[mma]

Hi,

Have you got an answer from any Intel engineer? I run into Pardiso size limitation many times, but never got a clear answer. My latest problem was a symmetric matrix with about 300,000 unknowns. I got a memory error message. However, it is hard to believe that Intel would collaborate with such a limited package.

Thanks

Ming

[funklair]

Hello Ming,
I am glad to see other people are finding similar problems to me. I have only found this problem in the last few days and noticed your earlier postings on Intel that referenced Intel Premier Support listings #337721 but I can't seem to access them.

I am using PARDISO for 3-D seismic wave modelling and am currently trying to solve about 680 000 equations with 33 million doublecomplex unknowns on a shared memory SGI Altix 3000 Intel Itanium-2 platform (64-bit) and I'm hoping to get a bit larger than that! Currently I observe 32-bit overflow on the L+U output report from PARDISO before the seg fault occurred. L and U have about 1.7 billion nnz! Golly is this correct? I don't know if you are also getting overflow.

I read in the PARDISO user manual about a 64-bit integer version for little-endian machines. I sent an email yesterday to pardiso@maillist.unibas.ch and got a very quick response from Olaf Schenk. Apparently there was once a version but the current version does not have 64-bit integers.

I hope there is some way of solving this problem soon. If anyone has any advice .....

Thanks,

Cathy

[funklair]

Woops, I meant 33 million nonzero values, 680 000 uknowns.

Cathy

[mma]

Hi, Cathy

I'm running the code on a Pentium-4, 3.2GHz, 2GB Ram and8GB free disk machine with Windows XP (32bit) system. The code was compiled with VS2005.Net C++. I used to have size problem on a 150,000 unknowns with MKL 8.0. With MKL 8.1, the new limit is about 250,000 unknowns from my experience. FYI, below was an error message from a "small" 481,000 unknowns problem.

Ming

================ PARDISO: solving a symm. posit. def. system ================

Summary PARDISO: ( reorder to reorder )
================

Times:
======

Time fulladj: 0.542644 s
Time reorder: 2.589200 s
Time symbfct: 1.929949 s
Time parlist: 0.070352 s
Time malloc : -0.115736 s
Time total : 7.883856 s total - sum: 2.867447 s

Statistics:
===========
< Parallel Direct Factorization with #processors: > 2
< Numerical Factorization with BLAS3 and O(n) synchronization >

< Linear system Ax = b>
#equations: 481135
#non-zeros in A: 14318675
non-zeros in A (%): 0.006185
#right-hand sides: 1

< Factors L and U >
#columns for each panel: 80
#independent subgraphs: 0
< Preprocessing with state of the art partitioning metis>
#supernodes: 51024
size of largest supernode:&n bsp; 3540
number of nonzeros in L 200512125
number of nonzeros in U 1
number of nonzeros in L+U 200512126

Reordering completed ...
Number of nonzeros in factors = 200512126
Number of factorization MFLOPS = 331698*** error PARDISO ( insufficient_memory) error_num= -1100
*** error pardiso (memory allocation) FACT_L&U, size to allocate: 1604097008 bytes
total memory wanted here: 1566501 kbyte
symbolic (max): 718597 symbolic (permanent): 169717 real(incl. 1 factor):

================ PARDISO: solving a symm. posit. def. system ================

Summary PARDISO: ( factorize to factorize )
================

Times:
======

Time total : 0.000200 s total - sum: 0.000200 s

Statistics:
===========
< Parallel Direct Factorization with #processors: > 2
< Numerical Factorization with BLAS3 and O(n) synchronization >

< Linear system Ax = b>
#equations: 481135
#non-zeros in A: 14318675
non-zeros in A (%): 0.006185
#right-hand sides: 1

< Factors L and U >
#columns for each panel: 80
#independent subgraphs: 0
< Preprocessing with state of the art par titioning metis>
#supernodes: 51024
size of largest supernode: 3540
number of nonzeros in L 200512125
number of nonzeros in U 1
number of nonzeros in L+U 200512126
gflop for the numerical factorization: 331.698293

ERROR during numerical factorization: -2

[funklair]

Hi Ming,

Yes, it looks like you are running out of RAM. A different problem to me because I have lots of RAM on the shared memory multiple processor system I am using. That's why I was so puzzled when it failed. It would seem your solution would be to run on a parallel system also, but then you will probably encounter the same problem as me. I think my problem is that the use of 32-bit integers within the compiled PARDISO library is inadequate for such large matrices. I think you also are at the very edge of what a 32-bit machine can cope with.

I'm not an expert on such things but I wonder when we run out of RAM, why can't we access that free disk space - is it paged memory? It would be slower, but our main concern is just to get the job done!

I have also tried SuperLU_DIST_2.0 on a parallel distributed memory system but my problem there I think was that as the size of the matrix got larger, the inter-process communication became prohibitive and the job never finished! I think I ran out of MPI requests. Unfortunately SuperLU_MT for a shared memory system doesn't as yet cater for double complex types. SuperLU is great because you can access the source code and alter the types to 64-bit if you need to. It would be too time consuming for me to try to write my own double complex version and I probably am not a good enough programmer yet! And it would not contribute directly to my PhD so I would be using up my research time...

Also - I have just received an email from Olaf Schenk (PARDISO) that maybe ILUPACK will be good for large matrices. I am going to try it. I know nothing about it yet. I wonder if it will help you?
http://www.math.tu-berlin.de/ilupack/

Cathy