topic MKL PARDISO - Large Scale Problem in Intel® oneAPI Math Kernel Library

MKL PARDISO - Large Scale Problem

Mistry__Mital — Tue, 02 Aug 2011 13:26:19 GMT

Hi,

I am solving = where both X and B are matrices. I am solving for X. Size of is 1,500,000 x 1,500,000. Aproximate non zeros are 60,000,000. I am able to run my code for the lower size problem. But for this size it does not run.

System configuration:

SGI - 8 processor with 32 GB memory.
Compiler - Intel 10.1.012.

I am using 10.0.1.014 intel mkl

When I am using iparm(60) = 2 it kill the process with -9 error

***************************FOLLOWING WAS ERROR******************************

The file ./pardiso_ooc.cfg was not opened
*** error PARDISO ( not enough mem. for OOC) error_num= -9

================ PARDISO: solving a symmetric indef. system ================

Summary PARDISO: ( reorder to reorder )
================

Times:
======

Time fulladj: 1.360909 s
Time reorder: 29.476320 s
Time symbfct: 34.058441 s
Time malloc : 0.000549 s
Time total : 64.896383 s total - sum: 0.000164 s

Statistics:
===========
< Parallel Direct Factorization with #processors: > 1
< Numerical Factorization with Level-3 BLAS performance >

< Linear system Ax = b>
#equations: 1460346
#non-zeros in A: 58022798
non-zeros in A (%): 0.002721
#right-hand sides: 1

< Factors L and U >
#columns for each panel: 10
#independent subgraphs: 0
< Preprocessing with state of the art partitioning metis>
#supernodes: 231589
size of largest supernode: 20746
number of nonzeros in L -905604445
number of nonzeros in U 1
number of nonzeros in L+U -905604444
test
check point 1
check point 2
check point 3
check point 5
check point 7
58022799
Reordering completed ...
The following ERROR was detected: -9

*****************************************************************************
Questions:

1) Is it possible to solve using current resources (8-Processors with 32GB RAM)?
2) If I want to run this size problem then what is the maximum RAM on single processor will I need? As we are planning to buy a system that has sufficient memory for this type of operations.

Thank you,

Mital Mistry.

MKL PARDISO - Large Scale Problem

Konstantin_A_Intel — Tue, 02 Aug 2011 14:32:34 GMT

Hi Mital,

I would suggest you to try to use MKL version 10.3 first of all. A lot of significant improvements has been implemented since MKL 10.0 in PARDISO.

Regards,

Konstantin

MKL PARDISO - Large Scale Problem

Mistry__Mital — Tue, 02 Aug 2011 14:49:01 GMT

Thanks Konstantin,

I will do that and also, I tried following,

export MKL_PARDISO_OOC_MAX_CORE_SIZE=16000

And the process did not stop but it kept running for almost 2 days and did nothing.

Also, the process is not possible by in-core right?

Regards,

Mital

MKL PARDISO - Large Scale Problem

Mistry__Mital — Tue, 02 Aug 2011 15:58:46 GMT

Konstantin,

We have IA-64 architecture. So is there any special needs for updating to MKL 10.3 version?

Regards,
Mital

Hi, the workstation of mine

yanpu_z_ — Mon, 29 Apr 2013 13:06:32 GMT

Hi, the workstation of mine is with 32G memery and two 6-core XEON E5-2630 cpu.

I wonder a sparse matrix with about 2million unknowns (roughly 20 million non-zero elements in the matrix) can be solved by the PARDISO within reasonable time.

Thanks a lot!

I'm afraid that the answer is

mecej4 — Mon, 29 Apr 2013 13:48:00 GMT

Yanpu:

I'm afraid that the answer is "It depends". Factorization of a highly sparse matrix (yours has a density of 0.000005) can be accompanied by substantial fill-in, and your system is far short of the memory that a dense matrix would take for the same number of unknowns.

You could try smaller problems and run tests to measure how the calculation scales, and project the results to the full size to obtain an estimate the run time.

Hi yanpu.

Alexander_K_Intel2 — Mon, 29 Apr 2013 14:33:06 GMT

Hi yanpu.

You can try to use pardiso with iparm(60) is equal to 1. In such case if pardiso need to use more than 32Gb of RAM OOC algorithm will be used. Just set environment variable MKL_PARDISO_OOC_MAX_CORE_SIZE to 30000 to provide information about existence RAM to pardiso before PARDISO call.

With best regards,

Alexander Kalinkin