Speed up PARDISO

[misch]

Hello,

I want to solve a linear system Ax=b. The matrix A is sparse and symmetric. Its is size [2 226 495 x 2 226 495] and 85 414 336 nnz. The RHS is a vector [2226495 x1] 5549 nnz. I am using Intel MKL (v2020.0.166) and PARDISO LDLT in the C++ Library Eigen. In my first attempts I didn't get any result. Since a few hours I was changing the iparm Parameters and finally I got the first (right) result but with bad timing (75s to solve). While solving my CPU usage was 50% and used RAM 28GB. Any advice for speeding it up more?

Im am using this Parameters:

 iparm[0] = 1

iparm[1] = 2

iparm[3] = 0

iparm[4] = 0

iparm[5] = 0

iparm[6] = 0

iparm[7] = 2

iparm[8] = 0

iparm[9] = 13

iparm[11] = 0

iparm[12] = 1 

iparm[13] = 0

iparm[17] = -1

iparm[18] = -1

iparm[19] = 0

iparm[20] = 0

iparm[26] = 0

iparm[34] = 1

iparm[36] = 0

iparm[59] = 1 

My system: Intel Xeon CPU E5-2687W@ 3,1GHz, 8 Cores, 16 logical Processors; WIN 10 x64; 128GB RAM

Thank you in advance. 

[Kirill_V_Intel]

Hello,

Can you try to set iparm[23]=1, iparm[12] = 0 and iparm[9] = 8. If your matrix is s.p.d, you can also set iparm[7] = 0. I would expect performance benefits mostly from switching to the two-level factorization (iparm[23]), but you need to switch off matching (iparm[12]) for it. I'd like to see your numbers for this case.

If it does not help, you can also try to do multithreaded reordering (iparm[1]=3).

Also, have you set your system properly? Is it Windows/Linux, how many cores, how much RAM? Did you set KMP_AFFINITY, do you have hyperthreading on/off? In a lot of cases suboptimal performance can be observed when the system was not configured for optimal performance.

Also, if you get any output from PARDISO, you can share it with us (in PARDISO API there is a paramter msglvl which enables statistics printing).

Best,
Kirill

[misch]

Thank you for your reply.

At first I tried to set iparm[1] = 3 -> time reduced from 75s to 51s. Changing the other parameters (also iparm[23]), didnt change anything. 

Here is my output after setting msglvl = 1 : Memory allocated on phase  11   4224.7972 MB
Number of non-zeros in L        1425805468
Number of non-zeros in U        1
Memory allocated on phase  22   15368.4958 MB

What does KMP_Affinity do and how to set it?

My system: Intel Xeon CPU E5-2687W@ 3,1GHz, 8 Cores, 16 logical Processors; WIN 10 x64; 128GB RAM, using Visual Studio 2019 16.3.6