Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.
6590 Discussions

Cluster Sparse Solver(cpardiso) reordering problem

GwangSeok_S_
Beginner
144 Views

Hello.

I tried to solve a large linear equation (1,000,000 x 1,000,000 / bandwidth = 100 or 1000) with cpardiso.

( the matrix type is real and symmetric indefinite. )

I have some problems about reordering time and memory.

CPARDISO's reordering phase is compare to slower than the other phase. So I checked event time using Traceanalyzer.

CPARDISO used only one process(rank 0) for reordering and Rank 0 collected information on the divided A matrix on each process.

As a result, When I solved the bandwidth 1,000 equation, It occurred insufficient memory error. (※bandwidth 100 equation was resolved)

 

Should CPARDISO do the reordering and collect the A matrix in only rank 0 ?

Does rank 0 must have a lot of memory to solve a large system?

How to solve this problem ?

 

The version of MKL is mkl 11.3, which was bundled with parallel studio xe 2016 cluster edition.

 

This is the setting for cpardiso

iparm[ 0] = 1;

iparm[ 1] = 0; (I also tried iparm[1]= 2 and 3)

iparm[ 5] = 0;

iparm[ 7] = 0;

iparm[ 9] = 8;

iparm[10] = 0;

iparm[12] = 0;

iparm[17] = 0;

iparm[18] = 0;

iparm[20] = 1;

iparm[26] = 0;

iparm[27] = 0;

iparm[34] = 1;

iparm[39] = input_value[1];

iparm[40] = input_value[2];

iparm[41] = input_value[3];

 

I used 4 nodes that are connected InfiniBand and Each node have 32 GB RAM.

Thanks.

0 Kudos
1 Reply
Gennady_F_Intel
Moderator
144 Views

Said M,  the reodering step is not threaded at this version of MKL. We are working on this improvement at this moment and will keep you informed when this feature will be implemented. thanks,

Reply