calling Pardiso parallel from Java on Linux environment

[eunsal]

Hi,

I am using Pardiso Sparse Matrix Solver and I call it from Java on Linux environment. Currently, I create the matrix elements in Java and I passed them to Pardiso in form of arrays. I have an interface to do this.Everything works fine.

I also have the opportunity to run my code on a cluster, but I couldnt manage to call Pardiso parallel. I tried

'setenv MKL_NUM_THREADS 4' command just before I run my executable, or I put the command in my batch file. But none of them made any difference. I still run my code on a single node, it didn't get any faster.

Does anybosy have any experience with a similar problem/ or any suggestions?

Thanks in advance,

Evren

[Todd_R_Intel]

Could you tell us what version of Intel MKL you are using?

What sort of problem are you solving with PARDISO and how big is the matrix?

Todd

[eunsal]

Hi Todd,

The MKL version is 10.0.4. I am working on a fluid mechanics problem and trying to solve the fluid flow equations in porous media. the medium is represented as points and I am solving for pressures at each point. Depending on the size of my porous medium I can have, at the moment, 500,000 points. So my sparse matrix size is 500,000 by 500,000. But later on I want to try larger matrices up to 2million perhaps. So I will really benefit from making the code parallel.

Evren

[Gennady_F_Intel]

Hi Evren,

PARDISO calculates the solution of a set of sparse linear equations on SMP systems only. So, thats why You cannot use PARDISO on Cluster System.

--Gennady

[Gennady_F_Intel]

What is nnz value of this problem?

The size of your solution is pretty big, and as I can guess, you have to use OOC version. Is it correct?

The version you are using is pretty aged version and many improvements regarding PARDISO (either in-core or OOC versions ) are available currently versus version of 10.0 you are using.

In particularly( among others):

Implemented high-level parallelization of out-of-core (OOC) PARDISO when operating on symmetric positive definite matrices.

Parallelized Reordering and Symbolic factorization stage in PARDISO/DSS

All these functionality avalible in the latest 10.2 update5 version.

--Gennady

[eunsal]

My sparse matrix is a banded matrix, I always have numbers on the main diagonal and 4 more numbers oncertain rows and 2 more numbers on the others. For very large matrices, the percentage of nonzero is very low.

At the moment I use for in-core version since I haven't yet moved up to very large matrices. Right now, I use for example, 50,000 by 50,000.

I already requested the latest version of MKL from my IT depeartment and it is being installed today. So I will be able to try the upgrades you suggested. I think I will be getting version 11.1.

Meantime, can we just assume that I use a workstation with 8 nodes on it and Isolve a smaller sparse matrix (10,000by10,000 orsmaller). How can I run it multi-threaded efficiently? In the terminal where I call my executable I write 'mkl_set_num_threads = 8' for example. It appears that Im allocated to8 nodes, but Iamactually using only 1. Because the usage profile shows only around %12. I dont get any speed-up at all.

I really want to make the best use of the MKL library, but I couldnt manage it. Mainly because I don't have any experience with it.

[Gennady_F_Intel]

Well, I agree with this conditions, the memory consumption will be relatively negligible.

1)What I d ask you to to:

Could you please check the execution time for each phase: Reordering and Symbolic Factorization, numerical factorization and solution?

2) second:

You wrote: In the terminal where I call my executable I write 'mkl_set_num_threads = 8' for example

You have to call this function from your application.

For example:

#include

int foo(){

int nthr = 8;

mkl_set_num_threads (Nthr );

pardiso()

}

--Gennady

[eunsal]

1) How can I get that information regarding the times in detail?

2) I now upgraded to the version 11.1 and everything is running fine, little bit faster compared to 10.0.4 actually. I tried to implementyour code:

#include

int foo(){

int nthr = 8;

mkl_set_num_threads (Nthr );

pardiso()

}

at different places. The wrapper has an interface in Java and the JNI in C. Am I supposed to create a new C file with this function in it? or implement it in the JNI C file? From JNI it didnt work. I also tried to pass it from Java interface to JNI , but no success either.

with pardiso(), are you refering to the Java interface or JNI file?