topic calling Pardiso parallel from Java on Linux environment in Intel® oneAPI Math Kernel Library

calling Pardiso parallel from Java on Linux environment

eunsal — Fri, 23 Jul 2010 17:14:33 GMT

Hi,

I am using Pardiso Sparse Matrix Solver and I call it from Java on Linux environment. Currently, I create the matrix elements in Java and I passed them to Pardiso in form of arrays. I have an interface to do this.Everything works fine.

I also have the opportunity to run my code on a cluster, but I couldnt manage to call Pardiso parallel. I tried

'setenv MKL_NUM_THREADS 4' command just before I run my executable, or I put the command in my batch file. But none of them made any difference. I still run my code on a single node, it didn't get any faster.

Does anybosy have any experience with a similar problem/ or any suggestions?

Thanks in advance,

Evren

calling Pardiso parallel from Java on Linux environment

Todd_R_Intel — Fri, 23 Jul 2010 22:41:08 GMT

Could you tell us what version of Intel MKL you are using?

What sort of problem are you solving with PARDISO and how big is the matrix?

Todd

calling Pardiso parallel from Java on Linux environment

eunsal — Mon, 26 Jul 2010 08:27:07 GMT

Hi Todd,

The MKL version is 10.0.4. I am working on a fluid mechanics problem and trying to solve the fluid flow equations in porous media. the medium is represented as points and I am solving for pressures at each point. Depending on the size of my porous medium I can have, at the moment, 500,000 points. So my sparse matrix size is 500,000 by 500,000. But later on I want to try larger matrices up to 2million perhaps. So I will really benefit from making the code parallel.

Evren

calling Pardiso parallel from Java on Linux environment

Gennady_F_Intel — Mon, 26 Jul 2010 13:10:42 GMT

Hi Evren,

PARDISO calculates the solution of a set of sparse linear equations on SMP systems only. So, thats why You cannot use PARDISO on Cluster System.

--Gennady

calling Pardiso parallel from Java on Linux environment

Gennady_F_Intel — Mon, 26 Jul 2010 13:25:28 GMT

What is nnz value of this problem?

The size of your solution is pretty big, and as I can guess, you have to use OOC version. Is it correct?

The version you are using is pretty aged version and many improvements regarding PARDISO (either in-core or OOC versions ) are available currently versus version of 10.0 you are using.

In particularly( among others):

Implemented high-level parallelization of out-of-core (OOC) PARDISO when operating on symmetric positive definite matrices.

Parallelized Reordering and Symbolic factorization stage in PARDISO/DSS

All these functionality avalible in the latest 10.2 update5 version.

--Gennady

calling Pardiso parallel from Java on Linux environment

eunsal — Mon, 26 Jul 2010 16:27:57 GMT

My sparse matrix is a banded matrix, I always have numbers on the main diagonal and 4 more numbers oncertain rows and 2 more numbers on the others. For very large matrices, the percentage of nonzero is very low.

At the moment I use for in-core version since I haven't yet moved up to very large matrices. Right now, I use for example, 50,000 by 50,000.

I already requested the latest version of MKL from my IT depeartment and it is being installed today. So I will be able to try the upgrades you suggested. I think I will be getting version 11.1.

Meantime, can we just assume that I use a workstation with 8 nodes on it and Isolve a smaller sparse matrix (10,000by10,000 orsmaller). How can I run it multi-threaded efficiently? In the terminal where I call my executable I write 'mkl_set_num_threads = 8' for example. It appears that Im allocated to8 nodes, but Iamactually using only 1. Because the usage profile shows only around %12. I dont get any speed-up at all.

I really want to make the best use of the MKL library, but I couldnt manage it. Mainly because I don't have any experience with it.

calling Pardiso parallel from Java on Linux environment

Gennady_F_Intel — Tue, 27 Jul 2010 15:32:55 GMT

Well, I agree with this conditions, the memory consumption will be relatively negligible.

1)What I d ask you to to:

Could you please check the execution time for each phase: Reordering and Symbolic Factorization, numerical factorization and solution?

2) second:

You wrote: In the terminal where I call my executable I write 'mkl_set_num_threads = 8' for example

You have to call this function from your application.

For example:

#include

int foo(){

int nthr = 8;

mkl_set_num_threads (Nthr );

pardiso()

}

--Gennady

calling Pardiso parallel from Java on Linux environment

eunsal — Thu, 29 Jul 2010 16:45:23 GMT

1) How can I get that information regarding the times in detail?

2) I now upgraded to the version 11.1 and everything is running fine, little bit faster compared to 10.0.4 actually. I tried to implementyour code:

#include

int foo(){

int nthr = 8;

mkl_set_num_threads (Nthr );

pardiso()

}

at different places. The wrapper has an interface in Java and the JNI in C. Am I supposed to create a new C file with this function in it? or implement it in the JNI C file? From JNI it didnt work. I also tried to pass it from Java interface to JNI , but no success either.

with pardiso(), are you refering to the Java interface or JNI file?