Intel® oneAPI Math Kernel Library
Ask questions and share information with other developers who use Intel® Math Kernel Library.

Pardiso Crash in MKL 2017

schulzey
New Contributor I
2,266 Views

After upgrading to MKL 2017 (as part of the 2017 Intel Fortran compiler for Windows) I get a Pardiso crash when calling it to solve a simple matrix. The crash doesn't happen in MKL 11.3.4 or in any other earlier versions.

I have created a test example that demonstrates the crash (see attachment). The data that gets passed to Pardiso (which is normally generated by my application) is read from a file in my test example which is also attached. The test example runs Ok in MKL 11.3.4 and earlier versions but crashes in MKL 2017.

I have been using Pardiso successfully for many years and don't think I am doing anything wrong.

Peter

0 Kudos
37 Replies
Gennady_F_Intel
Moderator
1,501 Views

Do you mean this is some sort of regression problem and this case passed well with previous updates? 

0 Kudos
Gennady_F_Intel
Moderator
1,501 Views

 

the light testing your example, shows no problems.  Win, 64 bit, lp64 mode. 

the only changes I did:  made the program instead of subroutine, added mkl_get_version_string and prints for output array.

here is the results:

_Forums\u696757>1.exe
 **********************************************
ntel(R) Math Kernel Library Version 2017.0.0 Product Build 20160801 for Intel(R) 64 architecture applications

 **********************************************
.... before Pardiso call .......
 after Pardiso call, phase == 13, ERROR ==            0
 **********************************************
 **********************************************
 after Pardiso call, Phase ==-1, ERROR ==            0
 ... X  ...==  0.000000000000000E+000 -1.130282459811037E-003
 3.526077288086858E-003 -3.592627831828867E-005  0.000000000000000E+000
 0.000000000000000E+000  0.000000000000000E+000 -5.602969108474065E-004
 2.777262958946455E-003 -6.643974549978017E-005  0.000000000000000E+000
 0.000000000000000E+000  1.149659736043504E-003  2.549216482510058E-003
-4.677457951176992E-003  0.000000000000000E+000  0.000000000000000E+000
 0.000000000000000E+000  4.259138634320509E-003  2.111724010591131E-003
-1.320197447645458E-002  0.000000000000000E+000  0.000000000000000E+000
 0.000000000000000E+000  5.987196295190753E-003  1.128306758578033E-003
-3.235162985183892E-002  0.000000000000000E+000  0.000000000000000E+000
 6.276281397749442E-003  1.021550220088535E-010 -5.476862158031406E-002
 0.000000000000000E+000  0.000000000000000E+000  8.459754026750180E-014
-1.128306554232226E-003 -3.235162985252373E-002  0.000000000000000E+000
 0.000000000000000E+000 -6.276281397759122E-003 -2.111723799051891E-003
-1.320197305531472E-002  0.000000000000000E+000  0.000000000000000E+000
 0.000000000000000E+000 -5.987196127432409E-003 -2.549216311189985E-003
-4.677461520114669E-003  0.000000000000000E+000  0.000000000000000E+000
 0.000000000000000E+000 -4.259139853882195E-003 -2.777262844894757E-003
-6.643974549908814E-005  0.000000000000000E+000  0.000000000000000E+000
-1.149659751248887E-003 -3.526077231058885E-003 -3.592627831794265E-005
 0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000
 5.602968956403245E-004  0.000000000000000E+000  1.130282444603389E-003

0 Kudos
schulzey
New Contributor I
1,501 Views

I'm not quite sure in what context you are using the term regression, but yes my test example works well in all previous versions of Pardiso. The problem only started in the 2017 version.

My test case is a small 65 x 65 symmetrical banded matrix with a half bandwidth of 12. I'm using CSR storage and calling Pardiso with mtype=2 (real and symmetric positive definite matrix).and phase=13. If I call Pardiso with phase=12 then it still crashes.
 

0 Kudos
schulzey
New Contributor I
1,501 Views

The results you got are identical to mine if I use an earlier version of Pardiso. Do you have any idea of why my version would be crashing? It is very consistent.

0 Kudos
Gennady_F_Intel
Moderator
1,501 Views

- regression means if 11.3.4 fails but the previous update 11.3.3 works as expected...

- I still don't see the problem on my side. here are the conditions which I use for validation your case:

window 8.1, MKL version 2017; command line build and execution; 

linking with /Qmkl or "mkl_core.lib mkl_intel_lp64.lib mkl_intel_thread.lib libiomp5md.lib"

CPU - AVX2 based system.

- Could you give the environment you use when the problem with 2017 happens?

 

 

0 Kudos
schulzey
New Contributor I
1,501 Views

My environment is Windows 10-64 with Visual Studio 2015. The VS command line is:

ifort /nologo /O2 /fpp /warn:noalignments /align:rec1byte /fp:strict /module:&quotx64\Release\\&quot /object:&quotx64\Release\\&quot /Fd&quotx64\Release\vc140.pdb&quot /libs:dll /threads /c /Qlocation,link,&quotC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\\bin\amd64&quot /Qm64 &quotC:\work\PardisoTest.FOR&quot

Could you send me your modified source code and command lines so that I can try your method?

0 Kudos
mecej4
Honored Contributor III
1,501 Views

No crashes with IPS2017 on W10-X64, run after compiling 32- or 64-bit EXEs.

0 Kudos
schulzey
New Contributor I
1,501 Views

C'mon guys, I know what regression means. I was asking about the context in case Gennady was asking if my application uses Pardiso to do some sort of regression analysis.

0 Kudos
Gennady_F_Intel
Moderator
1,501 Views

the attached is your example I used.

compile and link lines:

ifort /Qmkl PardisoTest_v1.for 

or

ifort /I "..\windows\mkl\include" PardisoTest_v1.for mkl_intel_lp64.lib mkl_intel_thread.lib  mkl_core.lib \libiomp5md.lib 

0 Kudos
schulzey
New Contributor I
1,501 Views

After further testing I have been able to establish that the Pardiso crash only happens in the 64-bit version if I use /Qmkl:parallel. If I use /Qmkl:sequential then it works Ok. I have also established that the problem occurs in MKL 11.3.4 and MKL 2017, whereas earlier versions seem to be Ok.

The problem for me is that running Pardiso in sequential mode is too slow for my calculation intensive application.

​Can you see if you can replicate this?

Peter

0 Kudos
schulzey
New Contributor I
1,501 Views

After a bit more testing I have found that the Pardiso crash only happens if I compile and link in Visual Studio but not if I do it with a command line prompt (eg. ifort /Qmkl:parallel PardisoTest.for). The problem therefore seems to be related to something in the Visual Studio 2015 implementation. I have used default VS settings everywhere except for turning on MKL with /Qmkl:parallel. I have attached my VS2015 solution which demonstrates the crash.

Peter

0 Kudos
Ferris_H_
Beginner
1,501 Views

I wonder if your issue is related to the one I just posted about. Try changing the OMP_NUM_THREADS to see if only certain values cause it to crash like with me.

Edit: Also try changing iparm[0]= 1 or 0 to see if one works.

0 Kudos
schulzey
New Contributor I
1,501 Views

Interesting... setting OMP_NUM_THREADS ​= 1 makes my example work with /Qmkl:parallel but anything greater than 1 (or leaving it out altogether) makes it crash. My system is an i7 3537U with 4 cores.

I think setting iparm[0]=1 might be a red herring. It just means that you have to set all the other iparm values yourself rather than using the default values. Are you calling PardisoInit before Pardiso and, if not, are you initializing pt to zeroes first?

I just noticed that someone else reported a similar problem back in November 2015 (see link below):

http://stackoverflow.com/questions/33539521/intel-mkl-pardiso-wont-run-parallel-in-fortran

0 Kudos
Ferris_H_
Beginner
1,501 Views

I am not using Pardisoinit, but I am initializing pt to 0's:

  for(i=0;i<64;i++){pt=0;}


For some reason setting iparm[0]=1 allows me to use any value for OMP_NUM_THREADS ( I do not set any other iparm values ). If I keep it at zero, then I get the crash for OMP_NUM_THREADS=2 or 3.

0 Kudos
Gennady_F_Intel
Moderator
1,501 Views

Peter, I reproduced the problem with the example you provided at #12. 

The exception happens when the code links with threaded versions of mkl as you said. In the case if of sequential libs or MKL_NUM_THREADS = 1, Pardiso works as expected. 

when you link with threaded versions of mkl's libs, as a temporarily workaround, we may recommend to set      call mkl_set_num_threads( 1 ) before the first Pardiso call and then restore the #of threads.

wbr, Gennady

0 Kudos
schulzey
New Contributor I
1,501 Views

I can't really use sequential or threads=1 because that will slow down my application too much.

Do you know why if I compile and link with a command prompt and still use /Qmkl:parallel it seems to work? Is there a particular VS2015 compiler setting that is causing the problem that I could possibly change to avoid the problem and get a result similar to the command prompt method (other than changing to sequential or one thread)?

0 Kudos
Alexander_K_Intel2
1,501 Views

Hi,

looks like your forgot to add include file in your example. After some slight change of file (in attach) to remove warning, example passed correctly on all mkl version that I have

Thanks,

Alex 

schulzey wrote:

I can't really use sequential or threads=1 because that will slow down my application too much.

Do you know why if I compile and link with a command prompt and still use /Qmkl:parallel it seems to work? Is there a particular VS2015 compiler setting that is causing the problem that I could possibly change to avoid the problem and get a result similar to the command prompt method (other than changing to sequential or one thread)?

0 Kudos
schulzey
New Contributor I
1,501 Views

Hi Alex, it still crashes in my Visual Studio application if I have the extra include line in there or not. You seem to be compiling it with a command line but have you tried it with my VS project (see post #12 above). Compiling it with a command line works for me too, but it is the VS implementation that is the problem.

0 Kudos
Alexander_K_Intel2
1,501 Views

Schulzey,

Can I ask you to send me project with link mkl which differ from default one? I see similar behaviour in case of linking non-default MKL without removing check mark in "Use Intel Math Kernel Library" field.

Thanks,

Alex

0 Kudos
schulzey
New Contributor I
1,445 Views

Alex,

I always have "Use Intel Math Kernel Library" set to "Parallel". I am able to change to an older version of the MKL library by changing the Include and Library paths in the Visual Studio "Tools" => "Options" settings for the Intel Visual Fortran compiler. When running the test program I also make sure I am using the MKL runtime DLLs to match the version of MKL I have compiled with. See attached images.

​If I use MKL 11.3.4 or MKL 2017 my test program crashes, but if I go back to any older version of MKL then it works Ok. I am using the Intel Fortran compiler 17.0.0.109.

​Changing the VS Tools => Options settings doesn't change the project and so there is no point uploading it again.

Regards,
Peter
 

0 Kudos
Reply