Example does not run

[Georgios_S_]

I am trying to run ScaLAPACK, through MKL. Since I have the student version, I had to install MPICH2, since what I want to do is to apply an inverse of a matrix, that can be done with Cholesky in a distributed environment.

I got this ex1.c from Intell 11. It compliles fine, but it fails to execute:

gsamaras@pythagoras:~/konstantis$ ../mpich-install/bin/mpicc -o test ex1.c -I../intel/mkl/include  ../intel/mkl/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group ../intel/mkl/lib/intel64/libmkl_intel_ilp64.a ../intel/mkl/lib/intel64/libmkl_core.a ../intel/mkl/lib/intel64/libmkl_sequential.a -Wl,--end-group ../intel/mkl/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -lpthread -lm -ldl
gsamaras@pythagoras:~/konstantis$ mpiexec -n 4 ./test
{    0,    0}:  On entry to 
DESCI{    1,    0}:  On entry to NIT parameter number    9 had an illegal value 
{    0,    0}:  On entry to 
DESCINIT parameter number    4 had an illegal value 
  0  0 vals   1.00000   0.20100   0.50100   0.60100   0.10200   2.00000   0.50200   0.60200   0.10500   0.20500   5.00000   0.60500   0.10600   0.20600   0.50600   6.00000

DESCINIT parameter number    4 had an illegal value 
  0  0 vals   1.00000   0.20100   0.50100   0.60100   0.10200   2.00000   0.50200   0.60200   0.10500   0.20500   5.00000   0.60500   0.10600   0.20600   0.50600   6.00000
{    0,    1}:  On entry to 
DESCINIT parameter number    6 had an illegal value 
{    0,    1}:  On entry to 
DESCINIT parameter number    6 had an illegal value 
  0  1 vals   0.10300   0.20300   0.50300   0.60300   0.10400   0.20400   0.50400   0.60400   0.10700   0.20700   0.50700   0.60700   0.10800   0.20800   0.50800   0.60800
{    1,    1}:  On entry to {   -1,   -1}:  On entry to 
PSGESV
 parameter number  602 had an illegal value 
*** Error in `./test': free(): invalid next size (fast): 0x00000000021a6ae0 ***
DESCINIT parameter number    6 had an illegal value 
{    1,    1}:  On entry to 
DESCINIT parameter number    6 had an illegal value 
  0  1 vals   0.10300   0.20300   0.50300   0.60300   0.10400   0.20400   0.50400   0.60400   0.10700   0.20700   0.50700   0.60700   0.10800   0.20800   0.50800   0.60800
{   -1,   -1}:  On entry to 
PSGESV parameter number  602 had an illegal value 
*** Error in `./test': free(): invalid next size (fast): 0x000000000195dae0 ***

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 13140 RUNNING AT pythagoras
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Aborted (signal 6)

Moreover, the same happens with Ying's example from here, which terrifies me:

gsamaras@pythagoras:~/konstantis$ ../mpich-install/bin/mpic++ -o test n.cpp -I../intel/mkl/include  ../intel/mkl/lib/intel64/libmkl_scalapack_ilp64.a -Wl,--start-group ../intel/mkl/lib/intel64/libmkl_intel_ilp64.a ../intel/mkl/lib/intel64/libmkl_core.a ../intel/mkl/lib/intel64/libmkl_sequential.a -Wl,--end-group ../intel/mkl/lib/intel64/libmkl_blacs_intelmpi_ilp64.a -lpthread -lm -ldl
n.cpp: In function ‘int main(int, char**)’:
n.cpp:96:48: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
     Cblacs_gridinit(&ictxt, "Row", nprow, npcol);
                                                ^
gsamaras@pythagoras:~/konstantis$ mpiexec -n 1 ./test
Returned:  Hello World! I am 0 of 1
here

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 13605 RUNNING AT pythagoras
=   EXIT CODE: 8
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Floating point exception (signal 8)

and this happens at this line of the code:

printf("here\n");
    locR = numroc_(&matrix_size, &block, &myrow, &izero, &nprow);
printf("here1\n");

I wanted to execute these examples, so that I know that I am good to go for developing my code. Is there a problem with the examples (which I do not really need to run), or is there a problem with MKL's ScaLAPACK?

[Ying_H_Intel · Accepted solution]

Hi Georgios, 

The int in C are supposed to be 32bit, you may try lp64 mode. 

 mpicc -o test_lp64 ex1.c -I/opt/intel/mkl/include  /opt/intel/mkl/lib/intel64/libmkl_scalapack_lp64.a -L/opt/intel/mkl/lib/intel64 -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_sequential.a -Wl,--end-group /opt/intel/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -lpthread -lm -ldl

Best Regards,

Ying 

[yhu5@prc-mic01 scalapack]$ mpirun -n 4 ./test_lp64                               0  0 vals   1.00000   0.20100   0.50100   0.60100   0.10200   2.00000   0.50200   0.60200   0.10500   0.20500   5.00000   0.60500   0.10600   0.20600   0.50600   6.00000
  1  0 vals   0.30100   0.40100   0.70100   0.80100   0.30200   0.40200   0.70200   0.80200   0.30500   0.40500   0.70500   0.80500   0.30600   0.40600   0.70600   0.80600
  0  1 vals   0.10300   0.20300   0.50300   0.60300   0.10400   0.20400   0.50400   0.60400   0.10700   0.20700   0.50700   0.60700   0.10800   0.20800   0.50800   0.60800
  1  1 vals   3.00000   0.40300   0.70300   0.80300   0.30400   4.00000   0.70400   0.80400   0.30700   0.40700   7.00000   0.80700   0.30800   0.40800   0.70800   8.00000
x=( 0 0)  0.9321  0.3773  0.0444  0.0074
x=( 1 0)  0.1924  0.0999 -0.0190 -0.0388

[Georgios_S_]

Hi Ying,

 thanks, that's solved the issue. Moreover, I was able to run your example, nice! Can you say me what we said to the compiler to do differently in your answer, so that I understand what I am doing.

  Also I noticed that you had a part that I didn't have in the compilation command, i.e. -L/opt/intel/mkl/lib/intel64, which in my case had to be -L../intel/mkl/lib/intel64. I noticed that things were OK with and without it. Why should we use it?

Thanks,

George

PS - damn you are good, it would take me much time to find out! :)

[TimP]

If you have installed and set up the intel compiler paths, the L path is taken care of.  Ilp64 libraries require long long int arguments while lp64 accepts 32 bit intel. N

[Ying_H_Intel]

Hi George, 

You are welcome. You may find the expected link line by Link advisor: https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/

for example, it will give you

 -m64  -I${MKLROOT}/include -Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -lmkl_scalapack_lp64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lmkl_blacs_intelmpi_lp64 -lpthread -lm. 

 Here -L dir,  means   Add directory dir to the list of directories to be searched for -llibrary

And it is fine to use the explicit path of library as your command. 

-I/opt/intel/mkl/include  /opt/intel/mkl/lib/intel64/libmkl_scalapack_lp64.a  -Wl,--start-group /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/mkl/lib/intel64/libmkl_core.a /opt/intel/mkl/lib/intel64/libmkl_sequential.a -Wl,--end-group /opt/intel/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a -lpthread -lm -ldl

So  you are exactly right the "-L../intel/mkl/lib/intel6" is not needed with the explicit path, please ignore it. 

Best Regards,

Ying