HELP! compile scalapack example1.f ok, run error

[geomech]

Hi!
I am trying to compile the example1.f (www.netlib.org/scalapack/examples/example1.f) using MKL (10.0.1.014). The compilation is error-free. But when I run the executable, a complaint showed up:

BLACS ERROR 'Illegal grid (2 x 3), #procs=1'
from {-1,-1}, pnum=0, Contxt=-1, on line -1 of file 'BLACS_GRIDINIT/BLACS_GRIDMAP'.
[0] MPI Abort by user Aborting program !
[0] Aborting program!
p0_29434: p4_error: : 1

The command I used to compile:
/opt/mpich/p4-intel/bin/mpif77 example1.f -L/opt/intel/mkl/10.0.1.014/lib/em64t -lmkl_scalapack_lp64 -lmkl_blacs_lp64 -lmkl_lapack -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lguide -lpthread

The command to run:
/opt/mpich/p4-intel/bin/mpirun -np 6 a.out

The machine is a Beowulf cluster with Intel Xeon Clovertown E5335 (Quad Core 2.0Ghz) processors. The system is Red Hat Enterprise ES Basic 64-bit Linux OS.

I have no clue at all what's the error message talking about. Really appreciate if somebody here can give me a hint. Thanks!

[Tabrez_Ali]

I dont see anything wrong here. In fact the example runs fine on my computer

$mpif77 example1.f $SCALAPACK
program EXAMPLE1
external subroutine MATINIT
external subroutine SL_INIT
324 Lines Compiled

$ mpirun -np 6 ./a.out
ScaLAPACK Example Program #1 -- May 1, 1997

Solving Ax=b where A is a 9 by 9 matrix with a block size of 2
Running on 6 processes, where the process grid is 2 by 3

INFO code returned by PDGESV = 0

According to the normalized residual the solution is correct.

||A*x - b|| / ( ||x||*||A||*eps*N ) = 5.72737686E-04



Are you sure you can run other mpi programs without any problems. Does a hello world example on 6 processors work okay?

Edit: The only other thing I would check is the linking part.

[geomech]

Hi tabrezali ,
Yes, I have run other mpi programs without problems. In fact, I compiled the example1.f using the libraries of Lapack, Blacs & Scalapack built individually and the resulting a.out gave me the same output as you showed. But I could do that only with g77, not with gfortran. My ultimate goal is to compile a home-grown code including f77 and f90 files. For that purpose I need something like gfortran or mpif90.

So I am trying Intel MKL now. The problem is that this Intel MKL was pre-installed by the company who sold us the cluster. I don't have an installation CD. I am not sure if the problem is caused by any incorrect parameter assigned during the pre-installation. Just a wild guess. I think I have followed the user guide of MKL. I asked the company but they had no clue neither what caused the error.

Actually I don't really understand the linking part you used. What's the "$SCALAPACK"? Yes, there might be some problem in my linking part.

Thank you very much!

[Tabrez_Ali]

mpif90/mpif77 are simply wrappers around compilers

If you want to use netlib's scalapack with gfortran then you first need to build mpich configured with gfortran/gcc and then build blacs/scalapack using mpich and blas/Lapack (MKL or netlib)

If you want to use MKL's scalapack with gfortran/gcc then you still need to first build mpich configured with gfortran/gcc and then link to MKL

In your original post it looks as if you were trying to use mpich configured with ifort/icc and linking to MKL's scalapack which is okay. The error usually means the number of processors the program is being run on is less than what is specified in the code (nprow x npcol)

[geomech]

tabrezali,
Your reply is very illuminating. I will follow what you said and see what happens.

Thanks a lot!

[geomech]

Hi,

Following tabrezali's advice, finally make example1.f run with gfortran. It should apply to ifort as well.

Just a reference forthose who need this:

1.) build MPICH with the configuration:

ARGDECL=" "

./configure -fc=gfortran -f90=gfortran

2.)use netlibscalapack installer (blas & lapack libraries pre-built with gfortran):

python setup.py --mpibindir=/root/MPICH/bin --mpiincdir=/root/MPICH/include --blaslib=/root/LAPACK/blas_LINUX.a --lapacklib=/root/LAPACK/lapack_LINUX.a --downblacs

3.)build example1.f with gfortran (blacsF77.a & blacs.a were obtained from the previous step; libmpich.a was generated in the 1st step)

gfortran example1.f libscalapack.a blacsF77.a blacs.a blacsF77.a blas_LINUX.a lapack_LINUX.a libmpich.a

4.) /root/MPICH/bin/mpirun -np 6 a.out

P.S. if anybody find out the cause to my problem with Intel MKL (inmy very first post), please let me know. I still have no clue how to fix it, though itdoesn't matter too much for now. Thanks!

[Tabrez_Ali]

I do have some notes to build mpich/scalapack with old Intel compilers (7.1) and MKL (7.2). The steps however should work for GNU compilers too.

They are available here

[rubel]

The problem is caused by undetermined number of processes, which is assumed to be 1 instead of 6 that generates the error. I managed to get example1.f running. I added the following part to SUBROUTINE SL_INIT:

INTEGER info
include "mpif.h"
call mpi_init(info)
Call MPI_COMM_SIZE(mpi_comm_world, nprocs, info)
Call MPI_COMM_RANK(mpi_comm_world, iproc, info)

in order to pass the 'nprocs' information from 'mpirun'. Then I compiled it using

/opt/intel/mpich/bin/mpif90 example1.f -I/opt/intel/mpich/include -L /opt/intel/mkl/10/lib/em64t -lmkl_scalapack_lp64
-lmkl_blacs_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lpthread

You need -I/opt/intel/mpich/include in order to pick up "mpif.h". Then you can run it

marc-hn:~/tmp/ScaLAPACK> mpirun -np 6 ./a.out
ScaLAPACK Example Program #1 -- May 1, 1997
Solving Ax=b where A is a 9 by 9 matrix with a block size of 2
Running on 6 processes, where the process grid is 2 by 3
INFO code returned by PDGESV = 0
According to the normalized residual the solution is correct.
||A*x - b|| / ( ||x||*||A||*eps*N ) = 0.00000000E+00

So, it works.

More important, I tried other ScaLAPACK tests, which come with MKL 10, and all of them fail to run due to the same reason. :-( Appropriate modification, as described above, helps to get them running.

Oleg Rubel

[geomech]

Oleg: Thank you very much !