topic ScaLAPACK for platform_mpi in Intel® oneAPI Math Kernel Library

ScaLAPACK for platform_mpi

xian-zhong_guous_cd- — Mon, 31 Jan 2011 18:06:10 GMT

I test a mpi code MUMPS which uses ScaLAPACK. MUMPS crashes as long as I use more than one processor. I suspect I didn't link the right version of ScaLAPACK and BLACS. I am using platform_mpi and couldn't find BLCAS lib for paltform_mpi. Instead i use intelmpi BLACS. I suspect intelmpi BLACS is not comptatible with platform_mpi. Here is my linking line for ScaLAPACK and BLACS:
-L/u/xeons03/people/xian/intel/composerxe-2011.1.107/mkl/lib/intel64/ -lmkl_scalapack_lp64 -lmkl_gf_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -L/u/xeons03/people/xian/intel/composerxe-2011.1.107/compiler/lib/intel64/ -liomp5 -lpthread -lm

Any idea what's going wrong?

Thanks

ScaLAPACK for platform_mpi

TimP — Mon, 31 Jan 2011 20:23:34 GMT

If you're using Intel Fortran, you can't use a gfortran library; in any case, you can't use the gfortran and Intel versions together. I don't expect compatibility between Platform MPI and any of the MPI versions supported by MKL. Of course, the MKL libraries which don't use MPI should be compatible.

ScaLAPACK for platform_mpi

xian-zhong_guous_cd- — Tue, 01 Feb 2011 21:35:54 GMT

Does this mean if I use platform_mpi, I have to recompile BLACS and ScaLAPACL using platform_mpi myself?

ScaLAPACK for platform_mpi

Konstantin_A_Intel — Wed, 02 Feb 2011 03:10:56 GMT

It seems you're right. The only way is to download ScaLAPACK & BLACS from NETLIB, build them with platform_mpi and change MKL scalapack and blacs libraries with the built ones, but leave other MKL libraries as is in order to use MKL BLAS.

Regards,

Konstantin

ScaLAPACK for platform_mpi

Alexei_Yakovlev — Fri, 04 Feb 2011 17:26:16 GMT

Actually, you can use the MKL's BLACS/ScaLAPACK if you use the Platform's include files and libraries from the MPICH1.2 subdirectory. That's assuming you use Platform-MPI 7.1 a.k.a HP-MPI. We've been using MKL with HP-MPI this way for several years now and had no trouble with this combo.

Alexei

ScaLAPACK for platform_mpi

xian-zhong_guous_cd- — Tue, 08 Feb 2011 18:05:46 GMT

Alexei,
I was able to combine MKL ScaLAPACK with platform_mpi using mpich2 wrapper. However, Platform suggests recompiling netlib BLACS/ScaLAPACK using platform_mpi due to overhead of mpich2 wrapper. Do you notice any ovehead?

Sam

ScaLAPACK for platform_mpi

Alexei_Yakovlev — Wed, 09 Feb 2011 14:18:34 GMT

Sam,
I did not notice any significant overhead. In response to my question to HP-MPI developers in 2007 was:

  As far as performance penalty, there IS a thin binary translation layer used, however it should not impact performance too much.

From simple ping-pong tests, I wasn't able to see any measurable effect of the wrappers either.

Alexei

P.S. For Windows, I did have to compile our own BLACS and ScaLAPACK. But then you can still link them with MKL.