Community
cancel
Showing results for 
Search instead for 
Did you mean: 
136 Views

ScaLAPACK for platform_mpi

I test a mpi code MUMPS which uses ScaLAPACK. MUMPS crashes as long as I use more than one processor. I suspect I didn't link the right version of ScaLAPACK and BLACS. I am using platform_mpi and couldn't find BLCAS lib for paltform_mpi. Instead i use intelmpi BLACS. I suspect intelmpi BLACS is not comptatible with platform_mpi. Here is my linking line for ScaLAPACK and BLACS:
-L/u/xeons03/people/xian/intel/composerxe-2011.1.107/mkl/lib/intel64/ -lmkl_scalapack_lp64 -lmkl_gf_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -L/u/xeons03/people/xian/intel/composerxe-2011.1.107/compiler/lib/intel64/ -liomp5 -lpthread -lm

Any idea what's going wrong?

Thanks
0 Kudos
6 Replies
TimP
Black Belt
136 Views

If you're using Intel Fortran, you can't use a gfortran library; in any case, you can't use the gfortran and Intel versions together. I don't expect compatibility between Platform MPI and any of the MPI versions supported by MKL. Of course, the MKL libraries which don't use MPI should be compatible.
136 Views

Does this mean if I use platform_mpi, I have to recompile BLACS and ScaLAPACL using platform_mpi myself?
136 Views

It seems you're right. The only way is to download ScaLAPACK & BLACS from NETLIB, build them with platform_mpi and change MKL scalapack and blacs libraries with the built ones, but leave other MKL libraries as is in order to use MKL BLAS.
Regards,
Konstantin
Alexei_Yakovlev
Beginner
136 Views

Actually, you can use the MKL's BLACS/ScaLAPACK if you use the Platform's include files and libraries from the MPICH1.2 subdirectory. That's assuming you use Platform-MPI 7.1 a.k.a HP-MPI. We've been using MKL with HP-MPI this way for several years now and had no trouble with this combo.

Alexei
136 Views

Alexei,
I was able to combine MKL ScaLAPACK with platform_mpi using mpich2 wrapper. However, Platform suggests recompiling netlib BLACS/ScaLAPACK using platform_mpi due to overhead of mpich2 wrapper. Do you notice any ovehead?

Sam
Alexei_Yakovlev
Beginner
136 Views

Sam,
I did not notice any significant overhead. In response to my question to HP-MPI developers in 2007 was:
  As far as performance penalty, there IS a thin binary translation layer used, however it should not impact performance too much.
From simple ping-pong tests, I wasn't able to see any measurable effect of the wrappers either.

Alexei

P.S. For Windows, I did have to compile our own BLACS and ScaLAPACK. But then you can still link them with MKL.

Reply