HPCC - use mkl=cluster and crashing in

[John_H_19]

I am trying to run the HPCC benchmark on our cluster.

The system has Intel compilers and MKL release 2015.5.223

I am compiling with Openmpi version 1.8.5 also (though I do not think this is the problem)

I am following https://software.intel.com/en-us/articles/performance-tools-for-software-developers-use-of-intel-mkl-in-hpcc-benchmark

I have compiled OK. This seems a bit out of date though.

With up to date MKL should I compile with    -mkl=cluster     for HPCC?

Are there some more up to date writeups please?

When I run it crashes in the MPIFFT portion of the run.

For what its worth:

[comp06:10941] *** Process received signal ***

[comp06:10941] Signal: Segmentation fault (11)

[comp06:10941] Signal code: Address not mapped (1)

[comp06:10941] Failing at address: 0x9b

[comp12:10910] [comp09:10876] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp09:10876] [ 1] [comp12:10928] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp12:10928] [ 1] [comp09:10890] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp09:10890] [ 1] [comp13:10948] Signal: Segmentation fault (11)

[comp13:10948] Signal code: Address not mapped (1)

[comp13:10948] Failing at address: 0x9b

[comp07:10876] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp07:10876] [ 1] [comp06:10945] *** Process received signal ***

[comp06:10945] Signal: Segmentation fault (11)

[comp06:10945] Signal code: Address not mapped (1)

[comp06:10945] Failing at address: 0x9b

/cm/shared/apps/openmpi/intel/64/1.8.5/lib64/libmpi.so.1(MPI_Comm_size+0x59)[0x2aaaab250969]

[comp12:10928] [ 2] ./hpcc[0x198e560]

[comp12:10928] [ 3] ./hpcc[0x4537f4]

[Evarist_F_Intel]

Hi John,

compiler option -mkl=cluster works for Intel MPI only, since it links an application with -lmkl_blacs_intelmpi_lp64. Since you are using OpenMPI, please specify mkl libraries explicitly, i.e. instead of -mkl=cluster use 

-L$MKLROOT/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_openmpi_lp64 ...

Please take a look at Intel MKL Link-line adviser: https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

[John_H_19]

Evarist,  thankyou very much for that advice.

I could use the Intel MPI.

What would I use for options then please?

[John_H_19]

Also, one other bit of advice.

When building HPCC should I use the ilp64  or the lp64 interface?

There seems to be some conflicting advice!  I have 380 cores to run on.

[Ying_H_Intel]

Hi John, 

By default, it is LP64.  (ILP64 is int64) 

could you please let me know what is conflicting advice? 

Thanks

Ying

${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_cdft_core.a ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a ${MKLROOT}/lib/intel64/libmkl_blacs_openmpi_lp64.a -Wl,--end-group -lpthread -lm

[John_H_19]

Ying,, thankyou for your reply.

Please have a look at the comments at the end of this article - it does say ilp64 !

https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/287304

[Ying_H_Intel]

Hi John, 

I see the problem. LP64 is 32bit integer, Long64bit Pointer64bit.  In most of case, we use  32bit integer, so LP64 library are used usually.

If you do set the option explicitly like the article https://origin-software.intel.com/en-us/articles/performance-tools-for-software-developers-use-of-intel-mkl-in-hpcc-benchmark ,

, -DMKL_INT=long -DLONG_IS_64BITS .   where MKL_INT=long is 64bit,  then yes, the correct library should be ILP64.  (all LP64 should be ILP64). 

Best Regards,

Ying