topic Evarist, thankyou very much in Intel® oneAPI Math Kernel Library

HPCC - use mkl=cluster and crashing in

John_H_19 — Tue, 17 Nov 2015 04:04:18 GMT

I am trying to run the HPCC benchmark on our cluster.

The system has Intel compilers and MKL release 2015.5.223

I am compiling with Openmpi version 1.8.5 also (though I do not think this is the problem)

I am following https://software.intel.com/en-us/articles/performance-tools-for-software-developers-use-of-intel-mkl-in-hpcc-benchmark

I have compiled OK. This seems a bit out of date though.

With up to date MKL should I compile with -mkl=cluster for HPCC?

Are there some more up to date writeups please?

When I run it crashes in the MPIFFT portion of the run.

For what its worth:

[comp06:10941] *** Process received signal ***

[comp06:10941] Signal: Segmentation fault (11)

[comp06:10941] Signal code: Address not mapped (1)

[comp06:10941] Failing at address: 0x9b

[comp12:10910] [comp09:10876] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp09:10876] [ 1] [comp12:10928] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp12:10928] [ 1] [comp09:10890] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp09:10890] [ 1] [comp13:10948] Signal: Segmentation fault (11)

[comp13:10948] Signal code: Address not mapped (1)

[comp13:10948] Failing at address: 0x9b

[comp07:10876] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2aaaaacde130]

[comp07:10876] [ 1] [comp06:10945] *** Process received signal ***

[comp06:10945] Signal: Segmentation fault (11)

[comp06:10945] Signal code: Address not mapped (1)

[comp06:10945] Failing at address: 0x9b

/cm/shared/apps/openmpi/intel/64/1.8.5/lib64/libmpi.so.1(MPI_Comm_size+0x59)[0x2aaaab250969]

[comp12:10928] [ 2] ./hpcc[0x198e560]

[comp12:10928] [ 3] ./hpcc[0x4537f4]

Hi John,

Evarist_F_Intel — Tue, 17 Nov 2015 07:13:05 GMT

Hi John,

compiler option -mkl=cluster works for Intel MPI only, since it links an application with -lmkl_blacs_intelmpi_lp64. Since you are using OpenMPI, please specify mkl libraries explicitly, i.e. instead of -mkl=cluster use

-L$MKLROOT/lib/intel64 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_openmpi_lp64 ...

Please take a look at Intel MKL Link-line adviser: https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

Evarist, thankyou very much

John_H_19 — Tue, 17 Nov 2015 07:41:05 GMT

Evarist, thankyou very much for that advice.

I could use the Intel MPI.

What would I use for options then please?

Also, one other bit of advice

John_H_19 — Tue, 17 Nov 2015 07:42:39 GMT

Also, one other bit of advice.

When building HPCC should I use the ilp64 or the lp64 interface?

There seems to be some conflicting advice! I have 380 cores to run on.

Hi John,

Ying_H_Intel — Tue, 17 Nov 2015 08:46:56 GMT

Hi John,

By default, it is LP64. (ILP64 is int64)

could you please let me know what is conflicting advice?

Thanks

Ying

${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_cdft_core.a ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_intel_thread.a ${MKLROOT}/lib/intel64/libmkl_blacs_openmpi_lp64.a -Wl,--end-group -lpthread -lm

Ying,, thankyou for your

John_H_19 — Tue, 17 Nov 2015 09:49:45 GMT

Ying,, thankyou for your reply.

Please have a look at the comments at the end of this article - it does say ilp64 !

https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/287304

Hi John,

Ying_H_Intel — Wed, 18 Nov 2015 06:18:03 GMT

Hi John,

I see the problem. LP64 is 32bit integer, Long64bit Pointer64bit. In most of case, we use 32bit integer, so LP64 library are used usually.

If you do set the option explicitly like the article https://origin-software.intel.com/en-us/articles/performance-tools-for-software-developers-use-of-intel-mkl-in-hpcc-benchmark ,

, -DMKL_INT=long -DLONG_IS_64BITS . where MKL_INT=long is 64bit, then yes, the correct library should be ILP64. (all LP64 should be ILP64).

Best Regards,

Ying