Computational layer library for Cluster FFT not loaded

[Jozsef]

Hello,

I get this run-time error message when I try to call any Cluster FFt function:

Computational layer library for Cluster FFT not loaded
Error: MKL DFTI ERROR: Internal error

The code compiles fine with a link line obtained with link advisor. What am I missing?

MKL library, 10.3.

Thanks,
Jozsef

[Gennady_F_Intel]

Jozsef,

This is an unknown problem.

How did you link the application?

Could you please try to build Cluster DFT examples ( see \examples\cdftc folder)first.

--Gennady

[Jozsef]

Hi Gennady,

test passes in dm_complex_2d_double_ex2.c after "make libem64t mpi=mpich2 example=dm_complex_2d_double_ex2" and mpiexec on 2 cpus.

Then I reproduce the error with the following code:

[cpp]#include "mpi.h"
#include "mkl_cdft.h"

#define ERR(s) {printf("Error: %sn",s); exit(-1);}

static int nprocs, rank;

int main( int argc, char *argv[] )
{
  DFTI_DESCRIPTOR_DM_HANDLE desc;


  if (MPI_Init(&argc,&argv) != MPI_SUCCESS)
    ERR("Can't initialize MPI!");


  MPI_Comm_size(MPI_COMM_WORLD, &nprocs);
  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
  if (rank == 0)
    printf(" * number of MPI processes: %dn",nprocs);


  MKL_LONG len[2];
  len[0]=10;
  len[1]=8;
  DftiCreateDescriptorDM(MPI_COMM_WORLD,&desc,DFTI_DOUBLE,DFTI_COMPLEX,2,len);


  DftiFreeDescriptorDM(&desc);


  MPI_Finalize();

  return 0;
}[/cpp]

Compile & link:
mpicc -c test.c; mpicc -o test test.o -lmkl_intel_ilp64 -lmkl_cdft_core -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_ilp64 -openmp -lpthread

Run:
$ mpiexec -n 2 ./test
* number of MPI processes: 2
Computational layer library for Cluster FFT not loaded
Computational layer library for Cluster FFT not loaded

That's, I suppose, for the two CDFT calls.

I use mpich2 compiled from source with CC=icc.
Let me know if/how I can give you more info.

Thanks,
Jozsef

[Gennady_F_Intel]

what would be if you add thegrouping symbols-Wl,--start-group and-Wl,--end-group into the linking line...

see what Linker Adviser recomends:

-Wl,--start-group $MKLroot/libmkl_intel_ilp64.a $MKLroot/libmkl_cdft_core.a $MKLroot/libmkl_intel_thread.a $MKLroot/libmkl_core.a $MKLroot/libmkl_blacs_intelmpi_ilp64.a -liomp5 -lpthread

[Jozsef]

Now I used:

mpicc -o test test.o -L$MKLROOT -Wl,--start-group -lmkl_intel_ilp64 -lmkl_cdft_core -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_ilp64 -Wl,--end-group -openmp -lpthread

Still not loaded.

[Jozsef]

Seems like it can create the descriptor fine, but using it fails:

I run the following:

[cpp]if (rank == 0)
    printf(" * number of MPI processes: %dn",nprocs);

  MKL_LONG size, status;
  MKL_LONG len[2];
  len[0]=10;
  len[1]=8;

  status = DftiCreateDescriptorDM(MPI_COMM_WORLD,&desc,DFTI_DOUBLE,DFTI_COMPLEX,2,len);
  printf("DftiCreateDescriptorDM status: %sn",DftiErrorMessage(status));
printf("--n");
  status = DftiGetValueDM(desc,CDFT_LOCAL_SIZE,&size);
  printf("DftiGetValueDM status: %sn",DftiErrorMessage(status));
  printf("size = %ldn",size);
printf("--n");
  status = DftiFreeDescriptorDM(&desc);
  printf("DftiFreeDescriptorDM status: %sn",DftiErrorMessage(status));[/cpp]

which gives the output:

* number of MPI processes: 1
DftiCreateDescriptorDM status: MKL DFTI SUCCESS: No error
--
Computational layer library for Cluster FFT not loaded
DftiGetValueDM status: MKL DFTI ERROR: Internal error
size = 140736815161303
--
Computational layer library for Cluster FFT not loaded
DftiFreeDescriptorDM status: MKL DFTI ERROR: Internal error

[Jozsef]

Alright, we are making progress. The above testcase only fails with dynamic linking. That also explains why the cdft examples work. So now I'm switching to static link, but it would be good to know why the dynamic fails.

Thanks,
Jozsef

[Vladimir_Petrov__Int]

Jozsef,

There are two ways to resolve the issue:
- when you run the executable set environment variable LD_DYNAMIC_WEAK to 1
- or change the libraries order to put -lmkl_core_cdft in front of -lmkl_intel_ilp64

The root cause is that glibc later than 2.1.91 does not allow weak symbols overriding by default.

Please note also that in order to use ILP64 interface of MKL you have to define MKL_ILP64.

Best regards,
-Vladimir

[Jozsef]

Vladimir,

the order -lmkl_cdft_core -lmkl_intel_ilp64 works.

Thanks for clearing the issue,
Jozsef