cuda + mkl

[ajay_panyala]

Hello,

I have a piece of code that uses both Cuda BLAS and regular BLAS calls from MKL.
When I try to compile (using nvcc - nvidia's C compiler) using the options

nvcc cudaMKL.cu -lcuda -lcudart -lcublas -I$MKL_INC $MKL_LIB/libmkl_intel_ilp64.a $MKL_LIB/libmkl_sequential.a $MKL_LIB/libmkl_core.a -lpthread -DMKL_ILP64 -m64

I get the following warnings and errors ( using MKL 10.3 update 4)

/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:3: warning: null character(s) ignored
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:3: warning: null character(s) preserved in literal
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:22: warning: null character(s) ignored
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:23: error: invalid preprocessing directive #b
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:23: warning: null character(s) ignored
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:24: error: invalid preprocessing directive #b
/opt/compilers/intel/icc/composerxe-2011.4.191/mkl/lib/intel64/libmkl_intel_ilp64.a:23: warning: null character(s) ignored

Is there a workaround for this issue ? I have attached the code (cudaMKL.cu) in case someone wants to have a look.

Thanks
Ajay

[Chao_Y_Intel]

Ajay,

thanks for report. Are you using the 64 bit integer interface or 32 bit interface?

If you are using the 64 bit integer, it should be -DMKL_ILP64 (not -DMKL_ILP64). It is 32 bit integer, the interface library is libmkl_intel_lp64.a Check the article here for some information: http://software.intel.com/en-us/forums/showthread.php?t=83366

We will have further check on the error message here.

Thanks,
Chao

[Chao_Y_Intel]

Hello,

our engineer expert have a check on the code, and it could be compiled by the following ways:

1. Renamed the source file to *.cpp

2. Used -ccbin /usr/bin/gcc in the compile line

nvcc -ccbin /usr/bin/gcc main.cpp $CUDA_DIR/lib64/libcublas.so I$MKLROOT/include $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_sequential.a $MKLROOT/lib/intel64/libmkl_core.a -lpthread -DMKL_ILP64 -m64

Thanks,
Chao

[Roman_D_Intel1]

Another workaroun is to tell nvcc that it should treat the MKL libraries as linker options via --linker-options flag:

nvcc \
    cudaMKL.cu -lcuda -lcudart -lcublas -m64 -DMKL_ILP64 -I$MKL_INC \
    --linker-options $MKL_LIB/libmkl_intel_ilp64.a,$MKL_LIB/libmkl_sequential.a,$MKL_LIB/libmkl_core.a,-lpthread

[ajay_panyala]

Hello Chao and Dubtsov,

Thanks a lot. Both the compilation suggestions work perfectly.

-Ajay

[Ernestina_M_]

Hello

  I am trying to compile a cuda code in NSight. The code uses mkl, cublas and cuda. The configuration of the project is the following;

  Project's include: ~/intel/mkl/include, /usr/local/cuda-6.0/include, /usr/local/cuda-6.0/lib

  Project's libraries: cublas mkl

  Project's library path as ~/intel/mkl/lib/intel64

  When I build the project I have got the following error:

Building target: MyUnmixing
Invoking: NVCC Linker
/usr/local/cuda-6.0/bin/nvcc --cudart static -L~/intel/mkl/lib/intel64 -link -o  "MyUnmixing"  ./src/GPUutil.o ./src/MyUnmixing.o ./src/PCA_Kernels.o ./src/io.o ./src/metodos.o ./src/util.o   -lcublas -lmkl
nvcc warning : The 'compute_10' and 'sm_10' architectures are deprecated, and may be removed in a future release.
/usr/bin/ld: no se puede encontrar -lmkl
collect2: error: ld returned 1 exit status
make: *** [MyUnmixing] Error 1

  How can I build the project which includes the Intel Math Kernel Library in NSight?

   Thanks in advance

   Ernestina

[Chao_Y_Intel]

Ernestina,

"-lmkl" is the options for Intel Compiler.  If you are not using Intel compiler, you can manually adding Intel MKL libraries into the linkage line, for example,  for 64 bit libraries, need to add:

-L~/intel/mkl/lib/intel64   --linker-options $MKL_LIB/libmkl_intel_ilp64.a,$MKL_LIB/libmkl_sequential.a,$MKL_LIB/libmkl_core.a,-lpthread ...

Thanks,
Chao

[Ernestina_M_]

    Thanks Chao. I have followed your indications but now I have more errors when I run

CUDA_PATH=/usr/local/cuda
MKLROOT=~/intel/composer_xe_2015.0.090/mkl
BUILD_DIR=./build

#Cuda includes
CUDA_INCLUDE_DIR=-I. -I$(CUDA_PATH)/include

#BLAS includes
BLAS_INCLUDE_DIR=-I. -I$(MKLROOT)/include
####################
#library search paths
####################
CUDA_LIB_DIR=-L$(CUDA_PATH)/lib64
BLAS_LIB_DIR=-L$(MKLROOT)/lib/intel64  -L$(MKLROOT)/../compiler/lib/intel64

####################
#libraries
####################
CUDALIBS=-lcublas
utilS=  -lpthread  -lm
####################
#other compilation flags
####################
CFLAGS= -Wwrite-strings
MKLFLAGS=-D __MKL
CUDAFLAGS=--gpu-architecture sm_20
LINKERFLAGS= -Wl,--start-group $(MKLROOT)/lib/intel64/libmkl_intel_lp64.a $(MKLROOT)/lib/intel64/libmkl_sequential.a $(MKLROOT)/lib/intel64/libmkl_core.a $(MKLROOT)/../compiler/lib/intel64/libiomp5.a -Wl,--end-group

  icc $(CFLAGS) -c -O3 io.c -o $(BUILD_DIR)/io.o
  icc $(CFLAGS) $(MKLFLAGS) $(BLAS_INCLUDE_DIR) -c -O3 util.c -o $(BUILD_DIR)/util.o
  nvcc $(CUDAFLAGS) $(CUDA_INCLUDE_DIR) -c -O3 metodos.cu  -o $(BUILD_DIR)/metodos.o
  nvcc $(CUDAFLAGS) $(CUDA_INCLUDE_DIR) -c -O3 GPUutil.cu  -o $(BUILD_DIR)/GPUutil.o    
  nvcc $(CUDAFLAGS) $(CUDA_INCLUDE_DIR) -c -O3 PCA_Kernels.cu  -o $(BUILD_DIR)/PCA_Kernels.o
  nvcc $(CUDAFLAGS) $(CUDA_INCLUDE_DIR) -c -O3 PCA.cu  -o $(BUILD_DIR)/PCA.o

  icc $(CFLAGS) $(BUILD_DIR)/io.o $(BUILD_DIR)/util.o $(BUILD_DIR)/metodos.o $(BUILD_DIR)/GPUutil.o $(BUILD_DIR)/PCA_Kernels.o   \
    $(BUILD_DIR)/PCA.o   \
    $(CUDA_LIB_DIR) $(BLAS_LIB_DIR) $(LINKERFLAGS) $(utilS) $(CUDALIBS)  -o PCA

  The execution of the last icc produces the following errors:

  ./build/metodos.o: En la función `VD(double*, double, int, int, int, int)':
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x1b5): referencia a `cudaConfigureCall' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x2f2): referencia a `cudaConfigureCall' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x81f): referencia a `cudaFree' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x82c): referencia a `cudaFree' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x839): referencia a `cudaFree' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x846): referencia a `cudaFree' sin definir
./build/metodos.o: En la función `PCA(float*, float*, int, int, int, int, int)':
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0xe54): referencia a `cudaConfigureCall' sin definir
./build/metodos.o: En la función `SPCA(float*, float*, int, int, float, int, int, int, int)':
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x1502): referencia a `cudaConfigureCall' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x1a3c): referencia a `cudaMemcpy' sin definir
tmpxft_00003dfc_00000000-3_metodos.cudafe1.cpp:(.text+0x1d2d): referencia a `cudaMemcpy' sin definir
./build/metodos.o: En la función `NFINDR(float*, int*, int, int, int, int, int)':
...

 I haven't listed all the errors but the following one are similar .

  Any idea where the problem is. I am using icc 15.0 y CUDA 6.0

   Thanks in advance

   Ernestina

[Chao_Y_Intel]

Ernestina, 

The missed symbols are from the cuda libraries. It seems some required cuda libraries are missed in your likage line, 

You may follow the example Roman provide before to linkage line: 
nvcc \
    cudaMKL.cu -lcuda -lcudart -lcublas -m64 -I$MKL_INC \
    --linker-options $MKL_LIB/libmkl_intel_lp64.a,$MKL_LIB/libmkl_sequential.a,$MKL_LIB/libmkl_core.a,-lpthread
    
Thanks,
Chao

[Najeeb_A_]

Roman Dubtsov (Intel) wrote:

Another workaroun is to tell nvcc that it should treat the MKL libraries as linker options via --linker-options flag:

nvcc \
    cudaMKL.cu -lcuda -lcudart -lcublas -m64 -DMKL_ILP64 -I$MKL_INC \
    --linker-options $MKL_LIB/libmkl_intel_ilp64.a,$MKL_LIB/libmkl_sequential.a,$MKL_LIB/libmkl_core.a,-lpthread

Thanks. This worked for me.