Modification of Makefile for Parallel Studio XE 2015

[burak_k_]

Dear smart MKL friends,
I have a very large C++ code written by someone else in MS visual studio. I converted the code in a way that it worked perfectly on UNIX clusters. Now, I want it to work on my Laptop’s Kubuntu Kernel. However, I couldn’t make it work and I decided to seek some online help.
Here is my software environment on my laptop:
•    Kubuntu 14.04 LTS
•    Compiler: Mpic++
•    Intel® Parallel Studio XE 2015 Composer Edition for Linux
•    Terminal (no Eclipse)

Necessary part of the makefile and how I linked the libraries can be found below:

#compiler
CC=mpic++
CFLAGS=-c -g -O0 -I./ParaSails
LFLAGS= -L/usr/global/intel/mkl/2011.4/lib/intel64/ 
-L/usr/global/intel/mkl/10.3.1.107/mkl/lib/intel64 -L./ParaSails 
-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread -lguide 
-openmp -lParaSails

all : cbm.x
cbm.x : frprop.o gmres.o input.o  mtxslv.o resvoir.o ccoal.o  
$(CC) $(LFLAGS) -fPIC frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o -o cbm.x -lParaSails

Linking part of the makefile for the solver (Parasails) can be found below:

BLASLIBFLAGS = -L/usr/global/intel/mkl/2011.4/lib/intel64/ 
-L/usr/global/intel/mkl/10.3.1.107/mkl/lib/intel64 
-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread –openmp

When I make my executable on my laptop, I receive the following fatal error:

./Parasails.c 1319: undefined reference to `dgels_'

I believe it is a problem with Lapack or Blas libraries. I tried different linkings to make it work but I couldn’t solve the problem. I spent good 4 days on this issue, searched every possible answer on the Internet, but I think it is time to seek for some help. I was wondering, maybe, someone could create sometime for my problem and help me to modify the links on my makefile for the given software environment above.
Thanks you in advance and best regards,
Burak

[TimP]

The long ago libguide and more recent libiomp5 are mutually exclusive. If your mpi was built with icpc, the openmp option takes care of the openmp and pthread requirements.

The authority on mkl linking is the mkl link advisor. 

if you continue to have difficulty, please quote at least the actual link command passed to ld.

[mecej4]

The following remarks may not apply to the old versions of ICC and MKL that you are using.

Your Parasails.c seems to contain one or more calls to the Fortran subroutine DGELS of Lapack, rather than calling LAPACKE_dgels(), which would be more natural for using from C. However, as long as you pass the arguments to dgels() that the Fortran subroutine expects, that should be OK.

I tried with an example code in C that contained a call to dgels_(), using just the simple -mkl option to icc. The program was compiled and linked with the Intel C compiler with no problems. Perhaps you should report the build messages in full.

[burak_k_]

Dear all,

I have been playing with the links by using Intel® Math Kernel Library Link Line Advisor. Still working on it. Here is the build messages in full.

$.../pcbm$ make
mpic++ -c -g -O0 -I./ParaSails  frprop.cpp
mpic++ -c -g -O0 -I./ParaSails  gmres.cpp
mpic++ -c -g -O0 -I./ParaSails  input.cpp
mpic++ -c -g -O0 -I./ParaSails  mtxslv.cpp
mpic++ -c -g -O0 -I./ParaSails  resvoir.cpp
mpic++ -c -g -O0 -I./ParaSails  ccoal.cpp
mpic++ -L/opt/intel/mkl/lib/intel64/ -L./ParaSails -lmkl_rt -lpthread -lParaSails -lm -fPIC frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o -o cbm.x -lParaSails
./ParaSails/libParaSails.a(ParaSails.o): In function `ComputeValuesNonsym':
/home/burakkulga/Phd/pcbm/ParaSails/ParaSails.c:1319: undefined reference to `dgels_'
collect2: error: ld returned 1 exit status
make: *** [cbm.x] Error 1

Thank you and best regards,

Burak

[mecej4]

I think that that the order in which the libraries are listed on line-8 of the shell commands in #4 is incorrect. The .o files must be listed before the library files. For example, try:

mpic++ -fPIC frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o -L/opt/intel/mkl/lib/intel64/ -L./ParaSails  -lParaSails -lmkl_rt -lpthread -lm  -o cbm.x

[burak_k_]

Dear mecej4,

I tried your last recommendation, I received the same error. Here is my makefile that was working on my university's UNIX clusters:

CC=mpic++
CFLAGS=-c -g -O0 -I./ParaSails
LFLAGS=-L/usr/global/intel/mkl/2011.4/lib/intel64/
-L/usr/global/intel/mkl/10.3.1.107/mkl/lib/intel64
-L./ParaSails
-lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread -lguide -openmp
-lParaSails

all : cbm.x

cbm.x : frprop.o gmres.o input.o  mtxslv.o resvoir.o ccoal.o  
    $(CC) $(LFLAGS) -fPIC frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o -o cbm.x -lParaSails
    rm -rf *.o
    rm -rf *.cbm *.txt

Thank you for your time. 

Burak

[mecej4]

If you are asking make to do the build, the line in the makefile that causes linking to occur needs to be fixed. 

$(CC) -fPIC frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o $(LFLAGS) -o cbm.x

[burak_k_]

Dear mecej4,

I have tried your suggestion. Unfortunately, I received the same error.

Do you think I should add the following linkers?

-lmkl_scalapack_ilp64 -lmkl_blacs_intelmpi_ilp64

Thanks,

Burak

[mecej4]

No, unless you are using 8-byte integer variables as arguments to MKL routines, you should not use ILP libraries.

Please post the output from make as in #4, after changing the makefile as suggested in #7.

[burak_k_]

Dear all,

I fixed my problem by fixing the makefile as follows:

MKLROOT = /opt/intel/mkl
LIBS = -L$(MKLROOT)/lib/intel64
INCLUDES = -m64 -I${MKLROOT}/include
#compiler
CC=mpic++

#cflags
CFLAGS=-c -g -O0 -I./ParaSails 

LFLAGS=  -Wl,--no-as-needed -L${MKLROOT}/lib/intel64 -L./ParaSails
  -lmkl_rt -lpthread -lm -lParaSails

all : cbm.x

cbm.x : frprop.o gmres.o input.o  mtxslv.o resvoir.o ccoal.o  
	$(CC) $(LIBS) $(INCLUDES) frprop.o gmres.o input.o resvoir.o mtxslv.o ccoal.o $(LFLAGS) -o cbm.x -lParaSails

I had to export some files as folows:

export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:$LD_LIBRARY_PATH

Thank you for helping me. You guys are awesome.

Happy Holidays!

Burak