topic An include path is specified in Intel® oneAPI Math Kernel Library

How to correctly compile Quantum Espresso with intel MKL especially intel FFTW

balabi_b_ — Mon, 11 Apr 2016 00:25:06 GMT

I wish to exploit the power of intel mkl in Quantum Espresso, which is a popular open-source first-principle calculation suite.

I found this tutorial named "Quantum ESPRESSO* for Intel® Xeon Phi™ Coprocessor", in the intel document, which is quite in detail, illustrate step by step the setup flow for the software using intel compiler and mkl. Though I don't have Intel® Xeon Phi, But I guess the setup also works for ordinary multicore computer, am I right?

But I encountered problems following that installation tutorial. This maybe due to that the version of my parallel studio is different from that in the tutorial. My version of parallel studio is 2016.2.181. After completely follow the steps, I got this when running "make pw"

ifort: command line warning #10006: ignoring unknown option '-/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/include/fftw'

I notice the FCFLAGS is exported as

export FCFLAGS="-O3 -xAVX -fno-alias -ansi-alias -g -mkl -$MKLROOT/include/fftw"

here -$MKLROOT/include/fftw is not right, because apparently the ifort doesn't know this option.

Investigate further, I found the FFT_LIBS="-L$MKLROOT/intel64" is also wrong, because in version 2016.2.181, there is no intel64 folder under $MKLROOT. So I changed it to $MKLROOT/lib/intel64_lin. However, this still not working. Also after running ./configure --enable-openmp --enable-parallel, there is actually no FFT_LIBS detected. The configure finally shows

The following libraries have been found:
BLAS_LIBS= -lmkl_intel_thread -lmkl_core
LAPACK_LIBS=-lmkl_blacs_intelmpi_lp64
SCALAPACK_LIBS=-lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64
FFT_LIBS=
Please check if this is what you expect.

Why FFT is not being detected? What is wrong? How to compile it correctly?

An include path is specified

TimP — Mon, 11 Apr 2016 01:09:39 GMT

An include path is specified by -I examples are given by mkl link advisor.

Quote:Tim P. wrote:

balabi_b_ — Mon, 11 Apr 2016 07:06:00 GMT

Tim P. wrote:

An include path is specified by -I examples are given by mkl link advisor.

Thank you! Change "-$MKLROOT/include/fftwto "-I$MKLROOT/include/fftw" seems working. When doing "make", the compiler recognized the option. However in the output info of "./configure", there is still no FFT_LIBS detected.

Also the new problem emerge. I got a huge list of undefined reference, which seems to mean all kinds of library is not properly set

mpiifort -static-intel -qopenmp -o pw.x \
pwscf.o libpw.a ../../Modules/libqemod.a ../../FFTXlib/libqefft.a ../../flib/ptools.a ../../flib/flib.a ../../clib/clib.a ../../iotk/src/libiotk.a -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_intel_thread -lmkl_core
libpw.a(update_pot.o): In function `extrapolate_wfcs':
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/PW/src/update_pot.f90:716: undefined reference to `zgesvd_'
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/PW/src/update_pot.f90:782: undefined reference to `zgesvd_'
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/PW/src/update_pot.f90:791: undefined reference to `zgemm_'

/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/FFTXlib/./fft_scalar.FFTW3.f90:128: undefined reference to `dfftw_execute_dft_'
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/FFTXlib/./fft_scalar.FFTW3.f90:124: undefined reference to `dfftw_execute_dft_'
../../FFTXlib/libqefft.a(fft_scalar.o): In function `fft_scalarcft_1z_mp_init_plan_':
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/FFTXlib/./fft_scalar.FFTW3.f90:153: undefined reference to `dfftw_cleanup_threads_'
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/FFTXlib/./fft_scalar.FFTW3.f90:154: undefined reference to `fftw_init_threads'

../../flib/flib.a(dylmr2.o): In function `dylmr2':
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/flib/dylmr2.f90:57: undefined reference to `dcopy_'
/fs10/home/qhw_wang/HPC-nj/quantum_espresso/espresso-5.3.0/flib/dylmr2.f90:81: undefined reference to `daxpy_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_real'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `ssteqr_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dsyr2k_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `cscal_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_op_sum'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `caxpy_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_byte'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_blacs_openmpi_lp64.so: undefined reference to `ompi_mpi_dblprec'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `dsteqr_'
/fs00/software/intel/ps2016u2/compilers_and_libraries_2016.2.181/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: undefined reference to `cbdsqr_'

What should I do?

Could you please change -lmkl

Evgueni_P_Intel — Mon, 11 Apr 2016 15:42:59 GMT

Could you please change -lmkl_blacs_openmpi_lp64 to -lmkl_intel_lp64 ?

If you don't care to study

TimP — Tue, 12 Apr 2016 12:51:00 GMT

If you don't care to study the recommendations of mkl link advisor, you must avoid static link.

Quote:Evgueni Petrov aka

balabi_b_ — Tue, 12 Apr 2016 15:10:10 GMT

Evgueni Petrov aka espetrov (Intel) wrote:

Could you please change -lmkl_blacs_openmpi_lp64 to -lmkl_intel_lp64 ?

Thank you very much!

Now I use

./configure MPIF90=mpiifort F90=mpiifort F77=mpiifort --enable-openmp --enable-parallel --with-scalapack=intel

And the make process is successful now.

But the problem is I still can't correctly invoke MKL's FFT. Quantum espresso still use its own FFT library. I tried several ways to set the library, all failed

Hi

Ying_H_Intel — Fri, 15 Apr 2016 09:18:52 GMT

I think you are right, the setup also works for ordinary multicore computer.

I notice in that article use two place to control the FFT

			
						export FFT_LIBS="-L$MKLROOT/intel64"
					
						14
						./configure --enable-openmp --enable-parallel

4. Edit the make.sys file in the top package directory and modify the DFLAGS variable in order to enable hybrid MPI+openmp mode, fftw3, and scalapack:

iew source

			
						1
						DFLAGS = -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK -D__OPENMP 
					
						2
						BLAS_LIBS = -lmkl_intel_thread -lmkl_core
					
						3
						LAPACK_LIBS = -lmkl_blacs_intelmpi_lp64

So about the FFTW switch may

Ying_H_Intel — Fri, 15 Apr 2016 09:22:47 GMT

So about the FFTW switch may include some trick.

generally,

export FFT_LIBS="-L$MKLROOT/intel64" it is only mkl path

Then It should be normal after configure, then FFT_LIB would be found.

Maybe you can try

FFT_LIBS="-L$MKLROOT/intel64 -lmkl"

and add

Edit the make.sys file in the top package directory and modify the DFLAGS variable in order to enable hybrid MPI+openmp mode, fftw3, and scalapack:

iew source

print ?

1
DFLAGS = -D__INTEL -D__FFTW3 -D__MPI -D__PARA -D__SCALAPACK -D__OPENMP

...

and see if it can work?

Best Regards,

Ying

Please have a look at https:/

Hans_P_Intel — Thu, 01 Dec 2016 10:54:07 GMT

Please have a look at https://software.intel.com/en-us/articles/quantum-espresso-for-the-intel-xeon-phi-processor, which not only focuses on the Intel® Xeon Phi™ x200 processor family (previously code-named "Knights Landing" or "KNL") but seamlessly applies to any Intel processor. The aforementioned article helps to build an optimized binary, which also yields optimized performance.

I have successfully compiled

Vahid_A_2 — Thu, 09 Mar 2017 00:05:51 GMT

I have successfully compiled QE-6.0 using the following modules:

1) intel/14.0.2.144 2) intel/ompi/1.6.5

To compile against the MKL FFT library (DFTI), I use the following:

./configure --prefix=/home/$username/bin/qe-6.0 --enable-shared --enable-parallel --with-scalapack CC=mpicc F77=mpif90 FC=mpif90

If I do not include the CC, F77 and FC options, the code compiles against the native QE FFTW library.

Vahid