Problem with Compiling HPL using MKL&MPI

[Jiang__Roger]

Hello everyone~

I've installed Intel MKL & Intel MPI for linux to run HPL, however, I have problem while compiling it.

 I use virtual machine to run CentOS, imformation of my computer and virtual machine setting is written below.

Oracle VirtualBox 5.2.20

OS: CentOS 6.10

CPU: Intel(R) Core(TM) i5-4200H *2

Memory(for Centos): 8192MB

gcc version: Red Hat 7.3.1-5

MKL: Intel®MKL 2019 Initial Release

MPI: Intel®MPI Library(Linux*packages) 2019 Initial Release

I reference the article 

https://software.intel.com/en-us/articles/performance-tools-for-software-developers-hpl-application-note

but I got an error while I do the make arch

I choose Make.Linux_PII_CBLAS to compile and rename it Make.intel64

below is the contents of my Makefile 

#------------------------------------------------

SHELL        = /bin/sh
#
CD           = cd
CP           = cp
LN_S         = ln -s
MKDIR        = mkdir
RM           = /bin/rm -f
TOUCH        = touch

ARCH         = intel64

TOPdir       = /root/hpl
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
#
HPLlib       = $(LIBdir)/libhpl.a
 

MPdir        = /opt/intel/impi/2019.0.117/intel64
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/release/libmpi.a 

LAdir        = /opt/intel/compilers_and_libraries/linux/mkl
ifndef  LAinc
LAinc        = -I$(LAdir)/include
endif
ifndef  LAlib
LAlib        = -L $(LAdir)/lib/intel64 \
               -Wl,--start-group \
               $(LAdir)/lib/intel64/libmkl_intel_lp64.a \
               $(LAdir)/lib/intel64/libmkl_intel_thread.a \
               $(LAdir)/lib/intel64/libmkl_core.a \
               -Wl,--end-group -lpthread -ldl -lm
endif

F2CDEFS      =
 

HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib)

HPL_OPTS     = -DHPL_CALL_CBLAS

HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)

CC           = /opt/intel/compilers_and_libraries_2019.0.117/linux/mpi/intel64/bin/mpicc
CCNOOPT      = $(HPL_DEFS)
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops

LINKER       = $(CC)
LINKFLAGS    = $(CCFLAGS)
#
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo
#
# --------------------------------------------

and the error I get

---------------------------

/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__trsm_drv.o): In function `omp_driver_iterative':
_trsm.c:(.text+0x7f): undefined reference to `__kmpc_global_thread_num'
_trsm.c:(.text+0x4b4): undefined reference to `__kmpc_ok_to_fork'
_trsm.c:(.text+0x4d5): undefined reference to `__kmpc_push_num_threads'
_trsm.c:(.text+0x510): undefined reference to `__kmpc_fork_call'
_trsm.c:(.text+0x52b): undefined reference to `__kmpc_serialized_parallel'
_trsm.c:(.text+0x57e): undefined reference to `__kmpc_end_serialized_parallel'
_trsm.c:(.text+0x123d): undefined reference to `omp_get_thread_num'
_trsm.c:(.text+0x12be): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__trsm_drv.o): In function `omp_driver_v2_leaf':
_trsm.c:(.text+0x3eed): undefined reference to `__kmpc_global_thread_num'
_trsm.c:(.text+0x3f02): undefined reference to `__kmpc_ok_to_fork'
_trsm.c:(.text+0x3f22): undefined reference to `__kmpc_push_num_threads'
_trsm.c:(.text+0x3f58): undefined reference to `__kmpc_fork_call'
_trsm.c:(.text+0x3f73): undefined reference to `__kmpc_serialized_parallel'
_trsm.c:(.text+0x3fc3): undefined reference to `__kmpc_end_serialized_parallel'
_trsm.c:(.text+0x4040): undefined reference to `omp_get_thread_num'
_trsm.c:(.text+0x40b7): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__trsm_drv.o): In function `omp_driver_recursive':
_trsm.c:(.text+0x7247): undefined reference to `__kmpc_global_thread_num'
_trsm.c:(.text+0x725c): undefined reference to `__kmpc_ok_to_fork'
_trsm.c:(.text+0x727d): undefined reference to `__kmpc_push_num_threads'
_trsm.c:(.text+0x72b9): undefined reference to `__kmpc_fork_call'
_trsm.c:(.text+0x72d4): undefined reference to `__kmpc_serialized_parallel'
_trsm.c:(.text+0x7327): undefined reference to `__kmpc_end_serialized_parallel'
_trsm.c:(.text+0x73a1): undefined reference to `omp_get_thread_num'
_trsm.c:(.text+0x741d): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_drv.o): In function `omp_parallel_acopy_lookahead':
_gemm.c:(.text+0x207a): undefined reference to `__kmpc_global_thread_num'
_gemm.c:(.text+0x280c): undefined reference to `__kmpc_critical'
_gemm.c:(.text+0x29db): undefined reference to `__kmpc_end_critical'
_gemm.c:(.text+0x2c7e): undefined reference to `__kmpc_critical'
_gemm.c:(.text+0x2e88): undefined reference to `__kmpc_end_critical'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_drv.o): In function `omp_simple_3d':
_gemm.c:(.text+0x738f): undefined reference to `__kmpc_global_thread_num'
_gemm.c:(.text+0x73a3): undefined reference to `__kmpc_ok_to_fork'
_gemm.c:(.text+0x73c3): undefined reference to `__kmpc_push_num_threads'
_gemm.c:(.text+0x749c): undefined reference to `__kmpc_fork_call'
_gemm.c:(.text+0x74bc): undefined reference to `__kmpc_serialized_parallel'
_gemm.c:(.text+0x755c): undefined reference to `__kmpc_end_serialized_parallel'
_gemm.c:(.text+0x770d): undefined reference to `__kmpc_for_static_init_8'
_gemm.c:(.text+0x816c): undefined reference to `__kmpc_for_static_fini'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_drv.o): In function `mkl_blas_dgemm':
_gemm.c:(.text+0x8aa0): undefined reference to `__kmpc_global_thread_num'
_gemm.c:(.text+0x8ab5): undefined reference to `__kmpc_ok_to_fork'
_gemm.c:(.text+0x8ada): undefined reference to `__kmpc_push_num_threads'
_gemm.c:(.text+0x8bad): undefined reference to `__kmpc_fork_call'
_gemm.c:(.text+0x8bce): undefined reference to `__kmpc_serialized_parallel'
_gemm.c:(.text+0x8c66): undefined reference to `__kmpc_end_serialized_parallel'
_gemm.c:(.text+0x8da8): undefined reference to `omp_get_thread_num'
_gemm.c:(.text+0x8de2): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_drv.o): In function `gemm_omp_driver_v2':
_gemm.c:(.text+0xa1de): undefined reference to `__kmpc_global_thread_num'
_gemm.c:(.text+0xa1f2): undefined reference to `__kmpc_ok_to_fork'
_gemm.c:(.text+0xa215): undefined reference to `__kmpc_push_num_threads'
_gemm.c:(.text+0xa277): undefined reference to `__kmpc_fork_call'
_gemm.c:(.text+0xa294): undefined reference to `__kmpc_serialized_parallel'
_gemm.c:(.text+0xa314): undefined reference to `__kmpc_end_serialized_parallel'
_gemm.c:(.text+0xa3b5): undefined reference to `omp_get_thread_num'
_gemm.c:(.text+0xa44b): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_drv.o): In function `use_smalln_kernels':
_gemm.c:(.text+0xdb83): undefined reference to `__kmpc_global_thread_num'
_gemm.c:(.text+0xdb98): undefined reference to `__kmpc_ok_to_fork'
_gemm.c:(.text+0xdbbd): undefined reference to `__kmpc_push_num_threads'
_gemm.c:(.text+0xdc90): undefined reference to `__kmpc_fork_call'
_gemm.c:(.text+0xdcb1): undefined reference to `__kmpc_serialized_parallel'
_gemm.c:(.text+0xdd49): undefined reference to `__kmpc_end_serialized_parallel'
_gemm.c:(.text+0xde32): undefined reference to `omp_get_thread_num'
_gemm.c:(.text+0xde6c): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__scal_drv.o): In function `mkl_blas_dscal':
_scal.c:(.text+0x1f3): undefined reference to `__kmpc_global_thread_num'
_scal.c:(.text+0x208): undefined reference to `__kmpc_ok_to_fork'
_scal.c:(.text+0x229): undefined reference to `__kmpc_push_num_threads'
_scal.c:(.text+0x25f): undefined reference to `__kmpc_fork_call'
_scal.c:(.text+0x27a): undefined reference to `__kmpc_serialized_parallel'
_scal.c:(.text+0x2b3): undefined reference to `__kmpc_end_serialized_parallel'
_scal.c:(.text+0x375): undefined reference to `omp_get_thread_num'
_scal.c:(.text+0x37d): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__iamax_drv.o): In function `mkl_blas_idamax':
_iamax.c:(.text+0x23a): undefined reference to `__kmpc_global_thread_num'
_iamax.c:(.text+0x24f): undefined reference to `__kmpc_ok_to_fork'
_iamax.c:(.text+0x270): undefined reference to `__kmpc_push_num_threads'
_iamax.c:(.text+0x2a6): undefined reference to `__kmpc_fork_call'
_iamax.c:(.text+0x2c1): undefined reference to `__kmpc_serialized_parallel'
_iamax.c:(.text+0x2fa): undefined reference to `__kmpc_end_serialized_parallel'
_iamax.c:(.text+0x4b8): undefined reference to `omp_get_thread_num'
_iamax.c:(.text+0x4bf): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__copy_drv.o): In function `mkl_blas_dcopy':
_copy.c:(.text+0x222): undefined reference to `__kmpc_global_thread_num'
_copy.c:(.text+0x237): undefined reference to `__kmpc_ok_to_fork'
_copy.c:(.text+0x258): undefined reference to `__kmpc_push_num_threads'
_copy.c:(.text+0x28e): undefined reference to `__kmpc_fork_call'
_copy.c:(.text+0x2a9): undefined reference to `__kmpc_serialized_parallel'
_copy.c:(.text+0x2e2): undefined reference to `__kmpc_end_serialized_parallel'
_copy.c:(.text+0x3a7): undefined reference to `omp_get_thread_num'
_copy.c:(.text+0x3af): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__axpy_drv.o): In function `mkl_blas_daxpy':
_axpy.c:(.text+0x24e): undefined reference to `__kmpc_global_thread_num'
_axpy.c:(.text+0x263): undefined reference to `__kmpc_ok_to_fork'
_axpy.c:(.text+0x284): undefined reference to `__kmpc_push_num_threads'
_axpy.c:(.text+0x2ba): undefined reference to `__kmpc_fork_call'
_axpy.c:(.text+0x2d5): undefined reference to `__kmpc_serialized_parallel'
_axpy.c:(.text+0x30e): undefined reference to `__kmpc_end_serialized_parallel'
_axpy.c:(.text+0x3d3): undefined reference to `omp_get_thread_num'
_axpy.c:(.text+0x3db): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_domain_get_max_threads':
mkl_threading.c:(.text+0x229): undefined reference to `omp_get_num_procs'
mkl_threading.c:(.text+0x297): undefined reference to `omp_in_parallel'
mkl_threading.c:(.text+0x340): undefined reference to `omp_get_max_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_dynamic':
mkl_threading.c:(.text+0x115a): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_omp_in_parallel':
mkl_threading.c:(.text+0x2681): undefined reference to `omp_in_parallel'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_num_stripes':
mkl_threading.c:(.text+0x270a): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_ncpus':
mkl_threading.c:(.text+0x33ea): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_ncorespercpu':
mkl_threading.c:(.text+0x40aa): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_ht':
mkl_threading.c:(.text+0x4d6a): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_threading_patched.o): In function `mkl_serv_get_nlogicalcores':
mkl_threading.c:(.text+0x5a1a): undefined reference to `omp_get_num_procs'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(mkl_print_verbose_patched.o): In function `mkl_serv_print_verbose_info':
mkl_print_verbose.c:(.text+0x1c3): undefined reference to `omp_get_thread_num'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d_ds_ger_omp.o): In function `mkl_blas_dger_omp':
ds_ger_omp.c:(.text+0x58): undefined reference to `__kmpc_global_thread_num'
ds_ger_omp.c:(.text+0x6a): undefined reference to `__kmpc_ok_to_fork'
ds_ger_omp.c:(.text+0x89): undefined reference to `__kmpc_push_num_threads'
ds_ger_omp.c:(.text+0x10c): undefined reference to `__kmpc_fork_call'
ds_ger_omp.c:(.text+0x124): undefined reference to `__kmpc_serialized_parallel'
ds_ger_omp.c:(.text+0x18c): undefined reference to `__kmpc_end_serialized_parallel'
ds_ger_omp.c:(.text+0x2d0): undefined reference to `__kmpc_for_static_init_8'
ds_ger_omp.c:(.text+0x3fd): undefined reference to `__kmpc_for_static_fini'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemv_omp.o): In function `mkl_blas_dgemv_omp':
_gemv_omp.c:(.text+0x11d): undefined reference to `__kmpc_global_thread_num'
_gemv_omp.c:(.text+0x132): undefined reference to `__kmpc_ok_to_fork'
_gemv_omp.c:(.text+0x153): undefined reference to `__kmpc_push_num_threads'
_gemv_omp.c:(.text+0x254): undefined reference to `__kmpc_fork_call'
_gemv_omp.c:(.text+0x275): undefined reference to `__kmpc_serialized_parallel'
_gemv_omp.c:(.text+0x32a): undefined reference to `__kmpc_end_serialized_parallel'
_gemv_omp.c:(.text+0x480): undefined reference to `omp_get_thread_num'
_gemv_omp.c:(.text+0x593): undefined reference to `omp_get_num_threads'
_gemv_omp.c:(.text+0x79e): undefined reference to `__kmpc_master'
_gemv_omp.c:(.text+0x7dc): undefined reference to `__kmpc_end_master'
_gemv_omp.c:(.text+0x7ee): undefined reference to `__kmpc_barrier'
_gemv_omp.c:(.text+0x8dc): undefined reference to `__kmpc_barrier'
_gemv_omp.c:(.text+0xb06): undefined reference to `__kmpc_barrier'
_gemv_omp.c:(.text+0xb1b): undefined reference to `__kmpc_master'
_gemv_omp.c:(.text+0xb3f): undefined reference to `__kmpc_end_master'
_gemv_omp.c:(.text+0xc66): undefined reference to `__kmpc_master'
_gemv_omp.c:(.text+0xccc): undefined reference to `__kmpc_end_master'
_gemv_omp.c:(.text+0xcde): undefined reference to `__kmpc_barrier'
_gemv_omp.c:(.text+0xdba): undefined reference to `__kmpc_barrier'
_gemv_omp.c:(.text+0xdcf): undefined reference to `__kmpc_master'
_gemv_omp.c:(.text+0xf7b): undefined reference to `__kmpc_end_master'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemmger_omp.o): In function `mkl_blas_dgemmger_omp':
_gemmger_omp.c:(.text+0x58): undefined reference to `__kmpc_global_thread_num'
_gemmger_omp.c:(.text+0x6a): undefined reference to `__kmpc_ok_to_fork'
_gemmger_omp.c:(.text+0x89): undefined reference to `__kmpc_push_num_threads'
_gemmger_omp.c:(.text+0xf7): undefined reference to `__kmpc_fork_call'
_gemmger_omp.c:(.text+0x112): undefined reference to `__kmpc_serialized_parallel'
_gemmger_omp.c:(.text+0x1b7): undefined reference to `__kmpc_end_serialized_parallel'
_gemmger_omp.c:(.text+0x335): undefined reference to `__kmpc_for_static_init_8'
_gemmger_omp.c:(.text+0x46d): undefined reference to `__kmpc_for_static_fini'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__trsm_omp_v1_drv.o): In function `mkl_blas_dtrsm_omp_driver_v1':
_trsm_omp_v1.c:(.text+0x108): undefined reference to `__kmpc_global_thread_num'
_trsm_omp_v1.c:(.text+0x233): undefined reference to `__kmpc_ok_to_fork'
_trsm_omp_v1.c:(.text+0x252): undefined reference to `__kmpc_push_num_threads'
_trsm_omp_v1.c:(.text+0x30a): undefined reference to `__kmpc_fork_call'
_trsm_omp_v1.c:(.text+0x328): undefined reference to `__kmpc_serialized_parallel'
_trsm_omp_v1.c:(.text+0x3b1): undefined reference to `__kmpc_end_serialized_parallel'
_trsm_omp_v1.c:(.text+0x56f): undefined reference to `__kmpc_ok_to_fork'
_trsm_omp_v1.c:(.text+0x58e): undefined reference to `__kmpc_push_num_threads'
_trsm_omp_v1.c:(.text+0x64a): undefined reference to `__kmpc_fork_call'
_trsm_omp_v1.c:(.text+0x668): undefined reference to `__kmpc_serialized_parallel'
_trsm_omp_v1.c:(.text+0x6f0): undefined reference to `__kmpc_end_serialized_parallel'
_trsm_omp_v1.c:(.text+0x89d): undefined reference to `omp_get_thread_num'
_trsm_omp_v1.c:(.text+0x95c): undefined reference to `omp_get_num_threads'
_trsm_omp_v1.c:(.text+0xb50): undefined reference to `omp_get_thread_num'
_trsm_omp_v1.c:(.text+0xc17): undefined reference to `omp_get_num_threads'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_omp_driver_v1':
_gemm_omp_v1.c:(.text+0x577): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x6ac): undefined reference to `__kmpc_ok_to_fork'
_gemm_omp_v1.c:(.text+0x6d1): undefined reference to `__kmpc_push_num_threads'
_gemm_omp_v1.c:(.text+0x7c3): undefined reference to `__kmpc_fork_call'
_gemm_omp_v1.c:(.text+0x7e4): undefined reference to `__kmpc_serialized_parallel'
_gemm_omp_v1.c:(.text+0x8a2): undefined reference to `__kmpc_end_serialized_parallel'
_gemm_omp_v1.c:(.text+0xa6d): undefined reference to `omp_get_num_threads'
_gemm_omp_v1.c:(.text+0xbcd): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0xc19): undefined reference to `omp_get_num_threads'
_gemm_omp_v1.c:(.text+0xdf2): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0xe41): undefined reference to `omp_get_num_threads'
_gemm_omp_v1.c:(.text+0x1527): undefined reference to `__kmpc_ok_to_fork'
_gemm_omp_v1.c:(.text+0x154c): undefined reference to `__kmpc_push_num_threads'
_gemm_omp_v1.c:(.text+0x15e1): undefined reference to `__kmpc_fork_call'
_gemm_omp_v1.c:(.text+0x15ff): undefined reference to `__kmpc_serialized_parallel'
_gemm_omp_v1.c:(.text+0x166e): undefined reference to `__kmpc_end_serialized_parallel'
_gemm_omp_v1.c:(.text+0x16d7): undefined reference to `__kmpc_ok_to_fork'
_gemm_omp_v1.c:(.text+0x16fc): undefined reference to `__kmpc_push_num_threads'
_gemm_omp_v1.c:(.text+0x1783): undefined reference to `__kmpc_fork_call'
_gemm_omp_v1.c:(.text+0x17a1): undefined reference to `__kmpc_serialized_parallel'
_gemm_omp_v1.c:(.text+0x1811): undefined reference to `__kmpc_end_serialized_parallel'
_gemm_omp_v1.c:(.text+0x1955): undefined reference to `__kmpc_single'
_gemm_omp_v1.c:(.text+0x1a27): undefined reference to `__kmpc_end_single'
_gemm_omp_v1.c:(.text+0x1a3c): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x1b00): undefined reference to `__kmpc_single'
_gemm_omp_v1.c:(.text+0x1b3c): undefined reference to `__kmpc_end_single'
_gemm_omp_v1.c:(.text+0x1b51): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_1D_row':
_gemm_omp_v1.c:(.text+0x2a87): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x2ba1): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x2bb1): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_1D_col':
_gemm_omp_v1.c:(.text+0x2d20): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x2e44): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x2e54): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_bsrc':
_gemm_omp_v1.c:(.text+0x2f88): undefined reference to `omp_get_thread_num'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_improved_bsrc':
_gemm_omp_v1.c:(.text+0x33ab): undefined reference to `omp_get_thread_num'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_bcopy':
_gemm_omp_v1.c:(.text+0x36ba): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x37c3): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x3a8a): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x3c21): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x3c36): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_1D_with_copy_0':
_gemm_omp_v1.c:(.text+0x3cf2): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x3dd5): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x3ffb): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x4199): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x41ae): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_abcopy_abx_m_km_par_p':
_gemm_omp_v1.c:(.text+0x43aa): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x43c6): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x43d6): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_xgemm_p':
_gemm_omp_v1.c:(.text+0x51b3): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x51d4): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x51e4): undefined reference to `__kmpc_barrier'
/opt/intel/compilers_and_libraries/linux/mkl/lib/intel64/libmkl_intel_thread.a(d__gemm_omp_v1_drv.o): In function `mkl_blas_dgemm_2D_acopy_n':
_gemm_omp_v1.c:(.text+0x5652): undefined reference to `__kmpc_global_thread_num'
_gemm_omp_v1.c:(.text+0x56f5): undefined reference to `omp_get_thread_num'
_gemm_omp_v1.c:(.text+0x5a8d): undefined reference to `__kmpc_barrier'
_gemm_omp_v1.c:(.text+0x5cc1): undefined reference to `__kmpc_barrier'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:76: dexe.grd] Error 1
make[2]: Leaving directory '/root/hpl/testing/ptest/intel64'
make[1]: *** [Make.top:68: build_tst] Error 2
make[1]: Leaving directory '/root/hpl'
make: *** [Makefile:73: build] Error 2
 

---------------------------------------------------------

What should I do to fix the problem?

Thanks!

[Kazushige_G_Intel]

Hi,

 Please modify CC and CFLAS as follows. it will replace mpicc with mpiicc and add OpenMP related flags.

CC           = /opt/intel/compilers_and_libraries_2019.0.117/linux/mpi/intel64/bin/mpiicc
CCFLAGS      = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops -qopenmp

Thanks