linking problem between gfortran and mkl10.2

[johannesrs]

Hi all.

I'm having a stupid problem in compiling gamess. Hope someone here can help me.

We are using gfortran together with mkl library. Basically, the first option when considering linux64 and intel machines (in this case, our new cluster of core2quads).

We are having problems in the linking stage, lked script. I get the following error messages:

gfortran -o ../gamess.01.x gamess.o unport.o zunix.o aldeci.o algnci.o
basccn.o basecp.o basext.o bashuz.o bashz2.o basn21.o basn31.o
baspcn.o bassto.o ccaux.o ccddi.o ccqaux.o ccquad.o ccsdt.o cee
is.o chgpen.o cisgrd.o cosmo.o cphf.o cpmchf.o cprohf.o ddilib.o
delocl.o demrpt.o dft.o dftaux.o dftexc.o dftfun.o dftgrd.o dftint.o
dgeev.o dgesvd.o dmulti.o drc.o ecp.o ecpder.o ecplib.o ecppot.
o efchtr.o efdrvr.o efgrda.o efgrdb.o efgrdc.o efgrd2.o efinp.o
efinta.o efintb.o efelec.o efpaul.o efpcm.o efpcov.o efpfmo.o eigen.o
elglib.o elgloc.o elgscf.o eomcc.o ewald.o ffield.o fmo.o fmoes
d.o fmogrd.o fmoint.o fmoio.o fmolib.o fmopbc.o fmoprp.o frfmt.o
fsodci.o globop.o gradex.o guess.o grd1.o grd2a.o grd2b.o grd2c.o
gugdga.o gugdgb.o gugdm.o gugdm2.o gugdrt.o gugem.o gugsrt.o gvb.o
hess.o hss1a.o hss1b.o hss2a.o hss2b.o inputa.o inputb.o inputc.o
int1.o int2a.o int2b.o int2c.o int2d.o int2f.o int2g.o int2r.o int2s.o
int2t.o int2u.o int2v.o int2w.o int2x.o iolib.o ivocas.o la
gran.o local.o loccd.o locpol.o mcpgrd.o mcpinp.o mcpint.o mcpl10.o
mcpl20.o mcpl30.o mcpl40.o mcpl50.o mcpl60.o mcpl70.o mcpl80.o mccas.o
mcjac.o mcqdpt.o mcqdwt.o mcqud.o mcscf.o mctwo.o mdefp.o
mexing.o mm23.o mp2.o mp2ddi.o mp2grd.o mp2gr2.o mp2ims.o mpcdat.o
mpcgrd.o mpcint.o mpcmol.o mpcmsc.o morokm.o mthlib.o nameio.o nmr.o
ordint.o ormas1.o parley.o pcm.o pcmcav.o pcmcv2.o pcmder.o p
cmdis.o pcmief.o pcmpol.o pcmvch.o prpel.o prplib.o prppop.o qeigen.o
qfmm.o qmfm.o qrel.o raman.o rhfuhf.o rohfcc.o rxncrd.o ryspol.o
scflib.o scfmi.o scrf.o sobrt.o soffac.o solib.o sozeff.o stat
pt.o surf.o svpchg.o svpinp.o svpleb.o symorb.o symslc.o tddft.o
tddefp.o tddfun.o tddfxc.o tddgrd.o tddint.o tdhf.o tdx.o tdxio.o
tdxitr.o tdxni.o tdxprp.o trans.o trfdm2.o trnstn.o trudge.o umpdd
i.o vibanl.o vscf.o vvos.o zapddi.o zheev.o zmatrx.o qmmm.o vbdum.o
neostb.o ../ddi/libddi.a -lpthread -L/home/johannes/lib/mkl/lib

/em64t -
lmkl_em64t -lmkl_lapack -lmkl
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `mkl_blas_dgemm':
__tmp_dgemm_omp.c:(.text+0x6c): undefined reference to
`__kmpc_global_thread_num'
__tmp_dgemm_omp.c:(.text+0x32c): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dgemm_omp.c:(.text+0x351): undefined reference to
`__kmpc_push_num_threads'
__tmp_dgemm_omp.c:(.text+0x3c3): undefined reference to
`__kmpc_fork_call'
__tmp_dgemm_omp.c:(.text+0x3dd): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x44f): undefined reference to
`__kmpc_end_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x53d): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dgemm_omp.c:(.text+0x562): undefined reference to
`__kmpc_push_num_threads'
__tmp_dgemm_omp.c:(.text+0x656): undefined reference to
`__kmpc_fork_call'
__tmp_dgemm_omp.c:(.text+0x670): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x743): undefined reference to
`__kmpc_end_serialized_parallel'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_mkl_blas_dgemm_570__par_loop2':
__tmp_dgemm_omp.c:(.text+0xaec): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0xd72): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_mkl_blas_dgemm_557__par_loop1':
__tmp_dgemm_omp.c:(.text+0xf3d): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0x10ec): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_mkl_blas_dgemm_601__par_loop3':
__tmp_dgemm_omp.c:(.text+0x12f3): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0x152a): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_mkl_blas_dgemm_467__par_loop0':
__tmp_dgemm_omp.c:(.text+0x166b): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0x1776): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `dgemm_1D':
__tmp_dgemm_omp.c:(.text+0x1931): undefined reference to
`__kmpc_global_thread_num'
__tmp_dgemm_omp.c:(.text+0x19ed): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dgemm_omp.c:(.text+0x1a12): undefined reference to
`__kmpc_push_num_threads'
__tmp_dgemm_omp.c:(.text+0x1aca): undefined reference to
`__kmpc_fork_call'
__tmp_dgemm_omp.c:(.text+0x1ae8): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x1bd9): undefined reference to
`__kmpc_end_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x1c2d): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dgemm_omp.c:(.text+0x1c52): undefined reference to
`__kmpc_push_num_threads'
__tmp_dgemm_omp.c:(.text+0x1d46): undefined reference to
`__kmpc_fork_call'
__tmp_dgemm_omp.c:(.text+0x1d60): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dgemm_omp.c:(.text+0x1e33): undefined reference to
`__kmpc_end_serialized_parallel'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_dgemm_1D_665__par_loop4':
__tmp_dgemm_omp.c:(.text+0x2022): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0x21c8): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dgemm_omp.o):
In function `L_dgemm_1D_678__par_loop5':
__tmp_dgemm_omp.c:(.text+0x23c9): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dgemm_omp.c:(.text+0x257a): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrmm_omp.o):
In function `mkl_blas_dtrmm':
__tmp_dtrmm_omp.c:(.text+0x68): undefined reference to
`__kmpc_global_thread_num'
__tmp_dtrmm_omp.c:(.text+0x1a4): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dtrmm_omp.c:(.text+0x1c9): undefined reference to
`__kmpc_push_num_threa ds'
__tmp_dtrmm_omp.c:(.text+0x280): undefined reference to
`__kmpc_fork_call'
__tmp_dtrmm_omp.c:(.text+0x29a): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dtrmm_omp.c:(.text+0x33f): undefined reference to
`__kmpc_end_serialized_parallel'
__tmp_dtrmm_omp.c:(.text+0x352): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dtrmm_omp.c:(.text+0x377): undefined reference to
`__kmpc_push_num_threads'
__tmp_dtrmm_omp.c:(.text+0x400): undefined reference to
`__kmpc_fork_call'
__tmp_dtrmm_omp.c:(.text+0x41e): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dtrmm_omp.c:(.text+0x4d2): undefined reference to
`__kmpc_end_serialized_parallel'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrmm_omp.o):
In function `L_mkl_blas_dtrmm_236__par_loop0':
__tmp_dtrmm_omp.c:(.text+0x730): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dtrmm_omp.c:(.text+0x897): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrmm_omp.o):
In function `L_mkl_blas_dtrmm_256__par_loop1':
__tmp_dtrmm_omp.c:(.text+0xa36): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dtrmm_omp.c:(.text+0xb97): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrsm_omp.o):
In function `mkl_blas_dtrsm':
__tmp_dtrsm_omp.c:(.text+0x68): undefined reference to
`__kmpc_global_thread_num'
__tmp_dtrsm_omp.c:(.text+0x1a2): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dtrsm_omp.c:(.text+0x1c7): undefined reference to
`__kmpc_push_num_threads'
__tmp_dtrsm_omp.c:(.text+0x27b): undefined reference to
`__kmpc_fork_call'
__tmp_dtrsm_omp.c:(.text+0x295): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dtrsm_omp.c:(.text+0x337): undefined reference to
`__kmpc_end_serialized_parallel'
__tmp_dtrsm_omp.c:(.text+0x34a): undefined reference to
`__kmpc_ok_to_fork'
__tmp_dtrsm_omp.c:(.text+0x36f): undefined reference to
`__kmpc_push_num_threads'
__tmp_dtrsm_omp.c:(.text+0x3f5): undefined reference to
`__kmpc_fork_call'
__tmp_dtrsm_omp.c:(.text+0x413): undefined reference to
`__kmpc_serialized_parallel'
__tmp_dtrsm_omp.c:(.text+0x4c4): undefined reference to
`__kmpc_end_serialized_parallel'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrsm_omp.o):
In function `L_mkl_blas_dtrsm_259__par_loop0':
__tmp_dtrsm_omp.c:(.text+0x722): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dtrsm_omp.c:(.text+0x88f): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.a(dtrsm_omp.o):
In function `L_mkl_blas_dtrsm_276__par_loop1':
__tmp_dtrsm_omp.c:(.text+0xa2e): undefined reference to
`__kmpc_for_static_init_8'
__tmp_dtrsm_omp.c:(.text+0xb8f): undefined reference to
`__kmpc_for_static_fini'
/home/johannes/lib/mkl/lib/em64t/
libmkl_intel_thread.a(mkl_threading.o): In function
`MKL_Get_Max_Threads':
__tmp_mkl_threading.c:(.text+0x55): undefined reference to
`omp_in_parallel'
__tmp_mkl_threading.c:(.text+0x77): undefined reference to
`omp_get_max_threads'
/home/johannes/lib/mkl/lib/em64t/
libmkl_intel_thread.a(mkl_threading.o): In function
`M KL_Domain_Get_Max_Threads':
__tmp_mkl_threading.c:(.text+0x855): undefined reference to
`omp_in_parallel'
__tmp_mkl_threading.c:(.text+0x874): undefined reference to
`omp_get_max_threads'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_ordered'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `omp_get_thread_num'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_barrier'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `omp_get_num_procs'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_critical'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_dispatch_fini_8'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_end_critical'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_dispatch_init_8'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_for_static_init_4'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_flush'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_dispatch_next_8'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_end_ordered'
/home/johannes/lib/mkl/lib/em64t/libmkl_intel_thread.so: undefined
reference to `__kmpc_atomic_fixed8_add'
collect2: ld returned 1 exit status
unset echo


IO found a few messages on this, but concerning pure use of intel compilers together with mkl. That would be an option, but not a good one, since would need to much hacking of the standard gamess scripts, which are all really long. :(

I'm using the last stable versions of both mkl and gamess. Does anybody has any clue on this? Have anybody please gone throw such a problem?

Thanks a lot to everybody in advance.

[TimP]

That's a dig, if there ever was one, but how are we to know which MKL you consider the "last stable version?" It does look like some 10.0 version.
When you use mkl_thread, you must link one of the corresponding OpenMP run-time libraries (-lguide, or, if using gfortran -fopenmp, -liomp5), and -lpthread. I don't think the docs are so poor in this respect.
If you are running as a cluster application, you should check whether others have been successful running this as a hybrid MPI/OpenMP application, with the MPI of your choice. For example, Intel MPI only just last week introduced environment variable options for this purpose. There is a reasonable chance that plain MPI with 1 process per node, each process running mkl_thread, should work.
The other option is to use the non-threaded MKL link option (mkl_sequential instead of mkl_thread).

[johannesrs]

Sorry, that was my mistake.

What I called "last stable version" of mkl is the "biggest" one available: 10.2. I did not downloaded any alpha or beta version, but I guess now that it is the 018 build (I can be mixing it up with with last comillers build, thow).

Basically, you mean I should add the flags -fopenmp -liomp5 -lpthread to the linking stage, all together?

Thanks a lot in advance, and sorry again for my "miswriting".

[TimP]

If you have gfortran 4.2 or later (so as to support -fopenmp), -fopenmp includes -lpthread automatically. It might work without -liomp5, but -liomp5 will perform better than -lgomp, which -liomp5 replaces. If you don't use -fopenmp, either -lguide -lpthread or -liomp5 -lpthread should work.

[johannesrs]

everybody know, you are right, the last tricks worked and it compiled perfectly! Thanks a lot!