Linking Error In Intel compilers used with WRF code

[Bhaskar__Sharad_S]

Hi all

I am facing some linking erroroccurringwith Intel compilers.

==============================================================

These are the error

My system is x86_64 .

==============================================================

lib.a -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/fftpack/fftpack5 -lfftpack -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib1 -lio_grib1 -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib_share -lio_grib_share -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_int -lwrfio_int /home/sharad/projects/WRF/wrf_files/WRFV3/external/esmf_time_f90/libesmf_time.a /home/sharad/projects/WRF/wrf_files/WRFV3/external/esmf_time_f90/libesmf_time.a /home/sharad/projects/WRF/wrf_files/WRFV3/external/RSL_LITE/librsl_lite.a /home/sharad/projects/WRF/wrf_files/WRFV3/frame/module_internal_header_util.o /home/sharad/projects/WRF/wrf_files/WRFV3/frame/pack_utils.o /home/sharad/projects/WRF/wrf_files/WRFV3/external/io_netcdf/libwrfio_nf.a -L/home/sharad/projects/WRF/wrf_files/WRFV3/netcdf_links/lib -lnetcdf -L/opt/intel/impi/4.0.1.007/intel64/lib-L/opt/intel/mpi-rt/4.0.1 -lmpi -lmpigf -lmpigi -lpthread -lpthread -lpthread -lpthread -lrt

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Err_return_comm'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Grequest_set_lang_f77'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_CommSetAttr'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Keyval_set_proxy'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_CommGetAttr'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Datatype_mem'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Add_finalize'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Wtick'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Datatype_builtin'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Datatype_set_contents'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Err_create_code'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_Process'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIU_Handle_get_ptr_indirect'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Datatype_direct'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_WinSetAttr'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_b_use_gettimeofday'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `i_malloc'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Type_contiguous'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `i_free'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_TypeSetAttr'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_WinGetAttr'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Type_commit'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPID_Wtime_todouble'

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.so: undefined reference to `MPIR_TypeGetAttr'

make[1]: [em_wrf] Error 1 (ignored)

make[1]: Leaving directory `/home/sharad/projects/WRF/wrf_files/WRFV3/main'

========================================================

mpiifort -link-info returns the following

--------------------------------------------------------

ifort -ldl -ldl -ldl -ldl -I/opt/intel/impi/4.0.1.007/intel64/include-I/opt/intel/impi/4.0.1.007/intel64/include-L/opt/intel/impi/4.0.1.007/intel64/lib-Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /opt/intel/impi/4.0.1.007/intel64/lib-Xlinker -rpath -Xlinker /opt/intel/mpi-rt/4.0.1 -lmpi -lmpigf -lmpigi -lpthread -lpthread -lpthread -lpthread -lrt

=========================================================

================================================================================================

After renaming libmpigf.so to libmpigf.so.bak

--------------------------------------------------------

mpif90 -f90=ifort -o wrf.exe -O3 -w -ftz -align all -fno-alias -fp-model precise -FR -convert big_endian -ip wrf.o ../main/module_wrf_top.o libwrflib.a -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/fftpack/fftpack5 -lfftpack -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib1 -lio_grib1 -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib_share -lio_grib_share -L/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_int -lwrfio_int /home/sharad/projects/WRF/wrf_files/WRFV3/external/esmf_time_f90/libesmf_time.a /home/sharad/projects/WRF/wrf_files/WRFV3/external/esmf_time_f90/libesmf_time.a /home/sharad/projects/WRF/wrf_files/WRFV3/external/RSL_LITE/librsl_lite.a /home/sharad/projects/WRF/wrf_files/WRFV3/frame/module_internal_header_util.o /home/sharad/projects/WRF/wrf_files/WRFV3/frame/pack_utils.o /home/sharad/projects/WRF/wrf_files/WRFV3/external/io_netcdf/libwrfio_nf.a -L/home/sharad/projects/WRF/wrf_files/WRFV3/netcdf_links/lib -lnetcdf -L/opt/intel/impi/4.0.1.007/intel64/lib-L/opt/intel/mpi-rt/4.0.1 -lmpi -lmpigf -lmpigi -lpthread -lpthread -lpthread -lpthread -lrt

/opt/intel/impi/4.0.1.007/intel64/lib/libmpigf.a(cart_createf.o): In function `pmpi_cart_create':

cart_createf.c:(.text+0x3e): undefined reference to `i_malloc'

cart_createf.c:(.text+0x9d): undefined reference to `i_free'

make[1]: [em_wrf] Error 1 (ignored)

==============================================================================================

These are libreries file we are using

Lib path

LIB_BUNDLED = \\

-L$(WRF_SRC_ROOT_DIR)/external/fftpack/fftpack5 -lfftpack \\

-L$(WRF_SRC_ROOT_DIR)/external/io_grib1 -lio_grib1 \\

-L$(WRF_SRC_ROOT_DIR)/external/io_grib_share -lio_grib_share \\

-L$(WRF_SRC_ROOT_DIR)/external/io_int -lwrfio_int \\

$(ESMF_IO_LIB) \\

$(ESMF_IO_LIB) \\

$(WRF_SRC_ROOT_DIR)/external/RSL_LITE/librsl_lite.a \\

$(WRF_SRC_ROOT_DIR)/frame/module_internal_header_util.o \\

$(WRF_SRC_ROOT_DIR)/frame/pack_utils.o

LIB_EXTERNAL = \\

$(WRF_SRC_ROOT_DIR)/external/io_netcdf/libwrfio_nf.a -L/home/sharad/projects/WRF/wrf_files/WRFV3/netcdf_links/lib -lnetcdf -L/opt/intel/impi/4.0.1.007/intel64/lib

-L/opt/intel/mpi-rt/4.0.1 -lmpi -lmpigf -lmpigi -lpthread -lpthread -lpthread -lpthread -lrt

LIB = $(LIB_BUNDLED) $(LIB_EXTERNAL) $(LIB_LOCAL)

=========================================================================================================

--

Thanks and Regards

Sharad

[James_T_Intel]

Hi Sharad,

Do you have another MPI installation on your system or is there an MPI library bundled with WRF? Based on the error messages you are seeing, I believe that your linker is finding a libmpi.a or libmpi.so that is not from the Intel MPI Library first. Check your LD_LIBRARY_PATH variable and all paths listedin the link line before /opt/intel/impi/4.0.1.007/intel64/lib for any other MPI libraries. If this is the case, I would recommend either adding the library path for the Intel MPI Library to the beginning of your LD_LIBRARY_PATH variable or rearranging the link line to have it appear before the other MPI library.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel Cluster Tools

[TimP]

If you are building and running with ifort and Intel MPI, you must use (only) the mpiifort compiler wrapper. mpif90 which comes with Intel MPI uses gfortran. Either of those will supply automatically the appropriate MPI headers and libraries, but you can't mix ifort and gfortran libraries. You must not specify linkage against libraries associated with another MPI.
It looks like you are mixing a makefile intended for some other MPI with the setup for Intel MPI. Perhaps one of the articles on building WRF with Intel tools may help, even if it's out of date. You will find several if you enter WRF in the "product index" search window.

[Bhaskar__Sharad_S]

My LD_LIBEARY_PATH is

/home/sharad/projects/CFS/COLA/cfs_v2/ESMF/esmf/lib/libO/Linux.intel.64.intelmpi.default:/home/sharad/projects/CFS/COLA/cfs_v2/lib/std:/opt/intel/impi/4.0.1.007/intel64/lib:/opt/intel/vtune_amplifier_xe_2011/lib64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/../compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/lib64:/lib:/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib:/home/root/projects/CFS/COLA/cfs_v2/ESMF/esmf/lib/libO/Linux.intel.64.intelmpi.default:/home/root/projects/CFS/COLA/cfs_v2/lib/std:/opt/intel/impi/4.0.1.007/intel64/lib:/opt/intel/vtune_amplifier_xe_2011/lib64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/../compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/lib64:/lib:/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib::/opt/intel/composer_xe_2011_sp1.6.233/debugger/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mpirt/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/debugger/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mpirt/lib/intel64

/home/sharad/projects/CFS/COLA/cfs_v2/ESMF/esmf/lib/libO/Linux.intel.64.intelmpi.default:/home/sharad/projects/CFS/COLA/cfs_v2/lib/std:/opt/intel/impi/4.0.1.007/intel64/lib:/opt/intel/vtune_amplifier_xe_2011/lib64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/../compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/lib64:/lib:/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib:/home/root/projects/CFS/COLA/cfs_v2/ESMF/esmf/lib/libO/Linux.intel.64.intelmpi.default:/home/root/projects/CFS/COLA/cfs_v2/lib/std:/opt/intel/impi/4.0.1.007/intel64/lib:/opt/intel/vtune_amplifier_xe_2011/lib64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/../compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/ipp/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:/lib64:/lib:/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib::/opt/intel/composer_xe_2011_sp1.6.233/debugger/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mpirt/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/debugger/lib/intel64:/opt/intel/composer_xe_2011_sp1.6.233/mpirt/lib/intel64

[James_T_Intel]

Hi Sharad,

Can you please look for a file name libmpi.a or libmpi.so in the following folders?

/home/sharad/projects/WRF/wrf_files/WRFV3/external/fftpack/fftpack5
/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib1
/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_grib_share
/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_int
/home/sharad/projects/WRF/wrf_files/WRFV3/external/io_int

If you find either file in either folder, try moving -L/opt/intel/impi/4.0.1.007/intel64/lib to the beginning of your link line to compile.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel Cluster Tools