Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Rahul_S_4
Beginner
126 Views

Installing NWchem using INTEL compilers + MKL libraries

Hello Intel Team,

 

 

    I am trying to install NWCHEM on the a local university cluster but I am having some problems in compiling NWchem. I am attaching my input script that I’m using to compile Nwchem.

 

export NWCHEM_TOP=/work/gb_lab/rahulsn/nwchem-6.6

export NWCHEM_TARGET=LINUX64

export ARMCI_NETWORK=OPENIB

export IB_HOME=/usr

export IB_INCLUDE=/usr/include/infiniband

export IB_LIB=/usr/lib64

export IB_LIB_NAME="-libumad -libverbs -lpthread"

export USE_MPI=Y

export USE_MPIF=Y

export USE_MPIF4=Y

export MPI_LOC=/shared/intel/impi_5.0.3

export MPI_INCLUDE="-I/shared/intel/impi_5.0.3/intel64/include"

export MPI_LIB="/shared/intel/impi_5.0.3/intel64/lib/release -L/shared/intel/impi_5.0.3/intel64/lib"

export LIBMPI="-lmpifort -lmpi -lmpigi -ldl -lrt -lpthread"

export NWCHEM_MODULES="all"

export LARGE_FILES=TRUE

export USE_NOFSCHECK=TRUE

export HAS_BLAS=yes

export USE_SCALAPACK=y

export MKLLIB=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/lib/intel64

export MKLINC=/opt/intel/compilers_and_libraries_2016.0.109/linux/mkl/include

export BLASOPT="-L$MKLLIB -lmkl_intel_ilp64 -lmkl_core -lmkl_sequential -lpthread -lm"

export LAPACK_LIBS="-L$MKLLIB -lmkl_intel_ilp64 -lmkl_core -lmkl_sequential -lpthread -lm"

export SCALAPACK="-L$MKLLIB -lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_core -lmkl_sequential -lmkl_blacs_intelmpi_ilp64 -lpthread -lm"

export SCALAPACK_SIZE=8

export BLAS_SIZE=8

export USE_64TO32=y

export FC=ifort

export CC=icc

 

make nwchem_config

make CC=icc FC=ifort FOPTIMIZE=-O3 -j4 

 

 

This is the a part of the error it is showing at the end. I think it is not able to locate SCALPACK libraries on Cyence.

 

    scalapack.F:(.text+0xd19c): undefined reference to `numroc_'

scalapack.F:(.text+0xd1c9): undefined reference to `numroc_'

scalapack.F:(.text+0xd363): undefined reference to `pdsyevd_'

scalapack.F:(.text+0xd719): undefined reference to `numroc_'

scalapack.F:(.text+0xd743): undefined reference to `numroc_'

scalapack.F:(.text+0xd76e): undefined reference to `numroc_'

scalapack.F:(.text+0xd79d): undefined reference to `numroc_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_pdsyevr_':

scalapack.F:(.text+0xdfa8): undefined reference to `blacs_gridinit_'

scalapack.F:(.text+0xdff9): undefined reference to `blacs_gridinfo_'

scalapack.F:(.text+0xe164): undefined reference to `descinit_'

scalapack.F:(.text+0xe1b5): undefined reference to `descinit_'

scalapack.F:(.text+0xe247): undefined reference to `numroc_'

scalapack.F:(.text+0xe26a): undefined reference to `numroc_'

scalapack.F:(.text+0xe3c7): undefined reference to `pdsyevr_'

scalapack.F:(.text+0xe5e8): undefined reference to `pdsyevr_'

scalapack.F:(.text+0xeb21): undefined reference to `numroc_'

scalapack.F:(.text+0xeb4c): undefined reference to `numroc_'

scalapack.F:(.text+0xeb76): undefined reference to `numroc_'

scalapack.F:(.text+0xeba0): undefined reference to `numroc_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_pzheevd_':

scalapack.F:(.text+0xf26a): undefined reference to `blacs_gridinit_'

scalapack.F:(.text+0xf2bb): undefined reference to `blacs_gridinfo_'

scalapack.F:(.text+0xf41e): undefined reference to `descinit_'

scalapack.F:(.text+0xf470): undefined reference to `descinit_'

scalapack.F:(.text+0xf593): undefined reference to `pzheevd_'

scalapack.F:(.text+0xf7db): undefined reference to `pzheevd_'

scalapack.F:(.text+0xfa00): undefined reference to `numroc_'

scalapack.F:(.text+0xfa2c): undefined reference to `numroc_'

scalapack.F:(.text+0xfa58): undefined reference to `numroc_'

scalapack.F:(.text+0xfa84): undefined reference to `numroc_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `ga_pzheevr_':

scalapack.F:(.text+0x10e61): undefined reference to `blacs_gridinit_'

scalapack.F:(.text+0x10eb2): undefined reference to `blacs_gridinfo_'

scalapack.F:(.text+0x1102d): undefined reference to `descinit_'

scalapack.F:(.text+0x1107d): undefined reference to `descinit_'

scalapack.F:(.text+0x11242): undefined reference to `pzheevr_'

scalapack.F:(.text+0x1152d): undefined reference to `pzheevr_'

scalapack.F:(.text+0x11835): undefined reference to `numroc_'

scalapack.F:(.text+0x11864): undefined reference to `numroc_'

scalapack.F:(.text+0x11893): undefined reference to `numroc_'

scalapack.F:(.text+0x118c2): undefined reference to `numroc_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `pxerbla_':

scalapack.F:(.text+0x11a9d): undefined reference to `blacs_gridinfo_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slexit4_':

scalapack.F:(.text+0x11b6b): undefined reference to `blacs_gridexit_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slexit3_':

scalapack.F:(.text+0x11b9b): undefined reference to `blacs_gridexit_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slexit2_':

scalapack.F:(.text+0x11be8): undefined reference to `blacs_gridexit_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slexit_':

scalapack.F:(.text+0x11c0b): undefined reference to `blacs_gridexit_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slinit4_':

scalapack.F:(.text+0x11ddb): undefined reference to `blacs_gridinfo_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slinit3_':

scalapack.F:(.text+0x11ee1): undefined reference to `blacs_gridinfo_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slinit2_':

scalapack.F:(.text+0x120e3): undefined reference to `blacs_gridinit_'

scalapack.F:(.text+0x12134): undefined reference to `blacs_gridinfo_'

/work/gb_lab/rahulsn/nwchem-6.6/src/tools/install/lib/libga.a(scalapack.o): In function `slinit_':

scalapack.F:(.text+0x1221a): undefined reference to `blacs_gridinit_'

scalapack.F:(.text+0x12247): undefined reference to `blacs_gridinfo_’

 

 

 

It would be really helpful if somebody can help me in preparing my input script. I think I am using the wrong locations of scalpack libraries but the path mentioned in the starting part of the script has all the libraries in the folder. Output of the $MKLLIB folder

 

libmkl_avx.so                   libmkl_blacs_openmpi_lp64.a  libmkl_gf_ilp64.so      libmkl_lapack95_ilp64.a    libmkl_vml_avx.so

libmkl_avx2.so                  libmkl_blacs_sgimpt_ilp64.a  libmkl_gf_lp64.a        libmkl_lapack95_lp64.a     libmkl_vml_avx2.so

libmkl_avx512.so                libmkl_blacs_sgimpt_lp64.a   libmkl_gf_lp64.so       libmkl_mc.so               libmkl_vml_avx512.so

libmkl_avx512_mic.so            libmkl_blas95_ilp64.a        libmkl_gnu_thread.a     libmkl_mc3.so              libmkl_vml_avx512_mic.so

libmkl_blacs_ilp64.a            libmkl_blas95_lp64.a         libmkl_gnu_thread.so    libmkl_rt.so               libmkl_vml_cmpt.so

libmkl_blacs_intelmpi_ilp64.a   libmkl_cdft_core.a           libmkl_intel_ilp64.a    libmkl_scalapack_ilp64.a   libmkl_vml_def.so

libmkl_blacs_intelmpi_ilp64.so  libmkl_cdft_core.so          libmkl_intel_ilp64.so   libmkl_scalapack_ilp64.so  libmkl_vml_mc.so

libmkl_blacs_intelmpi_lp64.a    libmkl_core.a                libmkl_intel_lp64.a     libmkl_scalapack_lp64.a    libmkl_vml_mc2.so

libmkl_blacs_intelmpi_lp64.so   libmkl_core.so               libmkl_intel_lp64.so    libmkl_scalapack_lp64.so   libmkl_vml_mc3.so

libmkl_blacs_lp64.a             libmkl_def.so                libmkl_intel_thread.a   libmkl_sequential.a        locale

libmkl_blacs_openmpi_ilp64.a    libmkl_gf_ilp64.a            libmkl_intel_thread.so  libmkl_sequential.so

 

Thanks for the help.

 

Rahul Singh

0 Kudos
4 Replies
Evarist_F_Intel
Employee
126 Views

Dear Rahul,

May the problem be in the SCALAPACK environment variable: not sure, but shouldn't it be SCALAPACK_LIBS?

The rest looks good. Could you please paste the whole link-line for the file where error happens?

Rahul_S_4
Beginner
126 Views

Thanks for your reply. I changed the SCALAPACK environment variable to SCALAPACK_LIBS. Now I'm not getting this error but a different one.

===========================================================================

 

Begin optimization report for: SD_E2_1

 

    Report from: Interprocedural optimizations [ipo]

 

INLINE REPORT: (SD_E2_1) [23] cr_ccsd_t_E.F(1209,18)

 

 

    Report from: Loop nest, Vector & Auto-parallelization optimizations [loop, vec, par]

 

 

LOOP BEGIN at cr_ccsd_t_E.F(1218,7)

   remark #15542: loop was not vectorized: inner loop was already vectorized

 

   LOOP BEGIN at cr_ccsd_t_E.F(1219,7)

      remark #25420: Collapsed with loop at line 1220 

      remark #15542: loop was not vectorized: inner loop was already vectorized

 

      LOOP BEGIN at cr_ccsd_t_E.F(1221,7)

         remark #15542: loop was not vectorized: inner loop was already vectorized

 

         LOOP BEGIN at cr_ccsd_t_E.F(1222,7)

         <Peeled>

         LOOP END

 

         LOOP BEGIN at cr_ccsd_t_E.F(1222,7)

            remark #25420: Collapsed with loop at line 1223 

            remark #15300: LOOP WAS VECTORIZED

 

            LOOP BEGIN at cr_ccsd_t_E.F(1223,7)

               remark #25421: Loop eliminated in Collapsing 

 

            LOOP END

         LOOP END

 

         LOOP BEGIN at cr_ccsd_t_E.F(1222,7)

         <Alternate Alignment Vectorized Loop>

         LOOP END

 

         LOOP BEGIN at cr_ccsd_t_E.F(1222,7)

         <Remainder>

            remark #15301: REMAINDER LOOP WAS VECTORIZED

         LOOP END

 

         LOOP BEGIN at cr_ccsd_t_E.F(1222,7)

         <Remainder>

         LOOP END

      LOOP END

 

      LOOP BEGIN at cr_ccsd_t_E.F(1220,7)

         remark #25421: Loop eliminated in Collapsing 

 

      LOOP END

   LOOP END

LOOP END

===========================================================================

make[3]: Leaving directory `/data009/gb_lab/rahulsn/nwchem-6.6/src/tce/ccsd_t'

make[2]: *** [optimized] Error 2

make[2]: Leaving directory `/data009/gb_lab/rahulsn/nwchem-6.6/src/tce/ccsd_t'

make[1]: *** [subdirs] Error 1

make[1]: Leaving directory `/data009/gb_lab/rahulsn/nwchem-6.6/src/tce'

make: *** [libraries] Error 1

TimP
Black Belt
126 Views

Opt report questions might be better dealt with in a compiler forum. I don't see a failure in what you quoted.
Evarist_F_Intel
Employee
126 Views

I am not sure opt-report is related to the problem. Rahul,could you please remove make option "-j4", to serialize the output. Most likely the problem is somewhere else.