Where are these references located? Installation error?

[Brandon_W_]

All,

Thank you for taking a look at this!

I have downloaded and installed composer xe 2015.3.187 and didn't see any obvious errors. However, when I try to run an often used code from other machines on my new machine, I get:

/usr/lib/gcc/x86_64-redhat-linux/4.8.3/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_group_failed'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_remote_group_failed'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `PMPIX_Comm_reenable_anysource'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `MPID_b_use_gettimeofday'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `i_malloc'
/home/bwheeler/intel/impi/5.0.3.048/intel64/lib/libmpifort.so: undefined reference to `i_free'
collect2: error: ld returned 1 exit status
make: *** [dmcblast_3d_local] Error 1

I've googled the issue, but I feel that 'main' isn't the problem, since the code works on other machines. I've also tried to google the underfined references, but can't find what library they're related to so that I can either download it or add the libraries to the PATH.

I have made sure that <install-dir>/intel/composer_xe_2015.3.187/bin/intel64  (for compilervars.sh) and <install-dir>/intel/impi/5.0.3.048/intel64/bin (for mpivars.sh) are in the PATH and LD_LIBRARY_PATH. I have also tested mpicc and mpiicc and seem to be getting the correct outputs, so I feel that the MPI installation and Libraries within /intel are working.

I feel that I may have downloaded the Intel Software incorrectly, but am unsure how. Could you please help me to either download the software correctly, with all libraries going to the right spots, or help me find the libraries these undefined references are in?

I would really appreciate it! Thank you! Please feel free to contact me with any further questions.

[Steven_L_Intel1]

These look like Intel MPI issues so I will move this thread to the Clustering tools forum.

[Ron_Green]

if you're using bash as your shell:

source <install-dir>/intel/bin/ifortvars.sh intel64

source <install-dir>/intel/impi/5.0.1.035/bin64/impivars.sh

Then use mpiifort for your compiler and linker, NOT ifort and ld.  

Ron

[James_T_Intel]

Please provide your compilation options and your command line for running your program.

[Brandon_W_]

Strange, I thought I replied to Ronald with info regarding my Makefile. My apologies! I had already sourced ifortvars.sh but hadn't sources impivars.sh. I then switched my compiler and linker.

  James/Ronald: I tried doing the things Ronald suggested, but with no success. Here is what my Makefile looks like. Included are my libraries and compilation options. Maybe I switched to mpiifort incorrectly? Make command: make -f Makefile     My run command is: ./rundmc-3d.opt (filename)

The Makefile:

# This Makefile is for f90 build with MPI, intel compiler.

LIBDIRS_INTEL= -L/usr/local/lib -L/home/bwheeler/intel/composer_xe_2015.3.187/compiler/lib/intel64 -L./lib -L/usr/local/bin

LIBDIRS_MPIF = -L/home/bwheeler/intel/impi/5.0.3.048/intel64/lib -L./lib

INCLUDE_MPIF = -I/home/bwheeler/intel/impi/5.0.3.048/intel64/include

LIBDIRS_SEACAS = -L/home/bwheeler/Downloads/SEACAS-2014-12-16/lib -L/home/bwheeler/Downloads/SEACAS-2014-12-16/lib/shared -L/home/bwheeler/Downloads/SEACAS-2014-12-16/TPL/hdf5/hdf5-1.8.14-linux-centos6-x86_64-gcc482-shared/lib -L./lib

INCLUDE_SEACAS = -I/home/bwheeler/Downloads/SEACAS-2014-12-16/inc

LDFLAGS  =  $(LIBDIRS_INTEL) $(LIBDIRS_SEACAS) $(LIBDIRS_MPIF) -lexoIIv2for -lexodus -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsupes -lsuplib -lifcore -lifcoremt -lsvml -limf -lintlc -lifport

#F90FLAGS = -g $(INCLUDE_SEACAS) $(INCLUDE_MPIF) -i8 -r8 -Dlinux -DADDC_ -fc=ifort (previous line)

F90FLAGS = -g $(INCLUDE_SEACAS) $(INCLUDE_MPIF) -i8 -r8 -Dlinux -DADDC_ -fc=mpiifort

#F90C= mpif90 (previous line)

F90C= mpiifort

.SUFFIXES: .o .f90

%.o: %.mod

.f90.o:
    $(F90C) $(F90FLAGS) -c $< -o $@

PROGRAM    = dmcblast_3d_local


default: $(PROGRAM)

OBJ = \
    mod_params.o mod_arrays.o mod_binsort.o mod_gas.o \
    ainit.o binsort.o bndset.o bndtrc.o contac.o dilate.o dis_pnt2lne.o \
    dis_pnt2lne_sa.o dist_pt_plane.o dmcblast-3d.o dmc_mpi.o dmpset.o dotprod.o exo1.o  \
    exo2.o expset.o fct2d.o forces.o fragstat.o gasfrc.o gasgrd.o  \
    gasimp.o gasini.o gastrm.o gbcond.o geosici.o geosjwl.o getbnd.o  \
    gradcal.o gridset.o iamax.o iamin.o igaspor.o indexx.o  \
    init_arrays_1.o init_arrays_2.o init_arrays_3.o \
    loaddk.o meshin.o mklst3.o \
    pexod1.o pexod2.o phazin.o plane.o rank.o readin.o rotvec.o  \
    salive.o solve.o srchge.o srchgt.o streng.o strlnk.o tracer.o  \
    veloce.o vert_extrm.o vlngth.o pnpoly.o flush.o

SRC = $(OBJ,.obj=.f90)

$(PROGRAM):: $(OBJ)
    touch mod*.f90
    $(F90C) $(MODULE_OBJ) $(OBJ) $(LDFLAGS) -o $(PROGRAM)

This Makefile has worked in the past on a different computer (though with the seacas and intel locations in a different place).

Thank you everyone for your help! I'm pretty lost and really appreciate your insight.

[Brandon_W_]

Sorry everyone for the late responses. I didn't realize that I had 'failed' the word verification! =P

Thank you again!!!!

[James_T_Intel]

A few comments and questions.  I'm not sure if any of these will help, but they shouldn't hurt to fix.

Using mpiifort, the -fc option doesn't do anything.  The mpiifort always uses ifort as the underlying compiler.

Also, when using mpiifort, if you have sourced mpivars.sh (recommended practice), you don't need to specify the include folder for the MPI include files or modules, or the library path for them.  The mpiifort wrapper will take care of this.

I would define the module objects in MODULE_OBJ.  That appears to be the purpose of that variable, and (for me at least) it makes tracking things easier.

Why touch the module source files in the linking phase?

What does rundmc-3d.opt do?

[Brandon_W_]

Why touch the module source files in the linking phase?

I'm not entirely sure. Unfortunately, I didn't write this make file, just updated the library paths for this specific computer.

What does rundmc-3d.opt do?

This code sets up a grid of spheres used to model an explosive blast and then calculates how those spheres will move over time. (I work for a mining company.)

Thank you for the suggestions on cleaning up the code! I like learning why things are actually done. However, it didn't fix this particular problem.

Thank you for your continued help! =D

[James_T_Intel]

It's interesting to know the goal of the program.  But I'm wondering if rundmc-3d.opt is a script that launches the program, or something else.  Can you provide output from

ldd dmcblast_3d_local

I'm trying to see if there is something odd in the environment preventing a library from being found.

[Brandon_W_]

rundmc-3d.opt is a shell script that runs the executable dmcblast_3d_local.

rundmc-3d.opt:

#!/bin/csh
rm  -f $argv[1].e
rm  -f $argv[1].gas
rm  -f $argv[1].o
rm  -f $argv[1].dbg
rm  -f $argv[1].lnk
setenv FOR005 $argv[1].i
setenv FOR007 $argv[1].o
setenv FOR008 $argv[1].rmi
setenv FOR009 $argv[1].g
setenv FOR010 $argv[1].rst
setenv FOR011 $argv[1].e
setenv FOR012 $argv[1].gas
setenv FOR013 $argv[1].lnk
setenv FOR014 $argv[1].dbg
idb dmcblast_3d_local

I can't yet run ldd dmcblast_3d_local on my new machine yet because I haven't been able to compile the Makefile to create the executable.

Here is the output from the other, working machine. Sorry, it's kinda long.

linux-vdso.so.1 =>  (0x00007fffc5517000)
    libdl.so.2 => /lib64/libdl.so.2 (0x0000003d53000000)
    libhdf5_hl.so.8 => /usr/lib/libhdf5_hl.so.8 (0x00007f94726b0000)
    libhdf5.so.8 => /usr/lib/libhdf5.so.8 (0x00007f94721fb000)
    libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x0000003d52c00000)
    libm.so.6 => /lib64/libm.so.6 (0x0000003d52800000)
    libz.so.1 => /lib64/libz.so.1 (0x0000003d53400000)
    libifcore.so.5 => /opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64/libifcore.so.5 (0x00007f9471fb7000)
    libifcoremt.so.5 => /opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64/libifcoremt.so.5 (0x00007f9471d46000)
    libsvml.so => /opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64/libsvml.so (0x00007f94715d3000)
    libmpi.so.4 => /opt/intel/impi/4.0.3.008/intel64/lib/libmpi.so.4 (0x00007f9471108000)
    libmpigf.so.4 => /opt/intel/impi/4.0.3.008/intel64/lib/libmpigf.so.4 (0x00007f9470fda000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9470dbd000)
    librt.so.1 => /lib64/librt.so.1 (0x0000003d53800000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x0000003d58c00000)
    libc.so.6 => /lib64/libc.so.6 (0x0000003d52400000)
    libimf.so => /opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64/libimf.so (0x00007f94709f0000)
    libintlc.so.5 => /opt/intel/composer_xe_2011_sp1.6.233/compiler/lib/intel64/libintlc.so.5 (0x00007f94708ad000)
    /lib64/ld-linux-x86-64.so.2 (0x0000003d52000000)
    libsz.so.2 => /usr/lib/libsz.so.2 (0x00007f9470799000)
 

As always, many, many thanks for your time and help. I am at a loss and really appreciate it!

[James_T_Intel]

Ah, I had a misunderstanding, I thought this error was showing up at runtime.  Are all of the object files listed in the Makefile being created?  Is there a source file (especially the one with the program declaration) missing from the object file list?

[Brandon_W_]

   It would appear that all of the .o are being made into .f90 files. The source file dmcblast-3d.o is being made as well. The problem seems to occur after the .f90 files are created and immediately after the PATHs and libraries are called out within the compilation.

The last line before the error reads: -lintlc -lifport -o dmcblast_3d_local (which is the execution file).

[James_T_Intel]

That shouldn't be the entire last line, it probably wrapped from an earlier line.  There should be more to it.  Is it possible to send me the source code and Makefile, along with the full output from your compilation?  If necessary, this can be done via private message.

[Brandon_W_]

You are correct. The  full last line is significantly longer. I should have said 'last part before the error takes place'. I'm happy to send you the Makefile and compilation output. I will have to double check with my supervisors regarding the source code.

Thank you for spending your time on this!!

[James_T_Intel]

Please send it to me here via private message.  If I am unavailable, other support engineers can see my private messages and respond if necessary.  I also removed your email address from the post, please try not to post personal information in the public forum.

[Brandon_W_]

I will do so as soon as I get out of my next meeting! Thank you.

[James_T_Intel]

Brandon,

I've compiled your code and it appears to be running as expected.  I would recommend cleaning out your object files and giving it another try.

James.

[Brandon_W_]

James!

   You're a genius! That's such wonderful news! =D

   I removed the .o files and reran the code, but still got the same error. May I ask what library/code versions you used to successfully run the code? I'll try downloading the same versions and see if that does anything different. If not, I would really appreciate a step-by-step process of what you did to make it work!

    Thank you so much again for all of your time and help. You rock.

[James_T_Intel]

Intel® Fortran Compiler 15.0.3.187
Intel® MPI Library 5.0.3.048
netCDF 4.3.3.1
HDF5-1.8.15-patch1
matio-1.5.2
SEACAS master

I compiled each of the components using the same base compiler and MPI.  The netCDF configure line included --disable-shared --enable-parallel-tests.  HDF5 had --enable-fortran --enable-parallel.  Others were default options (except for finding directories).

[Brandon_W_]

I'll give those a try! Thank you.

[Brandon_W_]

James,

    I was off to a pretty good start installing the list you sent me, but my computer restarted and gave me this today (picture attached). I haven't been able to find anything on how to fix this. Could it have been something I did when removing the other other versions of code? Any idea how I can fix this and whether, once it's working again, I should ever restart my computer or not? I wish I could tell you what might have caused this, but I really have no clue. However, now the installation disc I was using doesn't work, I can't access the grub at all, and don't know how to restore a grub! Ah!

Thank you for your help! I know with your help that I'll get this code working!!

Best Regards,

Brandon

If the picture is too small, it says:

error: not a correct XFS inode. (repeated a ton of times)

error: file '/grub2/i386-pc/normal.mod' not found

grub rescue>