the MPI compilation of my code results in the following error:
ld: MPIR_Thread: TLS definition in /opt/apps/intel13/impi/4.1.0.030/intel64/lib/libmpi_mt.so section .tbss mismatches non-TLS definition in /opt/apps/intel13/impi/4.1.0.030/intel64/lib/libmpi.so.4 section .bss
/opt/apps/intel13/impi/4.1.0.030/intel64/lib/libmpi.so.4: could not read symbols: Bad value
I have sent the systems admin. several e-mails but he just can't seem to figure it out. Any help on how to fix this problem would be appreciated.
How are you compiling and linking? At a guess, it looks like you are mixing the standard and thread-safe MPI libraries. If you are using the compiler scripts, look for -mt_mpi, -openmp, -parallel, -threads, -reentrancy, or -reentrancy threaded. Any of these options will link to the thread-safe library (libmpi_mt). If you are using them, make certain that they are used on all compliation and linking commands. If you are using one of the flags other than -mt_mpi and want to restrict it to only the necessary routines, then adding -mt_mpi to others should be sufficient.
Technical Consulting Engineer
Intel® Cluster Tools
Hope you all don't mind me resurrecting this one.
But I have the exact same, or at least similar, error.
ld: MPIR_Thread: TLS definition in /opt/gridware/c8000/intel/impi/4.1.0.024/lib64/libmpi_mt.so section .tbss mismatches non-TLS definition in /opt/gridware/c8000/intel/impi/4.1.0.024/lib64/libmpich.so section .bss
/opt/gridware/c8000/intel/impi/4.1.0.024/lib64/libmpi_mt.so: could not read symbols: Bad value
I'm using -openmp with ifort, and it appears at all compilation and linking commands.
I've since also added -mt_mpi, as well as "-L<mpi_install> -lmpi_mt".
I still get the error.
Here is a shortened version of my final compiler command:
mpiifort -o <executable> -I./Includes -I/opt/gridware/c8000/intel/impi/4.1.0.024/include64 -openmp -mt_mpi -mcmodel=medium -O2 -mieee-fp -align dcommons -extend_source -convert big_endian <many objects> -L/opt/gridware/c8000/intel/impi/4.1.0.024/lib64 -lmpich -lrt -lpthread -libverbs -libumad -lmpi_mt -L/opt/gridware/lib64 -libcommon
and mpiifort -show =
ifort -ldl -I/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/include -I/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/include -L/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/4.1 -lmpi -lmpigf -lmpigi -lrt -lpthread
Anybody have any ideas?
Thanks for having a look.
LD_LIBRARY_PATH = /opt/gridware/applications/PBS/mvapich2/lib:/opt/gridware/compilers/intel/icsxe/composer_xe_2013/lib/intel64:/opt/gridware/compilers/intel/icsxe/itac/8.1.0.024/intel64/slib:/opt/gridware/compilers/intel/icsxe/itac/8.1.0.024/intel64/lib:/opt/gridware/compilers/intel/icsxe/tbb/lib/intel64:/opt/gridware/compilers/intel/icsxe/ipp/lib/intel64:/opt/gridware/compilers/intel/icsxe/mkl/lib/intel64:/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/lib64:/opt/gridware/users/nag/lib64:/opt/gridware/users/nag/lib:/opt/pbs/default/lib/:/opt/gridware/lib64
But that's just for runtime right?
I should've mentioned this first, but the model I use makes use of C for a few routines (primarily the MPI stuff). My mpicc -show gives:
gcc -ldl -I/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/include -L/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/4.1 -lmpi -lmpigf -lmpigi -lrt -lpthread
And in that makefile I've used:
LIB = -L$(MPI_INST)/lib64 -lmpich -lrt -lmpi_mt
CFLAGS = -O2 -mcmodel=medium -I $(INC)
I've noticed that -lmpi_mt is only used in the final linking process.
Is there an -mt_mpi option for gcc?
I've tried to attach the log of the make here but I think my java is too old.
Can I mail it somewhere?
You mean change -lmpi to -lmpi_mt in my mpicc wrapper?
So that mpicc -show will show a gcc compile command with -lmpi_mt INSTEAD of -lmpi?
I didn't compile mpi for our cluster. How do I change what's given by the wrapper without recompiling?
I guess instead of mpicc I could manually use gcc with all the necessary mpicc commands and there change -lmpi to -lmpi_mt.
Adding -lmpi_mt to each use of mpicc should replace the default libmpi with libmpi_mt every time, as James suggested. It won't change the showme result.
You and your sysadmin don't have the option of rebuilding Intel MPI. You could make your own copy of the python script in the Intel mpi installation and make a change there, but your sysadmin would be entirely justified in refusing to support you.
If you have an installation of icc and mpiicc, do you have a reason for using gcc?
My initial attempts (post #3 and #6) with adding -L<impi install>/lib64 -lmpi_mt to the ifort and gcc compiler commands did not solve the problem:
ld: MPIR_Thread: TLS definition in /opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/lib64/libmpi_mt.so section .tbss mismatches non-TLS definition in /opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/lib64/libmpich.so section .bss
/opt/gridware/compilers/intel/icsxe/impi/4.1.0.024/lib64/libmpi_mt.so: could not read symbols: Bad value
Note that this happens for the final linking phase using mpiifort. The mpicc compilation itself seems to go ok (i.e. no errors), although there is no linking done, just ar to create a library. Which is later on linked in the final mpiifort linking that's giving the problem.
Indeed your solution should have worked, since you pointed out that it will replace (or supersede?) the -lmpi shown in the wrapper.
Yes we do have icc on the cluster (13.0.1) but my code specifies gcc in the makefile. Shown below is the makefile for the C codes.
MPI_INST = /opt/gridware/compilers/intel/icsxe/impi/4.1.0.024
LIBNAME = libparlib.a
CC = $(MPI_INST)/bin64/mpicc
LIB = $(MPI_INST)/lib64 -lmpich -lrt -lmpi_mt
INC = $(MPI_INST)/include64
####CFLAGS = -O2 -m32 -I $(INC)
CFLAGS = -O2 -mcmodel=medium -I$(INC) -L$(LIB)
OBJS= par_assoc_buff.o \
$(LIBNAME) : $(OBJS)
$(CC) $(CFLAGS) -c $(?:.o=.c)
ar -rv $@ $?
chmod 644 $@
rm -rf *.o $(LIBNAME)
I don't know to what extent anyone has been able to make a successful MPI application using the rudimentary MIC gcc, or if mpicc is even tested, as gcc isn't intended to support satisfactory performance. For example, there is an intentionally non-working mpif90 there.
Your attempt to use libmpi_mt implies that you are trying to make a threaded gcc application. Were you able to do that in a simple case, not using MPI?
If your point is that your icc is too old to have been tested with a current MIC MPSS, and you may as well use one untested tool as another, that may be true, but MIC programming is difficult enough using tested development tools.
Hi Tim and James,
Thanks for your help.
I could make an omp and non-omp executable without MPI.
I could also make a non-omp executable with MPI.
It was just the omp + MPI I couldn't do.
But James was correct in his last post. I just forgot to remove -lmpich :->.
Didn't even occur to me.
So now I can make an omp + MPI executable!
Edit: The reason for the old library linking in the makefile is that I'm using a relatively old version of a model, which used mpich2.