myricom MX udapl problems with intel MPI

[Rene_S_1]

Hi,

We have intel cluster toolkit compiler edition 3.2 installed and we are trying to use the intel mpi libraries in ICT to run on Myricom's MX.

I have downloaded the udapl library from from source forge and installed the DAPL 2.0 version in package dapl-myrinet-20-0.4.tar.gz.

when I try to run mpi this is what i get.
hpcp7761(salmr0)61:mpiexec -perhost 1 -genv I_MPI_DEVICE rdma:mx -np 2 -env I_MPI_DEBUG 10 ../bin/x86_64/IMB-MPI1.intelmpi PingPong
[1] MPI startup(): cannot open dynamic library libdat2.so
[1] my_dlopen(): Look for library libdat2.so in /usr/local/udaplmx/lib64/,/hpc/soft/intel/x86_64/ict-3.2.020/impi/3.2.0.011/lib64,/hpc/soft/intel/x86_64/ict-3.2.020/itac/7.2.0.011/itac/slib_impi3,/hpc/soft/intel/x86_64/ict-3.2.020/fc/11.0.074/lib/intel64,/hpc/soft/intel/x86_64/ict-3.2.020/cc/11.0.074/lib/intel64,/hpc/soft/intel/x86_64/ict-3.2.020/mkl/10.1.0.015/lib/em64t/usr/X11R6/lib64/Xaw3d,/usr/X11R6/lib64,/usr/X11R6/lib/Xaw3d,/usr/X11R6/lib,/usr/x86_64-suse-linux/lib,/usr/local/lib,/opt/kde3/lib,/opt/gnome/lib,/lib64,/lib,/usr/lib64,/usr/lib,/usr/local/lib64,/opt/kde3/lib64,/opt/gnome/lib64,include /etc/ld.so.conf.d/*.conf,/usr/local/syscfg/lib,,/lib,/usr/lib
[1] MPI startup(): cannot open dynamic library libdat2.so


hpcp7761(salmr0)59:ls /usr/local/udaplmx/lib64/
libdapl.a libdapl.so libdat2.so libdat.a libdat.so


hpcp7761(salmr0)65:cat /etc/dat.conf
#
# DAT 2.0 configuration file
#
# Each entry should have the following fields:
#
# \
#
#
mx u2.0 nonthreadsafe default /usr/local/udaplmx/lib64/libdapl.so ofa.2.0 "mx0 0" ""



Any ideas as to what the problem is?

Thanks
Rene

[g23754]

I'm not an Intel MPI expert, but I speculate that the problem is either the "/" on the end of "/usr/local/udaplmx/lib64/" in your search path, or /usr/local/udaplmx/lib64 is not available on a remote compute node, and the message is coming from there. If not, I bet help@myri.com can help.

--Glenn Brown (a Myricom developer)

[TimP]

Quoting - g23754

I'm not an Intel MPI expert, but I speculate that the problem is either the "/" on the end of "/usr/local/udaplmx/lib64/" in your search path, or /usr/local/udaplmx/lib64 is not available on a remote compute node, and the message is coming from there. If not, I bet help@myri.com can help.

--Glenn Brown (a Myricom developer)

According to the message, /usr/local/udaplmx/lib64 wasn't included in the search path LD_LIBRARY_PATH, on the node where the search failed. Intel MPI couldn't be expected to add (or subtract) that directory from the search path.

[Rene_S_1]

Quoting - tim18

According to the message, /usr/local/udaplmx/lib64 wasn't included in the search path LD_LIBRARY_PATH, on the node where the search failed. Intel MPI couldn't be expected to add (or subtract) that directory from the search path.

Yep that was it. Thanks. I added setenv LD_LIBRARY_PATH /usr/local/udaplmx/lib64:${LD_LIBRARY_PATH}
to my .cshrc file and now things work. Which is kind of strange because I have

cat /etc/ld.so.conf.d/udaplmx.ld.conf
/usr/local/udaplmx/lib64

setup system wide so there should be no need for me to set LD_LIBRARY_PATH.

Thanks

Rene