Error in loading libiomp5.so

[White_B_]

Hi!

When I run a program on mic0 after compiled with -mmic option, it returned

error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory

I've read this thread.

When I used find / -name "libiomp5.so" command, it returned

/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/intel64/libiomp5.so

/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/mic/libiomp5.so

/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/ia32/libiomp5.so

Which one should I add to the LD_LIBRARY_PATH?

Do I need to use export LD_LIBRARY_PATH= command? If so, what should be typed after the = ?

Thanks!

[White_B_]

In the original post, it referred to this thread.

[Kevin_D_Intel]

When running on the coprocessor, if your /opt/intel is mounted on the coprocessor then to LD_LIBRARY_PATH you need to add the path for the "mic" libraries (i.e. /opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/mic).

If /opt/intel is only accessible on the host, then as was noted in the cited thread in the last post, you scp the “mic” version of the library to the coprocessor in a location of your choice, connect to the coprocessor and set LD_LIBRARY_PATH on the coprocessor according to where you placed the library, then "cd" to where your executable is and run it.

Refer to Building a Native Application for Intel® Xeon Phi™ Coprocessors for more helpful details.

[robert-reed]

Would this program also use OpenMP? And is the OpenMP runtime (libiomp5.so) provisioned on your coprocessor?  If you're running in a cluster or other sort of institutional access to a pool of machines, they may have already defined some infrastructure where this lib is available.  If so, then the advise in the other thread will apply, but the LD_LIBRARY_PATH setting will have to be made in the coprocessor environment in order to be available to the execution file when it needs it.

Alternatively, you may be working on a solo/isolated machine and have full control (read: responsibility) over the placement of the MIC libiomp5.so upon the coprocessor FS.  If you have sudo access on the host, you could execute the following:

sudo scp /opt/intel/composerxe/lib/mic/libiomp5.so root@your_host_name_here-mic0:/lib64

"composerxe" is a symbolic link to the most recently installed version of the compiler, which in your case looks like 2013 SP1.2.144.  Placing the runtime library in that directory puts it in a default location for the loader, so no LD_LIBRARY_PATH would be needed in this case.

[White_B_]

Thank you!

Since the mic0 user can indeed get access to /opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/mic/libiomp5.so. The problem is solved by one line of command on mic0:

export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/mic/

[White_B_]

Hi Kevin,

I used export command and solved the problem temporarily.

How ever, when I log off and log on again, I need to reset it.

Previously, I don't need to set such a thing every time when I log on. Do you know how to make a "permanent" setting?

Thanks!

[Kevin_D_Intel]

Glad that helped.

I don't details of your host/coprocessor user access setup so I can’t speak to why something changed. It sounds like normal user login accounts are enabled on the coprocessor and that the host and coprocessor may share user home directories (in addition to /opt/intel). As you likely know, individual user shell setup scripts (e.g. like .bash_profile, .bashrc for the bash shell) resides in a user’s home area. Thus, you could setup in your private shell initialization scripts a setting specific to the coprocessor only where you assign the LD_LIBRARY_PATH variable accordingly.

For example, assuming you use the default MPSS setup, if your default working shell is bash, at the end of your private .bashrc, you could do this:

$test $HOSTNAME = <your_host_name_here>-mic0 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/composer_xe_2013_sp1/lib/mic

Where you replace <your_host_name_here> with your host’s hostname and you follow the earlier advice to use the generic Composer XE 2013 SP1 link so your shell setup script always uses the latest installed compiler run-time libs on your system. This would add the compiler RTL path to the variable only on your “mic0” coprocessor when logging in under the associated user id where you make this change.

[White_B_]

Hi,
By "a user's home area", do you mean the home area on the coprocessor?
On the coprocessor, as a sudo user newuser on the coprocessor mic0, I tried the following commands:
cat ~/.bashrc
cat ~/.bash_profile
cat ~/.profile
cat /etc/bash.bashrc
None of them exist. The only related file is /etc/profile

By hostname command on the coprocessor, I got mic0.local.
I tried adding
$test $HOSTNAME = mic0.loca-mic0 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/composer_xe_2013_sp1/lib/mic
$test $HOSTNAME = mic0.loca-mic0 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/composer_xe_2013_sp1.2.144/lib/mic
Neither of them worked.
Then I tried creating a file ~/.bashrc on the co-processor, with the only content:
$test $HOSTNAME = mic0.loca-mic0 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/composer_xe_2013_sp1/lib/mic
$test $HOSTNAME = mic0.loca-mic0 && export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/composer_xe_2013_sp1.2.144/lib/mic
Again, neither worked.

Is it because my ~/.bashrc doesn't have the correct format? What does $test mean here? It seems $test is used nowhere.

I believe my MPSS uses the default setup.

On the coprocessor, by echo $0, I got -bash; by echo $SHELL, I got /bin/bash and by ps -ef | grep $$ | grep -v grep, it returned
newuser       15932  15931  0 05:48 pts/0    00:00:00 -bash
newuser       15938  15932  0 05:49 pts/0    00:00:00 ps -ef

Thanks!

[robert-reed]

The files you want to have permanence on the coprocessor face a problem: the coprocessor "filesystem" is ephemeral as well, and gets rebuilt each time you restart MPSS.  So there's a couple of ways around this.  If you've created newuser as a coprocessor account using micctrl --useradd then there should be a "permanent" version on the host filesystem, used for recreating the virtual version during MPSS reboot.  For MPSS 3, the default location for those home directories for the first coprocessor on the system is /var/mpss/mic0/home.  If you have created this user, you should find a /var/mpss/mic0/home/newuser and it probably will have a .profile file contained therein.  You could modify that file to add the environment settings you want to appear in login shells on the coprocessor, or add .bash_profile and .bashrc files there, but recognize that even these host files are temporary: a reinstall of MPSS or executing some of the options available in micctrl can wipe out these files.  

A safer solution is to create user accounts in some non-system area of the host file system and then mount the "local home" on the coprocessor, say using NFS.  On several of the machines I take care of, I've created a directory, /home/michome, in which I construct home directories for installation on the coprocessor, and then use a line like the following in /var/mpss/mic0/etc/fstab to ensure the partition will be mounted when the coprocessor boots up:

host:/home/michome /home        nfs     rw,nolock               0 0

This has the advantage that though there might be a little work involved to reestablish these links after upgrading MPSS or performing other filesystem affecting operations on the host, the files themselves will not be affected by MPSS changes but will make the home directories when running on the coprocessor persistent--changes made in the home directory will propagate back to the host filesystem.