Intel® oneAPI Base Toolkit
Support for the core tools and libraries within the base toolkit that are used to build and deploy high-performance data-centric applications.

dlopen libirng.so error

Xiaochang_W_Intel
2,931 Views

Check the attached reproducer. I would like to manually package and load some oneapi libs (from /opt/intel/oneapi/compiler/2021.3.0/linux/compiler/lib/intel64_lin) in the dependency order. When I use dlopen to open libirng.so with path, dlopen return OK. in second time, I use dlopen without path and with RTLD_NOLOAD to check if it's loaded, failed.

For other lib like libsvml.so, First time I use dlopen with path, second time without path, it can pick up the first loaded lib with not error.

I wonder anything special related to libirng.so? I tried LD_DEBUG to debug but nothing found. Thanks in advance!

Output:

Loading ./lib/libintlc.so.5

Loading ./lib/libsvml.so

Loading ./lib/libirng.so

Checking libintlc.so.5

Checking libsvml.so

Checking libirng.so
libirng.so is not loaded!

0 Kudos
13 Replies
SantoshY_Intel
Moderator
2,891 Views

Hi,

 

Thanks for reaching out to us.

 

We are able to reproduce your issue at our end. However, you can try giving the path of libirng.so to LD_LIBRARY_PATH as below:

export LD_LIBRARY_PATH=path-of-libirng.so:$LD_LIBRARY_PATH

 

Now run the code to get the desired output.

 

>>"I wonder anything special related to libirng.so? I tried LD_DEBUG to debug but nothing found."

Regarding this we will get back to you soon.

 

Best Regards,

Santosh

 

0 Kudos
Xiaochang_W_Intel
2,889 Views

@SantoshY_Intel  Thanks for the quick response! We understood LD_LIBRARY_PATH can work however we have restricted environment that can't freely set LD_LIBRARY_PATH. We have to use dlopen to load the libs dynamicly. Must figure out why the method can't work in this single lib not the others. 

0 Kudos
SantoshY_Intel
Moderator
2,815 Views

Hi,


We are working on your issue internally. We will get back to you soon.


Thanks,

Santosh


0 Kudos
cw_intel
Moderator
2,788 Views

Hi,

It seems check_lib() calls are missing “ccl_root” .

If renamed these 3 libs , then check if these are loaded, as below shows. All of them failed.

    open_lib(ccl_root + "cc_libintlc.so.5");

    open_lib(ccl_root + "cc_libsvml.so");

    open_lib(ccl_root + "cc_libirng.so");

    check_lib("cc_libintlc.so.5");

    check_lib("cc_libsvml.so");

   check_lib("cc_libirng.so");

The output is, ​

Loading ./lib/cc_libintlc.so.5

Loading ./lib/cc_libsvml.so

Loading ./lib/cc_libirng.so

Checking cc_libintlc.so.5

cc_libintlc.so.5 is not loaded!

​This confirms that dlopen() did not search into ./lib to find the files to load. In the original case, it may be just lucky to find libsvml.so and libintlc.so.5 elsewhere in the PATH.

​After specify the path when calling check_lib(), there is no error.

Loading ./lib/libintlc.so.5


Loading ./lib/libsvml.so


Loading ./lib/libirng.so


Checking ./lib/libintlc.so.5


Checking ./lib/libsvml.so


Checking ./lib/libirng.so



0 Kudos
Xiaochang_W_Intel
2,779 Views

@cw_intel  Thanks for looking into details. the check_lib is not using path intentionally. I don't have other libs with the same name in the PATH. When not using a path, it will check if the lib is loaded. However, when you rename the filename, the following still work.

open_lib(ccl_root + "cc_libintlc.so.5");

check_lib("libintlc.so.5");                            // OK, it will still open ./lib/cc_libintlc.so.5 which has been loaded by the previous openlib.

check_lib("cc_libintlc.so.5");                      // Failed

check_lib(ccl_root + "cc_libintlc.so.5");    // OK by using path

This can also be comfirmed with export LD_DEBUG="files"

Loading ./lib/cc_libintlc.so.5
23347:
23347: file=./lib/cc_libintlc.so.5 [0]; dynamically loaded by ./dlopen_test [0]
23347: file=./lib/cc_libintlc.so.5 [0]; generating link map
23347: dynamic: 0x00007f58af2eb6a8 base: 0x00007f58af077000 size: 0x0000000000277680
23347: entry: 0x00007f58af07fa40 phdr: 0x00007f58af077040 phnum: 8
23347:
23347:
23347: calling init: ./lib/cc_libintlc.so.5
23347:
23347: opening file=./lib/cc_libintlc.so.5 [0]; direct_opencount=1
23347:
Checking libintlc.so.5
23347: opening file=./lib/cc_libintlc.so.5 [0]; direct_opencount=2

 

So it still can't explain why libirng.so is not behave like others. Maybe this lib is renamed? And pls also notice the libs have dependencies if you rename them. 

 

0 Kudos
Xiaochang_W_Intel
2,778 Views

Hi, I have some news, if you look at other libs, there is a SONAME read from readelf, but for libirng, there is not. Even when you rename the file, the SONAME will not change, I think dlopen will use the SONAME name. 

 

$ readelf -d lib/cc_libintlc.so.5

Dynamic section at offset 0x746a8 contains 28 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libintlc.so.5]
0x000000000000000c (INIT) 0x7d60
0x000000000000000d (FINI) 0x68588
0x0000000000000019 (INIT_ARRAY) 0x2742a0
0x000000000000001b (INIT_ARRAYSZ) 16 (bytes)
0x000000000000001a (FINI_ARRAY) 0x2742b0
0x000000000000001c (FINI_ARRAYSZ) 8 (bytes)
0x0000000000000004 (HASH) 0x228
0x000000006ffffef5 (GNU_HASH) 0xd18
0x0000000000000005 (STRTAB) 0x40e8
0x0000000000000006 (SYMTAB) 0x1820
0x000000000000000a (STRSZ) 6861 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x2748a8
0x0000000000000002 (PLTRELSZ) 4848 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x6a70
0x0000000000000007 (RELA) 0x5f90
0x0000000000000008 (RELASZ) 2784 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x0000000000000018 (BIND_NOW)
0x000000006ffffffb (FLAGS_1) Flags: NOW
0x000000006ffffffe (VERNEED) 0x5f20
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x5bb6
0x000000006ffffff9 (RELACOUNT) 88
0x0000000000000000 (NULL) 0x0

 

$ readelf -d lib/libirng.so

Dynamic section at offset 0x1569a8 contains 23 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libintlc.so.5]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
0x0000000000000004 (HASH) 0x260
0x000000006ffffef5 (GNU_HASH) 0x6c0
0x0000000000000005 (STRTAB) 0x1920
0x0000000000000006 (SYMTAB) 0xb70
0x000000000000000a (STRSZ) 3609 (bytes)
0x000000000000000b (SYMENT) 24 (bytes)
0x0000000000000003 (PLTGOT) 0x356b58
0x0000000000000002 (PLTRELSZ) 1728 (bytes)
0x0000000000000014 (PLTREL) RELA
0x0000000000000017 (JMPREL) 0x7590
0x0000000000000007 (RELA) 0x2880
0x0000000000000008 (RELASZ) 19728 (bytes)
0x0000000000000009 (RELAENT) 24 (bytes)
0x0000000000000018 (BIND_NOW)
0x000000006ffffffb (FLAGS_1) Flags: NOW
0x000000006ffffffe (VERNEED) 0x2860
0x000000006fffffff (VERNEEDNUM) 1
0x000000006ffffff0 (VERSYM) 0x273a
0x000000006ffffff9 (RELACOUNT) 816
0x0000000000000000 (NULL) 0x0

0 Kudos
cw_intel
Moderator
2,749 Views

Hi,


Thanks for providing the details. I will figure out this issue, and get back to you soon.


Thanks.


0 Kudos
cw_intel
Moderator
2,723 Views

Hi,


It seems like libirng.so don't be created with -Wl,-soname.

I confirmed with our linker expert, need to call dlclose after each dlopen. Otherwise the previously dlopend library will be used if their SONAMEs match.



0 Kudos
Xiaochang_W_Intel
2,699 Views

@cw_intel 

 

It seems like libirng.so don't be created with -Wl,-soname.

>> Is it possible to fix this by adding soname in the linker for libirng.so ?

 

I confirmed with our linker expert, need to call dlclose after each dlopen. Otherwise the previously dlopend library will be used if their SONAMEs match.

>> It's exactly what we want. We first load the lib with absolute path and then the following dlopen will reuse the previous lib without a absolute path. 

0 Kudos
cw_intel
Moderator
2,678 Views

Hi,

Is it possible to fix this by adding soname in the linker for libirng.so ?

>> I have delivered this request the relevant team. I'll let you know when I get the feedbacks.


Thanks.


0 Kudos
Xiaochang_W_Intel
2,289 Views

Hi, @cw_intel 

Could I know if any progress for this? I still need to use patchelf to set correct SONAME in our project. 

0 Kudos
cw_intel
Moderator
2,278 Views

This is fixed in the next release 2022.0 version. 

0 Kudos
Xiaochang_W_Intel
2,255 Views

@cw_intel Thanks for fixing that and prompt response!

0 Kudos
Reply