create static dpc++ application for deployment?

[CFR]

The explicit question is, what's the right/best way to build/link/deploy a dpc++ application where the target environment can not have the OneAPI toolkit(s) installed.  In my experience this is common in HPC/cluster installations where the compute node are "lean" and admins are reluctant to install beta system software on production machines.

Environment: Intel(R) oneAPI DPC++ Compiler 2021.1-beta08 (2020.7.0.0715)

I was experimenting with creating a static dpc++ application for deployment to another machine/cluster.  I have a development environment on my laptop (were I have privileges to install contemporary OS and tools) and want to deploy to another machine/cluster where I have no special privileges to install anything.  So I've run into two problems:

1) my target machine has older libstdc++ than what libsycl wants:

[~/OneAPI]$ ./a.out
./a.out: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./libsycl.so.2)
./a.out: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./libsycl.so.2)
./a.out: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by ./libsycl.so.2)

So my first thought was to try to make libstdc++ static (per https://software.intel.com/content/www/us/en/develop/documentation/oneapi-dpcpp-compiler-dev-guide-and-reference/top/compiler-reference/compiler-options/compiler-option-details/linking-or-linker-options/static-libstdc.html

user@t570: ~/OneAPI/SYCL$ cat libstatic.cpp

int
main(int argc, char *argv[])
{
}

user@t570: ~/OneAPI/SYCL$ dpcpp libstatic.cpp
user@t570: ~/OneAPI/SYCL$ ldd a.out
	linux-vdso.so.1 (0x00007fffec7b6000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f8a9c17a000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8a9bddc000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f8a9bbc4000)
	libsycl.so.2 => /local/opt/inteloneapi/compiler/latest/linux/lib/libsycl.so.2 (0x00007f8a9b964000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8a9b573000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f8a9c503000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8a9b36f000)
	libOpenCL.so.1 => /local/opt/inteloneapi/compiler/latest/linux/lib/libOpenCL.so.1 (0x00007f8a9c706000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8a9b150000)
	libimf.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libimf.so (0x00007f8a9aacd000)
	libsvml.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libsvml.so (0x00007f8a9901b000)
	libirng.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libirng.so (0x00007f8a98cb1000)
	libintlc.so.5 => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00007f8a98a37000)

user@t570: ~/OneAPI/SYCL$ rm a.out

user@t570: ~/OneAPI/SYCL$ dpcpp -static-libgcc -static-libstdc++ libstatic.cpp
user@t570: ~/OneAPI/SYCL$ ldd a.out
	linux-vdso.so.1 (0x00007ffe53b93000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3bf39a6000)
	libsycl.so.2 => /local/opt/inteloneapi/compiler/latest/linux/lib/libsycl.so.2 (0x00007f3bf3746000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3bf3355000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3bf3d44000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3bf3151000)
	libOpenCL.so.1 => /local/opt/inteloneapi/compiler/latest/linux/lib/libOpenCL.so.1 (0x00007f3bf3f47000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3bf2f32000)
	libimf.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libimf.so (0x00007f3bf28af000)
	libsvml.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libsvml.so (0x00007f3bf0dfd000)
	libirng.so => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libirng.so (0x00007f3bf0a93000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3bf070a000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3bf04f2000)
	libintlc.so.5 => /local/opt/inteloneapi/compiler/latest/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00007f3bf0278000)

I've never tried doing this before, but I would have expected libgcc and libstdc++ to _not_ be in the ldd listing. which seems to be the case with plain old g++:

user@t570:~/OneAPI/SYCL$ cat libstatic.cpp

#include <iostream>
int
main(int argc, char *argv[])
{
  std::cout << "Hello World" << std::endl;
}
user@t570:~/OneAPI/SYCL$ g++ libstatic.cpp
user@t570:~/OneAPI/SYCL$ ldd a.out
	linux-vdso.so.1 (0x00007ffdfcd7a000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3327b50000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f332775f000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f33273c1000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f33280db000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f33271a9000)

user@t570:~/OneAPI/SYCL$ g++ libstatic.cpp -static-libstdc++ -static-libgcc
user@t570:~/OneAPI/SYCL$ ldd a.out
	linux-vdso.so.1 (0x00007ffffe154000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb729d24000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fb72a3fc000)

2) The second issue is that there don't seem to be any static OneAPI libraries.  (I don't see any .a files in the compiler directories and using -static seems to do nothing).  I do think I can probably pick up the necessary libraries but it's still going to be a pain to schlep all that stuff around to deploy the application.

There is an additional issue regarding the SYCL/kernel/runtime part of an application but I'm assuming that "Ahead of Time" compilation https://software.intel.com/content/www/us/en/develop/documentation/oneapi-dpcpp-compiler-dev-guide-and-reference/top/compilation/ahead-of-time-compilation.html will address that.

[AbhishekD_Intel]

Hi,

We are also getting the same results with static libstdc++ and libgcc.

Will you please give us the output for readelf -d a.out (both static and dynamic)?

Meanwhile, we will look into this and will give you more details.

Warm Regards,

Abhishek

[CFR]

user@t570:~/OneAPI/SYCL$ cat libstatic.cpp

#include <iostream>
int
main(int argc, char *argv[])
{
  std::cout << "Hello World" << std::endl;
}

user@t570:~/OneAPI/SYCL$ dpcpp libstatic.cpp
user@t570:~/OneAPI/SYCL$ readelf -d a.out

Dynamic section at offset 0xde0 contains 28 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libsycl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000c (INIT)               0x400758
 0x000000000000000d (FINI)               0x400a24
 0x0000000000000019 (INIT_ARRAY)         0x600db8
 0x000000000000001b (INIT_ARRAYSZ)       24 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x600dd0
 0x000000000000001c (FINI_ARRAYSZ)       16 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x400278
 0x0000000000000005 (STRTAB)             0x400458
 0x0000000000000006 (SYMTAB)             0x4002c0
 0x000000000000000a (STRSZ)              401 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x601000
 0x0000000000000002 (PLTRELSZ)           192 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x400698
 0x0000000000000007 (RELA)               0x400650
 0x0000000000000008 (RELASZ)             72 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x400610
 0x000000006fffffff (VERNEEDNUM)         2
 0x000000006ffffff0 (VERSYM)             0x4005ea
 0x0000000000000000 (NULL)               0x0

user@t570:~/OneAPI/SYCL$ dpcpp -static-libgcc -static-libstdc++ libstatic.cpp
user@t570:~/OneAPI/SYCL$ readelf -d a.out

Dynamic section at offset 0x121da8 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libsycl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000000c (INIT)               0x451608
 0x000000000000000d (FINI)               0x4ec174
 0x0000000000000019 (INIT_ARRAY)         0x71b8a8
 0x000000000000001b (INIT_ARRAYSZ)       72 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x71b8f0
 0x000000000000001c (FINI_ARRAYSZ)       16 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x4002b0
 0x0000000000000005 (STRTAB)             0x41df20
 0x0000000000000006 (SYMTAB)             0x406e98
 0x000000000000000a (STRSZ)              200099 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x722000
 0x0000000000000002 (PLTRELSZ)           2376 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x450cc0
 0x0000000000000007 (RELA)               0x450c00
 0x0000000000000008 (RELASZ)             192 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x450b80
 0x000000006fffffff (VERNEEDNUM)         2
 0x000000006ffffff0 (VERSYM)             0x44ecc4
 0x0000000000000000 (NULL)               0x0

[AbhishekD_Intel]

Hi,

We are forwarding this issue to the experts. They will give you more details about it.

Warm Regards,

Abhishek

[Viet_H_Intel · Accepted solution]

As you see libsycl.so.2 has dependency on these shared libs already, I dont think we can eliminate them.

$ ldd /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/lib/libsycl.so.2

       linux-vdso.so.1 (0x00007ffe161a4000)

       libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f437a6b5000)

       libOpenCL.so.1 => /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/lib/libOpenCL.so.1 (0x00007f437a6ac000)

       libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f437a687000)

       libimf.so => /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/compiler/lib/intel64_lin/libimf.so (0x00007f437a004000)

       libsvml.so => /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/compiler/lib/intel64_lin/libsvml.so (0x00007f43784d8000)

       libirng.so => /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/compiler/lib/intel64_lin/libirng.so (0x00007f437816e000)

       libstdc++.so.6 => /cts/tools/gcc/new/gcc-8.1.0/lib64/libstdc++.so.6 (0x00007f4377dea000)

       libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f4377c9b000)

       libgcc_s.so.1 => /cts/tools/gcc/new/gcc-8.1.0/lib64/libgcc_s.so.1 (0x00007f4377a83000)

       libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4377891000)

       /lib64/ld-linux-x86-64.so.2 (0x00007f437a936000)

       libintlc.so.5 => /cts/tools/oneapi/2021.1-beta08/compiler/latest/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00007f4377617000)

$

Intel libraries are redistribute-able, so you can ship the required Intel libs to run your application with your products.