- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
https://gcc.gnu.org/wiki/Offloading
"Patches enabling OpenMP 4.0 offloading to Intel MIC are merged to trunk. They include general infrastructure changes, mkoffload tool, libgomp plugin, Intel MIC runtime offload library liboffloadmic and an emulator. This emulator lies under liboffloadmic and reproduces MIC's HW and SW stack behavior allowing to run offloaded code in a separate address space using the host machine. The emulator consists of 4 shared libraries which replace COI and MYO libraries from Intel Manycore Platform Software Stack (MPSS). In case of real offloading, user is supposed to specify path to MPSS libraries in LD_LIBRARY_PATH, this will overload emulator libraries on runtime."
I am trying to use the new GCC 6.2 release to offload OpenMP blocks to the Intel MIC (i.e. the Xeon Phi) KNC.
The GCC wiki says it supports emulation, but I want to run the OpenMP program on a real MIC (KNC).
As the wiki says, in case of real offloading, user is supposed to specify path to MPSS libraries in LD_LIBRARY_PATH, this will overload emulator libraries on runtime. I tried it, however, the execution failed at CheckProcessBinary() in line 879 of mpss-coi-3.6/src/mechanism/process/process_source.cpp (MPSS-3.6) when creating a process on MIC (_COIProcess::_COIProcess()).
So what's wrong with it? Is it possible to offload OpenMP program to a real Knights Corner Xeon Phi card using GCC? If yes, how? Thank you.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please look at the top of that page:
GCC 5 and later support two offloading configurations:
* OpenMP to Intel MIC targets (upcoming Intel Xeon Phi products codenamed KNL) as well as MIC emulation on host.
[...]
So, offloading to KNC is not supported.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you so much for your reply! I understand KNC is not supported, but what is exactly missing to support KNC except modifying the cross (offload) compiler? i.e. If I want to port GCC to support KNC, what else changes should I apply to liboffloadmic, libgomp and so on? Thank you!
Ilya V. wrote:
Please look at the top of that page:
GCC 5 and later support two offloading configurations:
* OpenMP to Intel MIC targets (upcoming Intel Xeon Phi products codenamed KNL) as well as MIC emulation on host.
[...]
So, offloading to KNC is not supported.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Xuhao C. wrote:
Thank you so much for your reply! I understand KNC is not supported, but what is exactly missing to support KNC except modifying the cross (offload) compiler? i.e. If I want to port GCC to support KNC, what else changes should I apply to liboffloadmic, libgomp and so on? Thank you!
GCC lacks support of KNC code-generation. Of course it can emit all basic x86 insns, however it doesn't support the main thing - KNC vector insns.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ilya V. wrote:
Quote:
Xuhao C. wrote:Thank you so much for your reply! I understand KNC is not supported, but what is exactly missing to support KNC except modifying the cross (offload) compiler? i.e. If I want to port GCC to support KNC, what else changes should I apply to liboffloadmic, libgomp and so on? Thank you!
GCC lacks support of KNC code-generation. Of course it can emit all basic x86 insns, however it doesn't support the main thing - KNC vector insns.
Thank you. So if my OpenMP program doesn't contain any vector instructions, theoretically, it can run successfully on KNC card with current GCC offload environment?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Xuhao C. wrote:
Thank you. So if my OpenMP program doesn't contain any vector instructions, theoretically, it can run successfully on KNC card with current GCC offload environment?
Theoretically yes. If you really want to try it, you should get this patch from MPSS GCC and apply it to GCC 6.2: https://github.com/apc-llc/gcc-5.1.1-knc/commit/cee3ea59cd33411db673dc639ee179bcf2d3fcef
But please note that offloading to KNC was never tested, and do not expect any performance gains, because of absent vector instructions.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ilya V. wrote:
Quote:
Xuhao C. wrote:Thank you. So if my OpenMP program doesn't contain any vector instructions, theoretically, it can run successfully on KNC card with current GCC offload environment?
Theoretically yes. If you really want to try it, you should get this patch from MPSS GCC and apply it to GCC 6.2: https://github.com/apc-llc/gcc-5.1.1-knc/commit/cee3ea59cd33411db673dc63...
But please note that offloading to KNC was never tested, and do not expect any performance gains, because of absent vector instructions.
Thank you so much. I tried to patch it to GCC-6.2, but failed with errors. However, I successfully patched it to GCC-5.1.0. Then when I build this patched GCC-5.1.0, how should I configure it?
Originally, for building accel compiler:
../configure --build=x86_64-intelmicemul-linux-gnu --host=x86_64-intelmicemul-linux-gnu --target=x86_64-intelmicemul-linux-gnu --enable-as-accelerator-for=x86_64-pc-linux-gnu
For building host compiler:
../configure --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-offload-targets=x86_64-intelmicemul-linux-gnu=/install/prefix
I tried to replace "x86_64-intelmicemul-linux-gnu" with "k1om-mpss-linux", and it works for building accel compiler, but compiling host compiler failed because it didn't recognize "k1om-mpss-linux".
Should I replace "intelmicemul-linux" with "intelmic-linux"?
Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Xuhao C. wrote:
Should I replace "intelmicemul-linux" with "intelmic-linux"?
No, you should make GCC understand "k1om-mpss-linux" as offload target by adding it to configure.ac (search for enable_offload_targets) and running autoconf.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ilya V. wrote:
Quote:
Xuhao C. wrote:Should I replace "intelmicemul-linux" with "intelmic-linux"?
No, you should make GCC understand "k1om-mpss-linux" as offload target by adding it to configure.ac (search for enable_offload_targets) and running autoconf.
Thank you. I added k1om-* in following files:
configure.ac
gcc/config.gcc
libgomp/plugin/configfrag.ac
liboffloadmic/plugin/configure.ac
liboffloadmic/configure.tgt
Now I can build this patched GCC-5.1.0 successfully:
$ ../gcc-5.1.0/configure --prefix=/home/cxh/work/gcc-install/gcc-5.1.0 --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=k1om-mpss-linux --enable-as-accelerator-for=x86_64-pc-linux-gnu --enable-languages=c,c++ --enable-threads=posix --disable-multilib --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1 --disable-libitm --with-sysroot=/opt/mpss/3.6/sysroots/k1om-mpss-linux
$ make -j16
$ make -j16 install
$ ../gcc-5.1.0/configure --prefix=/home/cxh/work/gcc-install/gcc-5.1.0 --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-offload-targets=k1om-mpss-linux=/home/cxh/work/gcc-install/gcc-5.1.0 --enable-languages=c,c++ --enable-threads=posix --disable-multilib --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1
$ make -j16
$ make -j16 install
However, when I tried to compile a simple OpenMP program using this GCC, I got this error:
$ export PATH=/home/cxh/work/gcc-install/gcc-5.1.0/bin:/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
$ export LD_LIBRARY_PATH=/home/cxh/work/gcc-install/gcc-5.1.0/lib64:/usr/local/mpc-0.8.1/lib:/usr/local/mpfr-2.4.2/lib:/usr/local/gmp-4.3.2/lib:/usr/local/cuda/lib64:/usr/local/lib
$ gcc vec_mult.c -o vec_mult -fopenmp
x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc: error: libgomp.spec: No such file or directory
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
My OS is RedHat 7 with Linux kernel 3.10.0-123.el7.x86_64.
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0/lto-wrapper
OFFLOAD_TARGET_NAMES=k1om-mpss-linux
Target: x86_64-pc-linux-gnu
Configured with: ../gcc-5.1.0/configure --prefix=/home/cxh/work/gcc-install/gcc-5.1.0 --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-offload-targets=k1om-mpss-linux=/home/cxh/work/gcc-install/gcc-5.1.0 --enable-languages=c,c++ --enable-threads=posix --disable-multilib --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1
Thread model: posix
gcc version 5.1.0 (GCC)
Why libgomp.spec exists, but can not be found?
$ find ../ -name libgomp.spec
../gcc-5.1.0/k1om-mpss-linux/lib64/libgomp.spec
../gcc-5.1.0/lib64/libgomp.spec
Thank you!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If libgomp wasn't built, on account of being "unsupported," you might try adding to configure --enable-libgomp. I would suspect then some feature of libgomp will be missing.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Tim P. wrote:
If libgomp wasn't built, on account of being "unsupported," you might try adding to configure --enable-libgomp. I would suspect then some feature of libgomp will be missing.
libgomp is already built, and the subdirectory lib64/ contains libgomp.so and other libraries.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Xuhao C. wrote:
$ gcc vec_mult.c -o vec_mult -fopenmp
x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc: error: libgomp.spec: No such file or directory
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Pretty good progress! This message means that host gcc is able to find and run k1om accel gcc. But accel gcc can't find its own libgomp (I don't know why). Maybe something like this will help:
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-L../gcc-5.1.0/k1om-mpss-linux/lib64/"
or
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-B../gcc-5.1.0/k1om-mpss-linux/lib64/"
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ilya V. wrote:
Quote:
Xuhao C. wrote:
$ gcc vec_mult.c -o vec_mult -fopenmp
x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc: error: libgomp.spec: No such file or directory
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
Pretty good progress! This message means that host gcc is able to find and run k1om accel gcc. But accel gcc can't find its own libgomp (I don't know why). Maybe something like this will help:
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-L../gcc-5.1.0/k1om-mpss-linux/lib64/"or
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-B../gcc-5.1.0/k1om-mpss-linux/lib64/"
Finally, it works. Putting it all together:
Building GCC for offloading KNC card
Gcc source code(patched): /home/cxh/work/gcc-5.1.0/
Gcc install path: /home/cxh/work/gcc-install/gcc-5.1.0/
OpenMP test program: /home/cxh/work/vec_mult.c (vector multiplication)
Environment setup: see Q5
MPSS version installed: 3.6
Accel compiler
../gcc-5.1.0/configure --prefix=/home/cxh/work/gcc-install/gcc-5.1.0 --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=k1om-mpss-linux --enable-as-accelerator-for=x86_64-pc-linux-gnu --enable-languages=c,c++ --enable-threads=posix --disable-multilib --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1 --disable-libitm --with-sysroot=/opt/mpss/3.6/sysroots/k1om-mpss-linux --with-ld=/opt/mpss/3.6/sysroots/x86_64-mpsssdk-linux/usr/bin/k1om-mpss-linux/k1om-mpss-linux-ld --with-as=/opt/mpss/3.6/sysroots/x86_64-mpsssdk-linux/usr/bin/k1om-mpss-linux/k1om-mpss-linux-as
Host compiler
../gcc-5.1.0/configure --prefix=/home/cxh/work/gcc-install/gcc-5.1.0 --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --target=x86_64-pc-linux-gnu --enable-offload-targets=k1om-mpss-linux=/home/cxh/work/gcc-install/gcc-5.1.0 --enable-languages=c,c++ --enable-threads=posix --disable-multilib --with-gmp=/usr/local/gmp-4.3.2 --with-mpfr=/usr/local/mpfr-2.4.2 --with-mpc=/usr/local/mpc-0.8.1
Questions and Answers
Q1: k1om-mpss-linux unsupported
A1: Get the patch from https://github.com/apc-llc/gcc-5.1.1-knc/commit/cee3ea59cd33411db673dc639ee179bcf2d3fcef
Add k1om-* in following files and run autoconf
configure.ac
gcc/config.gcc
libgomp/plugin/configfrag.ac
liboffloadmic/plugin/configure.ac
liboffloadmic/configure.tgt
Q2:
$ gcc vec_mult.c -o vec_mult -fopenmp
x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc: error: libgomp.spec: No such file or directory
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
export PATH=/home/cxh/work/gcc-install/gcc-5.1.0/bin:/usr/local/cuda/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
export LD_LIBRARY_PATH=/home/cxh/work/gcc-install/gcc-5.1.0/lib64:/usr/local/mpc-0.8.1/lib:/usr/local/mpfr-2.4.2/lib:/usr/local/gmp-4.3.2/lib:/usr/local/cuda/lib64:/usr/local/lib
A2:
Add -foffload="-B/home/cxh/work/gcc-install/gcc-5.1.0/k1om-mpss-linux/lib64/" when compiling the OpenMP program
Q3:
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-B/home/cxh/work/gcc-install/gcc-5.1.0/k1om-mpss-linux/lib64/"
collect2: fatal error: cannot find 'ld'
compilation terminated.
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
A3: add --with-ld=/opt/mpss/3.6/sysroots/x86_64-mpsssdk-linux/usr/bin/k1om-mpss-linux/k1om-mpss-linux-ld when building GCC
Q4:
$ gcc vec_mult.c -o vec_mult -fopenmp -foffload="-B/home/cxh/work/gcc-install/gcc-5.1.0/k1om-mpss-linux/lib64/"
/opt/mpss/3.6/sysroots/x86_64-mpsssdk-linux/usr/bin/k1om-mpss-linux/k1om-mpss-linux-ld: cannot find crtbeginS.o: No such file or directory
/opt/mpss/3.6/sysroots/x86_64-mpsssdk-linux/usr/bin/k1om-mpss-linux/k1om-mpss-linux-ld: cannot find -lgcc
collect2: error: ld returned 1 exit status
mkoffload-intelmic: fatal error: x86_64-pc-linux-gnu-accel-k1om-mpss-linux-gcc returned 1 exit status
compilation terminated.
lto-wrapper: fatal error: /home/cxh/work/gcc-install/gcc-5.1.0/libexec/gcc/x86_64-pc-linux-gnu/5.1.0//accel/k1om-mpss-linux/mkoffload returned 1 exit status
compilation terminated.
/usr/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
A4:
$ find ./ -name "crtbeginS.o"
./gcc-install/gcc-5.1.0/lib/gcc/k1om-mpss-linux/5.1.0/accel/k1om-mpss-linux/crtbeginS.o
./gcc-install/gcc-5.1.0/lib/gcc/x86_64-pc-linux-gnu/5.1.0/crtbeginS.o
$cp gcc-install/gcc-5.1.0/lib/gcc/k1om-mpss-linux/5.1.0/accel/k1om-mpss-linux/* gcc-install/gcc-5.1.0/lib/gcc/x86_64-pc-linux-gnu/5.1.0/accel/k1om-mpss-linux/
Q5:
$ ./vec_mult
offload error: cannot start process on the device 0 (error code 9)
A5:
This is because device libraries are not found. Note that some libraries (e.g. libgomp) have the same name for both host and device. Make sure the correct libraries are found for the device (The order in LD_LIBRARY_PATH matters). The correct environment setup:
export LD_LIBRARY_PATH=/opt/mpss/3.6/sysroots/k1om-mpss-linux/usr/lib64:/opt/mpss/3.6/sysroots/k1om-mpss-linux/lib64:/home/cxh/mpss-3.6/src/mpss-coi-3.6/install-host/usr/local/lib64:/home/cxh/work/gcc-install/gcc-5.1.0/k1om-mpss-linux/lib64:/home/cxh/work/gcc-install/gcc-5.1.0/lib64:/usr/local/mpc-0.8.1/lib:/usr/local/mpfr-2.4.2/lib:/usr/local/gmp-4.3.2/lib:/usr/local/cuda/lib64:/usr/local/lib
Maybe we can set SINK_LD_LIBRARY_PATH to make it more clear? but for now, the above setup works at least.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page