Mini-app with random123 library : Runtime Error

[Christos_Kotsalos]

Hello,

We are using Random123 library to generate random numbers for our solvers. In this self-contained mini-app (see attached folder) that I am trying in the devcloud, while I am able to compile it, there is a runtime error. For your information I have compiled and executed this mini-app with NVIDIA compilers and I had no runtime issues (after applying a small workaround).

In the attached mini-app, there is everything that you need in order to compile and run it.

A description of the mini-app:

This mini-app shows a small example of the use of OpenMP offloading with random123 in the context of the CoreNEURON application. We have removed most of the unnecessary code and simplified the handling of the random number generator, but kept the essential bits.

In order to build the example codes here, you need to first clone random123 inside this directory:

• git clone https://github.com/DEShawResearch/random123.git

A rough breakdown of the code:

- The main driver code is in `main.cpp`

- `mini.hpp` contains struct and function declarations used in the driver and the test codes.

- `mini.cpp` this is the test code that currently fails with nvhpc. The issue is described in detail also here: https://forums.developer.nvidia.com/t/implicit-omp-declare-target-for-functions-outside-namespace-not-accepted-by-nvc/198582

- `mini_workaround.cpp` this code implements the workaround we found that requires adding a helper function, which compiles and runs correctly.

Any feedback on why we cannot make it run with Intel-oneAPI would be really appreciated.

Thank you very much in advance.

Best,
Christos

[VarshaS_Intel · Accepted solution]

Hi Christos,

Thanks for posting in Intel Communities.

Please try using the LLVM based compilers(icx,icpx,dpcpp) as only these compilers support the OpenMP Offload feature.

We have tried your code with icpx and dpcpp compiler for both the codes(mini_workaround.cpp and mini.cpp), we are able to compile and run the code without any errors.

For more details, please find the below screenshots:

Thanks & Regards,

Varsha

[Christos_Kotsalos]

Hi Varsha,

Thank you very much for your response.

I have fixed my Makefile to use the compilers that you told me but I still have the same issue.

The steps that I follow (connected to the devcloud) are the following:

1. In the nrn123 folder -> qsub -I -l nodes=1:gpu:ppn=2 -d .

qsub: waiting for job 1989259.v-qsvr-1.aidevcloud to start
qsub: job 1989259.v-qsvr-1.aidevcloud ready########################################################################
# Date: Mon 19 Sep 2022 11:33:33 PM PDT
# Job ID: 1989259.v-qsvr-1.aidevcloud
# User: u149321
# Resources: neednodes=1:gpu:ppn=2,nodes=1:gpu:ppn=2,walltime=06:00:00
########################################################################

2. export OMP_TARGET_OFFLOAD=MANDATORY

3. make mini_intel_icpx ->

icpx -O2 -g -qopenmp -fopenmp-targets=spir64 -Irandom123/include -DR123_USE_INTRIN_H=0 mini.cpp main.cpp -c
icpx -O2 -g -qopenmp -fopenmp-targets=spir64 -Irandom123/include -DR123_USE_INTRIN_H=0 mini.o main.o -o out

4. ./out ->

Libomptarget error: Unable to generate entries table for device id 0.
Libomptarget error: Failed to init globals on device 0
Libomptarget error: Run with
Libomptarget error: LIBOMPTARGET_DEBUG=1 to display basic debug information.
Libomptarget error: LIBOMPTARGET_DEBUG=2 to display calls to the compute runtime.
Libomptarget error: LIBOMPTARGET_INFO=4 to dump host-target pointer mappings.
unknown:16:16: Libomptarget fatal error 1: failure of target construct while offloading is mandatory
Aborted

5. LIBOMPTARGET_DEBUG=1 ./out -> see attached error file

I have re-attached the code with the updated Makefile.

Am I missing something?

Thank you very much for your time!

[VarshaS_Intel]

Hi,

Thanks for providing the information.

At our end, we are able to run successfully without any errors. Please find the below screenshot:

Could you please let us know the node details you are using in Intel DevCloud? And also, please provide the output for "sycl-ls".

Thanks & Regards,

Varsha

[Christos_Kotsalos]

Hi Varsha,

Actually I just tried the steps that I posted in my previous comment and I could compile & run my mini-app without any issue. It is weird because yesterday following the exact same steps in the dev cloud I was getting the error that I posted previously.

FYI, I typed sycl-ls and I got the following:

[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2022.13.3.0.16_160000]
[opencl:cpu:1] Intel(R) OpenCL, Intel(R) Xeon(R) E-2176G CPU @ 3.70GHz 3.0 [2022.13.3.0.16_160000]
[opencl:gpu:2] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics P630 [0x3e96] 3.0 [22.10.22597]
[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) UHD Graphics P630 [0x3e96] 1.3 [1.3.22597]
[host:host:0] SYCL host platform, SYCL host device 1.2 [1.2]

Therefore, your earlier comment about the right compilers resolved my issue.

Thank you very much for helping me figuring it out.

Best,

Christos

[VarshaS_Intel]

Hi,

>>Therefore, your earlier comment about the right compilers resolved my issue.

Glad to know that your issue is resolved.

Thanks for accepting our solution. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks & Regards,

Varsha

[Jason21]

Hello Christos,

Thank you for providing the details and the mini-app. Based on the error message you provided in the attached file, it seems like there is an issue with the offload code generated by Intel-oneAPI compiler for the mini.cpp file. Specifically, the error message mentions that "implicit omp declare target for functions outside namespace not accepted by nvc".

This error message is related to the way Intel-oneAPI compiler handles offload code with OpenMP directives. One potential workaround is to modify the mini.cpp file by adding a helper function as you did for the NVIDIA compiler. Another potential workaround is to use a different compiler option for offload code generation. You can try adding the -qopenmp-offload=nvptx option to the CXXFLAGS variable in the Makefile to see if it resolves the issue.

I hope this helps. Let me know if you have any further questions or concerns.