DO CONCURRENT offload to Intel MAX GPU on DevCloud not compiling

[caplanr]

Hi,

I am on devcloud testing a Fortran DO CONCCURENT code.

I have sucessfully compiled it and ran it multi-threaded on the Xeon CPU nodes using "-qopenmp".

I am now trying to compile it for use with the MAX GPU.

I am using the following flags:

-O3 -fp-model precise -heap-arrays -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64

I get the following:

INTEL-DEVCLOUD: ~/diffuse_experimental $ make
ifx -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 number_types.f
ifx -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 zm_parse_modules.f
ifx -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 zm_parse.f
ifx -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 zm_sds_modules.f
ifx -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 zm_sds.f
ifx -o diffuse_experimental_gpu.o -c -O3 -fp-model precise -heap-arrays -I/home/u178585/ext_deps/deps/hdf5/include -fiopenmp -fopenmp-target-do-concurrent -fopenmp-targets=spir64 diffuse_experimental.f
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x146924a]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x11501a2]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x11b6082]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x11b61b0]
/lib/x86_64-linux-gnu/libc.so.6(+0x3ef20)[0x7f92d2c7df20]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2a9ecaf]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2a9c8b2]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2a999d1]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x222312d]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1544355]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x221e2ad]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1d0f68f]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2231f2d]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1d0b1ec]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2022e7d]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1d0d78c]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x202322d]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1d0c5d3]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x202303d]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1542f8e]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x201fffe]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x2220fed]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1542f8e]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x10fcf51]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0x1259aec]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f92d2c60b97]
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/compiler/2022.2.1/linux/bin-llvm/xfortcom[0xeeba29]

/home/u178585/tmp/ifxqXVsvi.bc: error #5633: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report. Note: File and line given may not be explicit cause of this error.
compilation aborted for /home/u178585/tmp/ifxqXVsvi.bc (code 3)
Makefile:43: recipe for target 'diffuse_experimental_gpu.o' failed
make: *** [diffuse_experimental_gpu.o] Error 3

Any ideas on what I am doing wrong?

Or is this a compiler bug?

The code is attached.

Note that this code was used in our publication:

https://link.springer.com/chapter/10.1007/978-3-030-97759-7_1

which has all the codes and build scripts included.

 - Ron Caplan

[Barbara_P_Intel]

Thanks for reporting this. I'll investigate. There's an awful lot of module files required to duplicate the ICE (Internal Compiler Error).

[Barbara_P_Intel]

@caplanr Can you please upload just the files needed to reproduce this ICE? Small reproducers are better than the 15GB file you point to (unless I'm looking in the wrong place.) A gzip'd tar file would be awesome.

[caplanr]

Sure!  I will try to get that to you later today.

Do you have access to my DevCloud account/folders?  That might be easiest?

The code is setup in:

/home/u178585/diffuse_experimental/

 - Ron

[Barbara_P_Intel]

Thank you. Can you create the tar ball? I don't have ready access to DevCloud to grab it.

[caplanr]

Hi,

Here is the tarball.

The code relies on the HDF5 library.

I have included the tar file for the version of HDF5 I am using.

It does not seem to compile with IFX and ICX, so I compiled it with IFORT and ICC/ICPC and that worked.

That shoudl not be a probelm as the code compiled and ran fine on the XEON CPU node.

You will need to modify the Makefile to point to wherever the HDF5 is installed to.

 - Ron

[Barbara_P_Intel]

Thank you! I appreciate this. I'll take a closer look.

[Barbara_P_Intel]

Good News! This ICE is resolved in the next release! This version of ifx will be part of oneAPI 2023.0 that is planned for release around the end of the year. DevCloud should have it available within a day or two of the announcement.

[caplanr]

Hi,

Ok cool!

I will give it a try after New Years

Thanks!

 -Ron

[Barbara_P_Intel]

This is the version I used.

$ ifx --version

ifx (IFORT) 2023.0.0 20221113

[Barbara_P_Intel]

I just confirmed the that Internal Compiler Error is gone in the release of ifx that was made available in December. Please try it!

[caplanr]

Hi,

The compiler works and does not seg fault anymore.

Thanks!

 - Ron