- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, I am a developer of AMReX (https://github.com/AMReX-Codes/amrex). We are currently porting AMReX to use DPC++ as a backend for Intel GPUs. For development, we are running on DevCloud. If I compile the following code with "-O0" and link to AMReX, an exception, "The program was built or 1 devices", is caught at runtime. The code below has no calls to any AMReX functions. But it runs fine if I compile it with "-O3". So why is this happening? What does the exception message mean? I have put the test code with a makefile at https://github.com/WeiqunZhang/dpcpp/tree/master/exception. Thanks! Weiqun #include <CL/sycl.hpp> namespace sycl = cl::sycl; int main (int argc, char* argv[]) { { sycl::gpu_selector my_selector; sycl::device my_device(my_selector); sycl::context my_context(my_device); sycl::ordered_queue q(my_context, my_selector); try { q.submit([&] (sycl::handler& h) { h.parallel_for(sycl::nd_range<1>(sycl::range<1>(8),sycl::range<1>(8)), [=] (sycl::nd_item<1> item) { // }); }); } catch (sycl::exception const& ex) { std::cout << "sycl::exception caught: " << ex.what() << std::endl; } } std::cout << "End of program\n"; }
- Tags:
- General Support
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Weiqun,
We build the AMReX from your specified git location and also tried building the Makefile specified in "https://github.com/WeiqunZhang/dpcpp/tree/master/exception" with your main.cpp code, but despite having USE_DPCPP = TRUE in Makefile its selecting g++ compiler and not dpcpp.
Please make changes in MakeFile and resend it to us.
We also manually tried to compile the main.cpp and linked with AMReX using -O0 and -O3 flag and with working fine, for more details we have specified the commands:
Compiling:
dpcpp -Werror=return-type -g -O0 -pthread -DNDEBUG -DAMREX_GIT_VERSION=\"20.02\" -DBL_GCC_VERSION=7.4.0 -DBL_GCC_MAJOR_VERSION=7 -DBL_GCC_MINOR_VERSION=4 -DAMREX_LAUNCH="" -DAMREX_DEVICE="" -DAMREX_CUDA_FORT_GLOBAL="" -DAMREX_CUDA_FORT_DEVICE="" -DAMREX_CUDA_FORT_HOST="" -DAMREX_CUDA_FORT_HOST_DEVICE="" -DBL_SPACEDIM=3 -DAMREX_SPACEDIM=3 -DBL_FORT_USE_UNDERSCORE -DAMREX_FORT_USE_UNDERSCORE -DBL_Linux -DAMREX_Linux -I. -I../..//Src/Base -c main.cpp -o tmp_build_dir/o/3d.gnu.EXE/main.o
Linking:
dpcpp -Werror=return-type -g -O0 -pthread -DNDEBUG -DAMREX_GIT_VERSION=\"20.02\" -DBL_GCC_VERSION=7.4.0 -DBL_GCC_MAJOR_VERSION=7 -DBL_GCC_MINOR_VERSION=4 -DAMREX_LAUNCH="" -DAMREX_DEVICE="" -DAMREX_CUDA_FORT_GLOBAL="" -DAMREX_CUDA_FORT_DEVICE="" -DAMREX_CUDA_FORT_HOST="" -DAMREX_CUDA_FORT_HOST_DEVICE="" -DBL_SPACEDIM=3 -DAMREX_SPACEDIM=3 -DBL_FORT_USE_UNDERSCORE -DAMREX_FORT_USE_UNDERSCORE -DBL_Linux -DAMREX_Linux -I. -I../..//Src/Base -L. -L/usr/lib/gcc/x86_64-linux-gnu/7/ -o main3d.gnu.ex tmp_build_dir/o/3d.gnu.EXE/main.o tmp_build_dir/o/3d.gnu.EXE/AMReX.o tmp_build_dir/o/3d.gnu.EXE/AMReX_error_fi.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BlockMutex.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParmParse.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parmparse_fi.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Utility.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Random.o tmp_build_dir/o/3d.gnu.EXE/AMReX_DistributionMapping.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelDescriptor.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ForkJoin.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelContext.o tmp_build_dir/o/3d.gnu.EXE/AMReX_VisMF.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Arena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_CArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_DArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_EArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_NFiles.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parstream.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuLaunch.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuControl.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuDevice.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuUtility.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuAsyncArray.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuElixir.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FabConv.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FPC.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IntConv.o tmp_build_dir/o/3d.gnu.EXE/AMReX_VectorIO.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Box.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxIterator.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IntVect.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IndexType.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Orientation.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Periodicity.o tmp_build_dir/o/3d.gnu.EXE/AMReX_RealBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_RealVect.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxList.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxArray.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxDomain.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FArrayBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IArrayBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BaseFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MFCopyDescriptor.o tmp_build_dir/o/3d.gnu.EXE/AMReX_iMultiFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FabArrayBase.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MFIter.o tmp_build_dir/o/3d.gnu.EXE/AMReX_CoordSys.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Geometry.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFabUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFabUtil_Perilla.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BCRec.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PhysBCFunct.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BCUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PlotFileUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PlotFileDataImpl.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLProfiler.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLBackTrace.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MemPool.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Machine.o tmp_build_dir/o/3d.gnu.EXE/AMReX_constants_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FILCC_3D.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLutil_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLProfiler_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_filcc_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_omp_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_acc_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_fort_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_error_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parmparse_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_string_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_bc_types_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_io_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelDescriptor_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_mempool_mod.o -lgfortran -lquadmath
And for both -O0 and -O3 its working fine.
Will you send me modified Makefile so that we can debug it more.
-Abhishek
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Weiqun,
We build the AMReX from your specified git location and also tried building the Makefile specified in "https://github.com/WeiqunZhang/dpcpp/tree/master/exception" with your main.cpp code, but despite having USE_DPCPP = TRUE in Makefile its selecting g++ compiler and not dpcpp.
Please make changes in MakeFile and resend it to us.
We also manually tried to compile the main.cpp and linked with AMReX using -O0 and -O3 flag and with working fine, for more details we have specified the commands:
Compiling:
dpcpp -Werror=return-type -g -O0 -pthread -DNDEBUG -DAMREX_GIT_VERSION=\"20.02\" -DBL_GCC_VERSION=7.4.0 -DBL_GCC_MAJOR_VERSION=7 -DBL_GCC_MINOR_VERSION=4 -DAMREX_LAUNCH="" -DAMREX_DEVICE="" -DAMREX_CUDA_FORT_GLOBAL="" -DAMREX_CUDA_FORT_DEVICE="" -DAMREX_CUDA_FORT_HOST="" -DAMREX_CUDA_FORT_HOST_DEVICE="" -DBL_SPACEDIM=3 -DAMREX_SPACEDIM=3 -DBL_FORT_USE_UNDERSCORE -DAMREX_FORT_USE_UNDERSCORE -DBL_Linux -DAMREX_Linux -I. -I../..//Src/Base -c main.cpp -o tmp_build_dir/o/3d.gnu.EXE/main.o
Linking:
dpcpp -Werror=return-type -g -O0 -pthread -DNDEBUG -DAMREX_GIT_VERSION=\"20.02\" -DBL_GCC_VERSION=7.4.0 -DBL_GCC_MAJOR_VERSION=7 -DBL_GCC_MINOR_VERSION=4 -DAMREX_LAUNCH="" -DAMREX_DEVICE="" -DAMREX_CUDA_FORT_GLOBAL="" -DAMREX_CUDA_FORT_DEVICE="" -DAMREX_CUDA_FORT_HOST="" -DAMREX_CUDA_FORT_HOST_DEVICE="" -DBL_SPACEDIM=3 -DAMREX_SPACEDIM=3 -DBL_FORT_USE_UNDERSCORE -DAMREX_FORT_USE_UNDERSCORE -DBL_Linux -DAMREX_Linux -I. -I../..//Src/Base -L. -L/usr/lib/gcc/x86_64-linux-gnu/7/ -o main3d.gnu.ex tmp_build_dir/o/3d.gnu.EXE/main.o tmp_build_dir/o/3d.gnu.EXE/AMReX.o tmp_build_dir/o/3d.gnu.EXE/AMReX_error_fi.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BlockMutex.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParmParse.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parmparse_fi.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Utility.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Random.o tmp_build_dir/o/3d.gnu.EXE/AMReX_DistributionMapping.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelDescriptor.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ForkJoin.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelContext.o tmp_build_dir/o/3d.gnu.EXE/AMReX_VisMF.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Arena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_CArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_DArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_EArena.o tmp_build_dir/o/3d.gnu.EXE/AMReX_NFiles.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parstream.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuLaunch.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuControl.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuDevice.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuUtility.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuAsyncArray.o tmp_build_dir/o/3d.gnu.EXE/AMReX_GpuElixir.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FabConv.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FPC.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IntConv.o tmp_build_dir/o/3d.gnu.EXE/AMReX_VectorIO.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Box.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxIterator.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IntVect.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IndexType.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Orientation.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Periodicity.o tmp_build_dir/o/3d.gnu.EXE/AMReX_RealBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_RealVect.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxList.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxArray.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BoxDomain.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FArrayBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_IArrayBox.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BaseFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MFCopyDescriptor.o tmp_build_dir/o/3d.gnu.EXE/AMReX_iMultiFab.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FabArrayBase.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MFIter.o tmp_build_dir/o/3d.gnu.EXE/AMReX_CoordSys.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Geometry.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFabUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MultiFabUtil_Perilla.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BCRec.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PhysBCFunct.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BCUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PlotFileUtil.o tmp_build_dir/o/3d.gnu.EXE/AMReX_PlotFileDataImpl.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLProfiler.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLBackTrace.o tmp_build_dir/o/3d.gnu.EXE/AMReX_MemPool.o tmp_build_dir/o/3d.gnu.EXE/AMReX_Machine.o tmp_build_dir/o/3d.gnu.EXE/AMReX_constants_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_FILCC_3D.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLutil_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_BLProfiler_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_filcc_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_omp_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_acc_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_fort_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_error_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_parmparse_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_string_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_bc_types_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_io_mod.o tmp_build_dir/o/3d.gnu.EXE/AMReX_ParallelDescriptor_F.o tmp_build_dir/o/3d.gnu.EXE/AMReX_mempool_mod.o -lgfortran -lquadmath
And for both -O0 and -O3 its working fine.
Will you send me modified Makefile so that we can debug it more.
-Abhishek
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for looking into this! The reason turns out to be `-DNDEBUG`, not `-O0` vs. `-O3`. We added `-DNDEBUG` in our make sytem when compiling for non-debug build. If `-DNDEBUG` is not used, it can also be fixed by linking with `/opt/intel/inteloneapi/compiler/latest/linux/lib/libsycl-glibc.o`. Similar things happen for cmath functions too, and it can be fixed by linking with `libsycl-cmath.o`. I am using oneAPI beta4.
Weiqun
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Glad to know your issue is resolved, we will look at your resolution.
-Abhishek
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you! The thread can be closed now.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your confirmation, you are always welcome to post a new thread if you have any issue.
-Abhishek

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page