OpenMP Segmentation Fault on XE Max GPU

lamb — Tue, 30 Mar 2021 18:14:05 GMT

I am trying to find the cause of a segmentation fault when running several of the SPEC Accel OpenMP benchmarks. Specifically, I have been working with the 552.ep (embarrassingly parallel) benchmark.

OS: x86_64 GNU/Linux
CPU: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
Accelerator: Intel Iris Xe MAX
Toolkit: Intel(R) oneAPI DPC++ Compiler 2021.1.2 (2020.10.0.1214)

Compilation commands:

$ ulimit -s unlimited $ export IGC_EnableDPEmulation=1 $ export OverrideDefaultFP64Settings=1 $ source /opt/intel/oneapi/setvars.sh $ icx -g -Wall -O3 -I. -fiopenmp -fopenmp-targets=spir64 *.c -o ep -lm $ gdb ./ep

Here is the result of running gdb (run). I can install the missing debuginfos and update the question if that would help in finding a solution.

(gdb) run Starting program: /552.pep/src/ep Missing separate debuginfos, use: yum debuginfo-install glibc-2.28-127.el8.x86_64 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". warning: Loadable section ".note.gnu.property" outside of ELF segments warning: Loadable section ".note.gnu.property" outside of ELF segments warning: Loadable section ".note.gnu.property" outside of ELF segments [New Thread 0x155550008700 (LWP 811436)] warning: Loadable section ".note.gnu.property" outside of ELF segments warning: Loadable section ".note.gnu.property" outside of ELF segments warning: Loadable section ".note.gnu.property" outside of ELF segments warning: Loadable section ".note.gnu.property" outside of ELF segments [New Thread 0x15554d9ac700 (LWP 811437)] Reading from input file ep.input NAS Parallel Benchmarks (NPB3.3-OPENMP-C) - EP Benchmark Number of random numbers generated: 67108864 Thread 1 "ep" received signal SIGSEGV, Segmentation fault. 0x000015553f455bca in vISA::G4_SrcRegRegion::computeLeftBound() () from /lib64/libigc.so.1 Missing separate debuginfos, use: yum debuginfo-install intel-gmmlib-20.4.1-i482.el8.x86_64 intel-igc-core-1.0.6410-i509.el8.x86_64 intel-igc-opencl-1.0.6410-i509.el8.x86_64 libedit-3.1-23.20170329cvs.el8.x86_64 libgcc-8.3.1-5.1.el8.x86_64 libstdc++-8.3.1-5.1.el8.x86_64 libxml2-2.9.7-8.el8.x86_64 ncurses-libs-6.1-7.20180224.el8.x86_64 xz-libs-5.2.4-3.el8.x86_64 (gdb) bt #0 0x000015553f455bca in vISA::G4_SrcRegRegion::computeLeftBound() () from /lib64/libigc.so.1 #1 0x000015553f34ef81 in vISA::IR_Builder::createSrcRegRegion(G4_SrcModifier, G4_RegAccess, vISA::G4_VarBase*, short, short, RegionDesc const*, G4_Type, G4_AccRegSel) () from /lib64/libigc.so.1 #2 0x000015553f360739 in VISAKernelImpl::CreateVISASrcOperand(_VISA_VectorOpnd*&, _VISA_GenVar*, VISA_Modifier, unsigned short, unsigned short, unsigned short, unsigned char, unsigned char) () from /lib64/libigc.so.1 #3 0x000015553f256b97 in IGC::CEncoder::GetSourceOperand(IGC::CVariable*, IGC::SModifier const&) () from /lib64/libigc.so.1 #4 0x000015553f25948f in IGC::CEncoder::DataMov(ISA_Opcode, IGC::CVariable*, IGC::CVariable*) () from /lib64/libigc.so.1 #5 0x000015553f297893 in IGC::EmitPass::UniformCopy(IGC::CVariable*, IGC::CVariable*&, IGC::CVariable*, bool) () from /lib64/libigc.so.1 #6 0x000015553f2b9a7b in IGC::EmitPass::emitVectorStore(llvm::StoreInst*, llvm::Value*, llvm::ConstantInt*) () from /lib64/libigc.so.1 #7 0x000015553f2c0a1c in IGC::EmitPass::runOnFunction(llvm::Function&) () from /lib64/libigc.so.1 #8 0x00001555404265ce in llvm::FPPassManager::runOnFunction(llvm::Function&) () from /lib64/libigc.so.1 #9 0x0000155540426e41 in llvm::FPPassManager::runOnModule(llvm::Module&) () from /lib64/libigc.so.1 #10 0x0000155540427298 in llvm::legacy::PassManagerImpl::run(llvm::Module&) () from /lib64/libigc.so.1 #11 0x000015553f0a9ef9 in void IGC::CodeGen<IGC::OpenCLProgramContext>(IGC::OpenCLProgramContext*, llvm::MapVector<llvm::Function*, IGC::CShaderProgram*, llvm::DenseMap<llvm::Function*, unsigned int, llvm::DenseMapInfo<llvm::Function*>, llvm::detail::DenseMapPair<llvm::Function*, unsigned int> >, std::vector<std::pair<llvm::Function*, IGC::CShaderProgram*>, std::allocator<std::pair<llvm::Function*, IGC::CShaderProgram*> > > >&) () from /lib64/libigc.so.1 #12 0x000015553f079635 in IGC::CodeGen(IGC::OpenCLProgramContext*) () from /lib64/libigc.so.1 #13 0x000015553ef40e67 in TC::TranslateBuild(TC::STB_TranslateInputArgs const*, TC::STB_TranslateOutputArgs*, TC::TB_DATA_FORMAT, IGC::CPlatform const&, float) [clone .part.314] () from /lib64/libigc.so.1 #14 0x000015553eff261d in IGC::IgcOclTranslationCtx<0ul>::Impl::Translate(unsigned long, CIF::Builtins::Buffer<1ul>*, CIF::Builtins::Buffer<1ul>*, CIF::Builtins::Buffer<1ul>*, CIF::Builtins::Buffer<1ul>*, CIF::Builtins::Buffer<1ul>*, CIF::Builtins::Buffer<1ul>*, unsigned int, void*) const () from /lib64/libigc.so.1 #15 0x00001555506df59a in NEO::CompilerInterface::build(NEO::Device const&, NEO::TranslationInput const&, NEO::TranslationOutput&) () from /lib64/libze_intel_gpu.so.1 #16 0x0000155550672a8e in L0::ModuleTranslationUnit::buildFromSpirV(char const*, unsigned int, char const*, char const*, _ze_module_constants_t const*) () from /lib64/libze_intel_gpu.so.1 #17 0x00001555506740ac in L0::ModuleImp::initialize(_ze_module_desc_t const*, NEO::Device*) () from /lib64/libze_intel_gpu.so.1 #18 0x0000155550674413 in L0::Module::create(L0::Device*, _ze_module_desc_t const*, L0::ModuleBuildLog*, L0::ModuleType) () from /lib64/libze_intel_gpu.so.1 #19 0x000015555065fa48 in L0::DeviceImp::createModule(_ze_module_desc_t const*, _ze_module_handle_t**, _ze_module_build_log_handle_t**, L0::ModuleType) () from /lib64/libze_intel_gpu.so.1 #20 0x000015555129748e in __tgt_rtl_load_binary () from /opt/intel/oneapi/compiler/2021.1.2/linux/lib/libomptarget.rtl.level0.so --Type <RET> for more, q to quit, c to continue without paging-- #21 0x00001555554ec6ba in DeviceTy::load_binary(void*) () from /opt/intel/oneapi/compiler/2021.1.2/linux/lib/libomptarget.so #22 0x00001555554f8159 in CheckDeviceAndCtors(long) () from /opt/intel/oneapi/compiler/2021.1.2/linux/lib/libomptarget.so #23 0x00001555554eeb63 in __tgt_target_data_begin_mapper () from /opt/intel/oneapi/compiler/2021.1.2/linux/lib/libomptarget.so #24 0x0000000400000000 in ?? () #25 0x0000000000000000 in ?? ()

Also, I'm intending to use the level0 backend, but I see "OpenCLProgramContext" in the gdb output. Does this mean I may be using the OpenCL backend by mistake?

Thank you!

Re: OpenMP Segmentation Fault on XE Max GPU

AbhishekD_Intel — Wed, 05 May 2021 13:19:45 GMT

Hi,

Thanks for reaching out to us.

We are looking into this issue internally, will get back to you with the updates.

Meanwhile please try the same example with the latest oneAPI toolkit (2021.2).

Warm Regards,

Abhishek

Re:OpenMP Segmentation Fault on XE Max GPU

Klaus-Dieter_O_Intel — Thu, 06 May 2021 11:05:50 GMT

The issue was forwarded to our engineering team.

Best regards

Klaus-Dieter

Re:OpenMP Segmentation Fault on XE Max GPU

Klaus-Dieter_O_Intel — Thu, 06 May 2021 13:56:05 GMT

Could you please provide kernel dumps? For the Linux environment the variable IGC_ShaderDumpEnable needs to be set. The dump should be available under /tmp/IntelIGC/<app>_<pid> directory.

Is this issue reproducible also when dumped .spv is compiled with ocloc tool? An example command line is:

ocloc compile -spirv_input -file input.spv -device skl

Re:OpenMP Segmentation Fault on XE Max GPU

Klaus-Dieter_O_Intel — Fri, 21 May 2021 10:44:24 GMT

Any update available on the kernel dump or the ocloc compilation?

topic Re:OpenMP Segmentation Fault on XE Max GPU in Intel® MPI Library

OpenMP Segmentation Fault on XE Max GPU

Re: OpenMP Segmentation Fault on XE Max GPU

Re:OpenMP Segmentation Fault on XE Max GPU

Re:OpenMP Segmentation Fault on XE Max GPU

Re:OpenMP Segmentation Fault on XE Max GPU