dpct's result of dot does not run successfully

[yhmtsai]

Hi,

I use dpct to convert the program with cublas dot product.
It can be compiled but it will lead the invalid argument failure.

Is it caused by my compile command?
The compile command: 
dpcpp {SOURCE_FILE} -lmkl_sycl -L${MKLROOT}/lib/in
tel64 -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -lsycl -lOpenCL -lpthread -lm -ldl 

If my compile command is correct, I will assume the following issue.
oneapi::mkl::blas::dot seems to require the result and the input are on the same memory. i.e. all are on device memory.
However, the cublas dot can allow the result on host memory (by default).
Thus, the conversion is failed.
Currently, need to manually fix it by allocating result on device memory.

The error message:

terminate called after throwing an instance of 'cl::sycl::runtime_error'
  what():  Native API failed. Native API returns: -50 (CL_INVALID_ARG_VALUE) -50 (CL_INVALID_ARG_VALUE)
/var/spool/torque/mom_priv/jobs/692649.v-qsvr-1.aidevcloud.SC: line 4: 16107 Aborted                 ./a.out

The expected result:

size 80000000 byte
result = 1e+07

The attachment contains three file,
1. dot.cu (the original cuda file)
2. dpct_result.dp.cpp (the result from dpct)
3. dot.dp.cpp (the correct version)

I am not sure whether oneMKL support the result on device memory or not.
If it does not support the result on device, it would be very helpful if dpct can gives alarm or the error message can point the memory issue.

Thanks,
Mike

[RahulV_intel]

Hi,

I'm unable to run your base code. It throws segmentation fault. Kindly re-check your CUDA source file.

Command:

nvcc dot.cu -lcublas && ./a.out


Output:

size 80000000 byte

Segmentation fault (core dumped)

Thanks,

Rahul

[yhmtsai]

Hi Rahul,

It still works on my system with K20 GPU.
I think the segmentation fault is related to the host memory size which require at least 8 * 1e7 * 2 byte.
Could you try the small num?
change the line 6 `constexpr int num = 1e7;` to smaller one like `constexpr int num = 1e4;` or something else.

Thanks,
Mike

[RahulV_intel]

Hi,

Could you let me know what is your oneAPI basetoolkit version?

I've tried the following steps with the latest beta09 release and the results are correct.

cd dot_report
dpct dot.cu --keep-original-code --extra-arg="-std=c++11"
cd dpct_output
dpcpp dot.dp.cpp -lmkl_sycl -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -o dot_mkl
./dot_mkl

//Result
size 80000000 byte
result = 1e+07

//If you wish to explicitly run this on iGPU:
SYCL_DEVICE_TYPE=GPU ./dot_mkl

Make sure that you source the setvars.sh script file to load the oneAPI environment.

By default, DPCT uses the default_selector, which will automatically select the best iGPU device, if present in your environment. To explicitly select the iGPU device, kindly set the env variable SYCL_DEVICE_TYPE=GPU (given above)

Let me know if you face any issues.

Regards,

Rahul

[yhmtsai]

Hi Rahul,

I use the default one on the devcloud.

dpct --version
Intel(R) DPC++ Compatibility Tool Version: 2021.1-beta09 codebase:(e2c9d9ea79576002cd035fd83a8aa7cb6e8f17ca)

dpcpp --version
Intel(R) oneAPI DPC++ Compiler Pro 2021.1 (2020.8.0.0827)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /glob/development-tools/versions/oneapi/beta09/inteloneapi/compiler/2021.1-beta09/linux/bi

Is this one the latest version?

I still get the same error when following your steps with adding `--cuda-include-path=<10.2-header-folder>`

run.sh

#!/bin/bash
source /opt/intel/inteloneapi/setvars.sh > /dev/null 2>&1
SYCL_DEVICE_TYPE=GPU ./dot_mkl

qsub -l nodes=1:gpu:ppn=2 -d . run.sh
qsub -l nodes=s001-n180:ppn=2 -d . run.sh

about SYCL_DEVICE_TYPE=GPU 
Is it possible to select FPGA or others as the iGPU? and then oneMKL does not support it yet?

Thanks,
Mike

[RahulV_intel]

Hi,

There was some issue with my previous environment (It was picking up CPU as device).

The migrated code runs successfully on CPU:

SYCL_DEVICE_TYPE=CPU ./dot_mkl (the results are correct on CPU)

When I tried this on iGPU, the program fails with the same error (like in your case).

Please note that you don't have to source anything on Devcloud. The oneAPI environment gets sourced automatically. Yes, beta09 is the latest version.

Currently, oneMKL only supports CPU, GPU and host devices. However not all APIs are supported on GPU. Please refer to this oneMKL reference: https://software.intel.com/content/www/us/en/develop/documentation/oneapi-mkl-dpcpp-developer-reference/top.html

I've escalated this issue to the concerned team for a fix. Thanks for reporting this.

Regards,

Rahul