Using depends_on with events returned from different queues

[JNorw]

dpc++ enables creating multiple queues on a device.

dpc++ enables dependencies specified with depends_on(ev1,ev2,...)

dpc++ compiler accepts depends_on with events returned by different queues, but apparently depends_on may not find the events.

I'm providing example code derived from the dpc++ book, modified to use different queues.  Using event.wait() works ok, but using depends_on(ev1, ...) statements, which are commented out in the example, fails to execute the associated kernels.

I'm using ubuntu 20.04 and the docker distribution

root@4578405bdff6:/workspaces/data-parallel-CPP-main/build# dpcpp --version
Intel(R) oneAPI DPC++/C++ Compiler 2021.3.0 (2021.3.0.20210619)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/intel/oneapi/compiler/2021.3.0/linux/bin
root@4578405bdff6:/workspaces/data-parallel-CPP-main/build#

root@4578405bdff6:/workspaces/data-parallel-CPP-main/build# sycl-ls
0. ACC : Intel(R) FPGA Emulation Platform for OpenCL(TM) 1.2 [2021.12.6.0.19_160000]

1. CPU : Intel(R) OpenCL 2.1 [2021.12.6.0.19_160000]
2. GPU : Intel(R) OpenCL HD Graphics 3.0 [21.23.20043]
3. GPU : Intel(R) Level-Zero 1.1 [1.1.20043]
4. HOST: SYCL host platform 1.2 [1.2]

I'm using make and the cmake from the dpc++ book examples.  To build, I use:

make fig_3_11_depends_on

with the Makefile generated by cmake that comes with the book examples.

I'm substituting my modified fig_3_11 code for the books sample code from

https://github.com/Apress/data-parallel-CPP 

I used cmake to create the makefile in a build directory with:

CXX=dpcpp cmake -D CMAKE_BUILD_TYPE=Debug -D CMAKE_CXX_FLAGS="-O0" -D NODPL=1 ../

The build and link commands with VERBOSE=1 are

[100%] Building CXX object samples/Ch03_data_management/CMakeFiles/fig_3_11_depends_on.dir/fig_3_11_depends_on.cpp.o
cd /workspaces/data-parallel-CPP-main/build/samples/Ch03_data_management && /opt/intel/oneapi/compiler/2021.3.0/linux/bin/dpcpp -O0 -g -fsycl -fsycl-unnamed-lambda -ferror-limit=1 -Wall -Wpedantic -std=gnu++17 -MD -MT samples/Ch03_data_management/CMakeFiles/fig_3_11_depends_on.dir/fig_3_11_depends_on.cpp.o -MF CMakeFiles/fig_3_11_depends_on.dir/fig_3_11_depends_on.cpp.o.d -o CMakeFiles/fig_3_11_depends_on.dir/fig_3_11_depends_on.cpp.o -c /workspaces/data-parallel-CPP-main/samples/Ch03_data_management/fig_3_11_depends_on.cpp
[100%] Linking CXX executable fig_3_11_depends_on
cd /workspaces/data-parallel-CPP-main/build/samples/Ch03_data_management && /usr/bin/cmake -E cmake_link_script CMakeFiles/fig_3_11_depends_on.dir/link.txt --verbose=1
/opt/intel/oneapi/compiler/2021.3.0/linux/bin/dpcpp -O0 -g CMakeFiles/fig_3_11_depends_on.dir/fig_3_11_depends_on.cpp.o -o fig_3_11_depends_on -lsycl -fsycl

[SantoshY_Intel]

Hi,

Thanks for reaching out to us.

>>" Using event.wait() works ok, but using depends_on(ev1, ...) statements, which are commented out in the example, fails to execute the associated kernels."

Could you provide a screenshot or an error log if it is throwing an error? Because we didn't notice any error in the make log you provided.

We see that you have used multiple queues in your attached code for ordering the tasks. But the DPC++ book provided an example with a single queue.

We modified the "fig_3_11_depends_on.cpp" code in which we used only a single queue to order the tasks using depends_on() and it works fine on Ubuntu 18.04 machine having Intel(R) oneAPI DPC++/C++ Compiler 2021.3.0. Please find the sample code attached below.

--

Best Regards,

Santosh

[JNorw]

If you uncomment the depends_on(event,...) statements, the printed results should be the same, but instead there are zeros in the printed results, indicating the kernels are not executed.  The kernels are updating an all zeros array with non-zero values.

for example, with depends_on_commented out, output is non_zero

 

while, with the the single depends_on in Q2.submit uncommented, there are zeros in the output, indicating that that kernel did not execute.

 

The example in the book used a single queue, yes.  It did not give an example with multiple queues, but I see no documentation that events returned from a different queue are not able to be used in depends_on.    If they cannot be used, it needs to be documented and/or the compiler should assert if you attempt to use an event returned from an invalid queue.

[SantoshY_Intel]

Hi,

We are working on your issue internally and we will get back to you soon.

Thanks & Regards,

Santosh

[JNorw]

ok, thanks.

I've modified the original problem code to check for each result value, mark the failed array values in the output and return 1 from main for failures.  I've uncommented a depends_on line that creates the failure in this modification.

[JNorw]

I've modified fig_7_8_accessors_simple example from the book to use multiple queues, handle.require() and accessors without any problems from multiple queues. 

In general, use of buffers, accessors and handle.require() seem to be a work-around alternative if the multi-queue depends_on(event) issue isn't resolved.

[Alex_Y_Intel]

Your question has been escalated to the developers and will be taken care of internally. We will respond when we have update.

[Alex_Y_Intel]

The issue does not present in oneAPI Base Toolkit version 2022.1.0 with DPC++ compiler 2022.0.0. Please use this version to work on your programs.