Embree cmake sycl not working

peizhao · ‎06-18-2023

Hi,

I try to build a cycle-based Embree application with Cmake as attached below:

cmake_minimum_required(VERSION 3.22.1)

set(CMAKE_CXX_COMPILER icpx)

set(CMAKE_C_COMPILER icx)

set(EMBREE_SYCL_SUPPORT ON)

project(HELLOEMBREE)

set(embree_DIR /mnt/d/embree-4.1.0.x86_64.linux/lib/cmake/embree-4.1.0/)

set(TBB_DIR /mnt/d/oneapi-tbb-2021.2.0/lib/cmake/tbb/)

FIND_PACKAGE(IntelSYCL REQUIRED)

FIND_PACKAGE(embree 4 REQUIRED)

add_executable(HELLOEMBREE embreeTest.cpp)

TARGET_LINK_LIBRARIES(HELLOEMBREE embree)

add_sycl_to_target(

TARGET HELLOEMBREE

SOURCES embreeTest.cpp

)

While the code I was running was one of the examples codes online

embreeTest.cpp

After Cmake generates the make file in the build directory, I try to build the code using make command the following error occurs:

/mnt/d/Embree_helloWorld/embreeTest.cpp:108:22: error: use of undeclared identifier 'rtcNewSYCLDevice'
RTCDevice device = rtcNewSYCLDevice(sycl_context, "");
^
/mnt/d/Embree_helloWorld/embreeTest.cpp:109:3: error: use of undeclared identifier 'rtcSetDeviceSYCLDevice'
rtcSetDeviceSYCLDevice(device,sycl_device);
^
/mnt/d/Embree_helloWorld/embreeTest.cpp:303:32: error: use of undeclared identifier 'rtcSYCLDeviceSelector'
sycl_device = sycl::device(rtcSYCLDeviceSelector);
^
/mnt/d/Embree_helloWorld/embreeTest.cpp:306:5: error: use of undeclared identifier 'embree'
embree::printAllSYCLDevices();
^
4 errors generated.
gmake[2]: *** [CMakeFiles/HELLOEMBREE.dir/build.make:76: CMakeFiles/HELLOEMBREE.dir/embreeTest.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/HELLOEMBREE.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

here is platform information:

System: window11 wsl2

cmake version: 3.22.1

dpcpp version: Intel(R) oneAPI DPC++/C++ Compiler 2023.1.0 (2023.1.0.20230320)

Thanks in advance cheers

FlorianR_Intel · ‎06-26-2023

Hi and thanks for reaching out,

I've verified that Embree should work with SYCL and WSL(2). When trying to reproduce your problem I've run into a different issue though. The TBB setup was not complete in your CMakeLists.txt. Here is the fixed CMakeLists.txt that worked for me:

cmake_minimum_required(VERSION 3.22.1)

set(CMAKE_CXX_COMPILER icpx)
set(CMAKE_C_COMPILER icx)
set(EMBREE_SYCL_SUPPORT ON)

project(HELLOEMBREE)

set(embree_DIR path-to-embree/lib/cmake/embree-4.1.0/)
set(TBB_DIR path-to-tbb/lib/cmake/tbb/)

FIND_PACKAGE(IntelSYCL REQUIRED)
FIND_PACKAGE(embree 4 REQUIRED)
FIND_PACKAGE(TBB REQUIRED)

add_executable(HELLOEMBREE embreeTest.cpp)
TARGET_LINK_LIBRARIES(HELLOEMBREE embree)
TARGET_LINK_LIBRARIES(HELLOEMBREE TBB::tbb)

add_sycl_to_target(
  TARGET HELLOEMBREE
  SOURCES embreeTest.cpp
)

Make sure to source the oneAPI DPC++ compiler environment before you run cmake, i.e

source /opt/intel/oneapi/compiler/latest/env/vars.sh

Then, you should be able to run cmake successfully. The output I get on my system is

freibold@freibold-intel:~/test$ cmake -B build
-- The C compiler identification is IntelLLVM 2023.1.0
-- The CXX compiler identification is IntelLLVM 2023.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/intel/oneapi/compiler/2023.1.0/linux/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/intel/oneapi/compiler/2023.1.0/linux/bin/icpx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found IntelSYCL: /opt/intel/oneapi/compiler/2023.1.0/linux/include (found version "202001")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/freibold/test/build

After that I could compile

freibold@freibold-intel:~/test$ cmake --build build
[ 50%] Building CXX object CMakeFiles/HELLOEMBREE.dir/embreeTest.cpp.o
[100%] Linking CXX executable HELLOEMBREE
[100%] Built target HELLOEMBREE

and run

freibold@freibold-intel:~/test$ build/HELLOEMBREE
0.330000, 0.330000, -1.000000: Found intersection on geometry 0, primitive 0 at tfar=1.000000
1.000000, 1.000000, -1.000000: Did not find any intersection.

everything fine inside WSL.

Please check if there are differences in the CMake log on your side. Also, make sure to fully remove the build folder and try again. The compilation error you reported (and I can't reproduce) could be due to a broken CMake configuration where you ran CMake with EMBREE_SYCL_SUPPORT=OFF first and then added the SET(EMBREE_SYCL_SUPPORT ON) line to the CMakeLists file. Maybe this confused CMake.

Also, be sure to install the correct drivers inside the Ubuntu subsystem as described here: https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html. You can (and probably should) skip step 2 and 3 if you are using Ubuntu via WSL and not natively. To check if the GPU is available within WSL install clinfo

sudo apt install clinfo

and run it. You should be able to find a "Intel(R) Graphics" device in the output. For example, on my system I get

freibold@freibold-intel:~/test$ clinfo | grep "Intel(R) Graphics"
  Device Name                                     Intel(R) Graphics [0x56a0]
    Device Name                                   Intel(R) Graphics [0x56a0]
    Device Name                                   Intel(R) Graphics [0x56a0]
    Device Name                                   Intel(R) Graphics [0x56a0]

The device number might be different for you, 0x56a0 is the code for an A770.

Hope this helps, if not, please report back the CMake output from your system.

Best,

Embree Team

peizhao · ‎06-26-2023

Hi Freibold,

Thank you very much for getting back to me.

I have modified my Cmake as below according to your instructions

However, the problem preserve.

``` cmake
cmake_minimum_required(VERSION 3.22.1)
set(CMAKE_CXX_COMPILER icpx)
set(CMAKE_C_COMPILER icx)
set(EMBREE_SYCL_SUPPORT ON)

project(HELLOEMBREE)
set(embree_DIR /lib/embree-4.1.0.x86_64.linux/lib/cmake/embree-4.1.0/)
set(TBB_DIR /lib/oneapi-tbb-2021.2.0/lib/cmake/tbb/)
FIND_PACKAGE(IntelSYCL REQUIRED)
FIND_PACKAGE(embree 4 REQUIRED)
FIND_PACKAGE(TBB REQUIRED)

add_executable(HELLOEMBREE embreeSyclTest.cpp)
TARGET_LINK_LIBRARIES(HELLOEMBREE embree)
TARGET_LINK_LIBRARIES(HELLOEMBREE TBB::tbb)

add_sycl_to_target(
TARGET HELLOEMBREE
SOURCES embreeSyclTest.cpp)
```

Then I checked my hardware driver by running

```console
peizhao@ENG-8502:~/embree_test/syclEmbree$ lspci -v |grep -A8 VGA
```

Which gives me the following error

```console
lspci: Unable to load libkmod resources: error -2
```

also, the clinfo for intel graphics doesn't give me anything

Is that mean I don't have the proper graph card to run sycl and therefore the system automatically set EMBREE_SYCL_SUPPORT to OFF.

So I can't run the sycl embree code on a non-intel machine even just in CPU (I can get sycl code running on the current environment just not GPU)?

Best regards,

FlorianR_Intel · ‎06-27-2023

Hi,

generally, you should be able to compile the SYCL applications also when you do not have a Intel GPU in your system. For Embree, you only need some version of the Intel DPC++ compiler. However, calling (during runtime) the SYCL related API of Embree (e.g rtcNewSYCLDevice) will only succeed when a Intel GPU is properly installed on the system.

I assumed you have an Intel GPU and want to run Embree in WSL with GPU support. Is that correct? Which version of Embree did you download/install exactly? We provide SYCL and non-SYCL versions of Embree here https://github.com/embree/embree/releases

and the non-SYCL version does not have the rtcNewSYCLDevice API call. Maybe you downloaded the wrong package?

If you still get the same ' error: use of undeclared identifier 'rtcNewSYCLDevice' error, can you please attach a complete log of your cmake configure and build steps?

When you can successfully compile Embree but still have issues running your application I suggest to try a Embree tutorial first. For example, embree_triangle_geometry and embree_triangle_geometry_ispc should always work, but embree_triangle_geometry_sycl only works, if a Intel GPU is available on the system. You can also check the output of embree_info which prints all Embree compatible and incompatible SYCL devices found on the system. On my WSL system with a A770 the output is:

freibold@freibold-intel:~/test/embree-4.1.0.sycl/bin$ source /opt/intel/oneapi/compiler/latest/env/vars.sh
freibold@freibold-intel:~/test/embree-4.1.0.sycl/bin$ ./embree_info

==================================================================================
  The SYCL support of Embree is in beta phase. Current functionality, quality,
  and GPU performance may not reflect that of the final product. Please read the
  documentation section "Embree SYCL Known Issues" for known limitations.
==================================================================================


Embree Ray Tracing Kernels 4.1.0-beta (0fcb306c9176221219dd15e27fe0527ed334948f)
  Compiler  : CLANG 16.0.0 (icx 2023.1.0.20230320)
  Build     : Release
  Platform  : Linux (64bit)
  CPU       : Unknown CPU (GenuineIntel)
   Threads  : 32
   ISA      : XMM YMM SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 POPCNT AVX F16C RDRAND AVX2 FMA3 LZCNT BMI1 BMI2
   Targets  : SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 AVX AVXI AVX2
   MXCSR    : FTZ=1, DAZ=1
  Config
    Threads : default
    ISA     : XMM YMM SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 POPCNT AVX F16C RDRAND AVX2 FMA3 LZCNT BMI1 BMI2
    Targets : SSE SSE2 SSE3 SSSE3 SSE4.1 SSE4.2 AVX AVXI AVX2  (supported)
              SSE2 SSE4.2 AVX AVX2 AVX512  (compile time enabled)
    Features: raymasks intersection_filter
    Tasking : TBB2021.9 TBB_header_interface_12090 TBB_lib_interface_12090

  Platform              : Intel(R) Level-Zero
    Device              : Intel(R) Graphics [0x56a0]
    Max Work Group Size : 1024
    Max Compute Units   : 512

Embree compatible SYCL device
 SYCL Device:
  Name: Intel(R) Graphics [0x56a0]
  Platform: Intel(R) Level-Zero
  Type: GPU
  Max Work Group Size : 1024
  Max Compute Units   : 512

Embree incompatible SYCL device
 SYCL Device:
  Name: Intel(R) Graphics [0x56a0]
  Platform: Intel(R) OpenCL Graphics
  Type: GPU
  Max Work Group Size : 1024
  Max Compute Units   : 512

From my experience, lspci won't show you a discrete Intel graphics card in WSL. The clinfo / embree_info tool are more helpful in this case.

If nothing helps: Have you tried to build/run Embree 4 on Windows natively without WSL? Did that work?

Best,

Embree Team

peizhao · ‎06-27-2023

Hi Freibold,

Thank you for the replay.

You are right that I installed the wrong Embree version without Sycl support .

The code is successfully compiled, However, the following error occurs during execution, even after installing the GPU driver:

No Embree compatible SYCL GPU device found.

Here is the information for my Intel GPU:

Just to confirm, this GPU is not sycl-Embree capable right?

Furthermore, I am currently working on a heterogenous computing project, so it would be interesting to know if Embree will support other vendors' GPUs in the future.

Best regard,

FlorianR_Intel · ‎06-28-2023

Yes, this (integrated) GPU is not supported by Embree because it does not have ray tracing hardware capabilities. Metor Lake will be the first generation of iGPU that Embree supports.

With regards to support of other vendors: We are looking into this and ideally would like to add support for other vendors, too. But there are many obstacles and we can give you no timeline or guarantee that we will achieve this.

peizhao · ‎06-27-2023

Also would be possible to code in Sycl and use Embree as a third-party library?

As I just need features like intersection checking and geometric define from Embree

FlorianR_Intel · ‎06-28-2023

I don't understand this question. How else would you use Embree if not as a third-party library?

peizhao · ‎06-28-2023

Sorry for the confusing question, what I am trying to say is.

Rather than use Sycl-based Embree functions such as rtcSYCLDeviceSelector

can I use functions such as initializeScene which is no-Sycl specified and use those functions in Sycl implementation as normal cpp function call?

The logic behind this thinking is Sycl can be compiled into heterogeneous devices while using a normal cpp library.

From my understanding, the non-Sycl Embree library can also be treated like a standard cpp library and therefore can be used in a Sycl implementation (SInce in these cases I am only interested in those optimized features in Embree such as creating a scene and finding the intersection between the ray and the scene)

Sorry if this makes no sense to you as I am quite a beginner for heterogeneous systems.