Building GROMACS-SYCL

[Viet-Duc]

Dear Support Team,

I chronicle problems arised when building process for experimental version of GROMACS-SYCL

https://manual.gromacs.org/documentation/2021-sycl/download.html

Typical cmake options are:

cmake ..                         \
    -DGMX_GPU=SYCL               \ 
    -DGMX_BUILD_OWN_FFTW=ON      \ 
    -DCMAKE_CXX_COMPILER=clang++ \ 
    -DCMAKE_C_COMPILER=clang

• First of all, since oneAPI's clang is identified as 'IntelLLVM 2021.2.0' instead of just 'Clang', 'FindLibStdCpp.cmake' must be modified as follow to correctly pass '-gcc-toolchain'

142     else() #Intel
143         set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} --gcc- toolchain=${GMX_GPLUSPLUS_PATH}")
144         #set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -gcc-name=${GMX_GPLUSPLUS_PATH}")
145     endif()
146 endif()

• The next issue is gcc/7.4 paths are not properly set up: 

/home/u66264/apps/src/gromacs/build/CMakeFiles/CMakeTmp/src.cxx:2:10: fatal error: 'string_view' file not found
#include <string_view>
         ^~~~~~~~~~~~~
1 error generated.

•  'string_view' is located in '/usr/include/c++/7', hence

cmake ..                                     \
    -DGMX_GPU=SYCL                           \ 
    -DGMX_BUILD_OWN_FFTW=ON                  \ 
    -DCMAKE_CXX_COMPILER=clang++             \ 
    -DCMAKE_C_COMPILER=clang                 \
    -DCMAKE_CXX_FLAGS='-I/usr/include/c++/7' 

• however

#include <bits/c++config.h>
         ^~~~~~~~~~~~~~~~~~
1 error generated.
CMakeFiles/cmTC_e2fa7.dir/build.make:78: recipe for target 'CMakeFiles/cmTC_e2fa7.dir/src.cxx.o' failed

• 'bits/c++config.h' is located in /usr/include/x86_64-linux-gnu/c++/7, hence

cmake ..                                     \
    -DGMX_GPU=SYCL                           \ 
    -DGMX_BUILD_OWN_FFTW=ON                  \ 
    -DCMAKE_CXX_COMPILER=clang++             \ 
    -DCMAKE_C_COMPILER=clang                 \
    -DCMAKE_CXX_FLAGS='-I/usr/include/c++/7 -I/usr/include/x86_64-linux-gnu/c++/7'

• However

/glob/development-tools/versions/oneapi/2021.2/inteloneapi/compiler/2021.2.0/linux/bin/clang++ -I/usr/include/x86_64-linux-gnu/c++/7 -I/usr/include/c++/7 --gcc-toolchain=/usr/bin/g++  CMakeFiles/cmTC_53942.dir/src.cxx.o -o cmTC_53942
/usr/bin/ld: cannot find crtbegin.o: No such file or directory
/usr/bin/ld: cannot find -lstdc++

• Both 'crtbegin.o' and 'libstdc++.so' are located in '/usr/lib/gcc/x86_64-linux-gnu/7'. The latter can be found by passing '-L/usr/lib/gcc/x86_64-linux-gnu/7' to CMAKE_CXX_FLAGS. But 'crtbegin.o' error still stand. So configure failed at the very beginnning of build process.

So  a QRD:

• Naming of oneAPI's clang will confuse cmake
• On DevCloud I think the gcc-related paths are not properly set up.

A way to successfully build GROMACS-SYCL is much appreciated !

Regards.

[NoorjahanSk_Intel]

Hi,

Thanks for reaching out to us.

we tried following the below steps and it worked fine

>> cmake .. -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON -DCMAKE_INSTALL_PREFIX=/path

>> make

>>make check

>> make install

>> source path/GMXRC

By default clang/clang++ compiler points to the oneAPI root directory. You may confirm the same using "which clang" command.

I did not find SYCL as a valid option for DGMX_GPU flag.

https://manual.gromacs.org/documentation/2021-sycl/install-guide/index.html

These are my Environment details:

GCC version: 9.3.0

BaseToolKit Version: 2021.2.0

Please do let us know if you have any issues

Thanks & Regards,

Noorjahan.

[Viet-Duc]

Since it is under development,  the document has not been updated to reflect the SYCL support. But 'CMakeLists.txt' does support SYCL.

203 gmx_option_multichoice(
204     GMX_GPU
205     "Framework for GPU acceleration"
206     OFF
207     OFF CUDA OpenCL SYCL)
...
574 if(GMX_GPU)
575 
576     string(TOUPPER "${GMX_GPU}" _gmx_gpu_uppercase)
577     if(${_gmx_gpu_uppercase} STREQUAL "CUDA")
578         include(gmxManageCuda)
579     elseif(${_gmx_gpu_uppercase} STREQUAL "OPENCL")
580         message(STATUS "GPU support with OpenCL is deprecated. It is still fully supported (and "
581             "recommended for AMD and Intel GPUs). It may be replaced by different approaches in "
582             "future releases of GROMACS.")
583         include(gmxManageOpenCL)
584     elseif(${_gmx_gpu_uppercase} STREQUAL "SYCL")
585         include(gmxManageSYCL)
586     endif()

I was motivated to test SYCL build following the presentation of Heinrich Bockhorst (Intel) at the oneAPI Dev Summit.

https://www.hlrn.de/doc/display/PUB/Joint+NHR@ZIB+-+INTEL+++oneAPI+Workshop

However, his cmake options, as indicated in the slides, mixed icx (dpcpp) and icpc (intel c++), leading to incompatible compilation error since they are two different compilers. 

If you omit GMX_GPU, by default the CPU version will be built. There would be no error since only standard gcc is required.

Regarding gcc, I don't have access to gcc/9.3

$ gcc --version
gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Could you try again with -DGMX_GPU=SYCL and clang compilers ?

Thanks.

[NoorjahanSk_Intel · Accepted solution]

Hi,

We have tried with CMAKE version 3.16.3 and gcc 9.3.0 on Devcloud using the below command

cmake -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_GPU=SYCL -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=dpcpp -DCMAKE_INSTALL_PREFIX=../.. ..

Surprisingly CMAKE identified compiler as clang 12.0 for me.

Could you share your CMAKE version and Devcloud node number?

Alternatively could you try the below command and let us know if it works.

cmake -DGMX_BUILD_OWN_FFTW=ON -DREGRESSIONTEST_DOWNLOAD=ON -DGMX_GPU=SYCL -DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc -DCMAKE_INSTALL_PREFIX=../.. ..

Thanks & Regards

Noorjahan

[Viet-Duc]

Dear Noorjahan,

Thanks for update. It is indeed an issue related to cmake. I had to downgrade my cmake to 3.16.3 for compilers to be identified.

Below is result with 3.20.2:

-- The C compiler identification is IntelLLVM 2021.2.0
-- The CXX compiler identification is IntelLLVM 2021.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /glob/development-tools/versions/oneapi/2021.2/inteloneapi/compiler/2021.2.0/linux/bin/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /glob/development-tools/versions/oneapi/2021.2/inteloneapi/compiler/2021.2.0/linux/bin/dpcpp - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test USING_LIBSTDCXX
-- Performing Test USING_LIBSTDCXX - Success
-- Performing Test CXX17_COMPILES
-- Performing Test CXX17_COMPILES - Failed
CMake Error at cmake/FindLibStdCpp.cmake:165 (message):
  GROMACS requires C++17, but a test of such functionality in the C++
  standard library failed to compile.  The g++ found at /usr/bin/g++ had a
  suitable version, so ;something else must be the problem
Call Stack (most recent call first):
  CMakeLists.txt:68 (find_package)

For the future reference, here are the neccessary steps to build GROMACS-SYCL:

1. build cmake/3.16.3

2. start an iteractive sessions to avoid OOM during compilation:    

$ qsub -I -l nodes=1:gen9:ppn=2

3. since dpcpp does not support OpenMP, the following should be used:  

$ cmake ..                    \ 
    -DGMX_BUILD_OWN_FFTW=ON   \
    -DGMX_GPU=SYCL            \  
    -DCMAKE_C_COMPILER=icx    \
    -DCMAKE_CXX_COMPILER=icpx \
    -DCMAKE_INSTALL_PREFIX=...

The result binary will work on Gen9 GPUs.

Once again, thank you for your time!

[NoorjahanSk_Intel]

Hi,

Thanks for accepting as a solution.

As this issue has been resolved, we will no longer respond to this thread.

If you require any additional assistance from Intel, please start a new thread.

Any further interaction in this thread will be considered community only.

Thanks & Regards

Noorjahan.