Unable to compile OpenCL kernel for FPGAs in Devcloud

[Opdenhövel__Jan-Oliv]

Hello all,

I'm trying to compile an OpenCL kernel for an FPGA in the Devcloud, using the supplied version of `aoc`. The kernel itself is attached to the post (It has a .c ending because the forum wouldn't allow me to upload a .cl file) and I've run the compilation task (`aoc gauss.cl`) on an fpga-compile node using `qsub -l nodes=1:fpga_compile:ppn=2 -d . build.sh`. The job ran for about an hour, but then failed with the following output:

aoc: Running OpenCL parser....
aoc: OpenCL parser completed 
aoc: Linking Object files....
aoc: Optimizing and doing static analysis of code...
aoc: First stage compilation completed successfully.
aoc: Compiling for FPGA. This process may take several hours to complete.  Prior to performing this compile, be sure to check the reports to ensure the design will meet your performance targets.  If the reports indicate performance targets are not being met, code edits may be required.  Please refer to the Intel FPGA SDK for OpenCL Best Practices Guide for information on performance tuning applications for FPGAs.
Error (213009): File name "output_files/afu_import.green_region.pmsf" does not exist or can't be read
Error: Quartus Prime Convert_programming_file was unsuccessful. 1 error, 0 warnings
Error (23031): Evaluation of Tcl script compile_script.tcl unsuccessful
Error: Quartus Prime Shell was unsuccessful. 1 error, 0 warnings
Error (23031): Evaluation of Tcl script build/entry.tcl unsuccessful
Error: Quartus Prime Shell was unsuccessful. 1 error, 0 warnings

I was able to reproduce this exact error multiple times, even by compiling SYCL programs for an FPGA using `dpcpp`. Is this a known error and what may be it's cause?

[Opdenhövel__Jan-Oliv]

There is a workaround: One of the first things the compiler does is to create a lot of files. Once possible, you need to copy the file "gauss/build/output_files/dcp.green_region.pmsf" to "gauss/build/output_files/afu_import.green_region.pmsf". After this is done, the compiler completes it's task without a problem.

However, I'm not able to run this kernel on an fpga node. I've attached the host code to the post, which I compiled with the command

gcc -lstdc++ -lOpenCL gauss.cpp -o gauss -DFPGA -DMATRIX_SIZE=4 -DDESIRED_PLATFORM_INDEX=1

Since the OpenCL headers aren't available in the Devcloud, I had to use my x86-64 computer running a current Arch Linux. I've installed `intel-compute-runtime` version 20.02.15268-1, `ocl-icd` version 2.2.12-3, `opencl-headers` version 2:2.2.20170516-2, and `gcc` version 9.2.0-4.

Once I uploaded the created executable and allocated an interactive FPGA node, I ran the executable, which created the following output:

Platform retrieved
Context created
Command queue created
###
1.4732e+09,2.94641e+09,4.41961e+09,5.89282e+09
7.36602e+09,8.83923e+09,1.03124e+10,1.17856e+10
1.32588e+10,1.4732e+10,1.62053e+10,1.76785e+10
1.91517e+10,2.06249e+10,2.20981e+10,2.35713e+10
###
Matrix loaded
Error initializing DMA: not found
Error initializing DMA
Kernel enqueued
Hardware CSR version ID differs from version expected by software.  Either:
   a) Ensure your compiled design was generated by the same ACL build
      currently in use, OR
   b) The host can not communicate properly with the compiled kernel.
Saw version=65535, expected=4.
gauss-host: acl_kernel_if.cpp:957: void acl_kernel_if_launch_kernel_on_custom_sof(acl_kernel_if*, cl_uint, acl_dev_kernel_invocation_image_t*, cl_int): Assertion `0' failed.
Aborted

To me, it looks like the DMA to load the kernel into the FPGA could not be found, which therefore lead to the failure of the program. Am I missing an environment variable or something?

[Michael_C_Intel1]

Hi Jan-Oliver,

Thanks for the feedback. We'll take a look for the best FPGA stakeholder route for this issue.

-MichaelC

[Opdenhövel__Jan-Oliv]

Oh, there's another update: Apparently, you have to tell aoc which BSP to use. Otherwise, it will use none or a wrong one and the computer can't communicate with the FPGA card once it was programmed. The setup is the same as described before, but I'm using

aoc gauss.cl -o gauss.aocx -board=pac_a10 -board-package=/opt/intel/inteloneapi/compiler/2021.1-beta03/linux/lib/oclfpga/board/intel_a10gx_pac

instead of

aoc gauss.cl -o gauss.aocx

Once the workaround from above is implemented (copying `gauss/build/output_files/dcp.green_region.pmsf` to `gauss/build/output_files/afu_import.green_region.pmsf` once possible) and the compilation is complete, the kernel can be used just fine.

The only problem that's left is the fact that the file `gauss/build/output_files/dcp.green_region.pmsf` is either non-existent or isn't generated.

[Lawrence_L_Intel]

Did you get this resolved? Note that the error:

Error (213009): File name "output_files/afu_import.green_region.pmsf" does not exist or can't be read

is a cryptic message that has been fixed with a patch. Refer to the github site:

https://github.com/intel/FPGA-Devcloud and navigate to the section on login script

Basically add this to your .bashrc

if [ -f /data/intel_fpga/devcloudLoginToolSetup.sh ]; then 

source /data/intel_fpga/devcloudLoginToolSetup.sh

fi

see the two functions in the script and try them out

Thanks