DevCloud: AOC Unable to compile kernels for device for any of the FPGA nodes: instantiates undefined entity "board" (reopened issue in new forum)

[PSath2]

Ask a question about a DevCloud FPGA resource in the DevCloud forum, get feedback over a week and then randomly locked and redirected elsewhere. Guess the other forum's first-tier support staff was tired of not being able to solve it? Either way as a customer it would be far easier to document a priori that for some reason FPGA DevCloud issues are handled elsewhere other than the DevCloud forum itself, rather than after the fact.

The original thread is here. https://software.intel.com/en-us/forums/intel-devcloud/topic/832012 (Contains my .bashrc and several solution attempts, and a week of debugging before they decided to randomly lock it and direct me here.)

The gist is the DevCloud FPGA nodes give this error (same as https://forums.intel.com/s/question/0D50P00004UAnXVSA1/how-to-use-paca10-with-intel-fpga-sdk-for-opencl-192, but on an Intel-configured machine, rather than a personally-configured): any time you try to compile a kernel:

• aoc: First stage compilation completed successfully.
• aoc: Compiling for FPGA. This process may take several hours to complete. Prior to performing this compile, be sure to check the reports to ensure the design will meet your performance targets. If the reports indicate performance targets are not being met, code edits may be required. Please refer to the Intel FPGA SDK for OpenCL Best Practices Guide for information on performance tuning applications for FPGAs.
• Error (16045): Instance "ccip_std_afu|bsp_logic_inst|board_inst" instantiates undefined entity "board" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/bsp_logic.sv Line: 133
• Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst|board_inst" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/bsp_logic.sv Line: 133
• Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/BBB_cci_mpf/hw/rtl/cci-mpf-if/cci_mpf_if.vh Line: 38
• Error (16185): Can't elaborate user hierarchy "ccip_std_afu" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/platform/green_bs.sv Line: 183
• Error (16186): Can't elaborate top-level user hierarchy
• Error: Flow failed:
• Error: Quartus Prime Synthesis was unsuccessful. 6 errors, 357 warnings
• Error (23035): Tcl error: ERROR: Error(s) found while running an executable. See report file(s) for error message(s). Message log indicates which executable was run last.
• Error (23031): Evaluation of Tcl script a10_partial_reconfig/flow.tcl unsuccessful
• Error: Quartus Prime Shell was unsuccessful. 13 errors, 357 warnings
• Error: Compiler Error, not able to generate hardware

[MEIYAN_L_Intel]

Hi,

I am checking this issues internally.

I will need some time and I will get back to you.

Thanks

[MEIYAN_L_Intel]

Hi,

According to https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/acceleration-card-arria-10-gx/getting-started.html, the Intel PAC with Arria10GX FPGA is only used in Quartus Prime Pro v17.1.1. For OpenCL SDK only can 2 versions higher than Quartus Prime pro version as stated in https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/hb/opencl-sdk/aocl_getting_started.pdf in Chpater 3.5.1.

Also, I would like to confirm which device you are using, are you using Intel PAC with Arria 10GX FPGA or using Intel Arria 10GX?

Thanks

[PSath2]

I am using the Intel FPGA DevCloud. So I'm using whatever is in the FPGA queue (batch@v-qsvr-fpga, nodes s001-n[137-139] and s001-n189). When given access I was instructed to provide beta feedback to Intel about their resource via the forums. The posts you link suggest that Intel has their own DevCloud configured in a way that prevent it from compiling OpenCL kernels.

This is the same feedback you gave in https://forums.intel.com/s/question/0D50P00004UAnXVSA1/how-to-use-paca10-with-intel-fpga-sdk-for-opencl-192, which is fine if a user has configured their own box with the wrong software versions, but again, this is Intel's machine. I did not configure it, I did not buy the parts, I do not have sudo. I am trying to escalate so that the admins in charge of it can fix the issue.

I am unclear on the distinction between those two devices, but here is the output of aoc -list-boards and aocl diagnose if that helps you determine which are in the Intel FPGA DevCloud nodes.

<userid>@s001-n137:~$ aoc -list-boards

Board list:

 pac_a10

   Board Package: /opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp

<userid>@s001-n137:~$ aocl diagnose

--------------------------------------------------------------------

Device Name:

acl0

Package Pat:

/opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp

Vendor: Intel Corp

Physical Dev Name  Status      Information

pac_a10_ec00001   Uninitialized   OpenCL BSP not loaded. Must load BSP using command:

                   'aocl program <device_name> <aocx_file>'

                   before running OpenCL programs using this device

DIAGNOSTIC_PASSED

--------------------------------------------------------------------

--------------------------------------------------------------------

Device Name:

acl1

Package Pat:

/opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp

Vendor: Intel Corp

Physical Dev Name  Status      Information

pac_a10_ec00000   Uninitialized   OpenCL BSP not loaded. Must load BSP using command:

                   'aocl program <device_name> <aocx_file>'

                   before running OpenCL programs using this device

DIAGNOSTIC_PASSED

--------------------------------------------------------------------

--------------------------------------------------------------------

Device Name:

acl2

Package Pat:

/opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp

Vendor: Intel Corp

Physical Dev Name  Status      Information

pac_a10_ec00002   Passed      PAC Arria 10 Platform (pac_a10_ec00002)

                   PCIe 175:00.0

                   FPGA temperature = 48 degrees C.

DIAGNOSTIC_PASSED

--------------------------------------------------------------------

Call "aocl diagnose <device-names>" to run diagnose for specified devices

Call "aocl diagnose all" to run diagnose for all devices

<userid>@s001-n137:~$

[MEIYAN_L_Intel]

Hi,

From the command here: https://software.intel.com/en-us/forums/intel-devcloud/topic/832012, I saw that you might be using Quartus/BSP and OpenCL SDK with 19.3 version.

Since the Intel PAC with A10GX FPGA need to be used BSP with version 17.1.1 and the OpenCL SDK version should be used as the same with BSP version or can only two version higher than BSP version.

Can you try to used BSP and OpenCL SDK with 17.1.1 version?

Thanks

[PSath2]

In that thread I have documented trying all three (18.1, 19.2, 19.3) versions available on the Intel FPGA Cloud Beta. As stated before, I cannot reconfigure the machine, it's Intel's. One would expect your resources to have the necessary toolkits installed to use your own hardware in your own system.

We have reached out directly to our contacts at Intel and they've notified us they've contacted the cloud provider in charge of the Intel FPGA DevCloud and have asked that the appropriate toolkits be installed. But this is on the queue with other sysadmin requests and will take some time. Will follow-up after Thanksgiving

[MEIYAN_L_Intel]

Hi,

I am understand your situation and I am checking internally and had reported these issues to the concern team.

I will reply here once I get reply from the concern team.

Thanks

[MEIYAN_L_Intel]

Hi,

I am setting up the Devcloud and will test it on my side. Since I am still have to wait for the access for node s001-n137 to n139, I will try to test in other FPGA node with aoc command.

Thanks

[MEIYAN_L_Intel]

Hi,

I would like to know which example you have compile?

I have try to compile on my side with other FPGA node for FPGA emulator and hardware with the command as below:

ssh decloud

git clone https://github.com/intel/BaseKit-code-samples.git

qsub -I -l nodes=1:fpga:ppn=2 -d .

cd BaseKit-code-samples/DPC++Compiler/vector-add/

vi build_fpga_emu.sh 

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

make fpga_emu -f Makefile.fpga

vi run_fpga_emu.sh 

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

make run_emu -f Makefile.fpga

vi build_fpga_hw.sh 

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

make hw -f Makefile.fpga

vi run_fpga_hw.sh 

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

make run_hw -f Makefile.fpga

qsub -I -l nodes=1:fpga:ppn=2 -d . build_fpga_emu.sh 

qsub -I -l nodes=1:fpga:ppn=2 -d . run_fpga_emu.sh 

qsub -I -l nodes=1:fpga:ppn=2 -d . build_fpga_hw.sh 

qsub -I -l nodes=1:fpga:ppn=2 -d . run_fpga_hw.sh 

You can have the output file by type the command cat run.sh.oXXXX.

Thanks

[PSath2]

Mylee,

I am just returning from vacation, haven't gotten to try anything on the non-FPGA-queue nodes.

As the previous thread shows (before I was moved to this forum), I am trying a canonical OpenCL VectorAdd like follows:

__kernel void vectorAdd(__global float *A, __global float *B, __global float *C, int nelem) { int tid = get_global_id(0); if (tid < nelem) C[tid] = A[tid] + B[tid]; }

The end goal is the 27 kernels in https://github.com/vetter/shoc/tree/master/src/opencl/level2/s3d

The examples you show are DPC++ which is a superset of SyCL, there are not OpenCL kernels in that repository. I am not sure if the DPC++ stack even uses AOC and all the old Altera stuff like the OpenCL stack does, so I am not sure how they're relevant. Our client has expressed that they want OpenCL, not SyCL, much less a vendor superset of it.

[MEIYAN_L_Intel]

Hi,

I would like to know which sample/example code you had used to do the compilation that make this error occured?

This is because I will try at one my side also.

Thanks

[PSath2]

What? I just provided 28 example kernels and they were also provided in the previous thread before it was locked?

The VectorAdd OpenCL kernel I posted yesterday that immediately precedes your response, as well as the 27 SHOC/S3D kernels linked in the same post. Literally just put those four lines in a file called vectorAdd.cl and then "aoc -v vectorAdd.cl"

[PSath2]

Interestingly, now that I've had a chance to check on the default queue (and not had my job terminated early, see (https://software.intel.com/en-us/forums/intel-devcloud/topic/840456), the vectorAdd example appears to make it substantially further, but still does not produce an aocx file. It does produce an aoco and aocr pair, however. quartus_sh_compile.log indicates that on the default queue it is correctly using Quartus Prime 17.1.1.

However, the pipeline still fails during some reporting step. Console and quartus_compile_report.log below:

1@s001-n091:~$ time aoc -v vectorAdd.cl aoc: Environment checks are completed successfully. aoc: Cached files in /var/tmp/aocl/ may be used to reduce compilation time You are now compiling the full flow!! aoc: Selected default target board pac_a10 aoc: Running OpenCL parser.... aoc: OpenCL parser completed successfully. aoc: Linking Object files.... aoc: Optimizing and doing static analysis of code... aoc: Linking with IP library ... Checking if memory usage is larger than 100% Compiler Warning: /home/<userId>/vectorAdd.cl:1: declaring global arguments 'A', 'B' and 'C' with no 'restrict' may lead to low performance for kernel 'vectorAdd' aoc: First stage compilation completed successfully. Compiling for FPGA. This process may take a long time, please be patient. Error: Quartus full compile: generating Quartus compile report FAILED. Refer to quartus_compile_report.log for details.     real 67m39.208s user 228m0.869s sys 11m19.219s------------------------------------------------ ERROR: Can't load report data for revision name: afu_import. Make sure the report database exists for the specified revision name.   while executing "load_report $project_rev" (procedure "generate_report" line 5) invoked from within "generate_report "kernel" $project_name $project_rev $report_file_name -1 $skip_entity_area_report $a_fmax $fmax2 $k_fmax $kernel_list" (procedure "generate_opencl_report" line 3) invoked from within "generate_opencl_report $project_name $project_rev $report_file_name $skip_entity_area_report $a_fmax $fmax2 $k_fmax $kernel_list" invoked from within "if {[string equal "aoc" [lindex $quartus(args) 0]]} { set arglen [llength $quartus(args)] if {$arglen < 9} { post_message "The minimum number ..." (file "/glob/development-tools/versions/intelFPGA_pro/18.1/hld/share/lib/tcl/quartus_compile_report.tcl" line 532) ------------------------------------------------ factured by Info: Intel and sold by Intel or its authorized distributors. Please Info: refer to the applicable agreement for further details. Info: Processing started: Thu Dec 5 10:38:09 2019 Info: Command: quartus_sh -t /glob/development-tools/versions/intelFPGA_pro/18.1/hld/share/lib/tcl/quartus_compile_report.tcl aoc dcp afu_import ./reports/lib/json/quartus.json 0 265 10000 265.67 vectorAdd,0 Info: Quartus(args): aoc dcp afu_import ./reports/lib/json/quartus.json 0 265 10000 265.67 vectorAdd,0 Info: Using INI file /home/u29791/vectorAdd/quartus.ini Error (23031): Evaluation of Tcl script /glob/development-tools/versions/intelFPGA_pro/18.1/hld/share/lib/tcl/quartus_compile_report.tcl unsuccessful Error: Quartus Prime Shell was unsuccessful. 1 error, 0 warnings Error: Peak virtual memory: 603 megabytes Error: Processing ended: Thu Dec 5 10:38:09 2019 Error: Elapsed time: 00:00:00 Error: Total CPU time (on all processors): 00:00:00

[MEIYAN_L_Intel]

Hi,

I had done the following steps to compile with OpenCL tool in Intel DevCloud:

1. Enter devloud: 

ssh devcloud

2. Enter the node with pac card:

qsub -q batch@v-qsvr-fpga -I -l nodes=s001-n137:ppn=2 

3. Source the OpenCL: 

source /opt/a10/inteldevstack/init_env.sh

source /opt/a10/inteldevstack/intelFPGA_pro/hld/init_opencl.sh

4. make a directory: 

eg: mkdir vector

5. add a file for vector_add:

vi vector.cl

6. Compile the example with the command below: 

aoc -march=emulator -v vector_add/vector.cl -o vector_add/vector.aocx

Thanks

[PSath2]

The issue appears isolated to the FPGA beta queue. When we use a default bashrc/bash_profile on the v-qsvr-1.aidevcloud queue, we can successfully build for hardware.

[MEIYAN_L_Intel]

Hi,

I will report the issue to the developer.

Thanks for your update.

Thanks