Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1626 Discussions

AOC Unable to compile kernels for device for any of the FPGA nodes.

PSath2
Beginner
1,800 Views

I've been struggling to get any kernels to compile for hardware on any of the nodes in the FPGA queue. I have struggled through a few solutions myself but have hit a sticking point I can't figure out.

I have succeeded in compiling for a10gx emulation on node s001-n189, but was unable to compile for hardware for the pac_s10_dc, as it complained about a missing license file. (The node is currently in use, I can edit this post with the exact error once I can get on it).

To try to sidestep the license issue with the Stratix card, I manually switch to an unused one of the older FPGA nodes (s001-n[137-139]) with Arria10. I am using the 19.3 toolkit in /glob/development-tools/versions/intelFPGA_pro/19.3/, and see the BSP for the Arria10 in /opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/ when I run `aoc -list-boards`. Specifically my .bashrc has the following to setup the tools. (Found by trial and error, the initial documentation for the FPGA nodes that was sent to me is now out of date.)

export FPGA_TOOL_ROOT=/glob/development-tools/versions/intelFPGA_pro/19.3/
export INTELFPGAOCLSDKROOT=$FPGA_TOOL_ROOT/hld
export QUARTUS_ROOTDIR_OVERRIDE=$FPGA_TOOL_ROOT/quartus
OPENCL=TRUE source /opt/a10/intelrtestack/init_env_nosudo.sh
export INTELFPGAOCLSDKROOT=$FPGA_TOOL_ROOT/hld
export ALTERAOCLSDKROOT=$INTELFPGAOCLSDKROOT
AOCL_BOARD_PACKAGE_ROOT=/opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp
OPENCL=TRUE source $INTELFPGAOCLSDKROOT/init_opencl.sh

(As an aside I also had to override the node's default python3 with python2 via a symbolic link in my PATH. Otherwise Quartus would error out early due to a missing 'constants' module in packager.pyz. Neither an alias or exported bash function seemed to correctly propagate into a non-interactive pbs job, even with the -V argument to qsub.)

At this point, it seems to progress with compilation/placement/routing to a point, where I encounter the error I'm currently blocking on:

aoc: First stage compilation completed successfully.
aoc: Compiling for FPGA. This process may take several hours to complete.  Prior to performing this compile, be sure to check the reports to ensure the design will meet your performance targets.  If the reports indicate performance targets are not being met, code edits may be required.  Please refer to the Intel FPGA SDK for OpenCL Best Practices Guide for information on performance tuning applications for FPGAs.
Error (16045): Instance "ccip_std_afu|bsp_logic_inst|board_inst" instantiates undefined entity "board" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/bsp_logic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst|board_inst" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/bsp_logic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/BBB_cci_mpf/hw/rtl/cci-mpf-if/cci_mpf_if.vh Line: 38
Error (16185): Can't elaborate user hierarchy "ccip_std_afu" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/platform/green_bs.sv Line: 183
Error (16186): Can't elaborate top-level user hierarchy
Error: Flow failed:
Error: Quartus Prime Synthesis was unsuccessful. 6 errors, 357 warnings
Error (23035): Tcl error: ERROR: Error(s) found while running an executable. See report file(s) for error message(s). Message log indicates which executable was run last.
Error (23031): Evaluation of Tcl script a10_partial_reconfig/flow.tcl unsuccessful
Error: Quartus Prime Shell was unsuccessful. 13 errors, 357 warnings
Error: Compiler Error, not able to generate hardware

This knowledgebase article is the only exact match I have found for the error, which inspired adding the explicit export of ALTERAOCLSDKROOT in my .bashrc: https://www.intel.com/content/www/us/en/programmable/support/support-resources/knowledge-base/embedded/2019/error--16045---instance--ccipstdafu-bsplogicinst-boardinst--inst.html However, that didn't help, nor did manually setting it to several other directories (including the /opt/a10 BSP directory) with an ip/board subtree. (This attempt was inspired by looking at build/iface.ipx, which seems to be the only generated file that uses ALTERAOCLSDKROOT.)

I'm at a loss, any suggestions on what to try next? My allocation expires next week, and I am unsure whether I should bother creating a DevMesh project and asking for an extension if I can't build code for the physical FPGAs we need. Thanks in advance!

0 Kudos
14 Replies
JEYANTHKRI_N_Intel
1,800 Views

Thanks for reaching out to us.

Please try using node 136 which supports FPGA

0 Kudos
PSath2
Beginner
1,800 Views

Ahh, I had been instructed to use the batch@v-qsvr-fpga queue. Purged output files and tried on s001-n136 and got the same result:

aoc: Environment checks completed successfully.
aoc: Cached files in /var/tmp/aocl/ may be used to reduce compilation time
You are now compiling the full flow!!
aoc: Selected target board pac_a10
aoc: Running OpenCL parser....
aoc: OpenCL parser completed
aoc: Linking Object files....
aoc: Optimizing and doing static analysis of code...
Compiler Warning: gr_base.cl:71: declaring global arguments 'P', 'T', 'Y' and 'C                                     ' with no 'restrict' may lead to low performance for kernel 'gr_base'
aoc: Linking with IP library ...
aoc: Checking if memory usage is larger than 100%...
aoc: Memory usage is not above 100.
Compiler Warning: addpipe in board_spec.xml is set to 1 which is no longer suppo                                     rted
Compiler Warning: global memory pipeline stage is now implemented in BSP instead
aoc: First stage compilation completed successfully.
aoc: Compiling for FPGA. This process may take several hours to complete.  Prior                                      to performing this compile, be sure to check the reports to ensure the design w                                     ill meet your performance targets.  If the reports indicate performance targets                                      are not being met, code edits may be required.  Please refer to the Intel FPGA S                                     DK for OpenCL Best Practices Guide for information on performance tuning applica                                     tions for FPGAs.
Error (16045): Instance "ccip_std_afu|bsp_logic_inst|board_inst" instantiates un                                     defined entity "board" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_                                     base/build/bsp_logic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst|board                                     _inst" File: /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/bsp_l                                     ogic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst" File                                     : /home/<userid>/P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/BBB_cci_mpf/hw/r                                     tl/cci-mpf-if/cci_mpf_if.vh Line: 38
Error (16185): Can't elaborate user hierarchy "ccip_std_afu" File: /home/<userid>/                                     P3HPC19/shoc/src/opencl/level2/s3d/gr_base/build/platform/green_bs.sv Line: 183
Error (16186): Can't elaborate top-level user hierarchy
Error: Flow failed:
Error: Quartus Prime Synthesis was unsuccessful. 6 errors, 357 warnings
Error (23035): Tcl error: ERROR: Error(s) found while running an executable. See                                      report file(s) for error message(s). Message log indicates which executable was                                      run last.
Error (23031): Evaluation of Tcl script a10_partial_reconfig/flow.tcl unsuccessf                                     ul
Error: Quartus Prime Shell was unsuccessful. 13 errors, 357 warnings
Error: Compiler Error, not able to generate hardware


real    3m39.082s
user    10m49.440s
sys     0m37.972s

 

0 Kudos
PSath2
Beginner
1,800 Views

For reference, this is the kernel I am trying to compile (as well as the other 26 from that application, which all work for FPGA emulation, Nvidia, AMD ROCm, AMD APP SDK, and POCL OpenCL implementations on multiple devices.)
https://github.com/vetter/shoc/blob/master/src/opencl/level2/s3d/gr_base.cl

0 Kudos
JEYANTHKRI_N_Intel
1,800 Views

Hi ,

We will check the concerned team and get back to you.

0 Kudos
PSath2
Beginner
1,800 Views

I've filed for and received an extension of my allocation to allow more time for resolution. Are there any additional channels I can pursue for more interactive debugging support?

 

The observed bug appears to be due to a lack of a definition of the "board" entity type that bsp_logic.sv is trying to instantiate. However, I don't have much direct FPGA experience to diagnose further. (I am a GPU OpenCL developer trying to move into the FPGA space.) One would think that wherever it's defined is either missing from my configuration, or missing from the machines entirely. I'm not intimately familiar with System Verilog, but that file only includes a single other, "cci_mpf_platform.vh", which can be found at /opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp/hardware/pac_a10/build/BBB_cci_mpf/hw/rtl/cci-mpf-if/cci_mpf_if.vh. However, neither that file, nor the one it includes "cci_mpf_platform.vh" includes the phrase "board" anywhere within them. From a C perspective, it thus makes sense that this "board" entity is undefined if it is nowhere within the include heirarchy.

Any other users that have been able to successfully compile an OpenCL kernel for hardware on the DevCloud nodes? If so, mind sharing your shell profile / environment? Thanks in advance.

0 Kudos
PSath2
Beginner
1,800 Views

Bump.

Sanity checked with a trivial vector add kernel, and received the exact same problem

Kernel code:

__kernel void vectorAdd(__global float *A, __global float *B, __global float *C, int nelem) {
        int tid = get_global_id(0);
        if (tid < nelem) C[tid] = A[tid] + B[tid];
}



Log:
 

<userid>@s001-n138:~$ time aoc -v vectorAdd.cl
aoc: Environment checks completed successfully.
aoc: Cached files in /var/tmp/aocl/ may be used to reduce compilation time
You are now compiling the full flow!!
aoc: Selected default target board pac_a10
aoc: Running OpenCL parser....
aoc: OpenCL parser completed
aoc: Linking Object files....
aoc: Optimizing and doing static analysis of code...
Compiler Warning: vectorAdd.cl:1: declaring global arguments 'A', 'B' and 'C' with no 'restrict' may lead to low performance for kernel 'vectorAdd'
aoc: Linking with IP library ...
aoc: Checking if memory usage is larger than 100%...
aoc: Memory usage is not above 100.
Compiler Warning: addpipe in board_spec.xml is set to 1 which is no longer supported
Compiler Warning: global memory pipeline stage is now implemented in BSP instead
aoc: First stage compilation completed successfully.
aoc: Compiling for FPGA. This process may take several hours to complete.  Prior to performing this compile, be sure to check the reports to ensure the design will meet your performance targets.  If the reports indicate performance targets are not being met, code edits may be required.  Please refer to the Intel FPGA SDK for OpenCL Best Practices Guide for information on performance tuning applications for FPGAs.
Error (16045): Instance "ccip_std_afu|bsp_logic_inst|board_inst" instantiates undefined entity "board" File: /home/<userid>/vectorAdd/build/bsp_logic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst|board_inst" File: /home/<userid>/vectorAdd/build/bsp_logic.sv Line: 133
Error (16185): Can't elaborate user hierarchy "ccip_std_afu|bsp_logic_inst" File: /home/<userid>/vectorAdd/build/BBB_cci_mpf/hw/rtl/cci-mpf-if/cci_mpf_if.vh Line: 38
Error (16185): Can't elaborate user hierarchy "ccip_std_afu" File: /home/<userid>/vectorAdd/build/platform/green_bs.sv Line: 183
Error (16186): Can't elaborate top-level user hierarchy
Error: Flow failed:
Error: Quartus Prime Synthesis was unsuccessful. 6 errors, 336 warnings
Error (23035): Tcl error: ERROR: Error(s) found while running an executable. See report file(s) for error message(s). Message log indicates which executable was run last.
Error (23031): Evaluation of Tcl script a10_partial_reconfig/flow.tcl unsuccessful
Error: Quartus Prime Shell was unsuccessful. 13 errors, 336 warnings
Error: Compiler Error, not able to generate hardware


real    2m21.248s
user    9m51.416s
sys     0m40.892s

 

0 Kudos
Lawrence_L_Intel
Employee
1,800 Views

The latest FPGA devcloud information is being kept up to date here:

https://github.com/intel/FPGA-Devcloud

 

For Arria 10 (n137-n139):

source /opt/a10/intelrtestack/init_env_nosudo.sh

 

For Stratix 10 (n189)

source /opt/intel/inteldevstack/nosudo_init_env.sh

 

See if you can complete the flow with these OpenCl setup scripts.

 

Regards

Larry

 

0 Kudos
PSath2
Beginner
1,800 Views

Thanks for the Github link, Larry. I will look at it in the AM.

I believe I am already sourcing that file in my bashrc. This is what I have in there for AOC right now:

export FPGA_TOOL_ROOT=/glob/development-tools/versions/intelFPGA_pro/19.3/
export INTELFPGAOCLSDKROOT=$FPGA_TOOL_ROOT/hld
export QUARTUS_ROOTDIR_OVERRIDE=$FPGA_TOOL_ROOT/quartus
OPENCL=TRUE source /opt/a10/intelrtestack/init_env_nosudo.sh
export INTELFPGAOCLSDKROOT=$FPGA_TOOL_ROOT/hld
export ALTERAOCLSDKROOT=$INTELFPGAOCLSDKROOT
AOCL_BOARD_PACKAGE_ROOT=/opt/a10/intelrtestack/a10_gx_pac_ias_1_2_pv/opencl/opencl_bsp
OPENCL=TRUE source $INTELFPGAOCLSDKROOT/init_opencl.sh

 

Originally I didn't have the QUARTUS_ROOTDIR_OVERRIDE (which i needed if I change to either the 19.2 or 18.1 stacks), the ALTERAOCLSDKROOT (suggested by an existing knowledgebase article linked above), or AOCL_BOARD_PACKAGE_ROOT ( I have forgotten where I found that suggestion), but the results were the same. Is there anything order dependent about init_env_no_sudo and init_opencl that could be going wrong?

 

0 Kudos
JEYANTHKRI_N_Intel
1,800 Views

Hi,

We urge you to post this question in the following url (https://forums.intel.com/s/topic/0TO0P0000001AUUWA2/intel-high-level-design?language=en_US) in order to get a faster response regarding FPGA .

0 Kudos
PSath2
Beginner
1,800 Views

Jeyanth, there is a user on those forums with an identical problem: https://forums.intel.com/s/question/0D50P00004UAnXVSA1/how-to-use-paca10-with-intel-fpga-sdk-for-opencl-192.

If the mismatched 17.1 BSP with 19.2 tools noted there are indeed the problem, then that is expected of a personally-configured machine. I raised the problem in the DevCloud forum, because I am encountering it on a DevCloud (i.e. Intel-configured) machine. Thus it should be escalated to the DevCloud system administrators to resolve the mismatch as the 17.1 tools are not installed, and the nearest installed version (18.1) encountered the same issue. However, I will cross-link the post on that end as they do appear related.

0 Kudos
PSath2
Beginner
1,800 Views

Re-reading the provided documentation at the GitHub site, I missed the "quartus_setup" script in step section 8.0, which itself calls init_opencl. I've adjusted my bashrc to the following

TOOL_VERSION=18.1
export QUARTUS_ROOTDIR_OVERRIDE=/glob/development-tools/versions/intelFPGA_pro/$TOOL_VERSION/quartus
OPENCL=TRUE source /opt/a10/intelrtestack/init_env_nosudo.sh
TOOL_VERSION=$TOOL_VERSION OPENCL=TRUE source $HOME/quartus_setup.sh

With this slightly tweaked quartus_setup.sh file. (The only change is to make the tool version a variable that can be set externally.)

#!/bin/bash

# source this file from .profile in Ubuntu home directory
# or.... source this file from .bash_profile in CentOS home directory
# typically this will be added to profile file:  source ~/tools.sh (assuming you put this file in your home directory where .profile or .bash_profile are located

# Set to LITE if using Quartus Lite tools: Cyclone and MAX FPGAs. Set to PRO if using Quartus Pro tools: Arria 10 and Stratix 10
QUARTUS_EDITION="PRO"

# Set OPENCL or HLS variables to true. Set one or the other as issues might arise if you set both to TRUE. These only work with the PRO version.
#OPENCL="TRUE"
HLS="TRUE"
if [ ! ${TOOL_VERSION+defined} ]; then TOOL_VERSION=19.3; fi
# location where all development tools will get installed
TOOLS_ROOT=/glob/development-tools/versions
if [ "$QUARTUS_EDITION" = "LITE" ]; then
   ACDS_ROOT="$TOOLS_ROOT/intelFPGA_lite/$TOOL_VERSION"
   MODELSIM_ROOT="$ACDS_ROOT/modelsim_ase/linuxaloem"
else
   ACDS_ROOT="$TOOLS_ROOT/intelFPGA_pro/$TOOL_VERSION"
   MODELSIM_ROOT="$ACDS_ROOT/modelsim_ae/linuxaloem"
fi

QUARTUS_ROOT="$ACDS_ROOT/quartus"
HLS_ROOT="$TOOLS_ROOT/intelFPGA_pro/$TOOL_VERSION/hls"
OPENCL_ROOT="$TOOLS_ROOT/intelFPGA_pro/$TOOL_VERSION/hld"
PATH="$ACDS_ROOT/quartus/bin:$ACDS_ROOT/nios2eds/bin:$MODELSIM_ROOT:.:$PATH"

export PATH=$PATH
export QUARTUS_ROOTDIR=$QUARTUS_ROOT
export QUARTUS_HOME=$QUARTUS_ROOT
export LM_LICENSE_FILE=/usr/local/licenseserver/quartus.lic
export ALTERAD_LICENSE=/usr/local/licenseserver/quartus.lic

if [ "$OPENCL" == "TRUE" ] && [ "$QUARTUS_EDITION" == "PRO" ]; then
   source $OPENCL_ROOT/init_opencl.sh
fi

if [ "$HLS" == "TRUE" ] && [ "$QUARTUS_EDITION" == "PRO" ]; then
   source $HLS_ROOT/init_hls.sh
fi

 

I still receive the exact same error though.

0 Kudos
JEYANTHKRI_N_Intel
1,800 Views

Hi ,

We will contact the admin team and get back to you .

0 Kudos
PSath2
Beginner
1,800 Views

There was a suggestion in the other thread (https://forums.intel.com/s/question/0D70P000006XMQCSA4) from another Intel representative to try setting 
ACL_ACDS_VERSION_OVERRIDE=17.1.1 as they assert that the pac_a10 BSP is frozen at 17.1.1. (The board_env.xml on the DevCloud in /opt/a10 has 17.1).

Doing due diligence to note that the suggestion was tried, but didn't change the output when tried on s001-n137.

0 Kudos
JEYANTHKRI_N_Intel
1,800 Views

Hi ,

Looking at your queue name, it looks like you have gained access to FPGA Intel AI DevCloud (https://software.intel.com/en-us/devcloud/FPGA ). Please check "get help" link under the section "Support" .
This is where you will get support for the mentioned issues. Or more specififcally, please post your queries in https://forums.intel.com/s/topic/0TO0P0000001AUUWA2/intel-high-level-design?language=en_US
Please note that even if the issues is related to the DevCloud setup, any questions on FPGA DevCloud will be answered in the afore-mentioned forum only 
 

0 Kudos
Reply