Programmable Devices
CPLDs, FPGAs, SoC FPGAs, Configuration, and Transceivers
20947 Discussions

Intel Devstack-OpenCL unable to detect PAC Cards

Saikiranbelana
Novice
‎02-07-2023 10:03 PM
1,917 Views

Intel Development Acceleration Stack for OpenCL is not detecting the Arria 10 PAC Cards on our server. It's looking for PAC drivers which cannot be found at the OpenCL_BSP installation path.

The Cards can be detected using `lspci` and `OPAE drivers` but not with `aocl diagnose` command:

 

System Specs:

OS: ubuntu - 18.04

 

The following is the log:

 

server2@ubuntu:~$ sudo fpgainfo fme
Board Management Controller, microcontroller FW version 26895
Last Power Down Cause: POK_CORE
Last Reset Cause: None
//****** FME ******//
Object Id                     : 0xEE00001
PCIe s:b:d:f                  : 0000:AF:00:0
Device Id                     : 0x09C4
Socket Id                     : 0x00
Ports Num                     : 01
Bitstream Id                  : 0x124000200000367
Bitstream Version             : 1.2.4
Pr Interface Id               : 38d782e3-b612-5343-b934-2433e348ac4c
Boot Page                     : user
Board Management Controller, microcontroller FW version 26895
Last Power Down Cause: POK_CORE
Last Reset Cause: None
//****** FME ******//
Object Id                     : 0xEE00000
PCIe s:b:d:f                  : 0000:86:00:0
Device Id                     : 0x09C4
Socket Id                     : 0x00
Ports Num                     : 01
Bitstream Id                  : 0x124000200000367
Bitstream Version             : 1.2.4
Pr Interface Id               : 38d782e3-b612-5343-b934-2433e348ac4c
Boot Page                     : user                    
server2@ubuntu:~$ source ~/inteldevstack/init_env.sh 
export QUARTUS_HOME=/opt/intelFPGA_pro/quartus_19.2.0b57/quartus
export OPAE_PLATFORM_ROOT=/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv
export AOCL_BOARD_PACKAGE_ROOT=/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp
source /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/libexec/setup_permissions.sh
$OPAE_PLATFORM_ROOT/bin is in PATH already
sudo cp /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/sw/afu_platform_info /usr/bin/
export INTELFPGAOCLSDKROOT=/opt/intelFPGA_pro/quartus_19.2.0b57/hld
export ALTERAOCLSDKROOT=/opt/intelFPGA_pro/quartus_19.2.0b57/hld
$QUARTUS_HOME/bin is in PATH already
source /opt/intelFPGA_pro/quartus_19.2.0b57/hld/init_opencl.sh
server2@ubuntu:~$ aocl diagnose
--------------------------------------------------------------------
ICD System Diagnostics                                              
--------------------------------------------------------------------

Using the following location for ICD installation: 
        /etc/OpenCL/vendors

Found 2 icd entry at that location:
        /etc/OpenCL/vendors/Altera.icd
        /etc/OpenCL/vendors/Intel_FPGA_SSG_Emulator.icd

the following OpenCL libraries are referenced in the icd files:
        libalteracl.so
        libintelocl_emu.so

checking LD_LIBRARY_PATH for registered libraries:
        libalteracl.so was registered on the system at /opt/intelFPGA_pro/quartus_19.2.0b57/hld/host/linux64/lib
        libintelocl_emu.so was registered on the system at /opt/intelFPGA_pro/quartus_19.2.0b57/hld/host/linux64/lib

Using the following location for fcd installations:
        /opt/Intel/OpenCL/Boards

Found 2 fcd entry at that location:
        /opt/Intel/OpenCL/Boards/a10_ref.fcd
        /opt/Intel/OpenCL/Boards/dcp_bsp.fcd

the following OpenCL libraries are referenced in the fcd files:
        /opt/intelFPGA_pro/quartus_19.2.0b57/hld/board/a10_ref/linux64/lib/libaltera_a10_ref_mmd.so
        /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/lib/libintel_opae_mmd.so

checking LD_LIBRARY_PATH for registered libraries:
        /opt/intelFPGA_pro/quartus_19.2.0b57/hld/board/a10_ref/linux64/lib/libaltera_a10_ref_mmd.so was registered on the system.
        /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/lib/libintel_opae_mmd.so was registered on the system.

Number of Platforms = 2 
        1. Intel(R) FPGA SDK for OpenCL(TM)                             | Intel(R) Corporation           | OpenCL 1.0 Intel(R) FPGA SDK for OpenCL(TM), Version 19.4
        2. Intel(R) FPGA Emulation Platform for OpenCL(TM)              | Intel(R) Corporation           | OpenCL 1.0 Intel(R) FPGA SDK for OpenCL(TM), Version 19.2
--------------------------------------------------------------------
ICD diagnostics PASSED                                              
--------------------------------------------------------------------
--------------------------------------------------------------------
BSP Diagnostics                                                     
--------------------------------------------------------------------
Error initializing DMA: no memory
Error initializing mmd dma
Error closing DMA
Error initializing bsp
Error initializing DMA: no memory
Error initializing mmd dma
Error closing DMA
Error initializing bsp
--------------------------------------------------------------------
Warning:
No devices attached for package:
/opt/intelFPGA_pro/quartus_19.2.0b57/hld/board/a10_ref
--------------------------------------------------------------------

Unable to find the kernel mode driver.

Please make sure you have properly installed the driver. To install the driver, run
      aocl install

DIAGNOSTIC_FAILED
--------------------------------------------------------------------
--------------------------------------------------------------------
Warning:
No devices attached for package:
/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp
--------------------------------------------------------------------
Error initializing DMA: no memory
Error initializing mmd dma
Error closing DMA
Error initializing bsp
Error initializing DMA: no memory
Error initializing mmd dma
Error closing DMA
Error initializing bsp
--------------------------------------------------------------------

Call "aocl diagnose <device-names>" to run diagnose for specified devices
Call "aocl diagnose all" to run diagnose for all devices

server2@ubuntu:~$ aocl install
Do you want to setup the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] y
aocl install: /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp is already registered in the system, please uninstall it before reinstalling

server2@ubuntu:~$

 

Please help us fix this @intel  @BoonBengT_Intel  @JohnT_Intel 

 

 

Thank you,

Saikiran

0 Kudos

Link Copied

16 Replies
JohnT_Intel
Employee
‎02-17-2023 02:04 AM
1,876 Views

Hi,


Can you try running "aocl uninstall" and "aocl install" to see if the issue is resolved?


0 Kudos
Copy link
Saikiranbelana
Novice
‎02-17-2023 03:45 AM
1,864 Views

When I try to reinstall, It says binary operator expected. Here's the log

 

server2@ubuntu:~$ aocl uninstall
Do you want to unset the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] Y
aocl uninstall: /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp is not installed, no need to uninstall.
server2@ubuntu:~$ aocl install
Do you want to setup the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] n
Please specify a directory to setup FCD:
/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/
aocl install: Setting up the FPGA Client Driver (FCD) to the system. This process may require admin privilege
Install the FCD file to /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp
aocl install: Adding the board package /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp to the list of installed packages
Installing the board package driver to the system.
aocl install: Running install from /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/libexec
/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/libexec/install: line 15: [: /dev/intel-fpga-port.0: binary operator expected
0 Kudos
Copy link
JohnT_Intel
Employee
‎02-19-2023 06:31 PM
1,831 Views

Hi,


Can you try to performed "lspci" and check if the driver is bind to the device? The reason is that when you are performing uninstall, the driver is not bind to the device.


0 Kudos
Copy link
Saikiranbelana
Novice
‎02-19-2023 09:26 PM
1,825 Views

As suggested, I have bound the device using `lspci`.  Below is the log.

server2@ubuntu:~$ sudo lspci -n -v -s 0000:af:00.0
af:00.0 1200: 8086:09c4
	Subsystem: 8086:0000
	Physical Slot: 5
	Flags: bus master, fast devsel, latency 0, IRQ 39, NUMA node 1
	Memory at dbffff00000 (64-bit, prefetchable) [size=512K]
	Memory at dbfffe00000 (64-bit, prefetchable) [size=1M]
	Capabilities: [68] MSI-X: Enable+ Count=7 Masked-
	Capabilities: [78] Power Management version 3
	Capabilities: [80] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [160] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [200] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [280] #19
	Kernel driver in use: intel-fpga-pci
	Kernel modules: intel_fpga_pci

server2@ubuntu:~$ sudo lspci -n -v -s 0000:86:00.0
86:00.0 1200: 8086:09c4
	Subsystem: 8086:0000
	Physical Slot: 7
	Flags: bus master, fast devsel, latency 0, IRQ 37, NUMA node 1
	Memory at d7ffff00000 (64-bit, prefetchable) [size=512K]
	Memory at d7fffe00000 (64-bit, prefetchable) [size=1M]
	Capabilities: [68] MSI-X: Enable+ Count=7 Masked-
	Capabilities: [78] Power Management version 3
	Capabilities: [80] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Capabilities: [160] Alternative Routing-ID Interpretation (ARI)
	Capabilities: [200] Single Root I/O Virtualization (SR-IOV)
	Capabilities: [280] #19
	Kernel driver in use: intel-fpga-pci
	Kernel modules: intel_fpga_pci

server2@ubuntu:~$ aocl install
Do you want to setup the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] Y
aocl install: Setting up the FPGA Client Driver (FCD) to the system. This process may require admin privilege
Install the FCD file to /opt/Intel/OpenCL/Boards
aocl install: Adding the board package /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp to the list of installed packages
Installing the board package driver to the system.
aocl install: Running install from /opt/intelFPGA_pro/quartus_19.2.0b57/hld/board/a10_ref/linux64/libexec
cp: cannot stat '/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/driver/*': No such file or directory
sh: 0: Can't open ./make_all.sh
aocl install: failed.
aocl install: Removing FCD
aocl install: Removing the board package /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp from the list of installed packages

 

For some reason, `aocl install` command is searching for the driver folder which is non-existent in the BSP.

 

I'm not sure how to diagnose this.

 

 

 

0 Kudos
Copy link
JohnT_Intel
Employee
‎02-20-2023 11:27 PM
1,810 Views

Hi,


Can you check your "'/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64"? I suspect that you have not un-archive or unzip the opencl BSP.


Thanks


0 Kudos
Copy link
JohnT_Intel
Employee
‎02-26-2023 11:29 PM
1,775 Views

Hi,


Have you check if the BSP is there?


0 Kudos
Copy link
Saikiranbelana
Novice
‎02-27-2023 01:24 AM
1,767 Views

Hi JohnT, Sorry for the delayed response.

The following are the available files in the linux64 folder

 

 

server2@ubuntu:~/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64$ tree
.
├── lib
│   └── libintel_opae_mmd.so
└── libexec
    ├── diagnose
    ├── flash
    ├── install
    ├── program
    ├── setup_permissions.sh
    ├── sign_aocx.sh
    └── uninstall

2 directories, 8 files

 

The Installer script extracts the opencl_bsp drivers.

0 Kudos
Copy link
JohnT_Intel
Employee
‎02-27-2023 01:29 AM
1,765 Views

Hi,

Have you setup your environment following https://www.intel.com/content/www/us/en/docs/programmable/683831/1-2-1/initializing-the-environment-for-with.html?


Thanks


0 Kudos
Copy link
Saikiranbelana
Novice
‎02-27-2023 01:40 AM
1,760 Views

yes, I have followed the same steps. The problem occurs the same.

 

The following is the output when I try to uninstall and reinstall the BSP:


server2@ubuntu:~$ aocl uninstall
Do you want to unset the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] Y
aocl uninstall: /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp is not installed, no need to uninstall.
server2@ubuntu:~$ aocl install
Do you want to setup the FCD at directory /opt/Intel/OpenCL/Boards [Y/n] n
Please specify a directory to setup FCD:
/home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/
aocl install: Setting up the FPGA Client Driver (FCD) to the system. This process may require admin privilege
Install the FCD file to /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp
aocl install: Adding the board package /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp to the list of installed packages
Installing the board package driver to the system.
aocl install: Running install from /home/server2/inteldevstack/a10_gx_pac_ias_1_2_1_pv/opencl/opencl_bsp/linux64/libexec
server2@ubuntu:~$ aocl list-devices
--------------------------------------------------------------------
Warning:
No devices attached for package:
/opt/intelFPGA_pro/quartus_19.2.0b57/hld/board/a10_ref
--------------------------------------------------------------------

 

 

0 Kudos
Copy link
JohnT_Intel
Employee
‎02-27-2023 04:13 PM
1,744 Views

Hi,


From the latest log, it looks like it is able to installed the driver. Can you double confirm using "lspci" to see if the driver is loaded into the FPGA card?


0 Kudos
Copy link
Saikiranbelana
Novice
‎02-28-2023 01:04 AM
1,728 Views

 

Hi @JohnT_Intel , Here's the output for lspci

server2@ubuntu:~$ lspci | grep acce
af:00.0 Processing accelerators: Intel Corporation Device 09c4

 

 

The FPGA is still available and only shows with lspci and fpgainfo. I think the problem is with the drivers. When I move the FPGA to a different system, It works there.

0 Kudos
Copy link
JohnT_Intel
Employee
‎02-28-2023 06:34 PM
1,718 Views

Hi,


Can you try "lspci -vv | grep acce" to get the kernel driver information.



0 Kudos
Copy link
Saikiranbelana
Novice
‎03-02-2023 03:56 AM
1,703 Views

Here's the output for `lspci -vv | grep acce`:

server2@ubuntu:~$ sudo lspci -vv | grep acce
af:00.0 Processing accelerators: Intel Corporation Device 09c4

 

for `lspci -vv` :

af:00.0 Processing accelerators: Intel Corporation Device 09c4
	Subsystem: Intel Corporation Device 0000
	Physical Slot: 5
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 39
	NUMA node: 1
	Region 0: Memory at dbffff00000 (64-bit, prefetchable) [size=512K]
	Region 2: Memory at dbfffe00000 (64-bit, prefetchable) [size=1M]
	Capabilities: [68] MSI-X: Enable+ Count=7 Masked-
		Vector table: BAR=0 offset=00009000
		PBA: BAR=0 offset=0000a000
	Capabilities: [78] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [80] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 4096 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x8, ASPM not supported, Exit Latency L0s <4us, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis-, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP- SDES+ TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
	Capabilities: [160 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [200 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 1, Total VFs: 1, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 1, stride: 1, Device ID: 09c5
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000dbfffd00000 (64-bit, prefetchable)
		Region 4: Memory at 00000dbffff80000 (64-bit, prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [280 v1] #19
	Kernel driver in use: intel-fpga-pci
	Kernel modules: intel_fpga_pci
0 Kudos
Copy link
JohnT_Intel
Employee
‎03-02-2023 05:17 PM
1,685 Views

Hi,


Thanks for providing the logfile. From the logfile, it looks like it automatically loaded with default PCIe driver. Please performed "modprobe -r intel-fpga-pci" then only performed "aocl install".


Thanks.


0 Kudos
Copy link
JohnT_Intel
Employee
‎03-14-2023 06:34 PM
1,565 Views

We do not receive any response from you to the previous question/reply/answer that I have provided. This thread will be transitioned to community support. If you have a new question, feel free to open a new thread to get the support from Intel experts. Otherwise, the community users will continue to help you on this thread. Thank you.



0 Kudos
Copy link
Saikiranbelana
Novice
‎04-17-2023 10:50 PM
1,494 Views
In response to JohnT_Intel

Hi,

 

Sorry for the delayed response.

 

I couldn't fix the error occurring. So, I had to reinstall the OS and install the inteldevstack on it. It works now.

 

Thanks, @JohnT_Intel  for the support.

In response to JohnT_Intel
0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.