Arria 10 Dev Kit, adjust OpenCL BSP a10soc2ddr to use only the FPGA Memory..

TLeng2 · ‎08-02-2019

I have a A10 Board with 2GB Memory for the HPS and 4GB Memory attached to the emif side of the FPGA, I want to get the Opencl system up and running without the shared memory stuff. ( boardtest.cl for verification )

For the Opencl code we want to use we need more than the 512 MB shared memory.

I looked into the PCIe examples for OpenCL which uses the FPGA Memory and also the a10soc example which doesnt use the FPGA external memory only shared, and the a10soc 2DDR which uses both of it. I struggle with connecting the correct DMA and also i am unsure if the sdram bridge is needed or not. In my understanding it works over the lwH2F AXI 32 bit as command and control and the H2F AXI 128 bit for Data transfer.

As i see the two DRAM Bridges are for Interfacing to the HPS external memory.

So i removed the Dual Port Splitter and the HPS Mem Pipeline bridge

Attached is the Block diagram of the a10soc 2ddr

I try to understand how the system works.

The System compiled and also linux is booting fine, Driver is adjusted to the propper Register numbers, Driverwise it loaded correctly and recognizes it correctly, but at device open ( seems to use dma for that ) i get kernel panic. (see attaached screenshots, yes timestamp is real, it was 2 AM :-) )

Screenshot description.

# modprobe aclsoc_drv.ko

Aug 2 02:49:36 armhf kernel: [ 272.867413] aclsoc_drv: loading out-of-tree module taints kernel.

Aug 2 02:49:36 armhf kernel: [ 272.874533] aclsoc_init (612):

Aug 2 02:49:36 armhf kernel: [ 272.874540] ----------------------------

Aug 2 02:49:36 armhf kernel: [ 272.874543]

Aug 2 02:49:36 armhf kernel: [ 272.874546] aclsoc_init (613):

Aug 2 02:49:36 armhf kernel: [ 272.874549] Driver version: 17.0.d41d8cd98f00b204e9800998ecf8427e

Aug 2 02:49:36 armhf kernel: [ 272.874551]

Aug 2 02:49:36 armhf kernel: [ 272.874669] init_chrdev (118):

Aug 2 02:49:36 armhf kernel: [ 272.874674] aclsoc = 249:0

Aug 2 02:49:36 armhf kernel: [ 272.874677]

Aug 2 02:49:36 armhf kernel: [ 272.878330] mapped region 0 (lw) to [f0be0000, f0c00000]. Size = 131072

Aug 2 02:49:36 armhf kernel: [ 272.878342] mapped region 2 (lw) to [f0b80000, f0bc0000]. Size = 262144

Aug 2 02:49:36 armhf kernel: [ 272.878400] init_irq (328):

Aug 2 02:49:36 armhf kernel: [ 272.878405] Virtualized IRQ (r_irq->start) requested: 18

Aug 2 02:49:36 armhf kernel: [ 272.878407]

Aug 2 02:49:36 armhf kernel: [ 272.878410] init_irq (335):

Aug 2 02:49:36 armhf kernel: [ 272.878414] Successfully requested IRQ #18, result 0

Aug 2 02:49:36 armhf kernel: [ 272.878416]

Aug 2 02:49:36 armhf kernel: [ 272.880400] cma: cma_alloc(cma c0b88b10, count 1, align 0)

Aug 2 02:49:36 armhf kernel: [ 272.880625] cma: cma_alloc(): returned efbf9980

Aug 2 02:49:36 armhf kernel: [ 272.881039] aclsoc_init (616):

Aug 2 02:49:36 armhf kernel: [ 272.881045] end aclsoc_init

Aug 2 02:49:36 armhf kernel: [ 272.881048]

# aocl diagnose

Aug 2 02:51:23 armhf kernel: [ 379.409993] aclsoc_open (150):

Aug 2 02:51:23 armhf kernel: [ 379.410005] aclsoc = ee15f800, pid = 1356 (diagnose)

Aug 2 02:51:23 armhf kernel: [ 379.410010]

Aug 2 02:51:23 armhf kernel: [ 379.410107] Unhandled fault: imprecise external abort (0x406) at 0xb5cd57b8

Aug 2 02:51:23 armhf kernel: [ 379.417044] pgd = edbbc000

Aug 2 02:51:23 armhf kernel: [ 379.419739] [b5cd57b8] *pgd=5fdbf831

Aug 2 02:51:23 armhf kernel: [ 379.423315] Internal error: : 406 [#1] SMP ARM

Aug 2 02:51:23 armhf kernel: [ 379.427739] Modules linked in: aclsoc_drv(O)

Aug 2 02:51:23 armhf kernel: [ 379.432021] CPU: 1 PID: 1356 Comm: diagnose Tainted: G O 4.9.78-ltsi-dirty #1

Aug 2 02:51:23 armhf kernel: [ 379.440246] Hardware name: Altera SOCFPGA Arria10

Aug 2 02:51:23 armhf kernel: [ 379.444928] task: ee2a6300 task.stack: ed56a000

Aug 2 02:51:23 armhf kernel: [ 379.449449] PC is at aclsoc_rw+0x1ac/0x590 [aclsoc_drv]

Aug 2 02:51:23 armhf kernel: [ 379.454651] LR is at 0x1285e

Aug 2 02:51:23 armhf kernel: [ 379.457521] pc : [<bf001610>] lr : [<0001285e>] psr: 200f0013

Aug 2 02:51:23 armhf kernel: [ 379.457521] sp : ed56be38 ip : c0b02d00 fp : ed56bec4

Aug 2 02:51:23 armhf kernel: [ 379.468944] r10: 00000001 r9 : 00000002 r8 : ed4c0000

Aug 2 02:51:23 armhf kernel: [ 379.474145] r7 : 00000008 r6 : 00000000 r5 : f0b84020 r4 : f0b84020

Aug 2 02:51:23 armhf kernel: [ 379.480642] r3 : d3792c3c r2 : c0b02d40 r1 : 00000001 r0 : 00001f03

Aug 2 02:51:23 armhf kernel: [ 379.487141] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none

Aug 2 02:51:23 armhf kernel: [ 379.494242] Control: 10c5387d Table: 2dbbc04a DAC: 00000051

Aug 2 02:51:23 armhf kernel: [ 379.499961] Process diagnose (pid: 1356, stack limit = 0xed56a218)

So it panics.

I think i have some error in my HW System.

Here are the settings in the driver, with the corrected register mappings due the fact that the alt_pr and the pipe_stage_alt_pr are not adresses with 0x0007, they by default have 0x003F per default and i tried everything to change it back to 0x0007 but it was not possible from qsys. so i needed to change the addressing slightly.

changes to original registers:

The clock_crossing_host is now at 0xCFB0 to 0xCFFF

Version_ID at 0xCFB0 to 0xCFB3

Temperature is at 0xCFB8 to 0xCFBF

The rest is still the same.

Clock crossing bridge 0x0000 ... 0x03FF

Kernel IF 0x4000 ... 0x5FFF

Kernel Clock gen 0xB000 ... 0xBFFF

Adrr Span expander 0xC870 ... 0xC877

PR_Base_ID 0xCF80 ... 0xCF83

And the IRQ are also same as original> 0xCF90...0xCF93, 0xCFA0...0xCFA3

Its weird a little bit....

Any help is apreciated

Thank You

Thomas

Douglas_P_Intel · ‎09-04-2019

Hi Thomas,

OpenCL on the Arria 10 SoC board can be very challenging. Have you gotten OpenCL working on the board with the existing BSP and boardtest.cl? (does aocl diagnose pass?) If not, I recommend starting there. You said you want to get up and running “without the shared memory stuff”. The shared memory allows some data movement between the FPGA and the HPS to be eliminated and I'm not sure if it's required by the driver. Exactly what configuration are you looking for? Remember that if you change the memory configuration, you may need to change the board_spec.xml file.

Douglas_P_Intel · ‎09-04-2019

I'm assuming that you are using this document to make your changes. https://www.intel.com/content/www/us/en/programmable/documentation/bba1475671648402.html#ska1506192907363

TLeng2 · ‎09-05-2019

Thank You for the Answer,

I worked the way through all these documents.

The Goal is to have the OpenCL working in the FPGA with the connected 4GB FPGA Memory, and the HPS working with its connected 2GB Memory. I made the proper changes in the Files and i got it up and running... unfortunately even without shared memory the Driver accesses only the host memory at the test which is weird. Also the aocl Program doesnt work with the actual linux kernels because the bridges are enumerated differently now. ( see my question about updated drivers and libraries. ) so it is compiling but partially working........

Douglas_P_Intel · ‎09-05-2019

TLeng2 · ‎09-05-2019

I already removed these Items in the BSP.

Compile works also system boot works.

but driver still accesses a maybe shared memory... and reprogram doesnt work either

Douglas_P_Intel · ‎09-05-2019

Did you delete all the components in red? (You have to modify the HPS component to remove the SDRAM0 and SDRAM1 bridges)

Did you delete the following from the board_spec.xml file?

This statement creates the AVMM Master on the left. The clock bridge on the left should have a port kernel_mem1.

Make sure the rebuild the drivers AND the utilities (like diagnose) after making hardware changes.

TLeng2 · ‎09-05-2019

Hello,

I have done the recomended changes before i made the question. I can send You the Data to look at.

Greetings Thomas

Hazlina_R_Intel · ‎11-14-2019

Hi,

My name is Hazlina and I am the Senior Manager within Intel applications engineering team. I have received an escalation for your case. Can you please let me know what are the outstanding questions pertaining to this? Is the previous issue still persists even after the recommended changes is applied? Please provide the info and I will get the engineer to respond to this inquiry.

TLeng2 · ‎11-16-2019

I write a new Topic, the issue persists but changed a bit. It is a Driver issue.