Programmable Devices
CPLDs, FPGAs, SoC FPGAs, Configuration, and Transceivers
20818 Discussions

Arria 10 GX: 64 bit atomic access to Avalon Bus through PCIe?

MehmetFide
Novice
1,166 Views

Hi,

 

We have an external processor Arm Cortex A53 with Linux 5.4 aarch64 connected to an Altera Arria 10 GX FPGA through a PCIe bus.

 

On the FPGA side, we have an Avalon bus, all addresses are set as 64bit. All implementation is done using Quartus 22.4 and QSys.

 

And I have no problem to access to the avalon bus though PCIe (DMA works, MSI interrupt works) on the Linux side as long as I use 8/16 or 32 bit accesses. Whenever I use 64 bit access for example using ioread64(), it returns 0xffffffffffffffff and I observe that PCIe Config page on the FPGA side gets corrupted. After that I have to restart FPGA to make it functional again.

 

My original question is here:

https://stackoverflow.com/questions/75097169/any-known-issue-with-ioread64-iowrite64-on-a-pcie-bus

 

Any idea that if Avalon bus and PCIe hard core in the Arria 10 GX together supports 64 bit io read and write operations?

 

Thank you.

 

0 Kudos
11 Replies
wchiah
Employee
1,116 Views

Hi,

 

Can I know which design example that you are using ?
AVST ? AVMM ? MCDMA ?

Regards,

Wincent_Intel

0 Kudos
MehmetFide
Novice
1,104 Views

Hello,

 

Our design is based on Arria 10 - AN690: PCIe Gen3x8 AVMM DMA ( here )

 

Regards,

Fide.

 

0 Kudos
wchiah
Employee
1,083 Views

Hi,


I am not familiar with the platform and code you are using.

Is that feature inside Quartus ? or it is from the driver ?


Why you need to change the IOread ?


Regards,

Wincent_Intel


0 Kudos
wchiah
Employee
1,056 Views

Hi,

 

I wish to follow up with you about this case.

Do you have any further questions on this matter ?

​​​​​​​Else I would like to have your permission to close this forum ticket

 

Regards,

Wincent_Intel


0 Kudos
MehmetFide
Novice
1,045 Views

Hi @wchiah,

 

All BAR registers definition and implementations are inside the Quartus.

When we try to access the memory area in FPGA from Linux, there are couple of options like Altera doing it here 

 

Those are

ioread8 -> Works fine

ioread16 -> Works fine

ioread32 -> Works fine

ioread64 -> Doesn't work and causes FPGA PCIe to be crashed

Avalon DMA -> Works fine

 

I'm trying to understand what is special with 64 bit direct access to make it non functional.

 

Thank you.

 

0 Kudos
wchiah
Employee
1,037 Views

Hi,


If you just want to use PCIe IP that interact with memory directly, then pls consider checking out below example design that use (PCIe) only, no DMA


Regards,

Wei Chuan


0 Kudos
wchiah
Employee
1,019 Views

Hi,

 

I wish to follow up with you about this case.

Do you have any further questions on this matter ?

​​​​​​​Else I would like to have your permission to close this forum ticket

 

Regards,

Wincent_Intel


0 Kudos
MehmetFide
Novice
1,007 Views

Hi @wchiah,

 

As you suggested, we have tested it with both internal RAM and external DDR ram without DMA.

The problem exists at both conditions. To be more specific, accessing bars as 8/16/32 bit data work fine. But 64 bit accesses (especially read attempt) corrupts the PCIe config page in the FPGA and it fully stops functioning.  It only recovers when we reconfigure the FPGA.

 

My question is that anyone on Intel ever tested the PCIe hard core in the Arria 10 GX to do 64 bit data access?

 

Thank you.

0 Kudos
wchiah
Employee
1,003 Views

Hi,

I check previous record, we had tested and it able to run.
I lay down some suggestion so that we can narrow down this further.

 

The issue might be caused by data packing limitation in interconnect on AXI/Avalon boundary. If your AXI is 32bit wide and PCIe Avalon interface is 64bit wide. In such situation multiple two 32bit transactions will be used to implement 64bit transfer.

If you want to further analyze, Singal TAP traces showing transactions on PCIe Avalon interface and HPS AXI should show what is going on.

 

Please try

  1.  can you please try disabling the "Enable Reordering" option (under the Controller tab)?
  2. At the same time, please try running the Calibrate Termination command in the EMIF Toolkit (Tools menu -> System Debugging Tools -> External Memory Interface Toolkit)  to verify the optimal ODT settings.

Regards,

Wincent_Intel

0 Kudos
wchiah
Employee
990 Views

Hi,

 

I wish to follow up with you about this case.

Do you have any further questions on this matter ?


Regards,

Wincent_Intel


0 Kudos
wchiah
Employee
968 Views

Hi

 

We have not hear from you and this Case is idling. It is not recommended to idle for too long.

Therefore following our support policy, I have to put this case in close status. My apologies if any inconvenience cause

Hence, This thread will be transitioned to community support.

If you have a new question, feel free to open a new thread to get support from Intel experts.

Otherwise, the community users will continue to help you on this thread. Thank you

If you feel your support experience was less than a 9 or 10,

please allow me to correct it before closing or let me know the cause so that I may improve your future support experience.

 

Regards,

Wincent_Intel


0 Kudos
Reply