FPGA to HPS bridge on Agilex 7

[Mikhail_a]

Hello 

I'm trying to read and write from and to HPS SDRAM on Agilex 7 board

I have this configuration of fpga2hps interface:

On the other side I have two msgdma ip cores for MM to Stream and Stream to MM transfers. But MM to Stream DMA doesnt work as its busy bit is permanently set to 1. As I understood from reading the forum it can be related to fpga2sdram bridge which is not initialized right.

So my question is:

1) how to make sure my  fpga2sdram bridge is in the right condition

2) what can be wrong with the MM to Stream DMA?

DMA settings:

FPGA to HPS AXI bus state:

[EBERLAZARE_I_Intel]

Hi,

For the mSGDMA IP core, you can refer to the Embedded Peripherals IP UG:

https://www.intel.com/content/www/us/en/docs/programmable/683130/23-1/modular-scatter-gather-dma-core.html

For the bridge setup, please refer here for the full documentation on what you can do:

https://www.rocketboards.org/foswiki/Projects/SettingUpAndUsingBridgesOnAgilex

If you have further questions on the setup, let me know.

[EBERLAZARE_I_Intel]

Hi,

Do you have any further questions?

[Mikhail_a]

Hello! 

Thank you for your answer. 

I've been able to run my fpga design, and it seems there was something wrong with u-boot.

The problem now is that when I run my board with Linux booted from SD card fpga design doesn't work and the system hangs when I try to read or write to FPGA memory space. But somehow when I make U-boot not able to read FPGA bitstream from the SD card and do reboot everything works completely fine. I use HPS boot first config, so my bitstream is located on SD card. To summarize the process:

1. Boot system as usual, FPGA will be programmed.

2. go to /run/media/mmcblk0p1/ directory where FPGA bitstream is located

3. rename fpga bitstream to any name, just  to make sure u-boot won't find it

4. reboot system and you will see that U-Boot wasn't able to program FPGA

Failed to load 'fpga_bitstream.rbf'
FPGA not ready. Bridge reset aborted!

5. wait until system boot is over, login

Now after these steps I can use FPGA.

For building SD card image I've used this link
https://www.rocketboards.org/foswiki/Documentation/BuildingBootloaderAgilex7?erpm_id=12888127_ts1687163227736

It seems something is wrong after FPGA programming and it can be fixed by something U-Boot does before programming FPGA. I've tried to add more 

bridge enable

or 

bridge disable

commands to CONFIG_BOOTCOMMAND but that didn't help. 

I use the latest commit of socfpga_v2022.10 branch in u-boot-socfpga repository.

[EBERLAZARE_I_Intel]

Hi,

Thanks for the info, I may need some time to digest it and I will get back to you.

Just to be sure again, you can do read/write to FPGA in Linux environment, if you do not configure the FPGA in U-boot in prior boot stage?

[Mikhail_a]

Yes, but at this moment FPGA should be already configured in the previous boot.

[Mikhail_a]

One more question has just recently arised. 

When I try to read from HPS SDRAM I get RRESP == 3. The address I use for read operation is provided by Linux driver (physical address like 0x2A377D802) and it is supposed to point to SDRAM region.

So my question is: do I need to somehow change read/write address before read or write through fpga2sdram bridge?

[EBERLAZARE_I_Intel]

Hi,

On the new questions, do you have a screenshot for that? I would like to take a look.

[Mikhail_a]

Hello

Unfortunately no, but it is a typical read transaction with RRESP == 3 and RDATA is set to all zeros.

[Mikhail_a]

I've set "Interface destination" option to CCU and got everything work alright with addresses Linux gives me. But for the final design I don't need my SDRAM access transactions to go through CCU. So it is ok as a temporary workaround but I still need to be able to access SDRAM directly.

[EBERLAZARE_I_Intel]

Hi,

Just checking again, if you do normal boot without interruption to Linux, it will boot just fine?

My guess would be the bridge must be disable again in Uboot before loading the Kernel and Linux.

[Mikhail_a]

Hello!

Sorry but I didn't really understand your question. What is interruption to Linux? 

[EBERLAZARE_I_Intel]

Hi,

Meaning to say, you would just let it boot upon power cycle and do not do anything in Uboot environment etc.

Also, want to check are you using the ATF? Arm Trusted Firmware.

[Mikhail_a]

Hello. 

Yes I don't do anything in Uboot. And actually I use default Uboot cmd from rocketboards page (link is above)

CONFIG_BOOTCOMMAND="load mmc 0:1 \${loadaddr} fpga_bitstream.rbf; bridge disable;fpga load 0 \${loadaddr} \${filesize};bridge enable;run mmcload;run linux_qspi_enable;run rsu_status;run mmcboot"

 And yes I use ATF which also was compiled in exactly the same manner as on rocketboards page

[EBERLAZARE_I_Intel]

Hi,

Apologies for the late responds, in the doc I provided last time, it mentioned at the most bottom page, we do need to disable all bridges in U-boot if we have enabled them manually, before booting to the kernel, there is an issue with the RBO server currently, you may access it another time and confirm:

https://www.rocketboards.org/foswiki/Projects/SettingUpAndUsingBridgesOnAgilex

I guess that demonstrate the behavior you're seeing? Do you see any other issue if you manually disable the bridges, every time your enabled it in U-boot?

[Mikhail_a]

Hello. 

Unfortunately I didn't find any document you told you provided before. Could you please send it one more time?

I tried to disable bridges in U-boot cmd console but without any success. Could you please provide the exact algorithm that can solve that issue?

[EBERLAZARE_I_Intel]

Hi,

The link is currently down:

https://www.rocketboards.org/foswiki/Projects/SettingUpAndUsingBridgesOnAgilex

There is something else I need to re-check, let me get back to you.

[EBERLAZARE_I_Intel]

Hi,

The rocketboards link is still down, also I found this KDB that might have relation with your issue:

https://www.intel.com/content/www/us/en/support/programmable/articles/000094603.html

Can you try to apply the git patch and see if that would fix your issue without renaming/removing the .rbf to be configured?

[Artavazd]

Hello!

Thank you for your answer. Unfortunately this article handles the problem with FPGA2HPS but I have a problem with HPS2FPGA. Anyways I will update ATF to the latest version and try it.

[EBERLAZARE_I_Intel]

Hi,

I believe the initial issue was the FPGA2HPS bridge, could you open a new thread regarding the new issue you're facing and we will work on the new thread on your new issue.

[Artavazd]

Hello,

You are right. Lets finish this thread. Thank you.