how to use separated the FPGA configuration and HPS booting?

[Rainwang]

As below figure shows, we use the 10AS016 chip, and we have 2 EPCQ flashes on our board, one for HPS booting and the other for FPGA configuration. We stored the FPGA configure file in the FPGA side EPCQ because we want to use the “remote system update” feature in our application to manage the factory image and app image.

With this structure, I think the separated FPGA configuration and HPS booting is suitable for us. But because we need to use DDR for our HPS, per the manual, If the hard memory controller or shared I/O are required by the HPS, then the FPGA fabric and I/O (FPGA, shared and hard memory controller) configuration must be complete before the HPS can boot. But I don’t know how to control the HPS booting start only after the FPGA configuration done, I checked several documents but found no information for this, so can you help to tell me how to do with this?

Thanks in advance.

[Kenny_Tan]

Kindly note that there will be some slowness on the first reply due to the public holiday, we will get back to you as soon as possible.

[EBERLAZARE_I_Intel]

Hi,

Maybe we can clear some of the info from your request, so you would want the HPS to boot first? Then do some sort of RSU to configure the FPGA?

[Rainwang]

hi, 

I am sorry for my late response due to my business travel with limited web access. as i mentioned, we have 2 EPCQ flashes on our board, one locates on the HPS side for HPS booting and the other locates on FPGA side for FPGA configuration.

From the document, i know the RSU IP can provide the remote upgrade feature of FPGA image and also provide the management of factory image and app image. Per my understanding, the RSU can only manage the FPGA side EPCQ device.

So, to use the RSU to manage our FPGA image, I store the HPS files such as uboot/OS/file system/app in the HPS side EPCQ, and store the FPGA image in the FPGA side EPCQ.

with this structure,  i need to use the "seperated HPS boot and FPGA configuration". I set the FPGA configuration as AS mode,  after power up, the FPGA will be configured from EPCQ, and then after the configuration done, the HPS can start to boot.

My question is, how to control the HPS booting wait untill the FPGA configuration completed? there shall have a hand-shake mechanism but i don't know. shall i connect the FPGA_config_done signal to HPS_RST pin ?

thanks 

[EBERLAZARE_I_Intel]

Hi,

Can you check on my previous respond for clarification?

[EBERLAZARE_I_Intel]

Hi,

Thanks, I see now, I will need to check on my side and get back to you.

[EBERLAZARE_I_Intel]

Hi,

Apologies for the late respond, my reply didn't went through.

First of all just to be clear, the FPGA boot first is not fully supported in our latest U-boot and require manual changes and we cannot confirm if it would work in our latest U-boot. Below is the old design flow:

So for the old design, the FPGA first boot up, and HPS wait after full FPGA config. We have the settings and config in Quartus, the old design (to be use as reference) looks like below:

https://www.rocketboards.org/foswiki/Documentation/UBootA10FPGABoot

With the settings enabled, the FPGA will configure fully first with the FPGA config that have been set in Quartus, or by Quartus Programmer manually.

After full FPGA config is done, then HPS will boot from it's sourced that is selected in Quartus + BSEL pins (flash boot devices).

These should be the flow of it if you are to do FPGA first config with U-boot edits.

As mentioned, the FPGA boot first feature is not fully supported in our latest U-boot for A10.

So all the supported feature and documentation, we recommend users to follow from our official Rocketboards A10 documentation page:

https://www.rocketboards.org/foswiki/Documentation/Arria10SoCGSRD

https://www.rocketboards.org/foswiki/Documentation/BuildingBootloaderCycloneVAndArria10

p/s: If any answer from the community or Intel Support are helpful, please feel free to give best answer or rate 4/5 survey.

[Rainwang]

hi, 

Per page 689 of Intel Arria 10 Hard Processor System Technical Reference Manual, there are 3 options for  Booting and Configuration Options, as below:

• The HPS boot and FPGA configuration occur separately.
• The HPS boots first and then configures the FPGA.
• The HPS boots from the FPGA after the FPGA is configured.

What I want is the option 1, the HPS boot and FPGA configuration occur separately, not the option 3. I checked the link you gave and it seems related to the option 3. We currently realized the option 2, and in option 2 the FPGA image shall be stored in HPS side EPCQ flash, we can not use the RSU IP to manage it. but as i mentioned, i want to store the FPGA image into FPGA side EPCQ so that i can use RSU IP to manage our factory image and app image (and we also need to upgrade the app image in system, this feature can also be supported by RSU).

If currently u-boot version can not support option 1, we can change it. but my question is, in option 1, even the booting and FPGA configuration occur separately, there is dependency between them. If the HPS booting runs to u-boot step and need to use the DDR, but at this time the FPGA configuration have not been done and the DDR have not been brought up, then the HPS booting shall be fail, right? To avoid this, how to control the HPS booting start after the FPGA configration done? Shall we need to connect the FPGA_init_done signal to HPS_RST pin so that the HPS booting can be triggered again after the FPGA configuation completed? Or if you have any example or document for option 1, please share to me.

Very appriciated.

[Rainwang]

will you please check my latest question and help to answer it? thanks.

[EBERLAZARE_I_Intel]

Hi,

I will need some time to look into it, I will get back to you soon.

[EBERLAZARE_I_Intel]

Hi,

So for your case, you would want the HPS to boot right after the FPGA have been configured, meaning there is a case where the FPGA have to be reset/power cycle?

If for separate boot sequence, the HPS can be a loop state in which it will wait for the FPGA to be configured before it will continue boot to SSBL/U-boot for e.g. So wherever your FPGA is to be configured, Quartus Programmer/HPS flash etc, the HPS will wait for it, only then boot to SSBL/U-boot. Just for this case the HPS will always be running and always wait for the FPGA to be configured and will show the UART terminal log of the "failure of the FPGA not configured" etc.

I believe I have test this on the latest U-boot, no much change needed nor any signal to be set/control, but Early IO Release feature must be disabled in Quartus, is this close to what you're trying to achieve?

[EBERLAZARE_I_Intel]

Hi,

Any update?

[Rainwang]

yes, when the FPGA is powered or reset, i want the HPS boot right after the FPGA have been configured, you are right.

From your description, we got below points.

1. Early IO release feature must be disabled in Quartus.

2. No other singal need to be set/control.

3. this feature is tested on latest U-boot.

these are the information we need.

and i have two more question,

1. you mentioned that the HPS can be in a loop to wait the FPGA configuration done, what step that the loop locates? in the boot ROM? or uboot-SPL?

2. we are now using the U-Boot 2021.04, is this version support the feature?

we will test it soon. 

thank you.

[EBERLAZARE_I_Intel]

Hi,

Yes, should be the Boot ROM as the settings set as Early IO release feature being disabled, it will need and keep checking for a Full FPGA config before booting the U-boot SPL, as the A10 does not have preloader.

Yes, we did tried that version and it was performing of what I mentioned previously.

Disclaimer, this feature is not fully tested, so you will need to use it at your own risk, anyway we may be able to give best support on any issue you'd have.

[Rainwang]

hi, 

today we tested this idea on our system, with the same SPL as HPS booting first then configuration FPGA, but we removed the periph.rbf from the DTB package of SPL. and we programed the FPGA configuration file(the early IO release is disabled) into the EPCQ on FPGA side, and config the FPGA with AS mode.

aftet power up, below information showed. 

it seems the booting didn't wait in the boot ROM, it enterred into the SPL. 

1. the SPL found FPGA is not configured, then it try to configure the FPGA, but we removed the periph.rbf from the DTB so it can not find the rbf file.

2. after FPGA AS configuration completed, as i mentioned, i connected the FPGA_INIT_DONE to the HPS_RST, then it trigger a cold reset, the HPS reset again, this time it found that the FPGA is configured and it go ahead to calibrate the DDR, run u-boot.......all OK.

3. from above info, it seems no wait loop in the boot ROM for waiting the FPGA config done. it still jump into the SPL step.

4. if 3 is right, I think the next step is, we need to add wait loop in SPL to wait for FPGA config done, with this, we need not connect the FPGA_INIT_DONE to the HPS_RST to trigger a cold reset after FPGA config completed.

thank you

[EBERLAZARE_I_Intel]

Hi,

Can you quick clarify the steps to disabled the Early IO Release?:

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an-a10-soc-fpga-early-io-release.pdf

[Rainwang]

i have no questions on early IO release, I unchecked this option in both HPS EMIF setting of platform designer and the device and pin options of Quartus, as i showed in below images with red highlighted.

my question is just to double confirm with you, per my test, the waiting loop is not in boot ROM and also not in SPL. So for my application, I need to add a wait loop in SPL by myself to let it wait until the FPGA config done, then the booting will jump out from the wait loop and go ahead and we need not trigger the HPS reset again after FPGA config done.

thanks.

[EBERLAZARE_I_Intel]

Hi,

Okay, can you see the below logs, below logs shown that the first two initial logs where it needs an FPGA configuration done before booting the SPL, after the two initial logs, the FPGA is programmed successfully using Quartus Programmer, you can see the U-boot SPL loading and go into U-boot user mode, did you see this behavior? Does this fits your design requirement?:

U-Boot SPL 2022.04-21230-gfcf317324c-dirty (Oct 20 2022 - 05:12:31 +0800)

FPGA: Checking FPGA configuration setting ...

FPGA: Checking FPGA configuration setting ...

FPGA: Checking FPGA configuration setting ...

WDT:  Started watchdog@ffd00300 with servicing (10s timeout)

Trying to boot from SPI

QSPI: Invalid input arguments txlen 64

SPL: failed to boot from all boot devices

### ERROR ### Please RESET the board ###

U-Boot SPL 2022.04-21230-gfcf317324c-dirty (Oct 20 2022 - 05:12:31 +0800)

FPGA: Checking FPGA configuration setting ...

FPGA: Checking FPGA configuration setting ...

FPGA: Checking FPGA configuration setting ...

WDT:  Started watchdog@ffd00300 with servicing (10s timeout)

Trying to boot from SPI

QSPI: Invalid input arguments txlen 64

SPL: failed to boot from all boot devices

### ERROR ### Please RESET the board ###

U-Boot SPL 2022.04-21230-gfcf317324c-dirty (Oct 20 2022 - 05:12:31 +0800)

U-Boot SPL 2022.04-21230-gfcf317324c-dirty (Oct 20 2022 - 05:12:31 +0800)

DDRCAL: Success

DDRCAL: Scrubbing ECC RAM (1024 MiB).

DDRCAL: SDRAM-ECC initialized success with 146 ms

WDT:  Started watchdog@ffd00300 with servicing (10s timeout)

Trying to boot from SPI

U-Boot 2022.04-21230-gfcf317324c-dirty (Oct 20 2022 - 05:12:31 +0800)socfpga_arria10

CPU:  Altera SoCFPGA Arria 10

BOOT: QSPI Flash (1.8V)

Model: Altera SOCFPGA Arria 10

DRAM: 1 GiB

Core: 69 devices, 16 uclasses, devicetree: separate

WDT:  Started watchdog@ffd00300 with servicing (10s timeout)

Loading Environment from SPIFlash... SF: Detected mt25qu01g with page size 256 Bytes, erase size 4 KiB, total 128 MiB

*** Warning - bad CRC, using default environment

In:  serial

Out:  serial

Err:  serial

Model: Altera SOCFPGA Arria 10

Net:

Warning: ethernet@ff800000 (eth0) using random MAC address - 52:6c:4d:8d:5c:15

eth0: ethernet@ff800000

Hit any key to stop autoboot: 0

=>

[Rainwang]

yes, we received similar information as yours, i attached mine as below.

per our understanding, the Boot ROM print nothing,  from the first line of printed message, the process has already entered into the SPL step.

and the content of first line also indicates the U-Boot SPL version, i don't think the Boot ROM knows this.

you can see the bold and italic message, which we added into SPL code to print more reference information for us, it was printed from the 2nd line.

U-Boot SPL 2021.04 (May 24 2023 - 11:38:08 +0800)
In early user mode!
FPGA: Checking FPGA configuration setting ...
not In mode!
2nd time not in user mode
FPGA: Checking FPGA configuration setting ...
WDT:   Started with servicing (10s timeout)
Trying to boot from SPI
QSPI: Invalid input arguments txlen 64
SPL: failed to boot from all boot devices
### ERROR ### Please RESET the board ###

U-Boot SPL 2021.04 (May 24 2023 - 11:38:08 +0800)
In user mode!
not In mode!
regular boot not valid

U-Boot SPL 2021.04 (May 24 2023 - 11:38:08 +0800)
In user mode!
not In mode!
regular boot valid
DDRCAL: Success
WDT:   Started with servicing (10s timeout)
Trying to boot from SPI

U-Boot 2021.04 (May 23 2023 - 18:00:07 +0800)socfpga_arria10

CPU:   Altera SoCFPGA Arria 10
BOOT:  QSPI Flash (1.8V)
Model: SOCFPGA Arria10 Dev Kit
DRAM:  1 GiB
WDT:   Started with servicing (10s timeout)
Loading Environment from SPIFlash... SF: Detected n25q00a with page size 256 Bytes, erase size 4 KiB, total 128 MiB
OK
In:    serial
Out:   serial
Err:   serial
Model: SOCFPGA Arria10 Dev Kit

[EBERLAZARE_I_Intel]

Hi,

Okay lets go back to your final question, which you mentioned that you "need not reset the HPS again after FPGA config is done" do you mean this on first time boot or at any time after the FPGA config/re-config will be initiate?

Again, the feature for boot HPS from FPGA is not really supported in our latest U-boot. Additionally, it takes time to program the FPGA first, as the FPGA need to be configured first, so that's one of the main cons on doing boot HPS from FPGA. We don't really recommend it.

If you need to have the HPS state the way it is and have interchangeable FPGA file, you can just do the normal booting for A10 QSPI, then do a TFTP in U-boot mode to just update/change the FPGA.itb file, plus you may more flexibility in U-boot:

https://www.rocketboards.org/foswiki/Documentation/BuildingBootloaderCycloneVAndArria10#Arria_10_SoC_45_Boot_from_QSPI

A10 Boot User Guide on Boot ROM:

https://www.intel.com/content/www/us/en/docs/programmable/683735/current/boot-rom-flow.html

[EBERLAZARE_I_Intel]

Hi,

Just checking if you have further questions.