FPGA Cyclone V cold boot -> crash / Soft reset ok => please help!

[qwitza]

Hello,

we are experiencing an issue with some of our boards that use the Cyclone V device.
On certain boards, the system crashes during a cold boot (first boot).

However, after performing a soft reset, everything seems to work as expected.

I can reliably reproduce the crash by booting into U-Boot and entering the following commands:

fpgaload=ext4load mmc 0:2 ${loadaddr} ${fpgafile}; run checkfpgafw;
checkfpgafw=if test ${filesize} -le 0; then echo cant load fpga firmare $fpgafile;else run fpgaloadandenable;fi;
fpgaloadandenable=fpga load 0 ${loadaddr} ${filesize};echo firmware $fpgafile written to fpga;bridge enable; echo bridges enabled;


mw 0xFFC25080 0x0
run fpgaload
mw 0xFFC2505C 0xA
mw 0xFFC25080 0xFFFF
mw 0xff200000 w

After an reboot the u-boot cmds are working, without an issue.

The issue is also in linux at the first cold boot, after accessing the bridge linux crash.

After the first cold boot into linux:

./devmem2 0xff200000 w
/dev/mem opened. 

After an soft reset

./devmem2 0xff200000 w
/dev/mem opened.
Memory mapped at address 0xb6f17000.
Value at address 0xFF200000 (0xb6f17000): 0x62656320

Can please someone help me, or assist howto debug?

thanks a lot

[qwitza]

hello,

here are some more information:

bridge disable
ext4load mmc 0:2 ${loadaddr} ${fpgafile}
fpga load 0 ${loadaddr} ${filesize}
md.l 0xFF706000 1
7007204 bytes read in 343 ms (19.5 MiB/s)
ff706000: 00000004
bridge enable
mw 0xff200000 0x1
<INTERRUPT> (i press ctrl+c, because i thougt it crashed, but it takes a long time)
mw 0xff200000 0x1

works, i test more but sometimes (cold boot) it took a very long time. after an soft boot its much faster

[qwitza]

Hello,

i try it again, and this time u-boot crash, no interrupt nothing.

i test it this way

bridge disable;ext4load mmc 0:2 ${loadaddr} ${fpgafile};fpga load 0 ${loadaddr} ${filesize};bridge enable;
mw 0xff200000 0x00
-> dead, crash

[khtan]

Hi qwitza,

Sorry for the delay, with your detailed description, from what I see this is mainly on the cold boot side and not warm reset, thus I suspect the FPGA fabric or the clock side/clock are not fully stable or still in reset state when bridge enable command is given. 

to quickly test:

bridge disable
ext4load mmc 0:2 ${loadaddr} ${fpgafile}
fpga load 0 ${loadaddr} ${filesize}

//just to check whether really is timing racing condition issue, for now just delay 5sec, when you mentioned wait long time , how long roughly the time, some few seconds long? just for testing wait 5 seconds or more just to really drill down to timing condition

sleep 5                                        

bridge enable
mw 0xff200000 0x1

When enabling bridges after cold reset, the recommended sequence is:

Wait for FPGA configuration done (nCONFIG/nSTATUS, or FPGA Manager state machine in U‑Boot/Linux)
Release FPGA–HPS interface resets via Reset Manager
Enable bridge control bits
Access FPGA peripherals

For warm reset, some of the reset sequencing are not needed thus in this case , why warm reset no issues but cold reset intermittently having issue. Is this issue occur on a particular board, or multiple or all boards that you have?

Thanks

Regards

Kian