EMIF Calibration Debug Toolkit "Firmware is not done running last command" error

[RichardWH]

We recently upgraded our Quartus Linux machine from Ubuntu version 18.04 to 20.04.

Now the EMIF Debug Toolkit returns the following errors when trying to re-calibrate. Used to work fine under Ubuntu 18.04

Firmware is not done running last command; command_status=0x00000000 (expecting 3)
Stack trace: tk_ie {Firmware is not done running last command; command_status=0x00000000 (expecting 3)}
An error occurred while running script "::emif_cal_dbg::calibration::rerun_cal_callback"

Steps to reproduce:

- Download bitstream with EMIF Debug Toolkit enabled using JTAG Blaster

- Quartus: Tools > System Debugging Tools > System Console

- Select and open the EMIF Calibration Debug Toolkit

- Activate selected interface

At this point the Calibration Report tab show that calibration was successful

- From Calibration tab click Run Re-Calibration

Above error is shown in messages

This happens with both Quartus 22.1 and 22.4. Running the Assert TG Reset (Run TG) works fine.

Interestingly when I run the same bitstream with Quartus 22.4 under Windows it will calibrate about 5 times successfully before throwing the above error.

Feels like some race problem with Quartus interacting with the OS.

Attached is the full message trace when going through the above steps.

[AdzimZM_Intel]

Hello RichardWH,

Thank you for submitting your question in Intel Community.

I am Adzim will assist you on this case.

Do you have more than one device on JTAG chain in this case?

If so try to reduce the number of devices and slightly increase the JTAG frequency.

Which device that you used?

Are you using your custom board or Intel development kit?

Regards,

Adzim

[RichardWH]

Hi Adzim,

There is only the one Stratix-10 device connected to the JTAG.

Quartus doesn't seem to allow me to increase the JTAG frequency beyond the default 24MHz to make it faster, I have tried with the "Auto-adjust frequency at chain scanning" on and off.

The device we are using is the Stratix 10 - 1SX280HU2F50E1VG.

The problem is with the device on our custom product which only has a single Stratix-10 device. We are attempting to test with the Quartus EMIF IP generated "ed_synth"  example design.

Richard

[RichardWH]

Hi Adzim,

Updating that it seems to be working now if I actually reduce the JTAG clock frequency from the default 24 MHz to 6 MHz (doesn't work with 16 MHz either). Revealing other issues but I'll leave that for another report when I have more information.

Richard

[AdzimZM_Intel]

Hi Richard,

Thank you for your update.

I will check with engineering team about the toolkit failure with firmware.

Just to confirm, you found the problem when using the EMIF IP example design right?

If yes, can you share the design file here?

Thank you.

Regards,

Adzim

[RichardWH]

Yes found the problem with the EMIF IP example - or at least a workaround. Works when the JTAG clock frequency is reduced to 6MHz.

Attached is the design that we were seeing the problem with.

Richard

[AdzimZM_Intel]

Hi Richard,

We have tried to replicate the issue with Stratix 10 SX Dev Kit but could not see similar issue.

We have already tested with both example design and your design.

Since the workaround is working, I will put this thread to community support.

Please login to 'https://supporttickets.intel.com', view details of the desire request, and post a feed/response within the next 15 days to allow me to continue to support you. After 15 days, this thread we be transitioned to community support. The community users will be able to help you on your follow-up questions.

Regards,

Adzim