EMIF link project error

SSilu · ‎02-27-2020

I encounter following error when try to link project to device. I enabled debug toolkit in emif ip. Quartus 19.3.

link_project_to_device -device_name {10AX115H1(.|E2|ES)|10AX115H2|..@1#1-2} -hardware_name {USB-Blaster on localhost (1-2)} -sof_file {/home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof}
Linking device 10AX115H1(.|E2|ES)|10AX115H2|..@1#1-2 on hardware USB-Blaster on localhost (1-2) using .sof file /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof.
Error occurred while running the System Console command design_load {/home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof}. System Console returned the result java.util.concurrent.ExecutionException: java.lang.Exception: design_load: Design with the same hash 720A09D3DA446F22D79 has been previously loaded at /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof
    invoked from within
"design_load /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof
"
    invoked from within
"interp eval $slave {
design_load /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof
 
}". You must shutdown the toolkit and restart.
ERROR: An error occurred while linking the project to the device.

sstrell · ‎02-27-2020

When you close and restart the toolkit, as it instructs, do you keep seeing the same issue?

#iwork4intel

Deshi_Intel · ‎02-28-2020

Hi,

Your issue looks similar like below known issue.

Due to a problem in the Intel® Quartus® Prime Pro Edition Software version 19.3, you may see a design link error in the EMIF Toolkit with the following message:

"An error occured while linking the project to the device"

Once a programming .sof file is linked through the "Link Project to Device" step in the EMIF Toolkit, and then unlinked through the "Unlink Project from Device" step, the same programming .sof file can't be re-linked through the "Link Project to Device" step again. If this is required, you will need to close the EMIF Toolkit and then re-open it.

Can you confirm and try out the workaround which is similar to Steve's suggestion as well.

The other possibility is to reduce JTAG frequency from default 16MHZ to maybe 16MHz or even 6MHz

Also you can try to create EMIF example design with just one EMIF interface to test out EMIF toolkit to help rule out is it Quartus design issue or board JTAG issue.

Thanks.

Regards,

dlim

SSilu · ‎02-28-2020

Tried toolkit restart, machine reboot, FPGA reprogram - nothing helped.

I also encountered new error:

link_project_to_device -device_name {10AX115H1(.|E2|ES)|10AX115H2|..@1#1-2} -hardware_name {USB-Blaster on localhost (1-2)} -sof_file {/home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof}
Linking device 10AX115H1(.|E2|ES)|10AX115H2|..@1#1-2 on hardware USB-Blaster on localhost (1-2) using .sof file /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof.
Error occurred while running the System Console command design_load {/home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof}. System Console returned the result java.util.concurrent.ExecutionException: java.lang.Exception: Filesystem is not ready
    invoked from within
"design_load /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof
"
    invoked from within
"interp eval $slave {
design_load /home/slawek/sofs/sdnbox_pcb_tester_ddr3.sof
 
}". You must shutdown the toolkit and restart.

File system is not ready exception seems like toolkit internal error? I'm running on SMP Debian 4.19.98-1. I can try on Ubuntu 18.04 if it might help.

Indeed JTAG connection might be unstable, but Transceiver Toolkit was working fine. I'm already using 6M as Terasic programmer does not support higher frequency (changing frequency neither). I will try to run EMIF toolkit on Terasic eval board to isolate problem.

Please find below output from jtagconfig -d:

slawek@sdnbox-3:~$ jtagconfig -d
1) USB-Blaster [1-2]
   (JTAG Server Version 19.3.0 Build 222 09/23/2019 SC Pro Edition)
  02E660DD   10AX115H1(.|E2|ES)/10AX115H2/.. (IR=10)
    Design hash    0720A09D3DA446F22D79
    + Node 0C206E00  JTAG PHY #0
    + Node 0C206E01  JTAG PHY #1
 
  Captured DR after reset = (02E660DD) [32]
  Captured IR after reset = (155) [10]
  Captured Bypass after reset = (0) [1]
  Captured Bypass chain = (0) [1]
  JTAG clock speed 6 MHz

Can I have both Transceiver toolkit and Emif toolkit active in the same sof? Or this might be source of problems?

EDIT: Ubuntu 18 also reports Filesystem is not ready exception.

Deshi_Intel · ‎03-02-2020

Hi,

Pls don't launch both transceiver toolkit and EMIF toolkit to avoid JTAG connection contention issue.

Just do it one at a time.

Also, have you tried out EMIF example design with just one memory interface in your whole Quartus Project to help isolate issue ?

Thanks.

Regards,

dlim

SSilu · ‎03-02-2020

I've tried with minimal project running on Terasic board and I managed to run emif toolkit. However I have another problem. Toolkit shows calibration status as failed, despite local_cal_success output from emif ip is in high state. Please see calibration report attached.

It also crashes somtimes with error:

Problem Details
Error:
 
*** Fatal Error: Segment Violation: faulting address=(nil), PC=0x7eff6e4f8eba : 0x7eff6e4f8eba: sld_emitt!EMITT_NF_EMIF_131_COMMAND_EXECUTOR_RECALIB::execute_command(EMITT_HARDWARE_DRIVER*, EMITT_CONNECTION*, std::d
Module: quartus_sh
Stack Trace:
     0xf935: ERR_UNWINDER_BACKTRACE::get_stack_trace(void const**, int, int, void*) + 0x101 (ccl_err)
    0x73360: msg_ie_get_call_stack(void*) + 0xfb (ccl_msg)
    0x74863: MSG_INTERNAL_ERROR::report_fatal(char const*, void*) + 0x3f (ccl_msg)
    0x118ed: err_report_fatal_exception(char const*, void*) + 0x71 (ccl_err)
    0x1244e: err_sigaction_handler + 0x149 (ccl_err)
    0x12890: (pthread)
   0x269eba: EMITT_NF_EMIF_131_COMMAND_EXECUTOR_RECALIB::execute_command(EMITT_HARDWARE_DRIVER*, EMITT_CONNECTION*, std::deque<unsigned int, std::allocator<unsigned int> > const*) + 0xc0 (sld_emitt)
    0xd0fa0: emitt_execute_connection_command + 0x6f3 (sld_emitt)
    0x4c942: TclNRRunCallbacks + 0x42 (tcl8.6)
    0x4de7b: TclEvalEx + 0x68b (tcl8.6)
    0xf3f0e: Tcl_FSEvalFileEx + 0x25e (tcl8.6)
    0xf3ffe: Tcl_EvalFile + 0x2e (tcl8.6)
    0x14916: qexe_evaluate_tcl_script(std::string const&) + 0x44c (comp_qexe)
    0x19a1c: qexe_do_tcl(QEXE_FRAMEWORK*, std::string const&, std::string const&, std::list<std::string, std::allocator<std::string> > const&, bool, bool) + 0x417 (comp_qexe)
    0x1d504: qexe_standard_main(QEXE_FRAMEWORK*, QEXE_OPTION_DEFINITION const**, int, char const**) + 0x506 (comp_qexe)
   0x402a5f: qsh_main(int, char const**) + 0x69 (quartus_sh)
    0x3ef00: msg_main_thread(void*) + 0x10 (ccl_msg)
    0x41114: msg_thread_wrapper(void* (*)(void*), void*) + 0x6e (ccl_msg)
    0x11f0c: mem_thread_wrapper(void* (*)(void*), void*) + 0x5c (ccl_mem)
     0xc728: err_thread_wrapper(void* (*)(void*), void*) + 0x27 (ccl_err)
     0x6d85: thr_thread_wrapper + 0x15 (ccl_thr)
    0x41c90: msg_exe_main(int, char const**, int (*)(int, char const**)) + 0x148 (ccl_msg)
   0x409d3b: main + 0x26 (quartus_sh)
    0x21b97: __libc_start_main + 0xe7 (c)
   0x402599: (quartus_sh)
 
End-trace
 
 
Executable: quartus
Comment:
None
 
System Information
Platform: linux64
OS name: Ubuntu 18.04.3
OS version: 18
 
Quartus Prime Information
Address bits: 64
Version: 19.3.0
Build: 222
Edition: Pro Edition

Deshi_Intel · ‎03-03-2020

HI,

It's good to know you manage to get toolkit running now after switching to another board.

Seems like toolkit is still not stable and crashing from time to time.

My advice would be :

Upgrade to Quartus v19.4 if possible. I worry v19.3 may not be that stable since there is known issue that I shared with you earlier
Also have you try out using EMIF example design as I suggested earlier ?
Lastly, feel free to check out attached EMIF calibration debug checklist to help you isolate to potential calibration failure root cause.

Thanks.

Regards,

dlim

SSilu · ‎03-03-2020

Hi I even managed to run emif toolkit on my custom board. One DDR controller at a time. It seems daisy chaining was the issue. I will debug it later and I will try Quartus 19.4. For the time being I have problem with interpretation of my first failing calibration reports. Calibration failed at last stage 4 according to https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/hb/external-memory/emi_ip.pdf page 478. Why read and write margins are set to 0? Moreover PAR_IN is uncalibrated, how could it reache stage 4 without PAR_IN working? Shouldn't it report failure at stage 1? Please see comparison with identical slot and sodimm where PAR_IN and ADD_14-16 are calibrated.

Deshi_Intel · ‎03-04-2020

HI,

Yes, I can see something odd with your ddr_b cal report.

add[16:14] and par_in pins result are uncalibrated
Did you see this weird result consistently over multiple ddr_b calibration (with either reseting DDR_b or after power cycle your your board) ?

I suggest you to look into below debug option first

Have you compare the Quartus design and DDR4 IP setting between ddr_a and ddr_b ? I can help you review as well if you can share with me your Quartus design QAR file
Also, is there any on board connection difference between ddr_a connectino vs ddr_b connection ? Again, I can help you review if you can share with me you board schematic pdf file

Thanks.

Regards,

dlim

Deshi_Intel · ‎04-16-2020

HI,

I don't hear back from you after my feedback explanation.

Hopefully you are doing well with your issue debug.

For now, I am setting this case to closure.

Thanks.

Regards,

dlim