Community
cancel
Showing results for 
Search instead for 
Did you mean: 
OHarb1
Novice
138 Views

Crash when programming .sof file

I saw this crash while programming a .sof file on Ubuntu:

 

Has anyone else run into this?

 

*** Fatal Error: Segment Violation at 0x22f700000076

Module: quartus_pgm

Stack Trace:

0xf8ea: rml::internal::internalPoolFree(rml::internal::MemoryPool*, void*) + 0x4a (tbbmalloc.so.2)

0x1729: operator delete(void*) + 0x9 (tbbmalloc_proxy.so.2)

0x2083e: S4_SII_BIT::~S4_SII_BIT() + 0x3e (pgm_build_npcf)

0x20901: S4_SII::~S4_SII() + 0x5b (pgm_build_npcf)

0x209de: NPCF_DEVICE_INFO::set_s4_s2_list(std::vector<S4_SII*, std::allocator<S4_SII*> >*, bool, bool) + 0x4c (pgm_build_npcf)

0x24655: NPCF_DEVICE_INFO::~NPCF_DEVICE_INFO() + 0x169 (pgm_build_npcf)

0x246e7: NPCF_INFO::~NPCF_INFO() + 0x41 (pgm_build_npcf)

0xe183c: FBGEN_GEN::~FBGEN_GEN() + 0x5c (pgm_fbgen)

0xe1e19: FBGEN_GEN::~FBGEN_GEN() + 0x9 (pgm_fbgen)

0x420be3: PGMIO_FBGEN_PROXY::~PGMIO_FBGEN_PROXY() + 0x13 (pgm_pgmio)

0x2c2d5c: PGMIO_F2P::create_bitstream(PGM_CHAIN_ELEMENT*, std::vector<std::string, std::allocator<std::string> >*, PGMIO_CCF*) + 0x1dc (pgm_pgmio)

0x284c14: PGM_CHAIN_ELEMENT::generate_bv_list(bool) + 0x104 (pgm_pgmio)

0x28868d: PGM_CHAIN_ELEMENT::create_chain_element(PGM_CHAIN_ELEMENT*, bool, FIO_PATH*, bool, PGMIO_CONFIG_SCHEME, bool, bool) + 0xd2d (pgm_pgmio)

0x232d9: PGME_PROGRAMMER::lookup_device(PGM_CHAIN_ELEMENT*, PGMIO_CONFIG_SCHEME, bool, bool, bool) + 0x29 (pgm_pgme)

0x2175d: QPGM_FRAMEWORK::create_element(std::string, std::string, unsigned int, unsigned int) + 0x601 (quartus_pgm)

0x23b91: QPGM_FRAMEWORK::process_operation(std::string*) + 0x1e93 (quartus_pgm)

0x24cde: QPGM_FRAMEWORK::post_check_arguments() + 0x2d6 (quartus_pgm)

0x1c08f: qexe_standard_main(QEXE_FRAMEWORK*, QEXE_OPTION_DEFINITION const**, int, char const**) + 0x1bc (comp_qexe)

0x1fd97: qpgm_main(int, char const**) + 0x5e (quartus_pgm)

0x40720: msg_main_thread(void*) + 0x10 (ccl_msg)

0x602c: thr_final_wrapper + 0xc (ccl_thr)

0x407df: msg_thread_wrapper(void* (*)(void*), void*) + 0x62 (ccl_msg)

0xa559: mem_thread_wrapper(void* (*)(void*), void*) + 0x99 (ccl_mem)

0x8f92: err_thread_wrapper(void* (*)(void*), void*) + 0x27 (ccl_err)

0x63f2: thr_thread_wrapper + 0x15 (ccl_thr)

0x427e2: msg_exe_main(int, char const**, int (*)(int, char const**)) + 0xa3 (ccl_msg)

0x1fe21: main + 0x26 (quartus_pgm)

0x270b3: __libc_start_main + 0xf3 (c.so.6)

 

0 Kudos
9 Replies
JohnT_Intel
Employee
77 Views

Hi,

 

May I know which device you are programming? Have you tried on other system?

OHarb1
Novice
77 Views

This is an Arria 10 device.

 

We've set up an automated test-server (to test incoming pull request) and we're seeing this failure occasionally on that server.

 

That this is not stable is problematic as it reduces the throughput of that test-server.

JohnT_Intel
Employee
77 Views

Hi,

 

May I know if you performed programming at the same time or you are running other thing? From the error message, the issue might be facing some memory issue which cause the Quartus Programmer to crash.

OHarb1
Novice
77 Views

Nothing else was running at the time. This is an automated script, so I know nothing else is going on on that machine. It is a Jenkins slave that's dedicated to regression teseting our FPGA bit file. The Arria 10 PC is a PCIe card in that machine and we're programming it using a loopback USB cable(we can't use CvP as we need to change the clock frequency of the design).

JohnT_Intel
Employee
77 Views

Hi,

 

What do you mean by loopback USB cable? Could you provide me more detail information on your script and how to duplicate the issue? May I know what is the probability of the issue occurring?

OHarb1
Novice
77 Views

I need more data on the frequency, but I guesstimate the chances of failure to be 1:10.

 

The loopback cable means that the the Arria 10 PCIe eval card is in the PC and that I have connected the USB Blaster of that eval card to the same PC.

 

The command I used to program the FPGA is the below. The second time I ran it, using the same .sof file, just 30 minutes afterwards without any other change that I'm aware of, it worked.

 

Hmm.... come to think of it... We've upgraded to Quartus 20.1 elsewhere, but the niso2-configure-sof command is still from Quartus 17.3 (as I recall).

 

# program FPGA

nios2-configure-sof top.sof

 

The output is:

 

Searching for SOF file:

in .

top.sof

 

 

*** Fatal Error: Segment Violation at 0x22f700000076

Module: quartus_pgm

Stack Trace:

0xf8ea: rml::internal::internalPoolFree(rml::internal::MemoryPool*, void*) + 0x4a (tbbmalloc.so.2)

0x1729: operator delete(void*) + 0x9 (tbbmalloc_proxy.so.2)

0x2083e: S4_SII_BIT::~S4_SII_BIT() + 0x3e (pgm_build_npcf)

0x20901: S4_SII::~S4_SII() + 0x5b (pgm_build_npcf)

0x209de: NPCF_DEVICE_INFO::set_s4_s2_list(std::vector<S4_SII*, std::allocator<S4_SII*> >*, bool, bool) + 0x4c (pgm_build_npcf)

0x24655: NPCF_DEVICE_INFO::~NPCF_DEVICE_INFO() + 0x169 (pgm_build_npcf)

0x246e7: NPCF_INFO::~NPCF_INFO() + 0x41 (pgm_build_npcf)

0xe183c: FBGEN_GEN::~FBGEN_GEN() + 0x5c (pgm_fbgen)

0xe1e19: FBGEN_GEN::~FBGEN_GEN() + 0x9 (pgm_fbgen)

0x420be3: PGMIO_FBGEN_PROXY::~PGMIO_FBGEN_PROXY() + 0x13 (pgm_pgmio)

0x2c2d5c: PGMIO_F2P::create_bitstream(PGM_CHAIN_ELEMENT*, std::vector<std::string, std::allocator<std::string> >*, PGMIO_CCF*) + 0x1dc (pgm_pgmio)

0x284c14: PGM_CHAIN_ELEMENT::generate_bv_list(bool) + 0x104 (pgm_pgmio)

0x28868d: PGM_CHAIN_ELEMENT::create_chain_element(PGM_CHAIN_ELEMENT*, bool, FIO_PATH*, bool, PGMIO_CONFIG_SCHEME, bool, bool) + 0xd2d (pgm_pgmio)

0x232d9: PGME_PROGRAMMER::lookup_device(PGM_CHAIN_ELEMENT*, PGMIO_CONFIG_SCHEME, bool, bool, bool) + 0x29 (pgm_pgme)

0x2175d: QPGM_FRAMEWORK::create_element(std::string, std::string, unsigned int, unsigned int) + 0x601 (quartus_pgm)

0x23b91: QPGM_FRAMEWORK::process_operation(std::string*) + 0x1e93 (quartus_pgm)

0x24cde: QPGM_FRAMEWORK::post_check_arguments() + 0x2d6 (quartus_pgm)

0x1c08f: qexe_standard_main(QEXE_FRAMEWORK*, QEXE_OPTION_DEFINITION const**, int, char const**) + 0x1bc (comp_qexe)

0x1fd97: qpgm_main(int, char const**) + 0x5e (quartus_pgm)

0x40720: msg_main_thread(void*) + 0x10 (ccl_msg)

0x602c: thr_final_wrapper + 0xc (ccl_thr)

0x407df: msg_thread_wrapper(void* (*)(void*), void*) + 0x62 (ccl_msg)

0xa559: mem_thread_wrapper(void* (*)(void*), void*) + 0x99 (ccl_mem)

0x8f92: err_thread_wrapper(void* (*)(void*), void*) + 0x27 (ccl_err)

0x63f2: thr_thread_wrapper + 0x15 (ccl_thr)

0x427e2: msg_exe_main(int, char const**, int (*)(int, char const**)) + 0xa3 (ccl_msg)

0x1fe21: main + 0x26 (quartus_pgm)

0x270b3: __libc_start_main + 0xf3 (c.so.6)

 

 

End-trace

 

Failed to program top.sof, erasing possibly corrupt top.sof

 

JohnT_Intel
Employee
77 Views

Hi,

 

Is it possible to change your script to utilize quartus_pgm rather than nios2-configure-sof? you can use the command "quartus_pgm -c 1 top.sof". Please refer to https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/manual/TclScriptRefMnl.pdf for more information.

OHarb1
Novice
77 Views

I'll see if I can do that. A good suggestion. Though... if you look at the stack trace, you'll find that the crash is in the quartus_pgm process, so nios2-configure-sof must be launching quartus_pgm.

 

I'll also upgrade to the latest quartus_pgm.

JohnT_Intel
Employee
77 Views

Sure. Let me know if you are still facing the issue

Reply