Error when generating hardware from .aoco file

[Altera_Forum]

I am trying to synthesize a kernel for a nallatech 385 board (pcie385n_d5) using the OpenCL toolchain 13.0. I am using the two step approach with the first step generating the .aoco file and the next one compiling it into a .aocx executable.  

 For the kernel I am trying to synthesize, I keep getting an error in the second step, i.e. going from .aoco to .aocx. I have attached an excerpt from from the log after the first reported error below. Also, the FPGA utilization is <40%, so I think this can't be caused by the design's size (if that's possibility)  Any clue what might be the cause here? The closest information I could find was an issue with QuartusII 12.1 toolchain for Cyclone devices [Cannot post the link to it due to forum restrictions]. Have you faced a similar issue?  Excerpt from the log:   --- Quote Start ---  Internal Error: Sub-system: DYGR, File: /quartus/ddb/dygr/dygr_place_info_body.cpp, Line: 2918 node_index >= 0 && node_index < static_cast<int>(m_place_block_list.size()) Stack Trace:  0xe3d9f: DYGR_PLACE_INFO_BODY::get_place_block(unsigned int) const + 0x9f (ddb_dygr)  0xeb5b3: DYGR_DIE_INFO_BODY::get_location(unsigned int, DEV_LOCATION&) const + 0xf3 (ddb_dygr)  0x95c1b: QTK_RCF_UTIL_IMPL::cache_constrained_iterms() + 0x146d (db_qtk)  0x96d91: QTK_RCF_UTIL_IMPL::QTK_RCF_UTIL_IMPL(CDB_ATOM_NETLIST*, RCF_INTERFACE_WRAPPER*, bool) + 0x11f (db_qtk)  0x96e47: QTK_RCF_UTIL::QTK_RCF_UTIL(CDB_ATOM_NETLIST*, RCF_INTERFACE_WRAPPER*, bool) + 0x55 (db_qtk)  0x7f700: FITCC_QIC_UTILITY::get_rcf_util(bool) + 0xd0 (fitter_fitcc)  0x1264d8: FITCC_INCREMENTAL_UTILITY_IMPL::create_postfit_rcf() + 0x118 (fitter_fitcc)  0x4c68a: FSV_EXPERT::fitter_preparation_post_fpp(bool) + 0x98a (fitter_fsv)  0x4da3f: FSV_EXPERT::fitter_preparation() + 0x5f (fitter_fsv)  0x404c9: FSV_EXPERT_BASE::fitter_preparation() const + 0x6e9 (fitter_fsv)  0x42597: FSV_EXPERT_BASE::invoke_fitter() const + 0x847 (fitter_fsv)  0x3cdee: fsv_execute + 0x28e (fitter_fsv)  0x2c8b2: fmain_start(CMP_FACADE*) + 0x562 (fitter_fmain)  0x1a640: qfit_execute_fit(QCU_FRAMEWORK*, QFIT_FRAMEWORK*) + 0x150 (comp_qfit_legacy_flow)  0x15680: QFIT_FRAMEWORK::execute() + 0x2a0 (comp_qfit_legacy_flow)  0x24f81: qfit_legacy_flow_run_legacy_fitter_flow + 0x1a1 (comp_qfit_legacy_flow)  0x2e8f6: TclInvokeStringCommand + 0x76 (tcl8.5)  0x32b1e: TclEvalObjvInternal + 0x2be (tcl8.5)  0x34310: TclEvalEx + 0x4f0 (tcl8.5)  0x34d13: TclEvalObjEx + 0x393 (tcl8.5)  0x3ac31: Tcl_EvalObjCmd + 0x91 (tcl8.5)  0x32b1e: TclEvalObjvInternal + 0x2be (tcl8.5)  0x73abf: TclExecuteByteCode + 0x151f (tcl8.5)  0xb5bc7: TclObjInterpProcCore + 0x107 (tcl8.5)  0x32b1e: TclEvalObjvInternal + 0x2be (tcl8.5)  0x73abf: TclExecuteByteCode + 0x151f (tcl8.5)  0xb5bc7: TclObjInterpProcCore + 0x107 (tcl8.5)  0x32b1e: TclEvalObjvInternal + 0x2be (tcl8.5)  0x34310: TclEvalEx + 0x4f0 (tcl8.5)  0x98c70: Tcl_FSEvalFileEx + 0x230 (tcl8.5)  0x98d6e: Tcl_EvalFile + 0x2e (tcl8.5)  0x101a2: qexe_evaluate_tcl_script(char const*) + 0x43b (comp_qexe)  0x13c5f: qexe_do_tcl(QEXE_FRAMEWORK*, char const*, char const*, _Dinkum_std::list<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> >, MEM_STL_ALLOCATOR<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> > > > const&, bool, bool) + 0x4e9 (comp_qexe)  0x14ab8: qexe_run_tcl_option(QEXE_FRAMEWORK*, char const*, _Dinkum_std::list<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> >, MEM_STL_ALLOCATOR<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> > > >*, bool) + 0x51d (comp_qexe)  0x37756: qcu_run_tcl_option(QCU_FRAMEWORK*, char const*, _Dinkum_std::list<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> >, MEM_STL_ALLOCATOR<_Dinkum_std::basic_string<char, _Dinkum_std::char_traits<char>, MEM_STL_ALLOCATOR<char> > > >*, bool) + 0x7b1 (comp_qcu)  0x17aaf: qexe_standard_main(QEXE_FRAMEWORK*, QEXE_OPTION_DEFINITION const**, int, char const**) + 0x403 (comp_qexe)  0x98b9: qfit2_main(int, char const**) + 0xa9 (quartus_fit)  0x361a0: msg_main_thread(void*) + 0x10 (ccl_msg)  0x7fac: thr_final_wrapper + 0xc (ccl_thr)  0x36d45: msg_thread_wrapper(void* (*)(void*), void*) + 0x5b (ccl_msg)  0x19135: mem_thread_wrapper(void* (*)(void*), void*) + 0xc5 (quartus_fit)  0xf408: err_thread_wrapper(void* (*)(void*), void*) + 0x27 (ccl_err)  0x838c: thr_thread_wrapper + 0x15 (ccl_thr)  0x48f37: msg_exe_main(int, char const**, int (*)(int, char const**)) + 0x96 (ccl_msg)  0x1ecdd: __libc_start_main + 0xfd (c.so.6)  0x9149: __gxx_personality_v0 + 0x309 (quartus_fit)   End-trace   Error: Flow compile (for project /home/kgeorgen/ocl/mreduce1/mreduce1/top) was not successful Error: ERROR: Error(s) found while running an executable. See report file(s) for error message(s). Message log indicates which executable was run last.   --- Quote End ---  

[Altera_Forum]

Were you an early access customer by any chance (i.e. using the tools before the 13.0 release)? I've seen this sort of thing happen when a version of Quartus other than 13.0 is being referenced by QUARTUS_ROOTDIR. If you are an early adapter I would try out 'which quartus' and 'which aoc' to make sure your enviornment is pointing at just to make sure older versions of the tools are not getting called.

[Altera_Forum]

Thanks for your reply, BadOmen.  

No, I was not an early access customer. Only the 13.0 release of the toolset was ever installed on this machine.

[Altera_Forum]

Try opening Quartus (just type 'quartus' at the command line) and navigate to the 'Help' menu option and select 'About Quartus'. It should say 13.0.0 Build 156. 

 Do any kernels compile correct? You could try compiling the vectorAdd or moving_average examples to see if it might be an issue with the particular kernel you are trying to compile. Those two designs compile relatively quickly.

[Altera_Forum]

--- Quote Start ---  Try opening Quartus (just type 'quartus' at the command line) and navigate to the 'Help' menu option and select 'About Quartus'. It should say 13.0.0 Build 156.  Do any kernels compile correct? You could try compiling the vectorAdd or moving_average examples to see if it might be an issue with the particular kernel you are trying to compile. Those two designs compile relatively quickly.  --- Quote End ---    That's a good point you brought up, BadOmen. Yes, in the past we have compiled a few kernels successfully, but it was not using my account. Hence, I tried compiling the moving_average this weekend and it succeeded. Afterwards, I repeated compiling my initial kernel, starting fresh from just the .cl file--previously, I kept using the same directory and didn't clean it completely before starting a new compilation. Anyway, this time, aoc never stopped compiling, even after waiting for more than 36 hours. quartus_fit was the process that kept running without terminating.   I suppose, the initial error I posted at the beginning of this tread was a temporal one, which can be rectified with a clean start. Nonetheless, I am still not able to compile my kernel. By the way, I am using an AMD llano machine (4 cores) with 16 GB of RAM. I am aware that this is below the recommended spec from Altera, of 24GB of RAM, but can it have such a profound effect? Please let me know if you have some idea. Meanwhile, we are trying again, and this time, we will be checking the pageout rate before killing the process; I'll post any new information we find.   Thanks again.