Error (293007): Current module quartus_fit ended unexpectedly

[StephanBorucki]

I have been working with Quartus Prime for many years and I teach students how to program FPGAs with Quartus. We use the DE10-Lite board with a MAX10 but also a self-developed board with MAX10.  Recently we have had a serious problem.

On some of our computers we can no longer compile even simple designs. I have attached the example.

We get the following error message:

Error (293007): Current module quartus_fit ended unexpectedly. Verify that you have sufficient memory available to compile your design. ...

The error occurs on powerful, newer computers that have 32 Gbyte to 128 Gbyte RAM with Windows 10 and Windows 11. After that, we can no longer fit even the Introtutorial from Altera. We have uninstalled and reinstalled the software. The third time the error occurs again.

On older computers with only 8 Gbyte memory or 16 Gbyte memory we can compile exactly the same design more than 10 times without errors.

The error occurs with Quartus Prime Lite Edition 19.1.0 but also with the latest version 24.1.

We have searched all answers in the community but have not found a solution.

We urgently need a solution, otherwise we will no longer be able to use Altera at the university!

Here is a description of the computers on which the error occurs:

Dell Precision 5820, 128 Gbyte RAM, Windows 10 Education 22H2, Intel Xeon Processor

Dell Precision 7550 , 32 Gbyte RAM, Windows 10 Enterprise 22H2, Intel Xeon Processor

Also a notebook of which I only know the following:

Notebook 32 Gbyte RAM, AMD Ryzen, Windows 11

And here are the computers on which the error does not occur:

Dell Optipelx 7050, 16 Gbyte RAM, Intel Core i7-6700 CPU, Windows 10

Dell Precision 360, 8 Gbyte RAM, Windows 10, Intel(R) Core(TM) i7-6700 CPU

Also a notebook of which I only know the following:

32 Gbyte RAM, Windows 11 Enterprise 24H2, Intel i7 1260p

[RichardTanSY_Altera]

128GB RAM is sufficient for Quartus Lite. Your design compile fine and finish within seconds in my PC.

Which devices are you using? How much disk space does your machine have? Minimum disk space recommended for Quartus lite is 14 GB. 

Additionally, go to Task Manager > Performance> Memory, to check and monitor the memory usage.

Here are troubleshooting steps that you may check during installation process: 

1. Before you download and install the software, please check that your machine system met the minimum Hardware and Software Requirements. Ensure that your Operating System is supported.

https://www.intel.com/content/www/us/en/docs/programmable/683472/25-1/system-requirements-and-prerequisite.html

https://www.intel.com/content/www/us/en/docs/programmable/683472/25-1/operating-system-requirements.html

Quartus 24.1std Software and Device Support Release Notes:

https://www.intel.com/content/www/us/en/docs/programmable/683593/24-1std/disk-space-and-memory-recommendations.html

2. Check that the disk space is sufficient for the installation of the Intel FPGA software. 

3. If you are running antivirus software, try to temporarily disable the antivirus software during the Quartus Prime software download and installation process.

4. Install the software with Administrator privileges. 

5. Try to download and auto install using the new Quartus installer.

6. If install using .tar, try to re-download the Quartus .tar installation package and install again. Check that the installation files .tar is not corrupted by checking the checksum. 

7. Check your OS history update, e.g. to check if the issue happen after some specific Window update. 

Regards,

Richard Tan

[StephanBorucki]

Thank you for your reply. The device is MAX 10, 10M50SAE144I7G.

We have followed all the instructions! I apologize for only replying now.

We have spent several days with several employees trying to find a cause for the problem. As already described, the problem usually occurs on very powerful computers with at least 32 Gbytes of memory (and more), sufficient hard disk capacity and CPUs with 8 cores.

On less powerful computers with only 4 cores the problem does not seem to occur. We have run the same design, which I have also sent, on many different computers. On some computers we can compile it 20 times. On other machines it occurs about the third time we compile it. If the problem occurs once, the Quartus installation is corrupted! Then you can no longer compile anything.

We then uninstalled Quartus completely and used CCleaner to search the registry several times for errors. Several errors were found here. When we had cleaned everything up, we reinstalled Quartus. Then the problem no longer occurred on the same computers where the problem previously occurred.

I have now found out that the error may occur when we compile this design on our network drive. After a complete reinstallation as described above, I was able to compile the project 5 times without errors when saving it to the local hard disk with both Prime Lite 24.1 and Prime Lite 19.1.

When I opened the project on the network drive, the error occurred on the 5th compile with 19.1.

This is the current status. The problem is very serious for us. If we don't get it solved, we will no longer use Quartus for teaching and research projects in our department.

[RichardTanSY_Altera]

Any update on this?

Do you able to resolve the issue?

Regards,

Richard Tan

[RichardTanSY_Altera]

Could you try deleting both the db and incremental_db folders from the project directory on the network drive after the error occurs? Then, try recompiling the project again to see if the issue reoccurs.

Additionally, in the Compilation Process Settings, try to set the Maximum processors allowed to 1, to disable the parallel compilation.

Regards,

Richard Tan

[StephanBorucki]

Hello,

I was able to prove through testing that the error only occurs when compiling on a network drive with a fast, powerful computer. I tried this with a Dell Precision 7540 with Windows 11, 128 Gbyte memory, Intel Core i9-9980HK CPU, 2.40 GHz.

I was able to compile the Blink design locally on the hard disk 5 times without any errors,  without any change to the design with use all available processors.

If I take the same design and compile it on the network drive (connection via the LAN of our university), the error occurs again the second time and the Quartus installation is corrupted! Fitting is no longer possible. (The first time I compiled, the computer even crashed with a blue screen).

I uninstalled Quartus, cleaned the registry with Ccleaner, reinstalled Quartus Prime and tried again.

Locally on the hard disk I can compile Blink 10 times without any changes.

Then I copied the project to the network drive and, as Richard Tan recommended, deleted the db and incremental_db directories and set Maximum processors allowed to 1.

The second time I compile, the same error occurs again!

The network drives at the university are sometimes slow, but I work with many other programs (Altium Designer, KiCad, field simulations with CST, …) that show no problems on the network drives.

[RichardTanSY_Altera]

May I know how do you install the Quartus? Is it using Quartus installer or .tar? If you try to copy the .tar to a network drive, there is a a chance that the .tar might get corrupted. 

Check that the installation files .tar is not corrupted by checking the checksum. 

Also, have you try to install the Quartus in the Network Drive? 

Regards,

Richard Tan

[RichardTanSY_Altera]

Hi,

May I know any update on this?

Regards,

Richard Tan

[StephanBorucki]

Hi Richard,

We have downloaded Quartus Prime Lite 19.1 as a tar file from Intel/Quartus and installed it locally on the hard disks of our computers. We have just checked the checksum. This is correct. We have unzipped tar and installed the software locally on the computer. (We have not installed the software on the network drive. That makes no sense for us. We only use the network drive to store project files. )

[StephanBorucki]

Hi Richard,

We have now repeated the test with the latest Quartus Lite 24.1 version, which we installed locally on the hard disk using the installer. It was a notebook with Windows 10, 128 Gbyte memory, Core i9. We were able to compile our project locally 10 times without errors. On the network drive, the error occurred again after we had compiled 7 times this time. The installation was then corrupted and the local project could no longer be compiled.

What should we do now? We have a workaround by only compiling locally, but if one of our employees or students does not follow this, we have to uninstall and reinstall the whole software again. This is not acceptable for us in the long term.

[RichardTanSY_Altera]

So assuming there are 20 computers (each computer has a few Quartus versions installed) sharing on one network drive, however some computers have different specification. Let say we divide into two different computer groups, A and B, more powerful A ( with issue) and a less powerful (with no issue) computer B.

When Computer A1 compiling a Quartus project on the network drive, the Quartus software locally installed in the Computer A1 get corrupted. Then, the Computer B1 is able to compile the same project (compiled before by A1) in the network drive without any issue. Then another Computer A2 compile the same project (compiled before by A1) will get corrupted as well after maybe 5 or 7 compile, in the network drive.

The Quartus corruption only happen in computer A1 and A2, causing them unable to compile project locally, while the Quartus in A3, A4 and so on, is still usable and able to compile the project locally.  

My guess is the network drive is used to share the project files to the students after each class.

Correct me if my understanding wrong at this point. 

My suggestion for now is to restrict the students permission to compile the project in the network drive and ask them to copy and compile the project in their local disk. Try to set up permissions on the network drive so that the students only have read access but not write or execute access. This allows them to copy the project files to their local drive while preventing them from modifying the original files on the network drive. 

Could you try to install a Quartus Standard or Pro version, setup an evaluation license and see if the issue persists? (to check if different Quartus version has an impact)

If possible, try to use a different network drive.

I am trying to duplicate the issue from my side but no success yet. Most likely the setup is different.

Processor Intel(R) Core(TM) Ultra 7 165U  1.70 GHz

Installed RAM 32.0 GB (31.4 GB usable)

System type 64-bit operating system, x64-based processor

Edition Windows 11 Enterprise

Version 23H2

OS build 22631.5335

Regards,

Richard Tan

[StephanBorucki]

Hi Richard,
thanks for your reply. Due to other important tasks I have to do, I won't get around to replying until Tuesday next week, May 27.
Best regards
Stephan

[NTA]

Hello Richard,

I'm a student / lab assistant who has been involved with trying to find the cause of this error.

There are even more different computers but they can basically be divided into "plenty powerful" (A) and "not so powerful" (B). All the B models will soon be replaced as they are not compatible with Windows 11; so all our computers could be affected in the near future.

---

When Computer A1 compiling a Quartus project on the network drive, the Quartus software locally installed in the Computer A1 get corrupted. Then, the Computer B1 is able to compile the same project (compiled before by A1) in the network drive without any issue. Then another Computer A2 compile the same project (compiled before by A1) will get corrupted as well after maybe 5 or 7 compile, in the network drive.

The Quartus corruption only happen in computer A1 and A2, causing them unable to compile project locally, while the Quartus in A3, A4 and so on, is still usable and able to compile the project locally.  

Exactly! The error only occurs on the A models and only while compiling a project which is stored on a network drive. Once the error has occurred, the Quartus installation on that machine is broken.

Maybe the network drive is not fast enough to keep up with more powerful computers which might explain the error message ("not enough memory", roughly).

In our lab courses the network drive is mostly used for distributing files and is not really meant as a working directory.

However, there are other projects that multiple people work on (not at the same time!) which stay on the network drive to keep everything synchronized. This has always been working fine and still works fine on all the B computers.

Copying the project to a local drive to work on and then uploading it back to the network drive afterwards could indeed serve as a workaround. But we still see the risk of someone unintentionally breaking the Quartus installation should they forget about this.

The fact that projects which are stored on the network drive cannot be compiled on certain computers is annoying but something we could live with, if it simply caused an unsuccessful compilation or even a crash of Quartus. What we see as a reason of potentially leaving the entire Altera/Intel FPGA platform is the tedious restoring process that is necessary once the error has occurred.

Simply uninstalling and reinstalling the software does not work and it took us quite a bit of experimentation to find a reliable restoring procedure:
Fully uninstall Quartus -> reboot -> delete all temporary files and all unresolved registry entries using CCleaner -> reboot -> reinstall Quartus.

We will test the latest version of Quartus Prime Pro in eval mode once we have some spare time next week.

The issue is not specific to a particular system and occurred on computers with...

- OS: Windows 10, Windows 11

- CPU: Intel Xeon W (Desktop), Intel Xeon W (Mobile), Intel i9 (Mobile), AMD Ryzen 7 (Mobile)

- RAM: 32 GB, 128 GB

Any university-specific setup (e.g. antivirus software, Dell management software) can also be ruled out as my personal laptop (AMD 8845HS, 32 GB RAM, Windows 11 Pro 24H2) behaves exactly the same when connected to the network drive.

Regards,

Nils

[RichardTanSY_Altera]

I will need to check with the internal team and see if there is any ways to debug this issue.

In the meantime, please use the workaround while we are looking into this.

Btw, could you help to provide the full error log with the error message?

Regards,

Richard Tan

[NTA]

Hello Richard,

here's the latest update:

1) We have installed Quartus Prime Standard 24.1 which showed exactly the same behavior as the Lite Edition. We did not perform any tests with Quartus Prime Pro 25.1 as this version does not support the Max 10 devices that are of interest to us.

2) Using Lite Edition 24.1 I compiled a basic demo project two times and saved the output_files folder with all the report files each time. The first compilation was successful, the second try led to the usual error (293007). I have attached those output_files folders as well as the project.

If there are any other tests that we can help you with, please let us know.

Kind regards,

Nils

[RichardTanSY_Altera]

Hi @NTA , 

After a machine is corrupted, does it fail with the same  "quartus_fit ended unexpectedly" messsage?

How is the compile launched?

1. Is the user using Quartus GUI to compile or 2. using quartus_sh --flow compile? or 3. Run quartus_ipgenerate, quartus_syn, quartus_fit directly?

Instead of:

     Fully uninstall Quartus -> reboot -> delete all temporary files and all unresolved registry entries using CCleaner -> reboot -> reinstall Quartus.

does simply rebooting the machine make Quartus work again?

Can you try quartus --reset_desktop to see if it helps?

In the C:\Users\<user>\ directory there is a quartus2.qreg file,  delete it and see if it helps.

How much free disk space is on the machine that crash?

Could setting the Quartus installation files to read-only be a potential workaround?

Regards,
Richard Tan

[RichardTanSY_Altera]

Hi @NTA ,

Additionally, please try the following:

-----------------------------------

In the, GUI, run Analysis & Synthesis.

Then exit the GUI.

On the command line, run quartus_fit.exe on the project.

-----------------------------------

What exactly is meant by "the quartus installation becomes corrupt and not even a re-install fixes it"? Is that referring to this specific error message (293007) from quartus_fit or is there any other symptom?

And to make sure I understand: after this error is generated, does the exactly same error occur for ANY other project located anywhere (network or local)?

-----------------------------------

What is the license setup?

-----------------------------------

Also, after the error, does running the Quartus menu command Project > Clean Project... followed by a fresh compilation fix the issue? If not, then try to manually delete all quartus-generated subdirectories and files in the project directory and try again. For example, delete the entire 'db', 'incremental_db', 'output_files' directories.

-----------------------------------

Have someone download the Free edition of Microsoft Visual Studio C++: https://visualstudio.microsoft.com/vs/community/

And then try to either run quartus_fit.exe from inside the IDE, or when quartus_fit.exe crashes, the Just-In-Time debugger prompt should appear to attach to the process in the IDE and hopefully we can get a callstack.

-----------------------------------

Regards,

Richard Tan

[NTA]

Hello @RichardTanSY_Altera ,

thank you for taking this issue seriously.

@StephanBorucki is currently not able to access his Intel account, but I will try to answer all of your questions.

1) We have always been using the GUI to start the compilation. Is using the command line worth a try?

2) Rebooting the computer has absolutely no effect once the error has occurred.

3) "Can you try quartus --reset_desktop to see if it helps?"

We will give it a try.

4)  "In the C:\Users\<user>\ directory there is a quartus2.qreg file,  delete it and see if it helps."

We will try that as well.

5) "How much free disk space is on the machine that crash?"

This varies from machine to machine, but at least 250 GB and in some cases > 1 TB.

6) "Could setting the Quartus installation files to read-only be a potential workaround?"

I assume, you are referring to the program files folder, correct?

7) Start Analysis & Synthesis from the GUI, Fitter from the command line.

We can also try that.

"What exactly is meant by..."
Once the error in question (293007) has occurred, it will always occur on that machine no matter what project we are trying to compile and no matter where that project is stored. Therefore it seems like the entire Quartus installation on that particular machine gets corrupted on the first occurrence.

From this point onward nothing seems to be able to revert the issue - not even uninstalling and reinstalling Quartus. The only way that we have found so far that fully restores Quartus on an affected computer is the previously described procedure using CCleaner.

9) "What is the license setup?"
We usually work with Lite Edition 19.1 and 24.1, so no license is required. When we installed Standard Edition 24.1 for testing purposes we used the 30 day trial, so also no license needed.

10) "Project > Clean Project..." and "manually delete all quartus-generated subdirectories"
This makes no difference either; the error persists.

11) Deugging the fitter with VS
We will look into this once we find some spare time.

For now this is all the information I can provide.

We will make sure to try out your suggestions, which we will probably not get around to before the end of next week.

Kind regards,

Nils

[RichardTanSY_Altera]

1) Yes, this is for debugging purposes.

6) Yes.

Regards,

Richard Tan

[StephanBoruckiPrivat]

Unfortunately, I am having problems logging into Intel with my work email. That is why I have just created a private Intel account and I am responding using this account.
I have tried the following two things:
3) "Can you try quartus --reset_desktop to see if it helps?"
I went to the installation directory c:\intelFPGA_lite\24.1std\quartus\bin64\ and opened a command line window there, where I entered quartus -reset_desktop. The error persists.
4)  "In the C:\Users\<user>\ directory there is a quartus2.qreg file,  delete it and see if it helps."
I have deleted the file. The error persists.

7) Run quartus_fit.exe on the project in the command line.
I can't find a quartus_fit.exe file,
only quartus_fit.pdb under c:intelFPGA_lite\24.1std\quartus\bin64\pdbmin\.

How should I start it? I can set a path to this file and than I can start quartus_fit.pdb in the project directory?

I will be in touch again next week.
Best regards,
Stephan

[StephanBoruckiPrivat]

This morning, I was able to verify that the quartus_fit.exe file is missing from the computers on which the fit function does not work. It cannot be found in the same location as on the computers where the fit function works. I checked this on several computers.
Best regards
Stephan