Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
28381 Discussions

Catastrophic error compiling Fortran project with oneAPI 2023.0.0

Scott_Boyce
New Contributor I
5,586 Views

Hi All,

 

I am a developer for a Fortran project called MODFLOW-OWHM. After updating my oneAPI from 2022.1.1 to 2023.0.0, I am now getting a:

catastrophic error: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report. 

message.

 

The project has a Fortran only option and I am compiling with the Intel Fortran Compiler Classic to create this error. In particular, Intel Fortran Compiler Classic (ifort) 2021.7.1 compiles fine, while 2021.8.0 raises the error.

I get the same error, but for a different file, if I try the Intel Fortran Compiler (ifx) 2023.0.0, but I have never been able to compile with that (previously ifx would raise errors saying feature not yet supported, but now it just says catastrophic error).

 

The source code is hosted at:

https://code.usgs.gov/modflow/mf-owhm 

(gitlab source repository)

with the main source under the src folder, a makefile for compiling in linux, and for windows a solution file located at ide/visual_studio/OneWater_Project.sln

 

For ifort, I get the same error on both linux and windows with the error happening in src/modflow_base/wel8.f90

I am using Windows Visual Studio 2019, my debug compliation, and the problem file has the following message:

ifort /nologo /debug:full /Od /assume:nocc_omp /Qdiag-error-limit:10 /Qdiag-disable:6717 /warn:declarations /warn:unused /warn:ignore_loc /warn:noalignments /warn:uncalled /warn:interfaces /real-size:64 /fpe:0 /fp:source /Qfp-stack-check /module:"obj\vs_mf-owhm_Debug_x64\\" /object:"obj\vs_mf-owhm_Debug_x64\\" /Fd"obj\vs_mf-owhm_Debug_x64\vc160.pdb" /traceback /check:pointer /check:bounds /check:shape /check:uninit /check:format /check:stack /libs:static /threads /dbglibs /c /Qlocation,link,"Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64" /Qm64 "src\modflow_base\wel8.f90"
fortcom: Fatal: There has been an internal compiler error (C0000005).
compilation aborted for src\modflow_base\wel8.f90 (code 1)

 

The Linux error says something similar:

/src/modflow_base/wel8.f90: catastrophic error: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report. Note: File and line given may not be explicit cause of this error.
compilation aborted for ./src/modflow_base/wel8.f90 (code 1)
make[1]: *** [makefile:756: obj/debug_ifort/wel8.o] Error 1

 

My office has several licenses to the intel compilers, but I do not have access to the intel accounts for posting support requests.

 

The code can be obtained from the gitlab repo, or I attached it here (I just quickly carved out code and zipped it up).

 

Any help with this would be greatly appreciated.

0 Kudos
33 Replies
Ron_Green
Moderator
4,419 Views

thanks for sending the source zip.  On linux I'm reproducing the ICE for wel8.f90.  Investigating this.

Scott_Boyce
New Contributor I
4,309 Views

Any luck finding the cause of the problem?

0 Kudos
JohnNichols
Valued Contributor III
4,254 Views

Intel time is bit slower than consulting time, ratio is about 5:1

 

0 Kudos
Scott_Boyce
New Contributor I
4,224 Views

What is funny is that I work for the USGS, but since we are soft money based (externally funded) so we operate at a really fast pace before funding for projects run out.

 

Im just getting nervous that a lot of large scale Fortran projects are in limbo (I know of 4 major ones I work on and 5 other projects that colleagues have told me about) because ifort seems to keep breaking with each release and ifx can't compile any of those projects (It either raises a feature not implemented error or just internal compiler error).

 

Its stressful because Ive worked on a lot of these projects for 10+ years, and its hard to tell end users to find a specific version of ifort because the newer ones wont compile anymore nor support Visual Studio.

Ron_Green
Moderator
4,200 Views

I will revisit this issue in the coming days. 

 

Ron

0 Kudos
JohnNichols
Valued Contributor III
4,189 Views

I downloaded the file, the wel8.f90 causes the error, although there are a lot of unused variables so you get a lot of lines on output. 

Can you pull out wel8?

0 Kudos
JohnNichols
Valued Contributor III
4,161 Views

I downloaded VS 2019 onto an old computer which does not have Fortran on it.   I then loaded 2020 old Fortran and your program compiled and ran. 

I can send you the exe, below is most of the system, it is to big for a zip alone. I will ship the second half along shortly,  

Not sure if it helps, 

John

0 Kudos
JohnNichols
Valued Contributor III
4,161 Views
0 Kudos
JohnNichols
Valued Contributor III
4,161 Views

We can talk by email if you need more assistance. 

 

0 Kudos
Ron_Green
Moderator
4,116 Views

I've got this cut down to just WEL_SUBROUTINES module and just GWF2WEL8AR, and just this 1 statement:

CALL IUNITRAMP%OPEN(LINE, LLOC, IOUT, IN, NOBINARY=TRUE)

 

That said, there are many similar CALLS that also trigger the ICE.  So commenting this 1 call is not enough, it'll just ICE in another call.  However, I think all these calls causing the ICE have the same root cause.  Stack traces are all the same.

 

with another few hours of time I should have it isolated to something I can send in a bug report.  More asap.

0 Kudos
Scott_Boyce
New Contributor I
4,085 Views

Thanks for looking into this.

What compiler options do you use to help diagnose the problem.
Im curious so I can do some of this on my own.


Right now I just comment out entire sections of code, then uncomment it until the compiler error is triggered.

 

0 Kudos
Steve_Lionel
Honored Contributor III
4,070 Views

@Scott_Boyce wrote:

Thanks for looking into this.

What compiler options do you use to help diagnose the problem.
Im curious so I can do some of this on my own.


Right now I just comment out entire sections of code, then uncomment it until the compiler error is triggered.

 


That's more or less what I would do when I was in Support.

  1. Reduce compile options for the build until I find a set where any change makes the bug go away (if there is one).
  2. Add back in /Od or -O0 to see if optimization is relevant.
  3. Reduce the set of files needed to reproduce the problem.
  4. Start commenting out chunks of code (I tended to use !DEC$ IF .FALSE. /!DEC$ END IF) to find the minimal source. If modules were used, I'll sometimes copy only needed declarations into the test source and then remove the USE. (Sometimes it is the module usage triggering the ICE, though!)

Note that while a minimal reproducer is always welcome, always submit the whole source or project as well. I encountered cases where the customer submitted only the minimal case, the bug was fixed for that, but the original build still failed from a different bug.

Ron_Green
Moderator
4,066 Views

We have an internal build of the compiler.  Unfortunately we cannot release this to our users.

It is far slower at compiling, but does give the line number of the statement causing the ICE.  It also has debug symbols to give a call stack.  This is how I found the line causing the ICE.  Commenting that out, I hit one after another CALLs with the same stack trace.  That is why I can say I think we have 1 bug causing multiple errors in that source file.

 

There was a project called "FREDUCE".  the goal was to automate trimming out code a little at a time until the ICE goes away.  Just automating what you (and I) are doing.  I never had much luck with it for advanced F2018 features so I use the manual method.

I pull up the code in a visual editor.  I cut executable code and then subroutines from the bottom up.  Save, compile, repeat.   had about 60+ iterations on this code to get me to the code below.  I still need to pull out the USEs and get to the core definitions.   it's manual.

the visual editor can UNDO repeatedly so I can rapidly undo the CUTted code. 

 


MODULE WEL_SUBROUTINES

  USE PAK_PROP_INTERFACE
  IMPLICIT NONE
  PRIVATE
  PUBLIC:: GWF2WEL8AR, GWF2WEL8RP
  
CONTAINS
  
SUBROUTINE GWF2WEL8AR(IN)

  USE GWFWEL2MODULE, ONLY : IUNITRAMP

  INTEGER,INTENT(INOUT):: IN
  INTEGER,POINTER:: IOUT
  INTEGER:: LLOC
  CHARACTER(768):: LINE    
  CALL IUNITRAMP%OPEN(LINE, LLOC, IOUT, IN, NOBINARY=.TRUE.)
END SUBROUTINE

SUBROUTINE GWF2WEL8RP(IN)

  USE GWFWEL2MODULE, ONLY: WELDATA
 
  INTEGER,INTENT(IN):: IN

  INTEGER:: NAUX
  INTEGER, POINTER :: IFREFM, NNPWEL, NWELVL, MXWELL, IPRWEL, IOUT
  CHARACTER(LEN=16),DIMENSION(:),POINTER,CONTIGUOUS  ::  WELAUX

  CHARACTER(75)::  LABEL

  CALL ULSTRDSTRUCT(NNPWEL,WELDATA,1,NWELVL,MXWELL,IN,IOUT,TRIM(LABEL),WELAUX,20,NAUX,IFREFM,1,1,IPRWEL)
 
END SUBROUTINE

END MODULE
0 Kudos
JohnNichols
Valued Contributor III
4,063 Views

If you do a compile in IFX you get multiple ICE's where as with IFORT you only get one.  It is not likely there is one from a quick look at the long code. 

 

+ 1500 unused variables

0 Kudos
JohnNichols
Valued Contributor III
4,057 Views

Screenshot 2023-02-01 140803.png

IFX ICE's and nothing says they have to be the same ICE as an underlying cause

0 Kudos
Scott_Boyce
New Contributor I
4,049 Views

That actually was going to be one of my questions.

I have three other projects that do not compile with ifx.

One of those two projects is in the bif_lib folder of the one that I shared here. That folder contains select files from my Batteries Included Fortran (BiF) library. All of BiF is located at: https://code.usgs.gov/fortran/bif

Two more software projects that do not compile with ifx are IWFM-2015 (IWFM: Integrated Water Flow Model (ca.gov)) and MODFLOW6 (GitHub - MODFLOW-USGS/modflow6: MODFLOW 6 Repository). Both have issues with the current version of ifort (2023.0.0/2021.8.0)

 

Should I make a separate post or add to this one?

Ive been separating out the code from BiF that ifx complains about.

MODFLOW6 would require more work, so I will have to post the full lib.

I kind of know some of the ifx problems. In past ifx raised "feature not implemented" error, but now it just says internal compiler error for the same code locations.

 

 

0 Kudos
Ron_Green
Moderator
4,009 Views

@Scott_Boyce  No need for a separate post.  I can work on BiF, LWFM-2015, and MODFLOW6

It seems I'm working for Scott for the next month    I'll see if Barbara, Devorah, or Shiquan can share the fun.

0 Kudos
Scott_Boyce
New Contributor I
3,274 Views

hah, thanks for the help, its really appreciated.

 

I just saw this message and already created a forum post specific to MODFLOW6 (ifort read file error and ifx internal compiler error for oneAPI 2023.0.0 for Modflow6 Software).

The BiF library is partially included in the MODFLOW-OWHM code added here. Its in the directory bif_lib, but I will make another reply here later this week with the full BiF library. It also contains some minor bug fixes that I caught last week.

 

Thanks again for all the help/support,

 

Scott

0 Kudos
Ron_Green
Moderator
4,049 Views

ICE is a catch-all for anything that crashes the compiler.  For the public version of the compiler, this is all you get.  Yes, I know, it's not helpful. 

0 Kudos
JohnNichols
Valued Contributor III
4,030 Views

I have been developing code to meet deadlines for three decades, you need to be realistic, this is potentially a major change and the chances it is one ICE must not be great, I looked at the ten lines, they are not the same, I realize the IFX and IFORT give you stuff all information, but I would accept that the camera in your hand is the one you are going to use at the moment and the old PS files work, you gain nothing worthwhile waiting except an ulcer.  

Intel will solve it, but I would start with the old.  Your client will not care. 

To paraphrase the old English "it haps." 

0 Kudos
Reply