Application works from VS2008 GUI, but fails when run as a stand alone console application

[rniswon]

I've had a very hard time tracking down an 'access violation' exception when running an application built in XE 2011 ver 12.0.x. The problem went unnoticed due to the fact there seemingly was no problem when running the fortran code from the visual studio 2008 GUI. Recently, however, I tried running the executable directly from a windows console application and got an access violation error. Based on suggestions from another forum (one of which was to post on this forum instead), I have tried the following:

1) right-click the executable and run it as an 'administrator'

2) from the debug menu, select 'exceptions' and switch 'on' pertinent exceptions since nearly all default to an 'off' status (I've even gone so far as switch nearly everything on)

3)right-clicked the project (from the Solution Explorer in Visual Studio), selected: Properties -> Linker -> System and bumped up "Heap Reserve Size" "Heap Commit Size" "Stack Reserve Size" & "Stack Commit Size" to 10^9 (previously 0).

4) constantly "cleaning" the solution (from the debug menu) and rebuilding it after each tweak to the code.

None of this has led to a final solution, however. Addtional items I have noticed are that and combination of 32-bit/64-bit and debug/release mode works when run from the GUI, but only the 32-bit debug-mode executable works when run from a console, and its much too slow for what I need it for (I really need to get a 64-bit release mode executable that works!). Since the debugger doesn't alert me to the problem, I've tried inserting WRITE statement throughout the code to clue me in on where it crashes, only to find out that by inserting write statements in different places causes the code to crash in different places. Baffled, I then tried contacting Intel support, only to land on the dreaded screen: "We are experiencing technical difficulty, we have alerted personnel and are sorry for the inconvenience."

So, alas, I'm turning to this forum as a last ditch effort to find a solution...how can I solve this problem?

Using the following:

1) Windows 7
2) MS VS2008 Version 9.0.21022.8 .NET Framework version 3.5 SP1
3) Intel Visual Fortran Composer XE 2011 Integration for Microsoft Visual Studio* 2008, 12.0.3470.2008

[Steven_L_Intel1]

Your symptom has all the hallmarks of accessing through an uninitialized location. You could have stack corruption, mismatched arguments or some other problem.

Have you tried enabling "Check Routine Interfaces" under Fortran > Diagnostics ? Can you attach a ZIP of the project for us to look at?

[rniswon]

Hi Steve, your suggestion has identified some problems. When trying to compile with "Check Routine Interfaces" turned on, the output window reports: "V:\File_Transfer\...\mt_adv5.for(2212):error #7836: If the actual argument is scalar, the corresponding dummy argument shall be scalar unless the actual argument is an element of an array that is not an assumed-shape or pointer array, or a substring of such an element. [ICBUND]"

I have attached a zip file containing the source fortran files and the VS solution.

If you're able to describe in a bit more detail the mistake I've made, that would help get me moving in the right direction sooner. Thanks for your time.

[rniswon]

Hi Steve, I was curious if you've had the opportunity to look at the zip file I attached (atyour request) when I responded to your poston this thread? I'm hopeful you can provide some additional direction on why "check interfaces" broke the solution and do I clean these types of problems up?

[mecej4]

No data files were included in the Zip file. Since the program has to be run in order to catch the errors, at least enough data is needed to allow the program to run upto the first error.

[rniswon]

I've attached an abbrevitated version of the model input files that will reproduce the problem. To run the compiled code from a windows console application (which is the way I typically do it unless running from the GUI), I type the name of the executable file followed by a space, followed by "Remodel3.nam". Remodel3.nam contains the information expected by the fortran code. Placing the model files (in the attachment) and the executable in the same directory is also required. After what is referred to as "stress period 3", the model will crash due to the cutting away of arrays worth of data to keep file sizes down in the attachment. However, the "access violation" occurs while reading in data in the first stress period.

Hopefully this is what you need? One curiousity, at least to me, is that with "Check Routine Interfaces" set to yes in the GUI (as suggested in a post above to this thread) the error occurs while compiling the code (also attached above) or well before any attempt to run the model is made. Thus, if you're able to make a suggestion on how to proceed with cleaning up the errors that are only reported with "check routine interfaces" turned on, I'm not sure you'll need the model files. But as requested, they are attached.

I appreciate your time and interest in helping me move past this problem.

[Steven_L_Intel1]

I did a debug build and, whether run from VS or from outside, I get an "end-of-file" error in Stress Period 4 reading ReModel3_Abbrev.ssm. I had to comment out one of your debug write statements as it had an out-of-bounds array reference, but that would not be your problem. I also did a build using the Static Security Analysis feature of Intel Fortran Studio and it found a number of possible errors, but I haven't yet found one that would lead to this problem.

If I build a Release configuration, I don't get an access violation but I don't get much output either. Strange. I'll see what I can figure out.

[rniswon]

Hi Steve, Yes, the end-of-file error would be expected for the input files I attached to a previous thread. I trimmed out quite a bit of data to keep the file sizes manageable. If you're able to build a release version that works past the first stress period, which seems to be the case since its coming up against a premature end-of-file problem, you have already gotten past the point of the access violation error I keep getting during execution of the first stress period. This is very curious. I'm wondering if I tried using you solution (*.sln) and/or project files, I would still get the same problem? I've long wondered if I have some setting that's buried deep within the GUI setwrong and by using your solution/project it would get set equivalent to what you have allowing everything to work as expected? I've got VS2010 on order (currently using VS2008) and will upgrade from Vis. Fortran XE 2011 12.0 to 12.1 once I get VS updated. I'm really surprised that the compiled code is working for you when built in 64-bit release mode? Now I'm really confused...

[Steven_L_Intel1]

Oh, I see what I did - I unpacked the first ZIP second, overwriting the file. In a debug build it seems fine - I used your project file after turning off generated interface checking.

In a Release build what I think I am seeing is a steady increase in memory consumption until the image just exits without an error. It may be a few days before I can get back to this.

[rniswon]

Hello Steve, I am curious if you've had any luck with the problem that motivated this thread? I received the following bit of advice from another post and wasn't sure if you are officially with Intel or not:

"The chief difference running something in the IDE (or GUI) is which of the standard libraries get linked to your executable (and sometimes run-time arguments). The run-time error you are seeing is likely occuring in one of these standard libraries, in particular, the i/o library, which is why trapping on the segfault doesn't work (as opposed to if was something directly in your code). Calling an I/O routine invokes a large chunk of outside code (just look at the source code in GCC for example). These standard libraries sometimes have bugs over which you have no control, but they just may not like something you are asking them to do. I would say your next move would be to try and get support from Intel - I've found them to be quite helpful in the past when I hit a dead end.

Posted by Jeff Felix "

Fortran serves as a means to an end, which is to say my knowledge base of the inner working of Fortran is rather finite. If things are working for you, might it be I just need to update my i/o library if there is a bug hanging out in there?

[Steven_L_Intel1]

I am definitely "officially with Intel".

Sorry, I got sidetracked from working on this. I will get back to it tomorrow.

[Steven_L_Intel1]

I have found part of the problem. You have set all the "Stack" options in the Linker properties to 1GB. This is not only unnecessary but can cause odd behavior because 1GB is the limit on the stack in Windows. Oddly, VS won't let me remove all those settings but I can set them to 1000000 and then the program runs "normally" (more or less). Now I get:

Transport Step: 1 Step Size: 0.5000 Total Elapsed Time: 0.50000
STOP. GCG SOLVER FAILED TO CONVERGE.

I suggest you take it from here and see where you get. Let me know if you need more help.

[rniswon]

Hi Steve, Any luck with the problem that originally motivated this thread?

[Steven_L_Intel1]

Well, yes, I thought I explained that, though maybe not well. It was the stack reserve setting that caused the original behavior.

[rniswon]

Hi Steve, I tried reposting to this thread yesterday to see if you had any luck with bug, but as I don't see that post appearing in the thread (I must've hit "cancel" instead of "Submit"), I'm reposting now...any luck?

[rniswon]

Nevermind, whereas I couldn't see your response before, I can see it now...not sure what happened. I'll follow your suggestions and see what I get.

[rniswon]

Hi Steve,

I'm still in need of some help, so much so I don't know what to do or where to turn. The code is still exhibiting the same strange behavoir where it executes fine from the VS GUI, but fails when running as a stand alone program. A couple of updates since I last posted: I have upgraded to VS2010 (from 2008) as well as to Intel Fortran to XE 12.1 (from 11.0). Sadly, $1,200 later it appears this did not help. Further, when I created a new project with the new software I made sure not to touch the stack options in the linker properties (based on one of your last posts), these remain in their original default state. I've also cleaned up all of the "Check Routine Interfaces" errors by turning this diagnositic on and making appropriate corrections.

Having little idea of what to do next, I thought I would start with the error message (that you mention in one of your last posts) that goes:

STOP. GCG SOLVER FAILED TO CONVERGE.

The GCG solver is attempting to solve Ax=b, where 'A' is a matrix and 'x' and 'b' are vectors. 'x' is the quantity being solved for. An idea I had was to write 'A' and 'b' to text files just before the code steps into the GCG solver and then cross-examine the text files written from the GUI by the text files written by the stand alone executable, and look for differences. Unfortunately, the stand alone version isn't even writing the text file? It is the very same code! This is intensly frustrating. Why do things work fine from the GUI but the executable seemingly has a mind of its own?

Also, the VS GUI does not stop at breakpointsin the 64-bit debug mode?

Is it possible to compile in a different compiler?, just to see if the stand alone executable created by a different compiler works?

I really need to get through this, its delaying my response to some final comments for a peer-reviewed journal article. Further, the code is meant to help others and so I'd like to distribute it now that it is nearly through review. This is all to say, I'm open to all possibilities to solve this problem. If there are hourly charges for direct support (through WEBEX for example), I'm certainly willing to consider it. I make to claim to being a Vis. Studio/Intel Fortran expert. These are simply the tools I use to help me in my profession and it would appear I need to defer to the experts of the tools themselves to get through this.

My direct email is emorway_at_usgs.gov if it's appropriate to pick this conversation up away from this forum. Confounded,

Eric

[Steven_L_Intel1]

Eric,

Debug breakpoints do work in 64-bit mode. Please attach a ZIP of your current project/solution files and I'll take a look. It is unlikely that using a different compiler is going to help you here.

[rniswon]

Hi Steve,

I have attached a zip ("UZF-MT3DMS_XE12.1_ver2.zip") with the solution/project included. The source file for the project are also in the zip file, located in the directory just above where the solution is located.

I've also attached a reduced set of model input files ("examp_model.zip") because I'm hoping you can trying running the model from with in the gui as well as running it directory from the compiled executable, and see if you are able to reproduce the problem.

The instructions for running the model are rather simple. You're undoubtedly fully aware of how to run the model, but I'll lay it out so you can see if I've got something wrong. From the GUI, I point the project to the directory where the model files reside by right-clicking the Project -> Properties -> Debugging and enter the path of where the model files reside on the "Working Directory" line and type "Cathy.nam" on the 'Command Arguments' line.

Running the simulation using the executable is even easier. After placing a copy of the UZF-MT3DMS_XE12.1_Ver2.exe executable in the same directory as the model files, I launch a windows console application, change the directory appropriately, and at the command line type "UZF-MT3DMS_XE12.1_Ver2.exe Cathy.nam"

I think you'll find that the model runs through 10 time steps (I had to trim the model files to keep zip file size down) using the GUI, but will crash using the executable. This is the crux of the problem. Lastly, you may need to comment out lines 1597-1620 in "mt_btn5_updated.for" before it'll run OK. This bit of code was meant to help me debug the problem...but as I mentioned the stand alone .exe doesn't seem to 'see' them.

I appreciate any insights you can offer. Gratefully, Eric

[Steven_L_Intel1]

Eric,

Thanks - I'm working on it now, but it may take me a while before I figure it out. I can see the bad behavior.

I did spot one problem, though. On line 118 of mt_ssm5_updated.for is this line:

IF(FUZF.AND.IETFLG) THEN !edm

IETFLG is an integer pointer in module mt_module, but it is not associated with any value when it gets to this statement. I would also caution you that using .AND. with an integer is a dangerous thing to do.

This error was one of many revealed by a run under Intel Inspector XE, one of our Parallel Studio XE tools. There were others I have not looked at yet - not all may be "real", but this one definitely is.

[Steven_L_Intel1]

Ran it through Static Analysis, another Parallel Studio XE feature. In addition to the IETFLG issue above, it complained about these:

mt_rct5_updated.for(377): error #12244: memory pointed by "IFMTRF" is uninitialized

This is another pointer in the module which may be allocated but is never assigned a value in the program. IETFLG is similar - it is allocated but never given a value. This means you are relying on uninitialized values.

I see that you use lots of POINTERs in the module, but only a couple are ever used in pointer assignment. The rest could be changed to ALLOCATABLE. With POINTER, and your lack of initialization to NULL(), the majority of the pointers are in an undefined state.

I see you also allocate a lot of the pointer arrays to bounds (1,1,1), etc., and then address them out of bounds. That's just asking for trouble.

I think I've given you a lot to work on.