Strange problem with /Qauto option

[gfellaqmulacuk]

Hi,

I have the following problem.

I have a rather large Fortran programme. I have recently parallelized a part of it using OpenMP.

The code builds and runs fine on my Xeon X5570 with the following versions of Visual Fortran

11.1.065
11.1.048
11.0.075

The same code builds but crashes on my coauthor's server running two e5430 processors with the following versions of Visual Fortran

11.1.065
11.0.074

I have isolated the nature of the problem. It happens when I include the option /Qauto even if I run serially. It happens also when we build the code with OpenMP with the option /Qopenmp. The option /Qopenmp uses /Qauto. It does not happen in a parallelized section of the code. The programme crashes without any message when trying to inizialize an array of size 10000. If it is of any help, the programme is very memory intensive. The moment it crashes it is already using 3GB of memory. My coauthor's server though as three times (24GB) the memory as my desktop.

All compiling options are the same on both systems, in particular we use

/F:100000000
/STACK:1000000000

We are at a loss. It seems that the problem is processor dependent which makes little sense to us.


Many thanks in advance
Giulio

[Steven_L_Intel1]

I know of no specific changes that would affect this. It sounds like a stack overflow to me. What happens if you build with /heap-arrays ?

[gfellaqmulacuk]

thanks for your reply!

The problem persists with

/heap-arrays-

[Steven_L_Intel1]

Um, I suggested "/heap-arrays" with no dash afterward.

[gfellaqmulacuk]

ditto with /heap-arrays

[Steven_L_Intel1]

Does the same EXE behave the same way - not just rebuilding it on the other system? Is it linked to static or dynamic libraries?

[gfellaqmulacuk]

The EXE built on my PC with option /QxSSE4.1 (which is the highest one compatible with my coauthor's processor) runs fine on my PC and still crashes (at the same point) on his server. Linked ibraries are dynamic.

Building with /check:all I get the warning


1>Space exceeded in Data Dependence Test in SIMUL
1>Subdivide routine into smaller ones to avoid optimization loss
1>An internal threshold was exceeded for routine SIMUL and optimization level may be reduced. See HTTP://SOFTWARE.INTEL.COM/EN-US/ARTICLES/INTERNAL-THRESHOLD-WAS-EXCEEDED for more information and advice.

on both my desktop and his server. SIMUL is indeed the subroutine where the crash takes place. Yet, I understand that error should not be fatal and indeed is not fatal on my desktop. Incidentally, the link in that error message is dead.

TIA

[Steven_L_Intel1]

The link should be lowercase - I'll report that. http://software.intel.com/en-us/articles/internal-threshold-was-exceeded/

These messages are just the optimizer saying "too much info!" and reducing optimizations. Probably the extra code added by /check:all did this.

Try building with static library linking and see what happens. I'm trying to reduce differences.

[gfellaqmulacuk]

Still the same.

Would declaring the variables in the subroutine as allocatable help? My understanding is that with the option /heap-arrays it shoud not make a difference.

[Steven_L_Intel1]

We don't yet know what the actual error is. I just theorized that it was a stack overflow, though this ususllu gives some sort of message. You might check the system event log to see if it has a record of the program exiting abnormally (not sure if it would.)

Can you identify the statement where it fails?

[gfellaqmulacuk]

I can.

it is initializing a real, double precision array of size 10000

zero=0.d0

[Steven_L_Intel1 · Accepted solution]

What happens if you replace this with a loop doing the initialization by element? It's possible the compiler is creating a temporary on the stack for this, though I'd expect better code for it.

[gfellaqmulacuk]

We are making progress

The following is relevant section of the code. The variable bcpeople is a real, double precision array with dimension (90,3,10000)

do id = 1,gensize
zero(id) = 0.0d0
one(id) = 1.0d0 ! VECTORS OF ZEROS AND ONES
swo(id)= 0 ! Initialize edu choice
sw(id) = 0 ! everybody has at least compulsory schooling
print *, id
enddo
pause
bcpeople = 0.d0

The programme now crashes after the pause. So your conjecture was correct. The crazy thing is that the programme crashes before reaching the pause if we comment out the line

print *, id



Also we have discovered the programme does not crash if we supplement the option

/STACK: 2000000000

with

/STACK: 2000000000, commit

Thanks a lot for your help!

[Steven_L_Intel1]

Clearly running into memory limits - especially with OpenMP which has to create its own thread stacks. I am surprised, though, that /heap-arrays doesn't help. Is bcspeople just an ordinary local variable, or might it be a POINTER array that was allocated? Is it mentioned in any OMP directives?

[gfellaqmulacuk]

What I cannot understand though is why the above loop crashes if I comment out the line

print *, id

bcpeople is just a local variable

The OMP section is in a subroutine called before the current one. The rest of the code is serial. I would hope that the thread stacks created by OMP have been cleared at this stage of the programme. The subroutine where the problem happens is serial.

[Steven_L_Intel1]

The serial portion runs in a thread anyway. Leaving out the print may affect inlining or other optimizations.

[gfellaqmulacuk]

Just to understand. You are saying that even a subroutine called AFTER closing !$END OMP PARALLEL
is running in a thread?

[Steven_L_Intel1]

Yes - the master thread.

[gfellaqmulacuk]

Gotcha. Many thanks for all your help!

Incidentally, one of the things I have learnt after all this going back and forth between the two systems is that X5570 and family are an amazing step forward. Congratulations to Intel!