topic Strange problem with /Qauto option in Intel® Fortran Compiler

Strange problem with /Qauto option

gfellaqmulacuk — Tue, 02 Nov 2010 17:09:51 GMT

Hi,

I have the following problem.

I have a rather large Fortran programme. I have recently parallelized a part of it using OpenMP.

The code builds and runs fine on my Xeon X5570 with the following versions of Visual Fortran

11.1.065
11.1.048
11.0.075

The same code builds but crashes on my coauthor's server running two e5430 processors with the following versions of Visual Fortran

11.1.065
11.0.074

I have isolated the nature of the problem. It happens when I include the option /Qauto even if I run serially. It happens also when we build the code with OpenMP with the option /Qopenmp. The option /Qopenmp uses /Qauto. It does not happen in a parallelized section of the code. The programme crashes without any message when trying to inizialize an array of size 10000. If it is of any help, the programme is very memory intensive. The moment it crashes it is already using 3GB of memory. My coauthor's server though as three times (24GB) the memory as my desktop.

All compiling options are the same on both systems, in particular we use

/F:100000000
/STACK:1000000000

We are at a loss. It seems that the problem is processor dependent which makes little sense to us.

Many thanks in advance
Giulio

Strange problem with /Qauto option

Steven_L_Intel1 — Tue, 02 Nov 2010 18:12:10 GMT

I know of no specific changes that would affect this. It sounds like a stack overflow to me. What happens if you build with /heap-arrays ?

Strange problem with /Qauto option

gfellaqmulacuk — Tue, 02 Nov 2010 18:32:56 GMT

thanks for your reply!

The problem persists with

/heap-arrays-

Strange problem with /Qauto option

Steven_L_Intel1 — Tue, 02 Nov 2010 18:39:38 GMT

Um, I suggested "/heap-arrays" with no dash afterward.

Strange problem with /Qauto option

gfellaqmulacuk — Wed, 03 Nov 2010 00:16:20 GMT

ditto with /heap-arrays

Strange problem with /Qauto option

Steven_L_Intel1 — Wed, 03 Nov 2010 15:31:37 GMT

Does the same EXE behave the same way - not just rebuilding it on the other system? Is it linked to static or dynamic libraries?

Strange problem with /Qauto option

gfellaqmulacuk — Wed, 03 Nov 2010 17:35:48 GMT

The EXE built on my PC with option /QxSSE4.1 (which is the highest one compatible with my coauthor's processor) runs fine on my PC and still crashes (at the same point) on his server. Linked ibraries are dynamic.

Building with /check:all I get the warning

1>Space exceeded in Data Dependence Test in SIMUL
1>Subdivide routine into smaller ones to avoid optimization loss
1>An internal threshold was exceeded for routine SIMUL and optimization level may be reduced. See HTTP://SOFTWARE.INTEL.COM/EN-US/ARTICLES/INTERNAL-THRESHOLD-WAS-EXCEEDED for more information and advice.

on both my desktop and his server. SIMUL is indeed the subroutine where the crash takes place. Yet, I understand that error should not be fatal and indeed is not fatal on my desktop. Incidentally, the link in that error message is dead.

TIA

Strange problem with /Qauto option

Steven_L_Intel1 — Wed, 03 Nov 2010 18:06:54 GMT

The link should be lowercase - I'll report that. http://software.intel.com/en-us/articles/internal-threshold-was-exceeded/

These messages are just the optimizer saying "too much info!" and reducing optimizations. Probably the extra code added by /check:all did this.

Try building with static library linking and see what happens. I'm trying to reduce differences.

Strange problem with /Qauto option

gfellaqmulacuk — Wed, 03 Nov 2010 18:34:19 GMT

Still the same.

Would declaring the variables in the subroutine as allocatable help? My understanding is that with the option /heap-arrays it shoud not make a difference.

Strange problem with /Qauto option

Steven_L_Intel1 — Wed, 03 Nov 2010 20:47:59 GMT

We don't yet know what the actual error is. I just theorized that it was a stack overflow, though this ususllu gives some sort of message. You might check the system event log to see if it has a record of the program exiting abnormally (not sure if it would.)

Can you identify the statement where it fails?

Strange problem with /Qauto option

gfellaqmulacuk — Wed, 03 Nov 2010 21:13:02 GMT

I can.

it is initializing a real, double precision array of size 10000

zero=0.d0

Strange problem with /Qauto option

Steven_L_Intel1 — Thu, 04 Nov 2010 00:17:32 GMT

What happens if you replace this with a loop doing the initialization by element? It's possible the compiler is creating a temporary on the stack for this, though I'd expect better code for it.

Strange problem with /Qauto option

gfellaqmulacuk — Thu, 04 Nov 2010 17:38:15 GMT

We are making progress

The following is relevant section of the code. The variable bcpeople is a real, double precision array with dimension (90,3,10000)

do id = 1,gensize
zero(id) = 0.0d0
one(id) = 1.0d0 ! VECTORS OF ZEROS AND ONES
swo(id)= 0 ! Initialize edu choice
sw(id) = 0 ! everybody has at least compulsory schooling
print *, id
enddo
pause
bcpeople = 0.d0

The programme now crashes after the pause. So your conjecture was correct. The crazy thing is that the programme crashes before reaching the pause if we comment out the line

print *, id

Also we have discovered the programme does not crash if we supplement the option

/STACK: 2000000000

with

/STACK: 2000000000, commit

Thanks a lot for your help!

Strange problem with /Qauto option

Steven_L_Intel1 — Thu, 04 Nov 2010 17:58:31 GMT

Clearly running into memory limits - especially with OpenMP which has to create its own thread stacks. I am surprised, though, that /heap-arrays doesn't help. Is bcspeople just an ordinary local variable, or might it be a POINTER array that was allocated? Is it mentioned in any OMP directives?

Strange problem with /Qauto option

gfellaqmulacuk — Thu, 04 Nov 2010 18:11:56 GMT

What I cannot understand though is why the above loop crashes if I comment out the line

print *, id

bcpeople is just a local variable

The OMP section is in a subroutine called before the current one. The rest of the code is serial. I would hope that the thread stacks created by OMP have been cleared at this stage of the programme. The subroutine where the problem happens is serial.

Strange problem with /Qauto option

Steven_L_Intel1 — Thu, 04 Nov 2010 18:56:45 GMT

The serial portion runs in a thread anyway. Leaving out the print may affect inlining or other optimizations.

Strange problem with /Qauto option

gfellaqmulacuk — Thu, 04 Nov 2010 19:42:55 GMT

Just to understand. You are saying that even a subroutine called AFTER closing !$END OMP PARALLEL
is running in a thread?

Strange problem with /Qauto option

Steven_L_Intel1 — Thu, 04 Nov 2010 20:43:40 GMT

Yes - the master thread.

Strange problem with /Qauto option

gfellaqmulacuk — Fri, 05 Nov 2010 16:04:44 GMT

Gotcha. Many thanks for all your help!

Incidentally, one of the things I have learnt after all this going back and forth between the two systems is that X5570 and family are an amazing step forward. Congratulations to Intel!