Inlining and OpenMP

[griflet]

I get an access violation from a subroutine call in intel Fortran compiled with OpenMP:

forrtl: severe (157): Program Exception - access violation

Image PC Routine Line Source

forrtl: severe (157): Program Exception - access violationImage PC Routine Line Source

The program runs fine when either of the following is true:

1 - inlining is deactivated.

2 - check:pointer and check:arg_temp_created are activated.

3 - The code is compiled without openmp.

My questions are:

- Is the problem in my code?

- Is openmp in Fortran buggy?

- What can I do to investigate this further? (Debugging is opted-out because inlining is automatically deactivated...)

I really need to get this right, because I'm heavily reliant on the performance of the code.

Thx in advance,

Guillaume

[jimdempseyatthecove]

Do the functions being inlined contain "local"arrays?

If so, declare those functions as being recursive (in this context the intent is to declare as reentrant - but there is no keyword for that). Compiling with openmp is supposed to make the functions recursive-able (reentrant-able). This is an easy enough test to perform. (and is innocous if not successful).

Jim Dempsey

[griflet]

Hello,

@jimdempseyatthecoveThank you for your answer.

I don't know which functions are being inlined. I simply add the option at compilation time. However, the code is heavily reliant on both "local" arrays and global (module scope) arrays. Thus,I will investigate the "intent reentrant"/recursiveness of the functions. It seems like a good tip/pointer.

I also have another issue: the matricial results are different (slightly different, but different nevertheless) when running with or without openmp. The difference in the results is worsened when I compile with single precision.

@jimdempseyatthecove:Do you think this could have anything to do with the recursiveness of the functions?

Thank you, again. I'll keep this thread informed of the follow up.

Guillaume Riflet

[TimP]

recursive declaration is good for any program unit likely to be called in a parallel region, if you don't want to depend on it being compiled with OpenMP compatible options.

[jimdempseyatthecove]

>>I also have another issue: the matricial results are different (slightly different, but different nevertheless) when running with or without openmp. The difference in the results is worsened when I compile with single precision.

When using floating point instructions and when the data (or results computed from the data) cannot be exactly represented, then the results generated may depend on the sequence in which the results are calculated.

An example of this is when your data contains very large values and very small values. When adding the large values first, the small values might extend past the precision of the floating point numbers held in your accumulators (holding accumulation of large numbers). If you were to accumulate (work with) the small numbers first, then large numbers second, you can maintain the precision of the smaller numbers for a longer duration of your calculation. This often results in different outputs (where one output is more accurate than the other).

The smaller the precision the more apparent the difference.

Jim Dempsey

[griflet]

Hello,

I went on parallelizing the Fortran95 700,000 lines long code (mohid.codeplex.com) but I disabled the "inline" option as sometimes I do get subroutines calls within prominent three-index do-loops (k,j,i) and I'm not sure how openmp handles "inlined" subroutines in what regards the privacy (PRIVATE) of local variables.

Basically, disabling "inlining" solved my "Access violation" problem I had several months ago, when I started this thread.

>

Meanwhile, I got more confident on PRIVATE/FIRSTPRIVATE/LASTPRIVATE issues, and developed a fairly efficient tactic consisting in parallelizing only do-loops. Whenever I needed to replicate pointed arrays for threading purposes, I would allocate each array once per thread. (See here for more detailshttp://guillaume-maretec.blogspot.com/2011/01/mohid-openmp-newsletter-1.html.It's in portuguese, mind you, but the code is universal and you can always use google translate...)

Eventually, I'd get different results between "Release Double" and "Release Double Openmp" configurations (I'm using VS2008). These were due to some missing PRIVATE variable (or pointed array) 99% of the time.

HOWEVER

I do am getting a troubling difference in the numerical results (I'm using intel fortran 11.1) of a time-serie between "Release Double" and "Release Double Openmp".

The error starts in the last decimal case, after a few iterations, and keeps creeping up in decimal cases. It looks like a missing PRIVATE variable issue ... BUT ...

i) the difference is *exactly* the same regardless of the number of threads I use (> set OMP_NUM_THREADS=n).

ii) If I comment *every* "!$OMP" directive (and remove every "!$" special line) then, supposedly, the code should be the same, no parallel zone should ever be created and yet, I still get exactly the same difference, listed below.

iii) I double checked this issue in two different computers accessing the same code over subversion. I get exactly these two time-series (one without /Qopenmp and the other with /Qopenmp), so it's probably not some goofy mistake in the configurations of "Release Double" or of the "Release Double Openmp" versions, but I'm open to suggestions...

What could it be? How can I fix this?

0.000000000000E+000 0.000000000000E+000
-0.105823438141E+000 -0.105823438141E+000
-0.101177683635E+000 -0.101177683635E+000
-0.965033372036E-001 -0.965033372036E-001
-0.916178948622E-001 -0.916178948622E-001
-0.866968377523E-001 -0.866968377523E-001
-0.851175220978E-001 -0.851175220978E-001
-0.834703111146E-001 -0.834703111146E-001
-0.843959497805E-001 -0.843959497805E-001
-0.852983885223E-001 -0.852983885223E-001
-0.865564116118E-001 -0.865564116118E-001
-0.878180942749E-001 -0.878180942749E-001
-0.883935902781E-001 -0.883935902781E-001
-0.889729628494E-001 -0.889729628494E-001
-0.886675796284E-001 -0.886675796284E-001
-0.883617100034E-001 -0.883617100034E-001
-0.874205153543E-001 -0.874205153543E-001
-0.864758951129E-001 -0.864758951129E-001
-0.856351794309E-001 -0.856351794309E-001
-0.847830882336E-001 -0.847830882336E-001
-0.845019748030E-001 -0.845019748030E-001
-0.842053654683E-001 -0.842053654683E-001
-0.841420242241E-001 -0.841420242241E-001
-0.840677382198E-001 -0.840677382198E-001
-0.838377807080E-001 -0.838377807080E-001
-0.836098825143E-001 -0.836098825143E-001
-0.835342988516E-001 -0.835342988516E-001
-0.834627734774E-001 -0.834627734774E-001
-0.840710621853E-001 -0.840710621853E-001
-0.846850223350E-001 -0.846850223351E-001
-0.859248288430E-001 -0.859248288433E-001
-0.871653859096E-001 -0.871653859099E-001
-0.883854141278E-001 -0.883854141283E-001
-0.896102930107E-001 -0.896102930115E-001
-0.901481478526E-001 -0.901481478536E-001
-0.906905625945E-001 -0.906905625957E-001
-0.903803808473E-001 -0.903803808483E-001
-0.900704347282E-001 -0.900704347289E-001
-0.893281813382E-001 -0.893281813376E-001
-0.885821591390E-001 -0.885821591371E-001
-0.879997466860E-001 -0.879997466820E-001
-0.874106226533E-001 -0.874106226472E-001
-0.872762768815E-001 -0.872762768745E-001
-0.871330394725E-001 -0.871330394646E-001
-0.872845351839E-001 -0.872845351783E-001
-0.874304524208E-001 -0.874304524176E-001
-0.875925939797E-001 -0.875925939818E-001
-0.877524458909E-001 -0.877524458984E-001
-0.878952917913E-001 -0.878952918036E-001

Thank you,

Guillaume Riflet