Different results when threads > 3 with openMP

[ByoungSeon_J_]

Hi, I am converting some legacy fortran code into openmp version. It uses some external functions like

interface
  function rotTens(x,mat)
  real,intent(in)::x(6),mat(3,3)
  real:: rotTens(6)
  end function
end interface
...
!$omp parallel do &
!$omp default(shared)&
!$omp private(I,j,x,y) &
do I=1, n_time
do j=1, neighbor(I)
   y = rotTens(x,mat)
end do
end do

Results are same up to threads 1~3, but 4 or more threads yield different answer every time (changes every time). The change is very small but noticeable (~0.01%). I checked variables of the function and it seems that memory is corrupted(? inside of the function, rotTens = x*mat  but printing rotTens and x*mat show different values). Such corruption is not found in all loops but in small fractions, randomly. In other words, error or corruption comes from non-consistent points. Very hard to predict or fix.

I haven't had this kind of experience before, and could anyone give me some advice or tip? The OpenMP loop is quite lengthy  (a few hundreds lines) . I tested ifort 13/14.1 and both yielded similar results. Any comments will be greatly appreciated.

Thanks,

ByoungSeon

[jimdempseyatthecove · Accepted solution]

Check to see if you have any function or subroutine that is in a library that was NOT compiled as an OpenMP compatible routine (with -openmp). If not, then if any of those functions or subroutines contain local arrays (real :: temp(3)), then those arrays would be SAVE and subject to race condition.

If that doesn't resolve the issue, then change your $omp default(shared) to $omp default(none) and fix the errors where you really intend the error-ing variable to be shared, and make those erroneously shared, private.

That failing, check to see if your loop has temporal dependencies (one such example is iteration n must precede iteration n+1).

Jim Dempsey

[ByoungSeon_J_]

I appreciate Jim's comment. The external function is located in a separate file, which is not compiled with -openmp option. It is a little bit weird for me that no OMP directive file is compiled with -openmp option but it resolve the problem! Thanks again.

Best regards,

ByoungSeon

[TimP]

You should have seen this explained many times.  If you don't want to set -auto or -openmp (which implies -auto) for compilation of procedures called in a parallel region, you can declare those procedures RECURSIVE.  The default of making local arrays implicitly SAVEd may be a relic of the years before OpenMP came with Fortran compilers, or maybe from the time before the standard required DATA to imply SAVE.