OpenMP FIRSTPRIVATE causes unusual behaviour

[Jozsef_K_]

Dear Community,

I'm asking your help with an OpenMP related parallelization problem. The following code is considered:

PROGRAM testfpr
  IMPLICIT NONE

  INTEGER :: i, j

  i = 1
  j = 2
  CALL sub1( i, j )

END PROGRAM testfpr

SUBROUTINE sub1( a, b )
  IMPLICIT NONE

  INTEGER, INTENT(IN) :: a, b
  INTEGER :: k

  !$OMP PARALLEL DEFAULT(PRIVATE) &
  !$OMP          FIRSTPRIVATE(a,b)
  !$OMP DO
  DO k = 1, 100
    !PRINT *,a,b
    CALL sub2( a, b )
  ENDDO  ! k
  !$OMP ENDDO
  !$OMP END PARALLEL

END SUBROUTINE sub1

SUBROUTINE sub2( c, d )
  IMPLICIT NONE

  INTEGER, INTENT(IN) :: c, d
  INTEGER, DIMENSION(1:10), PARAMETER :: aa = (/ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 /)

  PRINT *, c, d
  PRINT *, aa(c), aa(d)

END SUBROUTINE sub2

Compiling with ifort -O0 -g -openmp -check all testfpr.f90 and running it on two threads does not print anything. If running it on a single thread it crashes. However, if the PRINT statement in subroutine sub1 is uncommented, it's working  and printing the expected values from SUBROUTINE sub2. A more complicated real life code shows similar behaviour. The compiler is v13.1.3, running on a Linux cluster 2.6.32-431.11.2.el6.x86_64.

After experimenting somewhat with the code above, I believe that the problem is related to the FIRSTPRIVATE clause. If the variable on which the clause is imposed is not directly used in the PARALLEL section, but it is passed to a subroutine, then something is going wrong and the value of the original variable is not copied to the thread-local instance of the variable. However, there may be some other problems as well, since sometimes SUBROUTINE sub2 seems not to be called at all. Furthermore, if the !$OMP PARALLEL directive is unified with the !$OMP DO directive into a single one, it is working well. The same problem does not show up with a GNU compiler (4.6.4).

Am I doing something wrong, or is this a bug somewhere?

Thank you for your help,

Jozsef

[Kevin_D_Intel]

I confirmed your findings when using newer compilers too. I do not know what is causing this behavior. I managed to trigger a segv when limiting to a single thread so that might shed more clues. It does not appear you've done anything wrong. I will look into this further and post again after learning more.

[jimdempseyatthecove]

What happens when using COPYIN?

Jim Dempsey

[Jozsef_K_]

Dear Kevin and Jim,

Thank you for your replies. Replacing FIRSTPRIVATE with COPYIN results wrong (uninitialized) c and d variables and runtime error with array boundary check. But, at least subroutine sub2 is running. On the other hand, I assumed that COPYIN shall be used with THREADPRIVATE variables, and these can only be global or common block variables.

Jozsef

[jimdempseyatthecove]

THREADPRIVATE variables are different from PRIVATE variables. COPYIN goes into PRIVATE variables. The PRIVATE variables are typically placed on the respective stacks of the threads participating in the parallel region.

THREADPRIVATE have thread global visibility and are usually declared in a module or common, but possibly could be subroutine SAVE as well (though I haven't tried THREADPRIVATE-ing a subroutine SAVE variable). There is a different mechanism for THREADPRIVATE variables. The privatization technique varies with the runtime system. The most efficient way uses one of the segment registers FS or GS which is then mapped to a different (virtual) base address within the process. Less efficient ways use a thread number to index a global table of context pointers.

Jim Dempsey

[Kevin_D_Intel]

Jozsef - Please accept my apologies. I just noticed that I failed to follow up with you on your issue earlier this year. At the time of your post I submitted the issue to Development (Internal tracking id: DPD200357738); but overlooked notifying about that. There was an underlying defect that has been fixed in the most recent Intel® Parallel Studio XE 2015 Update 1 now available from the Intel Registration Center.

I apologize again for my oversight.