[DEADLOCK] in target regions using OMP parallel and parameters

[Grund__Aalexander]

Look at the following code:

subroutine changeDevice2(vals,xMax,val)
  implicit none
  
  integer :: xMax, val, i
  integer,  dimension(xMax) :: vals
  
  !$omp target
  !$omp parallel do
  do i=1,xMax
    if(vals(0).EQ.val) then
      vals(1)=2
    endif
  enddo
  write(*,*) "Try returning from 2"
  !$omp end target
  write(*,*) "Returned from 2"
end subroutine

subroutine changeDevice1(val)
  implicit none
  
  integer :: val,i
  integer, allocatable :: vals(:)
  integer :: xMax
  
  xMax = 100  
  allocate(vals(xMax))
  vals = 1

  !$omp target map(vals)
  !$omp parallel do
  do i=1,xMax
    if(vals(0).EQ.val) then
      vals(1)=2
    endif
  enddo
  write(*,*) "Try returning from 1"
  !$omp end target
  write(*,*) "Returned from 1"
end subroutine

subroutine changeDeviceWorks(val)
  implicit none
  
  integer :: val,valLoc,i
  integer, allocatable :: vals(:)
  integer :: xMax
  
  xMax = 100  
  allocate(vals(xMax))
  vals = 1
  valLoc=val

  !$omp target map(vals)
  !$omp parallel do
  do i=1,xMax
    if(vals(0).EQ.valLoc) then
      vals(1)=2
    endif
  enddo
  write(*,*) "Try returning from works"
  !$omp end target
  write(*,*) "Returned from works"
end subroutine

program fortTest

  implicit none
  
  integer, allocatable :: vals(:)
  integer :: xMax

  xMax = 100  
  allocate(vals(xMax))
  vals = 1
  
  call changeDeviceWorks(1)
  
  call changeDevice1(1)
  
  !$omp target data map(vals)
  call changeDevice2(vals,xMax,1)
  !$omp end target data
end

The changeDevice1 and changeDevice2 methods will never return from their target regions.
Using a local variable instead of the parameter works (see changeDeviceWorks)
Also removing the !$omp parallel do makes it return.

I consider this as critical as it renders quite some applications not working and occasionally even hangs the whole login session when trying to abort the program.

[Kevin_D_Intel]

Thank you for the report Alexander. I reproduced the symptoms described and reported this to Development (internal tracking id below) and keep you updated about a fix.

(Internal tracking id: DPD200255289)

(Resolution Update on 07/08/2014): This tracking id has been closed and two new tracking ids created (see subsequent post for details) to track separately the associated MPSS defect and Compiler enhancement request.)

[Grund__Aalexander]

Update: There are similar problems, if the parameter is used in a loop like:

do i=minX+myParam, maxX+myParam

Update2: Also simple assignments can cause this problem.

However there is a workaround: Using the parameter in a map(to:myParam) makes it work.

[Kevin_D_Intel]

The hang involving the constant (and associated dummy argument val) relates to the compiler assigning the default map-type of tofrom to the associated dummy argument. We witnessed a similar issue with the Intel offload. I believe the compiler should be able to assign the to map-type in this scenario. Both changeDevice1 and changeDevice2 complete successfully when assigning the dummy argument the map-type of ‘to’ (e.g. add map(to:val) to the omp target directives).

You didn’t show it, but I assume myParam was declared in the caller with the “parameter” attribute.  If so, then yes, it is the same situation occurring as with passing the constant. The constant and parameter should be in read-only memory and I believe the compiler can detect this and assign the to map-type instead of the default tofrom.

I included a note in the internal tracking report about the parameter scenario also.

[Grund__Aalexander]

Yes you were right, this happens only with constants and "parameters". Passing a variable does work.

Reason seems to be indeed that copying the value out of the target region fails due to it being a constant. A normal assignment would cause a SEGFAULT, which would be better behavior than a hang as it would be debugable.

However this could be considered a programming error as according to the spec, the variable has to have an implicit tofrom.

I think best way would be to throw a warning at compile time, that this might not be what is intended.

BTW: OpenACC is a bit smarter there. It does distinguish if a variable is read and/or written and applies the best implicit map clause.

[Kevin_D_Intel]

Just an update. The original tracking id created earlier (DPD200255289) was closed and two new tracking ids were created as described below.

The run-time hang root cause was traced back to an issue inside MPSS. There is a fix in progress (see HSD# tracking id noted below) on the MPSS side to address the run-time hang. I will notify you about the availability in a new MPSS release once available.

A separate compiler feature enhancement (see internal tracking id below) was created for consideration/investigation of assigning the best implicit default map assignment for a variable based on what is known about the variable.

I will update you again as I learn more.

(Internal tracking id: HSD 4869261 - COIBufferCopy hang when referencing dummy argument associated with constant)
(Internal tracking id: DPD200533452 - Assign best implicit default MAP assignment for variables)