Offload_transfer one variable on host to another on MIC (Fortran)

[james_B_8]

I have a problem where I need to copy an allocatable array on the host into a different allocatable array on the MIC.

I've been experimenting with different ways of trying to do it but the only way I've manage to get to work is the following:

[fortran]

module My_arrays
    implicit none

    real, allocatable :: a(:)
    !DIR$ ATTRIBUTES OFFLOAD : mic :: b
    real, allocatable :: b(:)

end module My_arrays

program Multi_dim_offload
    use My_arrays
    implicit none

    !DIR$ ATTRIBUTES OFFLOAD : mic :: SIZE
    integer, parameter :: SIZE=10
    integer :: i, j

    ! allocate on host
    allocate(a(SIZE))
    allocate(b(SIZE))

    ! give values on MIC
    do i=1,SIZE
        a(i) = i
    end do

    ! allocate b on the mic
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                    NOCOPY(b : length(SIZE) alloc_if(.TRUE.) &
                                free_if(.FALSE.) )

    ! send a values over into b on the MIC
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                        in(a : into(b) alloc_if(.FALSE.) free_if(.FALSE.) )


    print *, a
    !DIR$ OFFLOAD BEGIN TARGET(mic:0) nocopy(b)
    print *, b
    !DIR$ end OFFLOAD
    print *, "Should be the same..."

    !deallocate on MIC
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                        nocopy(b : alloc_if(.FALSE.) &
                                    free_if(.TRUE.) )

    !deallocate on host
    deallocate(a,b)

end program Multi_dim_offload

[/fortran]

My problem is that you apparently have to allocate the memory on the Host side even if the array is only to be used on the MIC side which is wasteful. Is there a better way of having an array that is only allocated on the MIC and you can do an OFFLOAD_TRANSFER INTO it?

Can anyone explain to me how this compiler managed MIC memory allocation actually works so I don't make any further misunderstandings, please?

[Kevin_D_Intel]

I still struggle with this topic myself. The variant on your program below allocates B only on the target.

You may have already seen these resources in the Fortran User Guide:

Managing Memory Allocation for Pointer Variables

Moving Data from One Variable to Another

There is an additional resource here: http://software.intel.com/en-us/articles/effective-use-of-the-intel-compilers-offload-features, specifically explicitly managed heap data. Unfortunately most content is C/C++ specific but some aspects can be exploited in Fortran too.

Let me know if there are still unanswered questions.

[fortran] module My_arrays

    implicit none

    !DIR$ ATTRIBUTES OFFLOAD : mic :: b,a

    real, allocatable :: a(:)

    real, allocatable :: b(:)

end module My_arrays

program Multi_dim_offload

    use My_arrays

    implicit none

    !DIR$ ATTRIBUTES OFFLOAD : mic :: SIZE

    integer, parameter :: SIZE=10

    integer :: i, j

    ! allocate on host

    allocate(a(SIZE))

    ! give values on MIC

    do i=1,SIZE

        a(i) = i

    end do

    ! allocate B on the MIC

    !DIR$ OFFLOAD BEGIN TARGET(mic:0) NOCOPY(b)

       allocate(b(SIZE))

    !DIR$ END OFFLOAD

    ! send a values over and assign to b on the MIC

    !DIR$ OFFLOAD begin TARGET(mic:0) &

                    IN(a : length(SIZE)) nocopy(b)

       b = a

    !dir$ end offload

    print *, a

    !DIR$ OFFLOAD BEGIN TARGET(mic:0) nocopy(b)

       print *, b

       deallocate(b)

    !DIR$ end OFFLOAD

    print *, "Should be the same..."

    !deallocate on host

    deallocate(a)

end program Multi_dim_offload

[/fortran]

[james_B_8]

Yeah that also works, but it also copies 'a' onto the target. The current problem I have is that I have some stupid struct that has an allocatable array in it and this array from this struct is referenced in an offload section. This makes things awkward for offloading it. Since I apparently can't offload a struct with an allocatable or pointer in it I figured I could just copy the array element by element over to the MIC using OFFLOAD_TRANSFER. So let's say 'a' is an allocatable array in some struct and 'b' is a temporary array that we can use in an offload region only on the MIC. So the code would just be identical to what I posted above except with 'a' replaced with 'Struct%a'.

To me more clear: the type that is used in the offload section is like this:

[fortran]

type myType

    integer :: npoints

    real, dimension(:), pointer :: points

end type

[/fortran]

Is there a way of copying an instantiation of 'myType' over to the mic?
 

[Sumedh_N_Intel]

I don't quite know how to rework your code without the host-side array. However, I may have a possible explanation why the compiler works the way it does. 

The Intel compiler uses hash-table like structure for memory allocated or de-allocated using the offload pragma/directive. The host-side pointer/array acts as a key to find the corresponding coprocessor-side array. For more details, please refer to the following blog : http://software.intel.com/en-us/blogs/2013/03/27/behind-the-scenes-offload-memory-management-on-the-intel-xeon-phi-coprocessor

I guess that you need to allocate host-side arrays to ensure that there are no collisions in the hash-table. Another point to note:  when transferring data to and from the coprocessors, the coprocessor-side source or destination arrays should be allocated using the offload runtime (either by offload or offload_transfer pragmas/directives). Hence, we cannot create arrays that are present only on the coprocessor by directlly allocating the array inside the offload. 

[Rajiv_D_Intel]

The array b needs to be allocated on the CPU because its CPU address is used as a handle to the MIC memory. Using the CPU address has advantages that the value can be passed around on the CPU between functions just like any other address, operated on, etc. and when needed, used as a handle to access the MIC memory.

However, as you've observed, this requires the array to be allocated on the CPU. Now, there is no need for it to be "fully" allocated. A size of 0 or 1 should suffice, but it doesn't because the offload library currently checks that the size allocated on the CPU matches the size requested on MIC. This check can probably be removed. We will look into it. I am filing a bug report to try and change this behavior.

[james_B_8]

@Sumedh. Thanks a lot for demystifying how it all works. 
@Rajiv Yes I tried to trick it like that already. But apparently the hash table stores the length of the array as well. Let me know if you find out why the length is needed also.
Cheers. 

[james_B_8]

Hi guys,

I managed to find a way to do it without allocating the same memory twice on the host side - by using pointers. Turns out you can make a pointer offloadable, point it at your allocated array (e.g. one in an awkward struct), then allocate and offload the pointer over to the MIC.

The ammended code is:

[fortran]

module My_arrays
    implicit none

    real, allocatable, target :: a(:)
    !DIR$ ATTRIBUTES OFFLOAD : mic :: b
    real, dimension(:), pointer :: b

end module My_arrays

program Alloc_offload
    use My_arrays
    implicit none

    !DIR$ ATTRIBUTES OFFLOAD : mic :: SIZE
    integer, parameter :: SIZE=10
    integer :: i, j

    ! allocate on host
    allocate(a(SIZE))

    ! give values on MIC
    do i=1,SIZE
        a(i) = i
    end do

    b => a

    ! allocate b on the mic
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                    NOCOPY(b : length(SIZE) alloc_if(.TRUE.) &
                                free_if(.FALSE.) )

    ! send a values over into b on the MIC
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                        in(b : alloc_if(.FALSE.) free_if(.FALSE.) )


    print *, a
    !DIR$ OFFLOAD BEGIN TARGET(mic:0) nocopy(b)
    print *, b
    !DIR$ end OFFLOAD
    print *, "Should be the same..."

    !deallocate on MIC
    !DIR$ OFFLOAD_TRANSFER TARGET(mic:0) &
                        nocopy(b : alloc_if(.FALSE.) &
                                    free_if(.TRUE.) )

    !deallocate on host
    deallocate(a)

end program Alloc_offload

[/fortran]

[james_B_8]

Even still though, I don't know why I can't offload a struct that has a pointer or an allocatable variable. You can offload pointers and allocatable variables if they exist outside of a struct, but once you place them inside one then it's no longer possible. You should be able to offload the struct anyway, but while treating the pointer / allocatable parts separately - ie. transferring them element by element. So the pointer / allocatable parts on the MIC could be left uninitialized, but the other stuff on the MIC would get the values. Then have unique offloads for the pointer / allocatable parts where you can copy them element by element.

Is it possible to get a feature request for that?

[Kevin_D_Intel]

Absolutely, James. We have an existing feature request (DPD200172750) to support offload of derived types with an allocatable component. I added your feedback and will keep you updated on any developments regarding this request.

I also wanted to note the internal tracking id (DPD200245090) Rajiv created for the current unnecessary limitation requiring the INTO array being fully allocated on the host. I keep you updated on any progress with this issue also.

(Internal tracking id: DPD200172750 - Support offload of derived types with allocatable components)
(Internal tracking id: DPD200245090 - Offload "in( a(n) : into b)" clause should not need b to be allocated of size n)

[james_B_8]

That's good to hear! Thanks, Kevin. 

For now, I can get around this problem not very cleanly, but more efficient on memory, by using pointers like in my latest example code.