two dimensional array offload issue

[andrei_k_]

i have two dimensional dynamic array that i offload to phi. i dont really pass any data all i want is to allocate mem via transfer and access that mem via nocopy each iteration later on

void foo()

unsigned int ** twoDimArray = new ... etc ... [n*m]

//allocate
#pragma offload_transfer target(mic:MIC_DEV) in(twoDimArray :length(n*m) alloc_if(1) free_if(0))

while (condition) {

//nocopy offload each iteration of external loop
#pragma offload target(mic:MIC_DEV) nocopy(twoDimArray :length(n*m) alloc_if(0) free_if(0))

}

//deallocate
#pragma offload_transfer target(mic:MIC_DEV) nocopy(twoDimArray :length(n*m) alloc_if(0) free_if(1))

i constantly get a  " offload error: process on the device 0 was terminated by signal 11 (SIGSEGV) "

[jimdempseyatthecove]

The problem you have is toDimArray is an array of pointers to memory on your local system. What you've done above is pass the array of pointers which is useless as the address spaces on host and MIC are not shared.

After allocating twoDimArray (which is a table of pointers to your data), you have two ways of filling out the intended data your twoDimArray on the host:

a) allocate one large block m*n*sizeof(int), then create a table of row pointers in twoDimArray, or
b) perform separate allocations for each row of twoDimArray

Choosing a) is better for MIC offloads since you can pass the raw block of data representing the content of the array. Then subsequently on MIC you can construct the pointers into a MIC instance of twoDimArray.

Choosing b) will require you to pass each row as separate offloads, incorporating the into clause.

Once data (and twoDimArray constructed) you can run the nocopy loop as you do above

I suggest you use a) if possible.

Jim Dempsey

[andrei_k_]

i should have been explicit but left out the obvious. i was already doing b) and this was not working. in addition, i also tried using in instead of nocopy with lengh(0) but it did not work either

//allocate
#pragma offload_transfer target(mic:MIC_DEV) in(twoDimArray :length(n*m) alloc_if(1) free_if(0))

while (condition) {

//nocopy offload each iteration of external loop
#pragma offload target(mic:MIC_DEV) in(twoDimArray :length(0) alloc_if(0) free_if(0))

}

//deallocate
#pragma offload_transfer target(mic:MIC_DEV) nocopy(twoDimArray :length(n*m) alloc_if(0) free_if(1))

[jimdempseyatthecove]

I misspoke, V15 reference manual states pointer arrays are supported, therefore b) is supported (assuming it is supported without bug).

#define ARRAY_SIZE 4
#define DATA_ELEMS 100

__declspec(target(mic)) int start[ARRAY_SIZE];
__declspec(target(mic)) int len[ARRAY_SIZE];
__declspec(target(mic)) int allocif[ARRAY_SIZE];
__declspec(target(mic)) int freeif[ARRAY_SIZE];
__declspec(target(mic)) int *p[ARRAY_SIZE];

int main() {
  int i, j;
  bool failed = false;

  for (i=0; i<ARRAY_SIZE; i++) {
    // Alloc ptr array elements; assume memory is available
    p = (int *)malloc(sizeof(int)*DATA_ELEMS);
    p[0:DATA_ELEMS] = i;
  }
 
  start[:] = 1;
  len[:] = 98;

  // Start is a section and length is also a section
  // Default values of free_if and align
  // alloc_if uses a vector and free_if a scalar that is expanded
  // Allocate only
  allocif[:] = 1;
  #pragma offload_transfer target(mic) \
  nocopy( p[0:ARRAY_SIZE] : \
  extent(start[0:ARRAY_SIZE]:len[0:ARRAY_SIZE]) \
  alloc_if(allocif[0:ARRAY_SIZE]) \
  free_if(0) )

  // Do the offload reusing memory
  #pragma offload target(mic) \
  inout( p[0:ARRAY_SIZE] : \
  extent(start[0:ARRAY_SIZE]:len[0:ARRAY_SIZE]) \
  alloc_if(0) \
  free_if(0) ) { 
    for (i=0; i<ARRAY_SIZE; i++) { p[start:len] += 1; }
  }

  // Free the memory
  // alloc_if uses a scalar, free_if a vector
  freeif[:] = 1;
  #pragma offload_transfer target(mic) \
  nocopy( p[0:ARRAY_SIZE] : \
  extent(start[0:ARRAY_SIZE]:len[0:ARRAY_SIZE]) \
  alloc_if(0) \
  free_if(freeif[0:ARRAY_SIZE]) )

 ...
 return 0;
}

In the above they are using the CEAN syntax.

In addition to the array of pointers on host, you also need to supply additional information regarding the array.

If alignment is required on target here is an example:

#define ARRAY_SIZE 4
#define DATA_ELEMS 100

__declspec(target(mic)) int start[ARRAY_SIZE];
__declspec(target(mic)) int len[ARRAY_SIZE];
__declspec(target(mic)) int align[ARRAY_SIZE];
__declspec(target(mic)) float *p[ARRAY_SIZE];
float *q[ARRAY_SIZE];

int main() {
  int i, j;
  bool failed = false;
  bool align_failed = false;

  for (i=0; i<ARRAY_SIZE; i++) {
    // Alloc ptr array elements; assume memory is available
    p = (float *)malloc(sizeof(float)*DATA_ELEMS);
    q = (float *)malloc(sizeof(float)*DATA_ELEMS);
    p[0:DATA_ELEMS] = i;
    q[0:DATA_ELEMS] = p[0:DATA_ELEMS];
  }
 
  start[0] = 0;
  start[1] = 1;
  start[2] = 1;
  start[3] = 0;
  len[0] = DATA_ELEMS;
  len[1] = DATA_ELEMS - 2;
  len[2] = DATA_ELEMS - 2;
  len[3] = DATA_ELEMS;
  align[0] = 2048;
  align[1] = 4096;
  align[2] = 8192;
  align[3] = 8;

  // Start is a section and length is also a section
  // Default values of alloc_if, free_if
  // Special alignment
  // Array allocations will start at element 0, but transfers will not
  // Update some elements and get them from MIC
  // UUUUUUUUUUUUUU
  // .UUUUUUUUUUUU.
  // .UUUUUUUUUUUU.
  // UUUUUUUUUUUUUU

  #pragma offload target(mic) \
  inout( p[0:ARRAY_SIZE] : \
  extent(start[0:ARRAY_SIZE]:len[0:ARRAY_SIZE]) \
  align(align[0:ARRAY_SIZE]) ) { 
    for (i=0; i<ARRAY_SIZE; i++) {
       if (((long long)&p[0] & (align-1)) != 0) {
          align_failed = true;
          printf("p[%d] failed alignment\n", i);
          fflush(0);
       }
    p[start:len] += 1.0;
    }
 }
 ...
 return 0;
}

The for loop they stuck in there is to illustrate to you that the data did get allocated aligned. You would remove that in your application.

Also, I assume you can rework those examples into using offload_transfer.

Sorry for the misinformation.

Run the test examples first, when working then rework the test example to use offload_transfer, when working then work it into your code.

Jim Dempsey