Does (simple) array slicing produce a copy or a pointer to the original array?

[Karanta__Antti]

I am trying to turn on interface checking in an old codebase. This requires quite a few changes, as many things that work and have (unfortunately) been used quite extensively are not according to the Fortran standard and cause compilation errors when using /gen-interfaces:nosource

Most are fairly straight forward to fix.

The case I am wondering about is passing a "subarray" as a parameter. As an example

! assume this subroutine is not in a module and the caller does not have an explicit interface
subroutine assign_zeros(a, n)
  integer a(*), n
  integer i

  do i=1,n
    a(i)=0
  end do
end subroutine

somewhere else:

integer x(10)

...

! are these equivalent?
! assign zeros to array x from index 5 to 10
call assign_zeros( x(5), 6 ) ! this way of passing a portion of an array as an array is common in the code base. Works fine as x(5) passes pointer to the fifth element in the array
call assign_zeros( x(5:), 6 )

To make the compilation pass when using interface checks, I thought I'd just change from the first form to the latter. But getting an access violation in a given spot in the code when simply adding the : seems to indicate that x(5:) actually creates a copy, where I thought it would just pass a pointer to the fifth element in the array. Is this what happens? Can I affect it somehow so no copy is made?
I do understand that e.g. in some more complex uses of slices the compiler must make a copy (e.g. 5:10:2), but for a simple case like x(5:) for a simple contiguous array x I thought this would be equivalent to doing something like this is C:

// C code
int x[10];
assign_zeros(x + 4, 6); // this would be equivalent to the above fortran calls

If Fortran semantics for slices actually do always make a copy, is there some other way I could obtain a "subarray" in the way that I need?
Changing all the routines so that instead of an array they take an array and a starting subscript is not possible, as it would require huge amount of work.

The change that caused an access violation is from this:

CALL DM_ARRAYCOPY32( II( IR + PROCESSED_WORDS ), TDBB( ISTT ), 1 )

To this

CALL DM_ARRAYCOPY32( II( IR + PROCESSED_WORDS: ), TDBB( ISTT ), 1 )

II having been defined as

INTEGER, PARAMETER :: DMSIZE = 2
INTEGER II
COMMON II(DMSIZE)

There is out of bounds access here, as II is used as a sort of base from where to point to memory of a given address. But that has been used for a long time and should not be the issue here, unless slicing somehow reacts badly to taking a slice that is out of the array bounds?

The called routine is simply

SUBROUTINE DM_ARRAYCOPY32( I1, I2, NW )
  IMPLICIT NONE 
  INTEGER, INTENT(IN) :: NW
  INTEGER*4, INTENT(IN) :: I1(*)
  INTEGER*4, INTENT(OUT) :: I2(*)

  INTEGER I
  DO I = 1, NW
    I2( I ) = I1( I )
  END DO
END

NB: there is also a datatype mismatch here, but that is intentional in the calling code and from the point of interface checking a separate solvable issue.

Environment: windows 10 x64, ifort 19.0.5.281 Build 20190815

[FortranFan]

@Karanta__Antti ,

Chances are highly you will NOT get anywhere with the information you're sharing.  It's incomplete, way too verbal and far less usable code, and you present no clear evidence of the problem.

You've 2 options: if your support allows, contact Intel OSC (https://supporttickets.intel.com/servicecenter?lang=en-US) and share all the code and build toolset details and they may be able to help.

Or work better to have a small, clear reproducer and perhaps readers here, especially the Black Belts like mecej4, can guide you.

Perhaps you can consider taking something like and modifying it just enough to reproduce your problem?

Your file 'DM_ARRAYCOPY32', say the file is named 'dm.for'

      SUBROUTINE DM_ARRAYCOPY32( I1, I2, NW )
        IMPLICIT NONE 
        INTEGER, INTENT(IN) :: NW
        INTEGER*4, INTENT(IN) :: I1(*)
        INTEGER*4, INTENT(OUT) :: I2(*)
        character(len=*), parameter :: fmtt = "(g0,t10,g0)"
      
        INTEGER I
        print *, "In DM_ARRAYCOPY32"
        print fmtt, "I", "LOC(I1(I))"
        DO I = 1, NW
           print fmtt, I, LOC(I1(I))
          I2( I ) = I1( I )
        END DO
      END

A work subroutine 'sub' to mimic the data processing: file 'sub.f90'

subroutine sub()
   interface
      SUBROUTINE DM_ARRAYCOPY32( I1, I2, NW )
        IMPLICIT NONE 
        INTEGER, INTENT(IN) :: NW
        INTEGER, INTENT(IN) :: I1(*)
        INTEGER, INTENT(OUT) :: I2(*)
      END SUBROUTINE
   end interface
   INTEGER, PARAMETER :: DMSIZE = 2
   INTEGER II
   COMMON II(DMSIZE)
   integer :: TDBB(5)
   integer :: IR, PROCESSED_WORDS, ISTT
   IR = 0
   PROCESSED_WORDS = 5
   ISTT = 1
   print *, "In sub: LOC( II( IR + PROCESSED_WORDS ) ) = ", LOC( II( IR + PROCESSED_WORDS ) )
   CALL DM_ARRAYCOPY32( II( IR + PROCESSED_WORDS: ), TDBB( ISTT ), 1 )
   print *, "Back in sub: TDDB( ISTT ) = ", TDBB( ISTT ), "; expected is 5"
end subroutine 

A driver program to try to reproduce the access violation error: file 'av.f90'

   INTEGER II
   COMMON II(10)
   II = [( ISTT, ISTT=1, size(II) )]
   call sub()
end 

Now the build steps using IFORT and program output:

C:\Temp>ifort /c /integer-size:64 /nologo dm.for

C:\Temp>ifort /c /integer-size:64 /nologo sub.f90

C:\Temp>ifort /c /integer-size:64 /nologo av.f90

C:\Temp>link dm.obj sub.obj av.obj /subsystem:console /out:av.exe
Microsoft (R) Incremental Linker Version 14.26.28806.0
Copyright (C) Microsoft Corporation.  All rights reserved.


C:\Temp>av.exe
 In sub: LOC( II( IR + PROCESSED_WORDS ) ) =        140695493870048
 In DM_ARRAYCOPY32
I        LOC(I1(I))
1        140695493870048
 Back in sub: TDDB( ISTT ) =                      5 ; expected is 5

C:\Temp>

So note from the output, there is neither an access violation error nor an array copy in effect given the addresses are identical at 140695493870048.

Note in this trivial reproducer I have tried to include all the intricacies you have brought up such as with array bound violation, interface mismatch in terms of types, and setting the default integer to Intel's  size 64.

Hence you can ask yourself why does this test case appear different from your actual code?  Investigate what's different.  And show here a similar reproducible case that readers can check and verify by themselves first.

[Karanta__Antti]

Hi FortranFan. Thanks for your reply.

I do understand your complaint that I have been maybe too verbal. That has come out of an effort to make it clear what I am doing, why and what is the problem.

However, I do not see how you can claim that I provide no usable code and no clear evidence of the problem.

In this message (https://community.intel.com/t5/Intel-Fortran-Compiler/Does-simple-array-slicing-produce-a-copy-or-a-pointer-to-the/m-p/1243130/highlight/true#M153704) I provide a full, admittedly a bit verbose, sample code that illustrates the problem. It can be compiled and run on its own and it demonstrates the problem.
The verbosity comes from me trying to be precise of what is wrong and as what is happening is quite esoteric, taking care to explain why I am trying to make it work that way.

For convenience, here is a much reduced version of that same code sample (which can be compiled and run on its own):

! ifort /integer-size:64 /nologo arraytest_minimal.f90
program Arraytest2

    implicit none

      INTEGER II

      COMMON II(2)
      
      integer :: offset
      integer, dimension(:), allocatable :: arr

    allocate(arr(10))

    offset = (loc(arr) - loc(ii)) / 8 ! the integers being eight bytes
    ! elements in arr can now be referenced by using ii as a base
    ! arr( N ) == ii( offset + N )

  write (*,*) "using the slice syntax as in the fourth call below results in a different address even though all four lines should point the the same memory location."
  write (*,*) "This seems to be due to array indexer in slice syntax only supporting what fits in a 32 bit (signed) integer when"
  write (*,*) "the size of the array is such that it can be addressed with such an integer (i.e. the size is less than 2^31 - 1 == 0x7FFFFFFF)"
  write (*,*) "To rephrase, the fouth expression fails when size of the array < 2^31 - 1 and indexer > 2^31 - 1"
  write (*,*) loc( arr(5) )
  write (*,*) loc( arr(5:) )
  write (*,*) loc( ii(offset + 5) )
  write (*,*) loc( ii(offset + 5:) )
  
end program

Result

C:\work>ifort /integer-size:64 /nologo arraytest_minimal.f90 && arraytest_minimal.exe
 using the slice syntax as in the fourth call below results in a different addre
 ss even though all four lines should point the the same memory location.
 This seems to be due to array indexer in slice syntax only supporting what fits
  in a 32 bit (signed) integer when
 the size of the array is such that it can be addressed with such an integer (i.
 e. the size is less than 2^31 - 1 == 0x7FFFFFFF)
         2393542545888
         2393542545888
         2393542545888
       140691489477088

[FortranFan]

@Karanta__Antti ,

The example you show in your latest post is of no value.  From the looks of it, it does NOT appear to model the legacy code you're grappling with.  In your attempt at this "minimal" reproducer, you have only a main program with a single declaration of a COMMON block with a component declaration of integer II(2).  The address space beyond II(2) has no meaning and the output you show with the "4th line" via an instruction "loc( ii(offset + 5:) )" is not revealing of any issue in your actual code.

It's highly likely the actual code has a COMMON block defined elsewhere with a larger address space BECAUSE  that was a memory management practice that is/was prevalent in quite a few legacy  FORTRAN codes.  So consider a modified case of your code:

A subprogram for the instructions you show: file 'arraytest.f90'

! ifort /integer-size:64 /nologo arraytest_minimal.f90
subroutine Arraytest2

    implicit none

      INTEGER II

      COMMON II(2)
      
      integer :: offset
      integer, dimension(:), allocatable :: arr

    allocate(arr(10))

    offset = (loc(arr) - loc(ii)) / 8 ! the integers being eight bytes
    ! elements in arr can now be referenced by using ii as a base
    ! arr( N ) == ii( offset + N )

  write (*,*) "using the slice syntax as in the fourth call below results in a different address even though all four lines should point the the same memory location."
  write (*,*) "This seems to be due to array indexer in slice syntax only supporting what fits in a 32 bit (signed) integer when"
  write (*,*) "the size of the array is such that it can be addressed with such an integer (i.e. the size is less than 2^31 - 1 == 0x7FFFFFFF)"
  write (*,*) "To rephrase, the fouth expression fails when size of the array < 2^31 - 1 and indexer > 2^31 - 1"
  write (*,*) loc( arr(5) )
  write (*,*) loc( arr(5:) )
  write (*,*) loc( ii(offset + 5) )
  write (*,*) loc( ii(offset + 5:) )
  
end subroutine

And a main program that defines the COMMON block with a larger space: file 'main.f90'

   INTEGER II, I
   COMMON II(10)
   II = [( I, I=1, size(II) )]
   call Arraytest2()
end 

Program build and output:

C:\Temp>ifort /c /integer-size:64 /nologo arraytest.f90

C:\Temp>ifort /c /integer-size:64 /nologo main.f90

C:\Temp>link main.obj arraytest.obj /subsystem:console /out:main.exe
Microsoft (R) Incremental Linker Version 14.26.28806.0
Copyright (C) Microsoft Corporation.  All rights reserved.


C:\Temp>main.exe
 using the slice syntax as in the fourth call below results in a different addre
 ss even though all four lines should point the the same memory location.
 This seems to be due to array indexer in slice syntax only supporting what fits
  in a 32 bit (signed) integer when
 the size of the array is such that it can be addressed with such an integer (i.
 e. the size is less than 2^31 - 1 == 0x7FFFFFFF)
 To rephrase, the fouth expression fails when size of the array < 2^31 - 1 and i
 ndexer > 2^31 - 1
         2427282921616
         2427282921616
         2427282921616
         2427282921616

C:\Temp>

So notice the issue you're trying to show is not reproduced with this variant.

Thus the point is when the COMMON block data is only declared as "II(2)", accessing the address space outside of the bounds of II is meaningless and you can either leave the legacy code as-is or refactor the code extensively to overcome the consequences of such meaningless instructions..

But more likely is the "tricks of the past" with 2 (or more) declarations of COMMON (I don't think that was ever conformant with the language standard but compilers were ok with it) are in effect as shown above and the the issue(s) are not what you are present when you write, "the fouth expression fails when size of the array < 2^31 - 1 and indexer > 2^31 - 1"

That's why I wrote you need to either share all the details of your code either with some support services (Intel?) or to readers here, or provide reproducers that model the legacy code adequately.  What you have shared thus far is neither of these.

[Karanta__Antti]

Naturally I have tried to distill my problem to as short a piece of code as possible. To me it very strongly seems that the crux of the matter is that

write (*,*) loc( ii(offset + 5) )
write (*,*) loc( ii(offset + 5:) )

produce different results in cases where offset + 5 > 2^31-1 (this limit being found by another small code sample in this thread).

To answer your questions and comments:

Originally II was declared larger size. In later history the declaration was changed so that the II was very small and only used as a base point to point to memory allocated by c malloc. This was to get rid of custom written memory allocation code and move to use c runtime memory allocation. This is what I tried to illustrate in my code sample. Although the fact that the memory allocation is done by using c malloc routines is irrelevant, the important point being that the base of II is far enough from the referenced memory.

The declaration of II in my post starting this thread is a copy-paste of the declaration of II in the actual code, as are the the shown calls to dm_arraycopy32 and the implementation of dm_arraycopy32

As to your modified sample not reproducing the problem - are you sure you are using x64 ifort compiler? Or are we using different ifort versions? Mine is

Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.0.5.281 Build 20190815

I tried your sample and get the output

C:\work>main
using the slice syntax as in the fourth call below results in a different addre
ss even though all four lines should point the the same memory location.
This seems to be due to array indexer in slice syntax only supporting what fits
in a 32 bit (signed) integer when
the size of the array is such that it can be addressed with such an integer (i.
e. the size is less than 2^31 - 1 == 0x7FFFFFFF)
To rephrase, the fouth expression fails when size of the array < 2^31 - 1 and i
ndexer > 2^31 - 1
2623662917328
2623662917328
2623662917328
140715451418320

which shows the problem is reproduced. If the compiler version is the same and you are using 64 bit compiler (not the 32 bit one) and you are on Windows, then I wonder what we might be doing differently?

[andrew_4619 · Accepted solution]

I get this:

ifort /c /integer-size:64 arraytest.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.1 Build 20201112_000000
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.


ifort /c /integer-size:64 main.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.1 Build 20201112_000000
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.


link main.obj arraytest.obj /subsystem:console /out:main.exe
Microsoft (R) Incremental Linker Version 14.16.27039.0
Copyright (C) Microsoft Corporation.  All rights reserved.


main.exe
 using the slice syntax as in the fourth call below results in a different addre
 ss even though all four lines should point the the same memory location.
 This seems to be due to array indexer in slice syntax only supporting what fits
  in a 32 bit (signed) integer when
 the size of the array is such that it can be addressed with such an integer (i.
 e. the size is less than 2^31 - 1 == 0x7FFFFFFF)
 To rephrase, the fouth expression fails when size of the array < 2^31 - 1 and i
 ndexer > 2^31 - 1
         2322159702160
         2322159702160
         2322159702160
         2322159702160

[FortranFan]

I forgot to add to my previous post: regardless of anything, the tests given by OP and variants thereof have aspects of non-standard legacy practices e.g., with COMMON block.  The program response under the circumstances is likely "processor dependent".  One can expect another compiler that doesn't have the legacy support built-in the way Intel Fortran does with its history going back to DEC FORTRAN, something like NAG compiler, might reject it entirely.

I think OP should strongly consider seeking formal support services, say Intel Premier Support, with actual code.  The use of this forum with the kind of code snippets shown can hardly provide reliable fixes for the legacy code in question.

[Karanta__Antti]

Thanks Andrew!

I downloaded ifort 2020.4, which is the latest release I seem to be able to access. Even though it is not as new as the one you were running, the problem seems to be fixed on that one, too!

Now I understand what the problem is about, know how to circumvent it and know that it is fixed on the newer ifort versions. Case closed.

A big thank you to all who responded!

[andrew_4619]

I have a paid licence and technical support however the latest OneAPI compilers are actually free of charge to anyone going forward so you can download and install those.