pointer assignments between two arrays of different shapes

[netphilou31]

Hi,

I am facing an issue that I am unable to solve (if it can be solved).

I have a memory block allocated with the malloc function and I want to use another variable that points to the same memory block but with a different shape and I don't know how to do this. Lets see what I have tried so far.

integer(4) N

real(8) Array1(*)
pointer (ptrArray1, Array1)

real(8), pointer :: Array2(:,:)

N = 8 ! or N defined somewhere else

ptrArray1 = malloc(N*sizeof(0.D0))

How to make Array2 pointing to the same location as Array1 ?

I have tried to add the target attribute to the Array1 declaration, like:

real(8), target :: Array1(*)

And tried

Array2(1:2,1:4) => Array1(1:N)

It compiles but does not work !

How can I do that ?

Thanks

Phil.

[mecej4]

Why are you mixing up two disparate ways of allocating memory dynamically? C and Fortran use different runtime libraries, and you should not be allocating memory using C's malloc() and then attempting to use that pointer as if it was a Fortran pointer or an ALLOCATEd Fortran array.

In the days of F77 and earlier, when Fortran had no provision for dynamic allocation, using malloc() + some clever tricks was a common practice. Do you really need to use malloc() instead of ALLOCATE?

The statement

pointer (ptrArray1, Array1)

rests on a non-standard extension of Fortran called the Cray or Integer Pointer extension. It is incompatible with Fortran 9X pointers and Fortran allocatable arrays. Many modern Fortran compilers do not support Cray pointers.

[FortranFan]

You asked a similar question not too long ago and you were shown a detailed example using standard Fortran capabilities now supported by Intel Fortran - see Message #3 in the link below.  It didn't make any impression on you since you're back attempting again the non-standard (and old-fashioned and potentially conflict-creating) approach of malloc and Cray pointers and so forth.   Will it register this time!?!

https://software.intel.com/pt-br/forums/intel-visual-fortran-compiler-for-windows/topic/595090

[Simon_Geard]

I didn't understand the code you posted - seemed to be a weird hybrid of C and Fortran.

You can do what you want in a more standard way which I hope you'll find self-explanatory:

program pointers
	use iso_c_binding
	implicit none
	integer, allocatable, target :: mem_block(:)
	type(c_ptr)                  :: mem_block_p
	integer                      :: i
	integer, pointer             :: shape1(:,:), shape2(:,:), shape3(:)
	
	allocate(mem_block(12),source=[(i,i=1,12)])
	
	mem_block_p = c_loc(mem_block)
	
	call c_f_pointer(mem_block_p,shape1,[6,2])
	call c_f_pointer(mem_block_p,shape2,[4,3])
	call c_f_pointer(mem_block_p,shape3,[12])
end program pointers

[FortranFan]

Just to clarify, with standard Fortran, one can have a one-dimensional target for a multidimensional pointer as shown below.  So one doesn't necessarily need to use C interoperability functions, as shown in Message #4, if the only need is such targeting:

program p

   implicit none

   integer, parameter :: n = 4
   integer, parameter :: m = 3

   integer, allocatable, target :: foo(:)
   integer, pointer             :: bar(:,:)
   integer :: i
   integer :: j

   allocate (foo(n*m), source=[(i,i=1,n*m)] )

   bar(1:n,1:m) => foo
   do j = 1, size(bar, dim=2)
      print *, "j = ", j, " bar(i,j) = ", (bar(i,j),i=1,size(bar, dim=1))
   end do

   bar => null()

   deallocate ( foo )

   stop

end program p

 j =  1  bar(i,j) =  1 2 3 4
 j =  2  bar(i,j) =  5 6 7 8
 j =  3  bar(i,j) =  9 10 11 12
Press any key to continue . . .

[netphilou31]

Hi guys,

Thanks for all your replies. However the problem is more complex than it appears (at least for me). To mecej4, I don't think I use the malloc function from the C library but the malloc function from the Fortran language (even if by the end the C library is called). Secondly the problem is that with the allocate statement you cannot allocate several time the same array with different sizes (same shape) and with different contents without loosing something and that is exactly my goal. I though that the only way to do this was to use integer pointers (I didn't know that it was an extension). I understand that allocating the same array sereval times could be a bit strange but for my purpose I have a dll wich may contain different sets of parameters and data and because the dll may be called a lot of time for different calculation conditions I don't want to reinitialize everything every time (and dumping the data to an external file as proposed in the other thread, is for me totally excluded). The Ideal solution would be to dump all the data in a dynamically allocated memory stream whose address will be returned to the calling process for subsequent uses.

Thanks again,

Best regards

Phil.

[FortranFan]

netphilou31 wrote:

Hi guys,

Thanks for all your replies. However the problem is more complex than it appears (at least for me). To mecej4, I don't think I use the malloc function from the C library but the malloc function from the Fortran language (even if by the end the C library is called). Secondly the problem is that with the allocate statement you cannot allocate several time the same array with different sizes (same shape) and with different contents without loosing something and that is exactly my goal. I though that the only way to do this was to use integer pointers (I didn't know that it was an extension). I understand that allocating the same array sereval times could be a bit strange but for my purpose I have a dll wich may contain different sets of parameters and data and because the dll may be called a lot of time for different calculation conditions I don't want to reinitialize everything every time (and dumping the data to an external file as proposed in the other thread, is for me totally excluded). The Ideal solution would be to dump all the data in a dynamically allocated memory stream whose address will be returned to the calling process for subsequent uses.

Thanks again,

Best regards

Phil.

It seems a problem here is that the code snippets you provide don't quite seem to jive with your description; perhaps it is an issue trying to express your code snippets in Fortran which may not be your programming language of choice - I don't know, I'm asking.

I had suggested previously to you, "If your "external application" is not in Fortran and your Fortran code is "library" type of code, an option might be to let the external application handle the memory part, as discussed in the above thread.  Note this is just a suggestion, especially because your original post involving malloc's and subsequent query suggest your expertise may be in C, C++ type of languages."  Note this suggestion would extent to other aspects as well i.e., you can setup all your arrays and pointers and associated data in the C, C++ side if that's your domain of expertise and have Fortran simply do the number-crunching.  With standard C interoperability features in Fortran, you should able to what you describe plus more.

Now if you want to continue with Fortran and not want to get into the mixed language mode, then you may want to become a lot more familiar with currently available standard facilities first before making many changes to code.  Your code snippets and statements such as "you cannot allocate several time the same array with different sizes (same shape) and with different contents without loosing something" imply you are not that familiar at all with Fortran or you have an impression based on a much earlier state, perhaps FORTRAN 77/66.

[netphilou31]

Hi,

I am sorry to not be clear enough about what I wanted to do. Be sure that I am quite familiar with Fortran because it was my main development language for more than 25 years now. May be I don't know all the recent possibilities but I think I am quite skilled in FORTRAN programming. Coming back to my main objective: I am working in CAPE (computer aided process engineering) and modelling / simulation of chemical engineering processes where a lot of thermodynamic calculations are required. In a current simulation one can use several sets of chemical species and thermodynamic models. The goal of the dll I am working on is to perform the thermodynamic calculations and it has to work with these several sets of compounds/models, must switch from one to the other without the need to reinitialize everything each time (because the main data are stored in common block). The actual situation is that the number of sets in a simulation is limited to 5 and I have in memory a copy of all 5 possible sets of commons from witch I switch every time it is needed (I set of common for the calculations + 5 copies in memory for backup). The problem is that this set of commons is representing a lot of memory (more than 170 MB, because they are sized at the max values) and I wanted to get rid of this problem by switching from common blocks to modules with dynamic allocation. Now the problem is that: How to save the data? As explained above I don't want to use external files (too time consuming) and I don't want to have another set of copies so I have imagined another way: all the variables that belong to a particular common are stored in a derived type in a Fortran module. For arrays, they will be allocated dynamically. All the pointers to the derived types associated to a common will be stored into another structure whose address will be returned to the calling application for persistency. So every time the calling process as to use a different thermo set it calls the calculation routines with the address of the right structure to use and that's all. Now the problem lies in the way to allocate the arrays. I would like to use the allocate statement but I when the dll is called at the beginning for the initialization of all the data for all the thermo sets I need to allocate another set of variables (because originally the data are stored in the same commons) for each thermo set and I don't think that the allocate statement allows that without deallocating the previous memory block. Furthermore, how to deallocate everything at the end of all the calculations? For me, it was better to use the malloc function which simply returns a pointer to the allocated memory block (no need to think about the Fortran array descriptors associated to the ALLOCATE statement). I already uses this in numerous subroutines with absolutely no problems (along with CopyMemory and ZeroMemory functions to play with the allocated memory blocks). Except if you show me another way to proceed I think that I have no other solution to solve the issue (I am totally open to any suggestion as long as they do not require a total rewriting of the existing code). So my initial question was related to how can I associate a two dimensional array to a single array whose pointer is passed as an argument. I have finally found that I can declare the Array1 array with two dimensions (even if their real size is not known at compile time) because when using an integer pointer association the pointee is not allocated until the pointer receive a value. The code I was looking for is something like:

integer(4) N

real(8) Array1(1,*)
pointer (ptrArray1, Array1)
 
N = 8 ! or N defined somewhere else
 
ptrArray1 = malloc(N*sizeof(0.D0))

do I=1,2
   do J=1,4
      Array1(I,J) = 0.D0
   end do
end do

is simply doing the job!

Thanks

Phil.

[jimdempseyatthecove]

The following is Fortran-esq:

! If your Species requires a mix of types
module SpeciesModule
  type Species_t
     real :: xProperty
     real :: yProperty
     integer :: iProperty
     ...
  end type Species_t

  ! use allocatable array of Species_t objects
  type(Species_t), allocatable :: Species(:)
end module
---
Program YourProgram
  use SpeciesModule
  ...
  nSpecies = getNumberOfSpeciesFromSomewhere()
  allocate(Species(nSpecies))
  do i=1,nSpecies
    call ReadSpeciesFromSomewhere(Species(i))
  end do
  ...
  ! process using individual data
  do i=1,nSpecies
    do j=1,nTests
      call Process(Species(i), Test(j), Results(i, j))
    end do
  end do
  ...

When your species data is of one type you can define Species as a two dimensional array. Something like this:

! If your Species requires a mix of types
module SpeciesModule
  integer, parameter :: nSpeciesProperties
  ! use allocatable array of Species
  real, allocatable :: Species(:,:)
end module
---
Program YourProgram
  use SpeciesModule
  ...
  nSpecies = getNumberOfSpeciesFromSomewhere()
  allocate(Species(nSpeciesProperties,nSpecies))
  do i=1,nSpecies
    call ReadSpeciesFromSomewhere(Species(:,i))
  end do
  ...
  ! process using individual data
  do i=1,nSpecies
    do j=1,nTests
      call Process(Species(:,i), Test(j), Results(i, j))
    end do
  end do
  ...

Jim Dempsey

[jimdempseyatthecove]

I forgot to mention that in the first case:

subroutine Process(species, test, results)
  use SpeciesModule
  implicit none
  type(Species_t) :: Species ! one instance of a species
  type(Test_t)    :: test    ! generic test parameters
  type(Results_t) :: results ! common results
  ... ! process code
end subroutine Process

Jim Dempsey

[netphilou31]

Hi Jim,

Thanks for the example. Unfortunately using this approach would require that I rewrite almost all of the existing code (~100 000 lines) and I have no time for that.

Thanks again and best regards,

Phil.

[FortranFan]

netphilou31 wrote:

.. Be sure that I am quite familiar with Fortran because it was my main development language for more than 25 years now. May be I don't know all the recent possibilities but I think I am quite skilled in FORTRAN programming. ..I already uses this in numerous subroutines with absolutely no problems (along with CopyMemory and ZeroMemory functions to play with the allocated memory blocks). ..

( and then a code snippet shown again with a Cray pointer and Intel-extension of the malloc function )

So it looks like you're in software business (with industrial customers?) and for your work, you're quite ok with (or perhaps even seek out) Intel-specific and Microsoft-specific solutions.  I say this because the "pointer (ptrArray1, Array1)" type of statements you keep showing as well as the malloc function you use are (at least, within your context) part of Intel Fortran package; and CopyMemory and ZeroMemory appear to be Microsoft Windows specific API functions.  As such, they are not part of standard Fortran or any standard revisions of C, C++ respectively.

You also state, "I am totally open to any suggestion as long as they do not require a total rewriting of the existing code".  Taking this into consideration along with the other description you provide indicates to me this is way beyond my ken and I better sign off this thread.  Also because I prefer to use standard language specifications as much as possible, especially when it involves something as important as memory management.  And anything I have to offer will likely imply a lot of code rewriting for you.

But my one final suggestion would be for you to consider Intel Premier Support, especially because in the original post you indicated something not working with Intel's implementation of Cray pointers and the Intel-provided malloc function.  You would be able to share your actual code securely with Intel engineers at the Premier Support site and they might be able to help you resolve the problems that involve your use of facilities provided by Intel.

Good luck,

[netphilou31]

Thanks for your support,

Concerning the original question, I have found that declaring the Array1 with two dimension solved the problem. Furthermore, I have a bit changed my mind about the use of the malloc function along with the integer pointer association because I have a lot of common blocks containing only scalar variables and it seems to be very time consuming and probably not efficient to allocate them manually one by one.

I will probably put the variables declarations into a derived type variable which will be allocated when needed and storing its address into the main structure. This will close the discussion about the non-standard features and probably facilitate the migration even if a lot of code would have to be adapted.

Thanks to all of you guys for your suggestions, comments and advices.

Best regards,

Phil.

[LRaim]

The roots of COMMON blocks are very deep: you can map a COMMON in assembler Language.

You simply decide that a group of variable are to be placed adjacent in storage: this is software architecture  and not a non-standard feature.

Regards

[mecej4]

Phil, 

Consider replacing COMMON blocks by modules containing type and variable declarations. Unlike COMMON, which imposes association by block name + position in the block, modules provide association by module name + variable name.