Link Copied
Based on the research, tests and discussions here and on the general fortran discourse forum, it is now evident that the proposed solution (in which the original f77 code is been modernized to use MODULE, INTENT, and DIMENSION constructs) avoids forcing a requirement for passing continuos (stride-1) arrays to the legacy function and allows the compiler to create optimal executable that does not require the creation or use of any additional temporary arrays in memory, that does NOT spend valuable CPU time copying data from the original structure into the function structures each time the method is run, and that runs much more faster than before, specially when large and complex data structures are in play.
The result above may be give significant improvements in both CPU usage, clock time and memory requirements and allow handling on even larger datasets.
The conclusion is that if you have old legacy fortran code and you have this scenario in where you need to pass columns of data structures to functions or methods that expect simple linear arrays and you are getting temporary array warnings from the compiler with -check all options, you may want to carefully look into this solution and invest a little bit of time upgrading your original fortran functions and methods to put them in Fortran Modules and updating their internal declaration of parameters to make sure the method accepts assumed array dimensions and ensuring that your methods explicitly define the INTENT of each parameter. This may not only remove your temporary array warnings, but it can also speed up your code by a factor of a 1000 as well as cut your memory use by half.
We also learned that this behaviour of the Intel Fortran, that allows you to get away to keep the old code humming without total rewrite, is not quite supported by other compilers, that do not seem to take advantage of the above constructs and still use temporary arrays.
For the purpose of cleaning this thread, I copy below the original, problem.f90 code, in is final test form:
INTEGER FUNCTION VSRCH(STR, LST, L)
IMPLICIT NONE
INTEGER L
CHARACTER(*) STR
CHARACTER(*) LST(L)
INTEGER I
VSRCH = -1
DO I=1,L
IF (LST(I) == STR) THEN
VSRCH = LOC(LST(I))
EXIT
END IF
END DO
END FUNCTION VSRCH
PROGRAM TEST
IMPLICIT NONE
INTEGER SIZ, LNA, LNB
PARAMETER (LNA = 10)
PARAMETER (LNB = 20)
PARAMETER (SIZ = 100000)
TYPE :: TableA
CHARACTER(LEN=LNA) :: A
CHARACTER(LEN=LNB) :: B
INTEGER D
REAL C
END TYPE
TYPE(TableA), DIMENSION(SIZ) :: list
INTEGER VSRCH
INTEGER IDX,I,N, OL, NL, CL
real T1,T2
CHARACTER(100) :: num
list%A = 'AA'
list(5)%A = 'A'
N=1
IF(COMMAND_ARGUMENT_COUNT().GE.1)THEN
CALL GET_COMMAND_ARGUMENT(1,num)
READ(num,*)N
END IF
OL = LOC(list(5)%A)
NL = OL
CL = 0
call cpu_time(T1)
DO I=1,N
IDX = VSRCH("A", list%A, SIZ)
IF (IDX .NE. NL) THEN
CL = CL + 1
NL = IDX
END IF
END DO
call cpu_time(T2)
PRINT *, "Found ", I, SIZ, OL, IDX, CL, T2-T1
END
When compiled with -check all the problem.f90 code produces warnings indicating that a temporary array is required and in deed, at run time the memory position allocated in the main program is different than the memory position found inside the function:
>ifort /check:all problem.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.
Microsoft (R) Incremental Linker Version 14.28.29913.0
Copyright (C) Microsoft Corporation. All rights reserved.
-out:problem.exe
-subsystem:console
problem.obj
>problem.exe 1
forrtl: warning (406): fort: (1): In call to VSRCH, an array temporary was created for argument #2
Image PC Routine Line Source
problem.exe 00007FF64154553C Unknown Unknown Unknown
problem.exe 00007FF641541712 Unknown Unknown Unknown
problem.exe 00007FF6415A157E Unknown Unknown Unknown
problem.exe 00007FF6415A1F98 Unknown Unknown Unknown
KERNEL32.DLL 00007FFD85917974 Unknown Unknown Unknown
ntdll.dll 00007FFD87E0A2D1 Unknown Unknown Unknown
Found 2 100000 1096726720 -23021592 1
3.1250000E-02
Also I copy here the final solution code, in where small changes to the are done to avoid the above problems:
MODULE SRCH
CONTAINS
INTEGER FUNCTION VSRCH(STR, LST, L)
IMPLICIT NONE
INTEGER, INTENT(IN) :: L
CHARACTER(*), INTENT(IN):: STR
CHARACTER(*), DIMENSION(:),INTENT(IN) :: LST
INTEGER :: I
VSRCH = -1
DO I=1,L
IF (LST(I) == STR) THEN
VSRCH = LOC(LST(I))
EXIT
END IF
END DO
END FUNCTION VSRCH
END MODULE
PROGRAM TEST
USE SRCH
IMPLICIT NONE
INTEGER SIZ, LNA, LNB
PARAMETER (LNA = 10)
PARAMETER (LNB = 20)
PARAMETER (SIZ = 100000)
TYPE :: TableA
CHARACTER(LEN=LNA) :: A
CHARACTER(LEN=LNB) :: B
INTEGER D
REAL C
END TYPE
TYPE(TableA), DIMENSION(SIZ) :: list
INTEGER IDX,I,N,OL,NL,CL
REAL T1,T2
CHARACTER(10) :: num
list%A = 'AA'
list(5)%A = 'A'
N=1
IF(COMMAND_ARGUMENT_COUNT().GE.1)THEN
CALL GET_COMMAND_ARGUMENT(1,num)
READ(num,*)N
END IF
OL = LOC(list(5)%A)
NL = OL
CL = 0
call cpu_time(T1)
DO I=1,N
IDX = VSRCH("A", list%A, SIZ)
IF (IDX .NE. NL) THEN
CL = CL + 1
NL = IDX
END IF
END DO
call cpu_time(T2)
PRINT *, "Found ", I, SIZ, OL, IDX, CL, T2-T1
END
When compiled with -check all the solution.f90 does not produce any warnings and in deed, the memory position allocated in the main program is the exactly the same position in memory that is later found inside the function:
>ifort /check:all solution.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.
Microsoft (R) Incremental Linker Version 14.28.29913.0
Copyright (C) Microsoft Corporation. All rights reserved.
-out:solution.exe
-subsystem:console
solution.obj
>solution.exe 1
Found 2 100000 435796160 435796160 0
0.0000000E+00
When the original problem.f90 is compiled with full optimization /O3, and executed 10k times using SIZ=100k array sizes, it takes 2.57 seconds to run, at it makes a copy of the data each time it invokes the method. Fortunatelly uses the exact same temporary memory location so there is not a lot of trashing, but it still needs twice as much memory as the originally allocated in the main program. The tests performed also confirm that the elapsed time grows linearly with the size of the memory required, so it takes 0.2 seconds if the array is reduced to SIZ=10k. Worst, if you try to run with SIZ=300k you get memory our of stack errors as the memory required is even larger than the CPU chache allows. Also the elapsed time grows linear with the number of executions, taking 25 seconds if you run 100k times:
>ifort /O3 problem.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.
Microsoft (R) Incremental Linker Version 14.28.29913.0
Copyright (C) Microsoft Corporation. All rights reserved.
-out:problem.exe
-subsystem:console
problem.obj
>problem.exe 10000
Found 10001 100000 890218688 553694600 1
2.578125
In contrast, when the proposed solution.f90 is compiled with full optimization /O3, and executed 10k times using SIZ=100k array sizes, it takes 0 seconds to run, makes no copies at all of the data each time it invokes the method. The tests performed also confirm that there is no copy done as that the elapsed time is constant with the memory required, so it still takes almost 0 seconds if the array is reduced to SIZ=10k. If you try to run with SIZ=300k it still runs as you are not duplicating the memory required:
>ifort /O3 solution.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.
Microsoft (R) Incremental Linker Version 14.28.29913.0
Copyright (C) Microsoft Corporation. All rights reserved.
-out:solution.exe
-subsystem:console
solution.obj
>solution.exe 10000
Found 10001 100000 -456939328 -456939328 0
0.0000000E+00
When running for 100k searches on an Intel i5 1.6Ghz laptop, one can see the difference in intensity of the CPU utilization in the original problem (33% for 24.8 secs) compared to practically nothing in the proposed solution:
>problem.exe 100000
Found 100001 100000 890218688 287815272 1
24.84375
>solution.exe 100000
Found 100001 100000 -456939328 -456939328 0
0.0000000E+00
Please note that the above results only measure the difference in the time wasted in moving data from the main program complex structure in memory into the function simple structure temporary memory, as the search itself inside the function was set up so that the string is found after only 5 iterations, which is the equivant to searching a string in a single array of 500k entries.
So this may be a significant extra time and memory requirements that you could shave of your program by doing a little upgrade to your old fortran code.
If people familiar with the Intel Fortran compiler behaviour could verify the above observations results, it would be most helpful. Thank you all in advance for your inputs on the above and thank you to all the people who has provided so much invaluable input on this issue.
