large array allocation failure 41

[BLaub1]

my code allocates a large array.  I've used Lahey fortran for years and no problem, but with intel, it won't allocate the array.  This is the allocation 

    REAL(8) , Allocatable, Dimension(:,:,:,:) ::	gas      ! Gas properties array		
    ALLOCATE( gas (ncham,nsteps,nspecies,nparam),stat=st)

ncham is 5, nsteps is 250,000, nspecies is 10, and nparam is 19.  my estimate is this is somewhere in the 20GB range.  My machine has 128 GB of RAM avail, but I'm getting the following error message:

 status after allocation of variable gas is           41
forrtl: severe (157): Program Exception - access violation
Image              PC        Routine            Line        Source
libifcoremd.dll    0FFF9F99  Unknown               Unknown  Unknown
libifcoremd.dll    10023738  Unknown               Unknown  Unknown
libifcoremd.dll    1005A4AF  Unknown               Unknown  Unknown
libifcoremd.dll    10058E40  Unknown               Unknown  Unknown
AIPPintel1.exe     0138C772  Unknown               Unknown  Unknown
AIPPintel1.exe     013A11CC  Unknown               Unknown  Unknown
AIPPintel1.exe     013DB70F  Unknown               Unknown  Unknown
AIPPintel1.exe     013DBC91  Unknown               Unknown  Unknown
KERNEL32.DLL       74CF8494  Unknown               Unknown  Unknown
ntdll.dll          770941C8  Unknown               Unknown  Unknown
ntdll.dll          77094198  Unknown               Unknown  Unknown

my virtual memory is set at 198 GB for all drives.  What could the problem be?  

thanks

Brian

[andrew_4619]

Is it a 32 bit application? There is a 2GB limit for 32bit your array is 1.9GB  but your program will have other memory needs to add to that....

[jimdempseyatthecove]

>>Is it a 32 bit application?

Image              PC        Routine            Line        Source
libifcoremd.dll    0FFF9F99  Unknown               Unknown  Unknown
 

Yes it is. The PC addresses are 32-bit.

Please compile as 64-bit application.

Additional note. Some of the older IVF compilers had an issue of allocating an array larger than 2GB when the allocation sizes arguments were passed in as 32-bit integers. The fix for those versions were to use 64-bit integers.

Jim Dempsey

[BLaub1]

how do I compile as 64 bit?  sorry, I'm used to Lahey

[mecej4]

Use a Parallel Studio command prompt for Intel64 from the Start Menu, or choose an X64 configuration in Visual Studio.

Lahey on Windows did not generate 64-bit EXEs.

[jimdempseyatthecove · Accepted solution]

In Visual Studio, click the Pull-Down of Platform. You should see Win32  and Configuration Manager. In Configuration Manager select <new>, then use pull down to select x64, (also check create new directory so to keep 64-bit object files separate from 32-bit files).

After you do this, the  click the Pull-Down of Platform you should see Win32, x64 and Configuration Manager.

Also note, In the Configuration Manager, there are check boxes to indicate if specific projects are to be built. When you add a new one, they usually are pre-checked, if you have problems with a build, look in the Configuration Manager to assure the project for Build (Debug, Release, yours, ...) is selected for the desired Platform(s).

Jim Dempsey

[BLaub1]

Hi Jim, thanks.  I didn't see x64 as an optioin so figured it was not available.  under LAHEY I needed to use the GNU Fortran compiler (if I remember correctly) to create 64 bit apps.  This worked.  I  also had another issue where later in a subroutine I would allocate another relatively small array, but it would crash immediately.  After a little research on error message 170, I found that I had to compile with the /stack:100000000 type of command line option and now everything is working.

the help is much appreciated.

thanks

Brian

[BLaub1]

also thanks to the others.  you were all correct as well.

[jimdempseyatthecove]

Brian,

>>I found that I had to compile with the /stack:100000000

Be careful of taking a "quick fix" that works at first, but later on you get hit by a surprise.

I am going to guess that you have some old code that you are porting to Windows (or Linux as well). This old code is most likely single threaded, and at some point soon, you will want to make it multi-threaded.

IIF this stack requirement will only be used by the main thread, then /stack:100000000, sets it for all threads, which can potentially make your program fail (out of memory). Consider using allocatable arrays and/or enabling heap arrays.

Jim Dempsey

[BLaub1]

Hi Jim, I've always used allocatable arrays and never had a problem with Lahey.  Moving to 64 bit was a little challenging but I got it working.  Now investigating intel fortran, again, a 64 bit issue, but that is resolved now.  As far as threading, I don't use hyperthreading since my understanding is the each core only has one math co-processor, so it really doesn't provide any advantage to heavy math codes like ANSYS FLUENT/Mechanical, etc.  My code is heavy on the math too, and is a single processor.  I have a few "forall" loops that could be parallelized, but they're such small loops that I doubt it would speed up the code at all.

I'm not exactly sure what you mean by single-threaded, is my interpretation correct?  I did a side/side comparison of the Intel and Lahey exe's and they were identical to the 12th decimal (or better), so I feel comfortable.  The intel compiler ended up about 2x faster.  Not sure if that was a default switch that is different during compile time or what.  

thanks

Brian

[jimdempseyatthecove]

>>As far as threading, I don't use hyperthreading since my understanding is the each core only has one math co-processor...

I am not specifically talking about hyperthreading. I am talking about multi-threading, which includes multi-core (as well as multi-threads within a core aka hyperthreads).

For any system performing serious work, it likely is multi-core. And for larger requirements multi-socket as well as each socket containing multi-cores (with or without HyperThreads per core).

>>they were identical to the 12th decimal (or better),

Be mindful, that different compiler code generation options present a trade-off between speed and precision. You may need to examine how these affect your results.

*** Do not assume the old results are the better results. You may need to specifically determine the accuracy using different means. 

Jim Dempsey

[JohnNichols]

*** Do not assume the old results are the better results. You may need to specifically determine the accuracy using different means. 

Precisely -- well say a delta of 1% on a good day 

But as Doolitle said in the recent movie -- get a 12 year old daughter and you see great movies -- trust me 

it is ok to be scared. 

[BLaub1]

I moved the .exe file over to a laptop and it didn't run.  I reduced the stack size significantly, and it still won't run.  Any ideas?  Is that the heap again?  What' is the syntax for applying the heap to the compile?

thanks

Brian

[jimdempseyatthecove]

>>I moved the .exe file over to a laptop and it didn't run.

Is the laptop 64-bit?
Did you install the necessary runtime libraries? (Use the Intel Fortran Redistributables)
Did you instruct the compiler to use the instruction set available on the host (build) machine but not on the laptop?

Jim Dempsey

[BLaub1]

it is an x64 based PC.  When I installed Visual Fortran, I installed the C++ libararies and fortran stuff as instructed earlier in this or other chains.  However, the second sentence above is foreign to me.  Instruct the compiler to use the instruction set avail on the host but not laptop?  not sure what you mean?

thanks

Brian

[jimdempseyatthecove]

When you port an application to a system that does not have the development platform (in your case you are doing so), then all the appropriate runtime libraries, whatever they are (Microsoft C++ RTL, Intel Fortran RTL, whatnot RTL), they must be installed too. These typically are called Runtime Redistributables and are available from the appropriate vendor.

When you use an application to a system that IS the development platform (in your case your build system is), then all the appropriate runtime libraries are generally present, and in such cases there is no need to install the Redistributables. Although in some cases they are not installed with the development software (e.g. MKL, MPI, Graphics Library, etc...).

Your post #13 read: I moved the .exe file over to a laptop and it didn't run. 

It did not mention anything about also installing the development tools.

Does anything appear that can be interpreted as an error message? If so, what?

Jim Dempsey

[Steve_Lionel]

And what does "did not run" mean? Was there an error message, and if so, what was it (complete and exact)?

[John_Campbell]

jimdempseyatthecove (Blackbelt) wrote:

Brian,

>>I found that I had to compile with the /stack:100000000

...

Consider using allocatable arrays and/or enabling heap arrays.

Jim Dempsey

I note the heap arrays option is /heap-arrays[:size]

Does the [:size] option work for automatic or private arrays ?

I am asking if this is implemented at compile time (easy for local arrays) or at run time so that small automatic arrays whose size (in kilobytes) < "size" are sent to the stack, but those larger are sent to the heap. This would require alternate deallocation, depending on where the array was allocated.

Does this option also influence allocation of ALLOCATE arrays or !$OMP PRIVATE copies of arrays, depending on size ?

Some compilers are vague as to if they apply this type of option at run time to decide for local, automatic or !$OMP PRIVATE arrays.

[LRaim]

For information. Out of curiosity I have performed some tests using the following code.

=======================================================================
      PROGRAM MAIN
      IMPLICIT  REAL*8(A-H,O-Z)
!
      REAL(8) , Allocatable, Dimension(:,:,:,:) ::    gas      ! Gas properties array
!
      INTEGER*4  ST
!
      NCHAM = 5
      NSTEPS = 250000
      NSPECIES = 10
      NPARAM = 19
!
      ALLOCATE( gas (ncham,nsteps,nspecies,nparam),stat=st)
!
      GAS(1,1,1,1) = 1.111111D00
      GAS(NCHAM,NSTEPS,NSPECIES,NPARAM) = 9.9999999
!
      STOP
      END
=============================================================
I have compiled (frequently used) and re-linked (it was many years since) with the Lahey compiler (1997 release).
I have compiled and linked with Intel VF 2018 (x86 solution, default settings for a console application) in Visual Studio 2017 Professional.
Both executables run normally. I am developing in W10 with I7 (6 cores) and 16 GB ram.

Regards  

[John_Campbell]

Could I suggest you try and adapt the following in your new environment.

You could use task manager to inspect the memory working set and Commit Size as the program runs.

      PROGRAM MAIN
      IMPLICIT  REAL*8 (A-H,O-Z)
!
      integer,parameter :: jp = selected_int_kind (15)          ! integer*8 kind (for SIZE)
      integer,parameter :: dp = selected_real_kind (10,100)     ! real*8 kind
      REAL(dp) , Allocatable, Dimension(:,:,:,:) ::    gas      ! Gas properties array
!
      INTEGER*4  ST
      character  ans
!
      NCHAM    = 18        ! increased for 6.4 GByte array
      NSTEPS   = 250000
      NSPECIES = 10
      NPARAM   = 19
!
      ALLOCATE ( gas (ncham,nsteps,nspecies,nparam), stat=st)   ! heap array
!
      GAS(1,1,1,1) = 1.111111D00
      GAS(NCHAM,NSTEPS,NSPECIES,NPARAM) = 9.9999999D0
!
      write (*,*) size (gas, KIND=jp) * (8./1024./1024./1024.), ' GByte'
      write (*,*) 'Finish ?'
      read  (*,*) ans
      END

[jimdempseyatthecove]

John,

Good suggestion about the task manager. To be nit picky, I would suggest the suffix D0 (and D00) with _dp.

Also, perhaps replacing dp with rk (real kind) or some other generic name for default real kind as declared within the application code.

This leave at issue is how to declare a literal in exponential notation format in the correct kindness. It is somewhat awkward to use

real(rk), parameter :: G = 6.7430_rk / (10_rk**11)

One would think you could always use 6.7430D-11, however, this would not work if rk is the kind real(16).

If someone can suggest a better way to do this it would be insightful.

Jim Dempsey