The size of the arrray dimension is too large error

[Murat_Yetkin]

Hi,

I have below code to use LAPACK in visual fortran compiler. But it gives the 'error #6930: The size of the array dimension is too large, and overflow occured when computing the array size '.

    program deneme
    
    implicit none
    integer I,J,N,M,NRHS, INFO, LWORK, BLOCKSIZE
    parameter (M=90001, N=6000, NRHS=1, BLOCKSIZE=16, LWORK=N+M*BLOCKSIZE)
    double precision A(M,N), B(M,NRHS), WORK(LWORK)
    
    open(unit=10, file='C:\Users\Murat\Desktop\u3.txt')
    read(10,*) ((A(I,J),J=1,N),I=1,M)
    close (10)
    
    
    
    open(unit=11, file='C:\Users\Murat\Desktop\bdeneme.txt')
    read (11,*) ((B(I,J),J=1,NRHS), I=1,M)
    close(11)
    
    
    
    call dgels('No transpose', M,N,NRHS,A,M,B,M,WORK,LWORK,INFO)
    
    
    open(unit=12, file='C:\Users\Murat\Desktop\result.txt')
    do I=1,N
        do J=1,NRHS
            write (12,*) B(I,J)
        end do
    end do
    close(12)
    

    
    end program deneme

[Johannes_Rieke]

Hi Murat,

it would help to make the data array allocatable (heap instead of stack). You might need to add in the linker '/LARGEADDRESSAWARE'.

program array_dim
  use, intrinsic :: ISO_FORTRAN_ENV, only : real64
  implicit none
  
  ! Variables
  integer, parameter        :: M=90001, N=6000
  !real(real64)              :: my_array(M,N)
  real(real64), allocatable :: my_array(:,:)
  
  allocate(my_array(M,N))
  
  ! Body of array_dim
  print *, 'Hello World'

end program array_dim

[rwg]

Hi Murat,

the solution of Johannes will not work if you create a 32-bit version.

Your array A is of size(90001,6000). This are 90001*6000*8 bytes = 4,023 GB (one double precision value has 8 bytes) . A 32 bit program can use max. 2 GB. Using /LARGEADDRESSAWRE gives you 3 GB on 32 bit windows and 4 GB on 64 bit windows. And you have to consider that your program and the other arrays and variables need some space too.

To use such large arrays you have to build a 64 bit version of your program. The 64 bit version doesn't need then linker option /LARGEADDRESSAWARE

[Murat_Yetkin]

Thank you for answers. 

My computer has 64 bit operating system but intel files are in programfiles(x86). How can I change my code to work as 64 bit?

[Steve_Lionel]

It doesn't matter where the Intel files are.

If you are using Visual Studio, select Build > Configuration Manager. Under "Active Solution Platform" select <New...> Change the new platform to "x64" and click OK. Now you are configured to build a 64-bit application.

If you are using the command line, start your command prompt session with the Intel-installed shortcut for "Compiler xx.x for Intel 64 ...:"

[Murat_Yetkin]

I did and built solution but  compilation aborted with no explanation. I attached build log as a text file.

[andrew_4619]

The buildlog explains. "catastrophic error: Variable DENEME$A too large for NTCOFF. Bigger than 2GB. Use heap instead"

The allocatable shown at post #2 will put the arrays on the heap. An alternative is the heap arrays option..

[Johannes_Rieke]

So was my thought/ understanding of the heap-array option also: /heap-array0 should work for x64. But it doesn't:

Build Log  
     Build started: Project: array_dim, Configuration: Debug|x64 
 
Output  
    Deleting intermediate files and output files for project 'array_dim', configuration 'Debug|x64'.
Compiling with Intel(R) Visual Fortran Compiler 18.0.1.156 [Intel(R) 64]...
ifort /nologo /debug:full /Od /heap-arrays0 /warn:interfaces /module:"x64\Debug\\" /object:"x64\Debug\\" /Fd"x64\Debug\vc140.pdb" /traceback /check:bounds /check:stack /libs:dll /threads /dbglibs /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\\bin\amd64" /Qm64 "D:\02_Fortran\99_test\array_dim\array_dim.f90"
x64\Debug\array_dim.obj: catastrophic error: Variable ARRAY_DIM$MY_ARRAY too large for NTCOFF.  Bigger than 2GB.  Use heap instead
compilation aborted for D:\02_Fortran\99_test\array_dim\array_dim.f90 (code 1)


array_dim - 1 error(s), 0 warning(s)

This is the result of the code from #2 with static array definition (line 7 un-commented, line 8 + 10 commented). Shouldn't it work this way?

[Steve_Lionel]

/heap-array is used only for temporary copies and "automatic arrays" (procedure local arrays with dimensions based on dummy arguments). It doesn't affect regular variables you declare. 

[Johannes_Rieke]

Hi Steve, thanks for clarification. So, the only way to work with large arrays >= 2 GB is to make them allocatable, if I understand it correctly.

[Steve_Lionel]

That is correct.

[Murat_Yetkin]

Hi,

Thank you for helping and answers.

I have changed coding platform to 64 bit and revised the code with 'allocatable'.

I get an error without any line explanation ( I dont understand which line created this error)

I added screenshot of error and code and buildlog file.

[Johannes_Rieke]

Without testing, I've seen that you mix single with double precision. A and B are defined as single (real, allocatable instead of real(real64), allocatable, compare with #2 line 8). Then you call *gels as double precision version (dgels). The interface cannot be checked by the compiler, because you have an implicit interface. A segfault is the result.

Compare this blog topic for interfaces: https://software.intel.com/en-us/blogs/2012/01/05/doctor-fortran-gets-explicit-again

And for precision this: https://software.intel.com/en-us/blogs/2017/03/27/doctor-fortran-in-it-takes-all-kinds

[Murat_Yetkin]

I fixed real-double precision issue than built the code with no warning. When I start to debug an error page has occured about 'mkl_avx2.dll' file. I checked the path of that file and it is in there. any idea about that error?  ( I added screen shots of code and errors)

[andrew_4619]

Dgels requires double precision arrays, A and B in your code are type real which is the same as real(4).  real(8) is DP. 

[Johannes_Rieke]

Hi, the first figure you attached contains the old, single precision version as Andrew said. Are you sure you changed it to double precision?

This one works for me including debug:

program array_dim
  use, intrinsic :: ISO_FORTRAN_ENV, only : real64
  USE lapack95, ONLY: GELS
  implicit none
  
  ! Variables
  !integer, parameter        :: M=90001, N=6000, NRHS=1
  integer, parameter        :: M=90, N=60, NRHS=1
  real(real64), allocatable :: A(:,:), B(:,:)
  ! Local Scalars
  INTEGER :: I, J, INFO
  
  allocate(A(M,N))
  allocate(B(M,NRHS))
    
  CALL RANDOM_SEED()
  CALL RANDOM_NUMBER(A)  
  CALL RANDOM_NUMBER(B)
  
  CALL GELS( A, B(:,1), 'T', INFO )

end program array_dim

Don't forget to add /Qmkl:parallel in compiler setting and e.g. mkl_lapack95_ilp64.lib and include path in linker settings. By inserting the 'use lapack95, only:GELS' you enables the compiler to make interface checking (explicit f90 interface for lapack).

[Destarina__Lisa]

Hi, regards,, 

I am very newbie to(dummiest) use fortran. I have program that's written using fortran 77 with the extension file *.f If I worked with larga data, it will be error and it said, catastrophic error ... > 2GB Use heap instead compilation.... I am using intel fortran compiler in windows 10 64 bit. I am confuse to use the heap, do we need to change the code or using command line and/or setting in visual studio.  I tried to use command prompt intel fortran compiler: ifort filename filename /link. Is there command that I can use to execute the files without error. Or I have to modify the code, here some code of program that I used

block data
      implicit real*8(a-h,o-z)
      common/fa01es/g
      common/ma27dd/u,lp,mp,ldiag
      common/ma27fd/iovflo,nemin,ifrlvl(20)
      common/csit1/licn,liag,nxs2
      common/csit2/xs2,
     *ana,anm,anws,anw,ans,air
     *aina,ainm,ainws,ainw,ains,aiir,beta
      common/csit3/shift,alamda,itrn,ip,iq,ip1,wd,we,ht,he
      common/csit4/irtot,iprint,llp,mmp
      common/csit5/alpha,tol,nimax,ifx,idiv,nbd,ifld,npen,alpha1
      common/csit6/aly1,aly2,alx,inf
      common/strucrad/r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12
      common/strucradel/nr1,nr2,nr3,nr4,nr5,nr6,nr7,nr8,nr9,
     *nr10,nr11,nr12

[Steve_Lionel]

Windows has a hard 2GB limit for static code and data. You don't show where you declare an array with "large data", but if it is really large you will run out of available address space, even if you build for 64-bit. The suggestion to use the heap applies if you are building for 64-bit. Then you can declare arrays as ALLOCATABLE, DIMENSION(:) and ALLOCATE them to the desired size in the main program. There is no compiler option that will magically do this for you.

See https://software.intel.com/en-us/articles/memory-limits-applications-windows for more information.

[mecej4]

Beginners do not usually write or attempt to run programs that involve over 2GB of data. I suspect that the error message is caused by an understandable, but incorrect diagnosis of  your source code, which possibly contains a simple typographical error. Consider one of the COMMON block declarations in your code:

      common/csit2/xs2,
     *ana,anm,anws,anw,ans,air
     *aina,ainm,ainws,ainw,ains,aiir,beta

You probably intended to have AIR and AINA as separate variables but, because no ',' was typed after AIR, the declaration is seen by the compiler as

common/csit2/xs2,ana,anm,anws,anw,ans,airaina,ainm,ainws,ainw,ains,aiir,beta

If your code assigns values to AIR and AINA but not AIRAINA, and the variables in the CSIT2 block are used elsewhere, incorrect (undefined) values will be used.