Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

The size of the arrray dimension is too large error

Murat_Yetkin
Beginner
3,654 Views

Hi,

I have below code to use LAPACK in visual fortran compiler. But it gives the 'error #6930: The size of the array dimension is too large, and overflow occured when computing the array size '.

   

    program deneme
    
    implicit none
    integer I,J,N,M,NRHS, INFO, LWORK, BLOCKSIZE
    parameter (M=90001, N=6000, NRHS=1, BLOCKSIZE=16, LWORK=N+M*BLOCKSIZE)
    double precision A(M,N), B(M,NRHS), WORK(LWORK)
    
    open(unit=10, file='C:\Users\Murat\Desktop\u3.txt')
    read(10,*) ((A(I,J),J=1,N),I=1,M)
    close (10)
    
    
    
    open(unit=11, file='C:\Users\Murat\Desktop\bdeneme.txt')
    read (11,*) ((B(I,J),J=1,NRHS), I=1,M)
    close(11)
    
    
    
    call dgels('No transpose', M,N,NRHS,A,M,B,M,WORK,LWORK,INFO)
    
    
    open(unit=12, file='C:\Users\Murat\Desktop\result.txt')
    do I=1,N
        do J=1,NRHS
            write (12,*) B(I,J)
        end do
    end do
    close(12)
    

    
    end program deneme

 

0 Kudos
18 Replies
Johannes_Rieke
New Contributor III
3,653 Views

Hi Murat,

it would help to make the data array allocatable (heap instead of stack). You might need to add in the linker '/LARGEADDRESSAWARE'.

program array_dim
  use, intrinsic :: ISO_FORTRAN_ENV, only : real64
  implicit none
  
  ! Variables
  integer, parameter        :: M=90001, N=6000
  !real(real64)              :: my_array(M,N)
  real(real64), allocatable :: my_array(:,:)
  
  allocate(my_array(M,N))
  
  ! Body of array_dim
  print *, 'Hello World'

end program array_dim

 

0 Kudos
rwg
Beginner
3,653 Views

Hi Murat,

the solution of Johannes will not work if you create a 32-bit version.

Your array A is of size(90001,6000). This are 90001*6000*8 bytes = 4,023 GB (one double precision value has 8 bytes) . A 32 bit program can use max. 2 GB. Using /LARGEADDRESSAWRE gives you 3 GB on 32 bit windows and 4 GB on 64 bit windows. And you have to consider that your program and the other arrays and variables need some space too.

To use such large arrays you have to build a 64 bit version of your program. The 64 bit version doesn't need then linker option /LARGEADDRESSAWARE

0 Kudos
Murat_Yetkin
Beginner
3,653 Views

Thank you for answers. 

My computer has 64 bit operating system but intel files are in programfiles(x86). How can I change my code to work as 64 bit?

0 Kudos
Steve_Lionel
Honored Contributor III
3,653 Views

It doesn't matter where the Intel files are.

If you are using Visual Studio, select Build > Configuration Manager. Under "Active Solution Platform" select <New...> Change the new platform to "x64" and click OK. Now you are configured to build a 64-bit application.

If you are using the command line, start your command prompt session with the Intel-installed shortcut for "Compiler xx.x for Intel 64 ...:"

0 Kudos
Murat_Yetkin
Beginner
3,653 Views

I did and built solution but  compilation aborted with no explanation. I attached build log as a text file.

0 Kudos
andrew_4619
Honored Contributor II
3,653 Views

The buildlog explains. "catastrophic error: Variable DENEME$A too large for NTCOFF. Bigger than 2GB. Use heap instead"

The allocatable shown at post #2 will put the arrays on the heap. An alternative is the heap arrays option..

0 Kudos
Johannes_Rieke
New Contributor III
3,653 Views

So was my thought/ understanding of the heap-array option also: /heap-array0 should work for x64. But it doesn't:

Build Log  
     Build started: Project: array_dim, Configuration: Debug|x64 
 
Output  
    Deleting intermediate files and output files for project 'array_dim', configuration 'Debug|x64'.
Compiling with Intel(R) Visual Fortran Compiler 18.0.1.156 [Intel(R) 64]...
ifort /nologo /debug:full /Od /heap-arrays0 /warn:interfaces /module:"x64\Debug\\" /object:"x64\Debug\\" /Fd"x64\Debug\vc140.pdb" /traceback /check:bounds /check:stack /libs:dll /threads /dbglibs /c /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\\bin\amd64" /Qm64 "D:\02_Fortran\99_test\array_dim\array_dim.f90"
x64\Debug\array_dim.obj: catastrophic error: Variable ARRAY_DIM$MY_ARRAY too large for NTCOFF.  Bigger than 2GB.  Use heap instead
compilation aborted for D:\02_Fortran\99_test\array_dim\array_dim.f90 (code 1)


array_dim - 1 error(s), 0 warning(s)

This is the result of the code from #2 with static array definition (line 7 un-commented, line 8 + 10 commented). Shouldn't it work this way?

0 Kudos
Steve_Lionel
Honored Contributor III
3,653 Views

/heap-array is used only for temporary copies and "automatic arrays" (procedure local arrays with dimensions based on dummy arguments). It doesn't affect regular variables you declare. 

0 Kudos
Johannes_Rieke
New Contributor III
3,653 Views

Hi Steve, thanks for clarification. So, the only way to work with large arrays >= 2 GB is to make them allocatable, if I understand it correctly.

0 Kudos
Steve_Lionel
Honored Contributor III
3,653 Views

That is correct.

0 Kudos
Murat_Yetkin
Beginner
3,653 Views

Hi,

Thank you for helping and answers.

I have changed coding platform to 64 bit and revised the code with 'allocatable'.

I get an error without any line explanation ( I dont understand which line created this error)

I added screenshot of error and code and buildlog file.

0 Kudos
Johannes_Rieke
New Contributor III
3,653 Views

Without testing, I've seen that you mix single with double precision. A and B are defined as single (real, allocatable instead of real(real64), allocatable, compare with #2 line 8). Then you call *gels as double precision version (dgels). The interface cannot be checked by the compiler, because you have an implicit interface. A segfault is the result.

Compare this blog topic for interfaces: https://software.intel.com/en-us/blogs/2012/01/05/doctor-fortran-gets-explicit-again

And for precision this: https://software.intel.com/en-us/blogs/2017/03/27/doctor-fortran-in-it-takes-all-kinds

0 Kudos
Murat_Yetkin
Beginner
3,653 Views

I fixed real-double precision issue than built the code with no warning. When I start to debug an error page has occured about 'mkl_avx2.dll' file. I checked the path of that file and it is in there. any idea about that error?  ( I added screen shots of code and errors)

0 Kudos
andrew_4619
Honored Contributor II
3,653 Views

Dgels requires double precision arrays, A and B in your code are type real which is the same as real(4).  real(8) is DP. 

0 Kudos
Johannes_Rieke
New Contributor III
3,653 Views

Hi, the first figure you attached contains the old, single precision version as Andrew said. Are you sure you changed it to double precision?

This one works for me including debug:

program array_dim
  use, intrinsic :: ISO_FORTRAN_ENV, only : real64
  USE lapack95, ONLY: GELS
  implicit none
  
  ! Variables
  !integer, parameter        :: M=90001, N=6000, NRHS=1
  integer, parameter        :: M=90, N=60, NRHS=1
  real(real64), allocatable :: A(:,:), B(:,:)
  ! Local Scalars
  INTEGER :: I, J, INFO
  
  allocate(A(M,N))
  allocate(B(M,NRHS))
    
  CALL RANDOM_SEED()
  CALL RANDOM_NUMBER(A)  
  CALL RANDOM_NUMBER(B)
  
  CALL GELS( A, B(:,1), 'T', INFO )

end program array_dim

Don't forget to add /Qmkl:parallel in compiler setting and e.g. mkl_lapack95_ilp64.lib and include path in linker settings. By inserting the 'use lapack95, only:GELS' you enables the compiler to make interface checking (explicit f90 interface for lapack).

0 Kudos
Destarina__Lisa
Beginner
3,653 Views

Hi, regards,, 

I am very newbie to(dummiest) use fortran. I have program that's written using fortran 77 with the extension file *.f If I worked with larga data, it will be error and it said, catastrophic error ... > 2GB Use heap instead compilation.... I am using intel fortran compiler in windows 10 64 bit. I am confuse to use the heap, do we need to change the code or using command line and/or setting in visual studio.  I tried to use command prompt intel fortran compiler: ifort filename filename /link. Is there command that I can use to execute the files without error. Or I have to modify the code, here some code of program that I used

block data
      implicit real*8(a-h,o-z)
      common/fa01es/g
      common/ma27dd/u,lp,mp,ldiag
      common/ma27fd/iovflo,nemin,ifrlvl(20)
      common/csit1/licn,liag,nxs2
      common/csit2/xs2,
     *ana,anm,anws,anw,ans,air
     *aina,ainm,ainws,ainw,ains,aiir,beta
      common/csit3/shift,alamda,itrn,ip,iq,ip1,wd,we,ht,he
      common/csit4/irtot,iprint,llp,mmp
      common/csit5/alpha,tol,nimax,ifx,idiv,nbd,ifld,npen,alpha1
      common/csit6/aly1,aly2,alx,inf
      common/strucrad/r1,r2,r3,r4,r5,r6,r7,r8,r9,r10,r11,r12
      common/strucradel/nr1,nr2,nr3,nr4,nr5,nr6,nr7,nr8,nr9,
     *nr10,nr11,nr12

0 Kudos
Steve_Lionel
Honored Contributor III
3,653 Views

Windows has a hard 2GB limit for static code and data. You don't show where you declare an array with "large data", but if it is really large you will run out of available address space, even if you build for 64-bit. The suggestion to use the heap applies if you are building for 64-bit. Then you can declare arrays as ALLOCATABLE, DIMENSION(:) and ALLOCATE them to the desired size in the main program. There is no compiler option that will magically do this for you.

See https://software.intel.com/en-us/articles/memory-limits-applications-windows for more information.

0 Kudos
mecej4
Honored Contributor III
3,653 Views

Beginners do not usually write or attempt to run programs that involve over 2GB of data. I suspect that the error message is caused by an understandable, but incorrect diagnosis of  your source code, which possibly contains a simple typographical error. Consider one of the COMMON block declarations in your code:

      common/csit2/xs2,
     *ana,anm,anws,anw,ans,air
     *aina,ainm,ainws,ainw,ains,aiir,beta

You probably intended to have AIR and AINA as separate variables but, because no ',' was typed after AIR, the declaration is seen by the compiler as

common/csit2/xs2,ana,anm,anws,anw,ans,airaina,ainm,ainws,ainw,ains,aiir,beta

If your code assigns values to AIR and AINA but not AIRAINA, and the variables in the CSIT2 block are used elsewhere, incorrect (undefined) values will be used.

0 Kudos
Reply