!$OMP and stack overflow problem--cannot solve

[Arm_N_]

Hi, 

I have a very basic code using openmp on intel fortran (Intel(R) Visual Fortran Compiler Integration for Microsoft Visual Studio* 2008, 11.1.3471.2008, Copyright (C) 2002-2010 Intel Corporation Microsoft Visual Studio 2008 Version 9.0.21022.8.RTM On the help page it said "Intel® Visual Fortran Compiler Professional Edition 11.1")  I have the stack overflow problem when the array k is increased to 5,000,000, which should not be that high number for the compiler.  The code works fine with something like 200,000, but for 500,000 it just breaks down with stack overflow problem.

I tried all solutions I could search, but i am not sure whether I did them correctly,

- Apparently "Call KMP_SET_STACKSIZE(2000000000)" does not work.
-OMP_get_thread_num()+1 cannot even be compiled.
- Setting Heap arrays to 0 does not work as well.

Could someone please help me out?
Thanks very much.

==========================================================

PROGRAM TestOPENMP

USE omp_lib

IMPLICIT NONE
INTEGER :: i,j,l
INTEGER, DIMENSION (500000) :: k

Call KMP_SET_STACKSIZE(2000000000)
CALL OMP_set_num_threads(OMP_get_max_threads()-1)


!$OMP PARALLEL DO PRIVATE(i,j) 	

!  a=OMP_get_thread_num()+1

DO i=1,500000
    j=i*i
    k(i)=i*i
    !PRINT*, 'i=',i,'  j=',j,'  k=',k(i)
END DO

!$OMP END PARALLEL DO

END PROGRAM

[Arm_N_]

Oops.... it seems this solution of Steve's works.  But here lies more problem

setting the project property Linker > System > Stack reserve size, or from the command line, put /link /stack:<value> at the end of the ifort line that links the program.

If I set to 600,000, it still does not work.  

If I set to 2,000,000,000, the runtime screen just blinks and disappears.

If I set to 200,000,000 then this works, BUT

If I set dimension of k to 50,000,000, then the runtime screen just appears as a black screen without any error message and stops working.

I am quite concerned about this, as I need to parallel the code with an array of dimension 200,000,000 (but with more than 1 dimension, .i.e., the dimension of array k might be k(260,260,2,260,6) and 260x260x2x260x6 is definitely larger than 50,000,000.

In short, the first problem posted is solved by setting new stack reserve size, but further problems are ever more worrisome since there is no error message at all but it just cannot be run.

Any solution now?

Thank you very much !

[John_Campbell]

You could try using allocatable arrays to avoid use of the stack. This is my preferred approach.

The following changes should fix the problem you are finding and also provide some flexibility for further investigation of !$OMP. I chose the size range of n to run on my available physical memory. task manager will show varying CPU, memory usage and threads as it runs.

USE omp_lib

 IMPLICIT NONE
 INTEGER :: i,j,l, mth, nth, n
 INTEGER, allocatable :: k(:)
 integer, parameter :: million = 1000000

 mth = OMP_get_max_threads()
 do nth = 1,mth-1
   CALL OMP_set_num_threads (nth)
   
   do l = 1000,1800,100
     n = l*million
     allocate ( k(n) )
     
!$OMP PARALLEL DO PRIVATE(i,j) SHARED (k,n) 
     
     !  a=OMP_get_thread_num()+1
     
     DO i=1,n
         j=i*i
         k(i)=i*i
         !PRINT*, 'i=',i,'  j=',j,'  k=',k(i)
     END DO
     
!$OMP END PARALLEL DO
     
     write (*,*) 'K(',n,') initialised with',nth,' threads'
     deallocate (k)
   end do ! l
 end do ! nth

END PROGRAM

[TimP]

John is correct in pointing out the advantage of allocatable array in preference to a large array which may be set as static and exceed the 2GB space for code and static data.

I'm not certain whether you have been trying to ignore the increase in requirement for stack reserve size which is implied by increasing KMP_STACKSIZE, which defaults to 4MB per thread (2MB in 32-bit mode).  You aren't likely to exceed the default KMP_STACKSIZE unless you use private arrays.  I hope you don't plan to stay for long on such an old compiler that OMP_STACKSIZE isn't supported.

The recommended usage of /heap-arrays is with no size number specified.  It may cause problems with such an old compiler, if you are allocating arrays in a parallel region.

[Steven_L_Intel1]

Stack space also comes out of the 2GB static code and data space, so setting the stack reserve to 2GB or so is never going to work.

[Arm_N_]

John Campbell wrote:

You could try using allocatable arrays to avoid use of the stack. This is my preferred approach.

Thanks John, but when I tried executing your code, it said "Insufficient virtual memory".

Steve Lionel (Intel) wrote:

Stack space also comes out of the 2GB static code and data space, so setting the stack reserve to 2GB or so is never going to work.

Yes, it worked when I set something like 1.9 GB !  Thanks!   However, when the dimension of the array is 500,000,000, my computer crashed.  I guess that's the limit ?

[John_Campbell]

Arm,

I presume "Insufficient virtual memory" indicates you are running on a 32-bit environment, with insufficient physical memory or virtual memory available.  Adjust line 12 of my post so that "n" is suitable for the memory and virtual memory you have available. I tested the program using a 64-bit compilation and 8gb of memory. ( using virtual memory is not a good combination with OpenMP )

"do l = 100,250,50" should work

When you get it working, you might also be interested in testing the following changes, as they produce significantly different performance. Initialising large arrays is not the best use of !$OMP, but does demonstrate some of the good and bad of selecting OpenMP.

!$OMP PARALLEL DO PRIVATE(i,j) SHARED (k,n)   &
!$OMP& SCHEDULE(STATIC)   

or

!$OMP PARALLEL DO PRIVATE(i,j) SHARED (k,n)   &
!$OMP& SCHEDULE(DYNAMIC)   

I hope my example provides some ideas for further investigation.

[Arm_N_]

John Campbell wrote:

I presume "Insufficient virtual memory" indicates you are running on a 32-bit environment, with insufficient physical memory or virtual memory available.

Thanks John for additional suggestion.  You were right about 32 bit operation system as well.