Increasing heap-array size causes trouble with debugging

[Randy_Direen]

I have Fortran codes that worked with an old Windows compiler and I'm trying to get them to work on a 64bit Windows machine with Parallel Studeo XE 2011 under Visual Studios 2008. An assertion error occured when I tried to call a function that had arrays passed to it:

Debug Assertion Failed!

Program:

...\\x64\\Debug\\PositionCorection.exe

File: f:\\dd\\vctools\\crt_bld\\self_64_amd64\\crt\\winsig.c

Line: 419

Expression: ("Invalid signal or error",0)

To work around the problem, I enabled heap space by going under properties Configureation Properties/Fortran/Optimization and setting Heap Arrays to 10000. Now when I step through my code in debug mode, when I try to step over a function call, it jumps over several lines and doesn't execute any of them.

Any ideas as to what might be causing this?

[Steven_L_Intel1]

Don't bother setting the Heap Arrays value to anything other than zero. Using a non-zero value doesn't buy you anything.

Typically when you get the "Debug Assertion Failed" error, there is a Fortran error in the console window which may be hidden behind the VS window. Look for it.

[Randy_Direen]

Thank you, I see the console window and it tells me that an exception has been thrown: a stack overflow. The assertion error occurs in debug mode when I try to step into a function that is located within another source file.

To fix it, I set Heap Arrays to 0 (understanding now that setting it to anything else doesn't buy me anything). Now when I'm debugging and I step through the code and come to a function call, the next step I make ends up jumping over several lines of code. I believe this is happening during regular execution too because it appears as if some of my variables are not being set due to these lines being skipped over.

Maybe it is not necessary that I use Heap Arrays? Is there something else I can do to keep the stack from overflowing?

Thank you for your help.

[mecej4]

This is just a conjecture, since I have not seen any code in this thread.

Sometimes, a missing or extra argument, or arguments in the wrong order, can cause stack overflow upon or soon after entry to a subroutine.

In such cases, one needs to resist the temptation to try the standard solutions: allowing a larger stack (or 'unlimited' stack), changing from stack allocation to heap or static allocation, and so on. These attempts will fail because the underlying error remains unchanged. The symptoms, however will show a rich variety as a consequence of the changes, and consume much effort.

It takes far less work to check that the subroutine arguments are correct. The compiler can be asked to help by generating and using interfaces.

[Steven_L_Intel1]

Do you get the error with Heap Arrays set to 0? If so, then please show us a test case.

[Randy_Direen]

I checked to see if I had things in the right order and I believe I do. I put a together a small program that reproduces the problem.

Here is the main program:

[fortran]module passdat
  integer :: nn=128,mm=128,nnp=1,mmp=1,npt, ntheta=140, nphi=258
  real (8) :: ak=16D0
  real (8), pointer :: xyz(:,:)
  complex (8), pointer :: tpmeas(:,:),psh(:,:),tsh(:,:)
end module passdat


! Solve least-squares problem by conjugate gradient on the normal equations
!
program lsqr
  use passdat
  implicit none
  integer :: n,maxiter,ierr,ia, i,j, cs, cf, rate, jmax
  character (20) :: filename='fake.dat'
  real (8) :: tol,radius=16D0,avgerr=0D0
  real (8) :: S, T
  complex (8) :: a
  complex (8), pointer :: b(:),x(:)
  complex (8), Dimension(:,:), Pointer       ::thfs, phfs, thtp, phtp
  complex (8), dimension (:,:), allocatable  :: save_tsh, difsh, ctp
  interface
     subroutine getdat(filename, npt,xyz,tpmeas)
       character (20) :: filename
       integer :: npt
       real (8), pointer :: xyz(:,:)
       complex (8), pointer :: tpmeas(:,:)
     end subroutine getdat
  end interface
  !
  ! Generate measurement data; write to file
  !
  cs=0
  cf=0
  rate=0
  jmax=4*mm+1
  Call System_Clock(COUNT_RATE=rate)
  allocate(psh(0:nnp,0:4*mmp+1),tsh(0:nn,0:4*mm+1))
  allocate(save_tsh(0:nn,0:4*mm+1), difsh(0:nn,0:4*mm+1))
  call fakedat(nn,mm,nnp,mmp,ntheta, nphi, psh,radius,ak,avgerr,tsh,filename)
  save_tsh = tsh
    do i=0,nn
       do j=0, 4*i+1
	      If(j .lt. jmax)then
          write(7, '(2i5, 2e16.6)')i,j,save_tsh(i,j)
		  endif
       enddo
    enddo
!
! Get measurement data from file
!
  call getdat(filename, npt,xyz,tpmeas)
!
 
  n=2*(2*mm+mm*mm)+(nn-mm)*(4*mm+2)
  maxiter=200
  tol=1D-5
  allocate(b(n),x(n))
  x=0
  Call System_Clock(cs)
  if (ierr/=0)  print *, 'WARNING: ierr=1'
  Call System_Clock(cf)
  Write(10, '("Iteration elapsed time in seconds: ", F10.2)')(cf-cs)/(1.0*rate)
!
  do i=0,nn
       do j=0, 4*i+1
	      If(j .le. jmax)then
          write(10, '(2i5, 2e16.6)')i,j,tsh(i,j)
		  endif
       enddo
  enddo
  difsh = tsh - save_tsh
    do i=0,nn
       do j=0, 4*i+1
	      If(j .le. jmax)then
          write(10, '(2i5, 2e16.6)')i,j,difsh(i,j)
		  endif
       enddo
  enddo
  Write(10, '(/,"T= ", e16.6)')T
  Write(10, '(/,"S= ", e16.6)')S
  Write(10, '(/, "RMS normalized difference= ", e16.6)') sqrt(S/T)

 
  Allocate(ctp(0:1023, 0:1))
  Open(13, file='phicut0.txt')
  Close(13)
!  Open(14, file='phicut90.txt')
!  Call T_ph_cut(thtp,phtp,64,ctp)
!  Call Print_1DCut(ctp, 14)
!  Close(14)
end program lsqr[/fortran]

The main program calls functions from this file,

[fortran]subroutine fakedat(nn,mm,nnp,mmp,ntheta, nphi,psh,radius,ak,avgerr, tsh,filename)
  implicit none
  integer :: nn,mm,nnp,mmp, i,j,npt, jmax, seed(1),ntheta, nphi, igrid
  character (20) :: filename
  real (4) ::ran
  real (8) :: radius,ak,avgerr, xyz(ntheta*nphi,3), xyzreg(ntheta*nphi,3), radius1, theta1, phi1, S
  real (8) :: theta,phi,one=1D0,pi
  complex (8) :: psh(0:nnp,0:4*mmp+1),tsh(0:nn,0:4*mm+1), rsh(0:nn,0:4*mm+1)
  complex (8) :: tpmeas(ntheta*nphi,2)
  COMPLEX(8), DIMENSION(0:1), PARAMETER      ::pol=(/(0.,0.),(1.,0.)/)
  pi=4*atan(one)
  !
  ! Spherical harmonic coefficients
  !
  seed=2
  Call Random_Seed
  Call Random_Seed(Put=Seed)
  tsh=0
  jmax=4*mm+1
!  do i=1,nn
!     do j=0,4*i+1
!	    If(j .le. jmax)then
!           tsh(i,j)=2*i*i+j-1
!           tsh(i,j)=1/tsh(i,j)
!		endif
!     end do
!  end do

! Generate maximum directivity antenna with random noise addition
!

tsh = tsh/SQRT(S)

  !
  ! Measurement positions  (no position errors yet)
  !
  npt=0
  If(ntheta .lt. nn)then
     Print *, 'Not enough samples in theta'
	 stop
  endif
  If(nphi .lt. 2*mm+2)then
     Print *, 'Not enough samples in phi'
	 stop
  endif
  do i=1, ntheta
     theta=i*pi/(ntheta+1)
     do j=0, nphi-1
        phi=j*2*pi/(nphi)
	    Call Random_number(Harvest=Ran)
        radius1 = radius
	    radius1 = radius1 + 6.0*(0.5-ran*1D0)*2*pi/ak
!		radius1 = radius1 + 0.2*2*pi/ak*cos(0.19*i)*cos(0.14*j)
	    Write(7, '("radius= ", f10.6, "  and should be ", f10.6)')radius1, radius
        Call Random_number(Harvest=ran)
		phi1=phi + 6.0*(0.5-ran*1D0)*2*pi/nphi
!        phi1 = phi + 0.28*cos(0.31*i)*cos(0.19*j)*2*pi/nphi
		Write(7, '("phi= ", f10.6, "  and should be ", f10.6)')phi, j*2*pi/(nphi)
        Call Random_number(Harvest=Ran)
        theta1=theta
	    theta1 = theta + 6.0*(0.5-ran*1D0)*pi/(ntheta+1)
!        theta1 = theta + 0.28*cos(0.44*i)*cos(0.81*j)*pi/(ntheta+1)
	    Write(7, '("theta= ", f10.6, "  and should be ", f10.6)')theta1, i*pi/(ntheta+1)
        npt=npt+1
        xyz(npt,1)=radius1*cos(phi1)*sin(theta1)
        xyz(npt,2)=radius1*sin(phi1)*sin(theta1)
        xyz(npt,3)=radius1*cos(theta1)
		xyzreg(npt,1)=radius*cos(phi)*sin(theta)
        xyzreg(npt,2)=radius*sin(phi)*sin(theta)
        xyzreg(npt,3)=radius*cos(theta)
     end do
  end do
  print *, 'fakedat: npt:',npt
  !
  ! Measurements
  !
 
  !
  ! Write positions and measurements to file
  !
  igrid = 1
  !If igrid = 0 write out results as if data were assumed to be at regular grid points.
  !  THIS GENERATES CORRUPTED RESULT.
  !If igrid .ne. 0, write out results using actual irregular grid points used to
  ! generate field.  
  open(unit=9,file=filename)
  write(9,'(i10)') npt
  do i=1, npt
     If (igrid .eq. 0)then
        write(9,'(3(e20.12),2(a,e20.12,a,e20.12,a))') (xyzreg(i,j),j=1,3),  &
          (' (',real(tpmeas(i,j)),',',imag(tpmeas(i,j)),')',j=1,2) 
     else
        write(9,'(3(e20.12),2(a,e20.12,a,e20.12,a))') (xyz(i,j),j=1,3),  &
          (' (',real(tpmeas(i,j)),',',imag(tpmeas(i,j)),')',j=1,2)
     endif
  end do
  close(9)
end subroutine fakedat[/fortran]

I simply set up a project to work with my 64bit machine, compiled and set a break point at line 40 (at the fakedat call). When I try to step over the function fakedat in debug mode I get the assertion error. Thank you for your help.

[Anonymous66]

Hello Randy,

This error is caused by a stack overflow when initializing the the automatic arrays in fakedat. They are being stored to the program stack that has a default size of 1mb which is not big enough for your arrays.

If you set the /heap-arrays property to 0, the program will run. Setting it too high can disable it, so it is better to set it to 0 and let the compiler allocate space automatically.

Another solution is to use allocatable arrays instead of pointers and allocate them to the desired size. This has the advantage of not depending on command line arguments.

Please see the following article for more information on memory allocation in Fortran - http://software.intel.com/en-us/forums/showpost.php?p=12913

[Randy_Direen]

Ok so this brings me to the real problem I was having. I wasn't sure if setting the /heap-array was the correct thing to do, so thank you for verifying that for me.

When I set /heap-array to 0 in the project using the sample program above, I get strange debugging results. I start debugging by setting a break point at line 40 (at the fakedat call) and then I run the debugger, when I try to step over fakedat the next line the debugger goes to is line 33? I then step back to and over line 40 and then things hang. Any ideas?

[Steven_L_Intel1]

Sometimes the compiler assigns line numbers to code that executes earlier in the program. The program runs with heap arrays enabled. But I don't see the behavior you describe when I build with 12.0.4.

[Randy_Direen]

I'm using 12.0.3. In fact, if I put a break point on line 33 and on line 40, the debugger skips over line 33 all together when I start the debugger and it stops on line 40. I know that the compiler assignes line numbers in funny orders when its trying to optimize things, but this shouldn't happen when I compile things in debug mode?

[Steven_L_Intel1]

Even in debug mode this can happen. There is some stack setup that happens at the beginning of the routine which is being assigned to line 40 - might have to do with the argumrent list for the call to fakedat, I am not sure. Whn I run it, though, it stops at 33 and then again at 40.

[Randy_Direen]

Hmmm. Could this have anything to do with the fact that I'm on a 64bit machine. Sorry, I'm grasping for anything here.

[Randy_Direen]

Could this have anything to do with me using a 64bit machine? Sorry, I'm grasping for anything here.

[Steven_L_Intel1]

I tried it both 32-bit and 64-bit and saw the same behavior as I described previously. What problem are you having now?

[Randy_Direen]

Same thing. I put a break point at line 33 and one at 40 and it just skips line 33 all together. When I try to step from 40 to 41 it hops back to line 33?

[Steven_L_Intel1]

I am not seeing that when I build and run. Can you attach a build log (Debug\buildlog.htm) from a full build?

[IanH]

I don't think there's a real problem here - at the start of your program (or each procedure) the compiler may need to set some things up for statements that occur later in the execution sequence of the program. It marks that setup code as belonging to the statement that it is more or less (the mapping of setup code to fortran source statement isn't unique and the compilers selection of the appropriate line is a bit bizarre at times) required for.

So there are two blocks of code (two instruction addresses) that are marked as belonging to line 40 - the setup and then the actual execution of the call statement. By default, when you set a breakpoint on a line (say by using F9) the debugger will break when either instruction address is hit. If you have a look in the breakpoints window (Debug > Windows > Breakpoints or similar depending on your VS version/edition) you will see that the breapoint for line 40 has a plus next to it and has two "child" breakpoints - one is for the setup instruction location, the other for the actual passing of arguments and transfer of execution to the procedure. If it really bothers you can toggle on and off the child components of the breakpoint from that window.

The setup code for line 40 is hit before the "real" code for line 33 - hence you appear to skip line 33. You step again from that setup code - the instruction pointer in the CPU moves to the start of the "real" code for the procedure - hence the source line in the debugger bounces up to the first executable statement. If you then keep stepping, finally the instruction pointer will make to the "real" point of execution of the call statement.

IO statements often result in this sort of bouncing around - there was another thread here recently where this was confusing someone. It's possible that the setup being done here is IO related - I had a quick look at the disassembly and set some data breakpoints - but I don't have the patience or skill to wade through it to see why the compiler is doing the setup that it is doing.

[Randy_Direen]

I'm not sure how to attach a file, but here is what I found in the Debug\buildlog.htm

Deleting intermediate files and output files for project 'Console1', configuration 'Debug|x64'.
Compiling with Intel Visual Fortran Compiler XE 12.0.3.175 [Intel 64]...
ifort /nologo /debug:full /Od /heap-arrays0 /warn:interfaces /module:"x64\Debug\" /object:"x64\Debug\" /Fd"x64\Debug\vc90.pdb" /traceback /check:bounds /libs:static /threads /dbglibs /c /Qvc9 /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64" "C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\Fakedat.f90"
ifort /nologo /debug:full /Od /heap-arrays0 /warn:interfaces /module:"x64\Debug\" /object:"x64\Debug\" /Fd"x64\Debug\vc90.pdb" /traceback /check:bounds /libs:static /threads /dbglibs /c /Qvc9 /Qlocation,link,"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64" "C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\Procdat.f90"
Linking...
Link /OUT:"x64\Debug\Console1.exe" /INCREMENTAL:NO /NOLOGO /MANIFEST /MANIFESTFILE:"C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\x64\Debug\Console1.exe.intermediate.manifest" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /DEBUG /PDB:"C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\x64\Debug\Console1.pdb" /SUBSYSTEM:CONSOLE /IMPLIB:"C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\x64\Debug\Console1.lib" "x64\Debug\Fakedat.obj" "x64\Debug\Procdat.obj"
Link: executing 'link'

Embedding manifest...
mt.exe /nologo /outputresource:"C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\x64\Debug\Console1.exe;#1" /manifest "C:\Users\direen\Documents\projects\PositionCorrect\Console1\Console1\x64\Debug\Console1.exe.intermediate.manifest"

Console1 - 0 error(s), 0 warning(s)

[Steven_L_Intel1]

Instructions for attaching files are in my signature below and in the "Useful Links" section on the forum main page. But what you have shown is fine. I was just wondering if you were using any unusal options.

Can you step through the code and have it execute correctly?