- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I am getting an error "forrtl: severe (151): allocatable array is already allocated" only in release configuration. This is not called within a loop and to try and get around this strange issue I am dellocating first but traceback is still throwing an error on the second of these two lines (but only in release not debug):
if (allocated(xreg2)) deallocate(xreg2)
allocate(xreg2(regPeriods, dimAboveMed, regPeriods-1+controls))
I'm a bit at a loss to try to trouble shoot this any more. If I deallocate without testing for allocatation first it gives an access violation on the first line and if I comment out the first line it gives an access violation later when I try and use xreg2 (both of which seem like correct errors because I am not allocating xreg anywhere else).
Thanks,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The observation about commenting out these two lines strongly suggests there is a memory problem. Have you tried building the program with various run-time checking options on, such as array bounds checking?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could this be masking a memory issue? What triggers this issue is increasing controls from 3 to 4 so it seems this might be a misleading error in relation to a memory issue. But not sure why it would only happen in release configuration. The other dimension variables have values regPeriods =24, dimAboveMed = 96940 and xreg is an arraoy of double precision reals.
Thanks,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It is quite difficult to give adequate advice about this. Have you got a small reproducer for the problem or is the complete program available?
You might want to try with the Intel Inspector tool from Intel oneAPI.
Alternatively, could it be that you first assign a value to xreg2 and then later do the explicit allocation? Though that error should have been solved by the deallocate statement.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is this a multi-threaded program .AND. xreg2 is shared .AND. this section of code is within a parallel region?
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jim,
It is multi-threaded (using omp) but this is not within a shared region which I thinks makes the question about whether shared void. I have also set the number of cores to run the paralllel code on to 1, so it wasn't really parallel and there could be no data race, and I got the same error.
Hard to create a minimal replicable example as I haven't figured out the conditions that it happens under but there is no prior assigment of xreg2. Here is the full subroutine it is contained in.
My hypothesis about it masking a memory error seems incorrect as I set dimAboveMed to 1947 and the reduction in that dimension didn't compensate for the increase in control. The strange thing is that if I comment out lines 46 and 47 xreg2 sucessfully allocates.
subroutine DIDreg(params,grids, modelObjects)
implicit none
!inputs
type (structparamstype), intent(in) :: params
!changing
type(gridsType), intent(inout) :: grids
type (modelObjectsType), intent(inout) :: modelObjects(numPointsType,Tperiods)
!local
real (kind=rk):: yreg(regPeriods, 3*numsims)
real (kind=rk) :: xreg(regPeriods, 3*numsims, regPeriods-1+controls), assetstore( 3*numsims)
logical :: mask(regPeriods,3*numsims) !, ols
real (kind=rk) :: beta(regPeriods+2, 1)
real (kind=rk), allocatable :: yreg2(:, :)
real (kind=rk), allocatable :: xreg2(:, :, :)
logical, allocatable :: mask2(:,:)
logical:: recal = .true.
real (kind=rk) :: ypath(Tperiods, numSims) !income
real (kind=rk) :: cpath(Tperiods, numSims) !consumption
integer :: lpath(Tperiods, numSims) !labour supply
integer :: apath(Tperiods + 1,numSims) !this is the path at the start of each period, so we include the 'start' of death
integer :: spa, i,j, dimAboveMed, counter, total
real (kind=rk) :: AIME(Tperiods + 1,numSims), medianA
xreg = 0.0
do SPA =1,3
if (rank == 0) write (*,*) ' SPA ', spa
call setSPA(SPA)
if (modelChoice==1 .OR. SPA==1) then
if (recal) call solveValueFunction( params, grids, modelObjects, .TRUE., .TRUE. )
end if
if (rank == 0) then
call simWithUncer(params, grids, grids%Simy, cpath, apath, lpath, ypath, AIME, SPA, modelObjects, .false. )
do i = firstReg, lastReg
!y variable is labour supply
yreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims) = lpath(i,:)
!Indicator below spa
xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,1) = abs(i<Tretire+SPA-1)
do j = 1,numsims
!Wealth
!xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,2) = grids%Agrid(1,1, apath(i,:) )
xreg(i-firstReg+1,j+(SPA-1)*numsims,2) = grids%AgridFine(1,1, apath(i,j) )
xreg(i-firstReg+1,j+(SPA-1)*numsims,3) = grids%AgridFine(1,1, apath(i,j) )**2
xreg(i-firstReg+1,j+(SPA-1)*numsims,4) = grids%AgridFine(1,1, apath(i,j) )**3
!!assetstore(1+(SPA-1)*numsims:SPA*numsims) = grids%AgridFine(1,1, apath(i,:) )
if (i==Tretire-1+SPA) then
assetstore(j+(SPA-1)*numsims) = grids%AgridFine(1,1, apath(i,j) )
end if
end do
!Constant
xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,controls) = 1.0
!Age dummy with first period excluded category
if (i>firstReg) xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,i-firstReg+controls) = 1.0
end do
end if
if (modelChoice==1 ) then
do i=1,Tperiods
do j=1,4
if (allocated(modelObjects(j,i)%EV) ) deallocate(modelObjects(j,i)%EV )
if (allocated(modelObjects(j,i)%policy) ) deallocate(modelObjects(j,i)%policy)
if (allocated(modelObjects(j,i)%V) ) deallocate(modelObjects(j,i)%V)
end do
end do
end if
end do
if (rank == 0) then
mask = .true.
call doReg(yreg, xreg, 3*numsims, regPeriods, regPeriods-1+controls, mask, .false., beta)
write (*,*) 'Treatment effect', beta(1,1)
medianA = 34869.00 !3.0*34869.00 !7.0*34869.00 !34869.00 ! 24846.00
write (*,*) "median assets", medianA
dimAboveMed = count(assetstore>medianA)
Total = count(assetstore > 0.0) !count(xreg(3,:,2)>0.0)
write (*,*) dimAboveMed, 'of', total, 'above med'
if (dimAboveMed > 10) then
counter = 0
allocate(yreg2(regPeriods, dimAboveMed))
if (allocated(xreg2)) deallocate(xreg2)
allocate(xreg2(regPeriods, dimAboveMed, regPeriods-1+controls))
write (*,*) "aloocated xreg2!"
if (allocated(mask2)) deallocate(mask2)
allocate(mask2(regPeriods,dimAboveMed))
do j = 1,3*numsims
if (assetstore(j) > medianA ) then
counter = counter + 1
yreg2(:, counter) = yreg(:, j)
xreg2(:, counter, = xreg(:, j, :)
mask2(:,counter) = mask(:,j)
end if
end do
write (*,*) "Mean wealth of those above median: ", sum(xreg2(3, :, 2))/dimAboveMed
call doReg(yreg2, xreg2, dimAboveMed, regPeriods, regPeriods-1+controls, mask2, .false., beta)
write (*,*) 'Treatment effect above median', beta(1,1)
deallocate(yreg2, xreg2, mask2)
end if
end if
end subroutine
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The observation about commenting out these two lines strongly suggests there is a memory problem. Have you tried building the program with various run-time checking options on, such as array bounds checking?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
>> The strange thing is that if I comment out lines 46 and 47 xreg2 sucessfully allocates.
That is strange...
Might lines 46 and/or 47 result in an index out of range writing to xreg that then causes a trashing of the array descriptor of xreg2 in such a manner that it looks both deallocated and allocated?
BTW firstReg and lastReg are not defined.
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes the debug configuration has nearly all of these set (details below). As I mentioned it does not produce this error in the debug configuration but it also does not throw any other error like out of bounds.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Perhaps use these extra options (but not the debugging option) in the Release configuration? I have noticed in the past that a debug version may hide problems such as an uninitialised variable because it is more likely to have a value zero than a random value like 234562323523 or some such.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using the extra option on the release configuration fixed the issue. That seems to be because turning on these checks turns of optimization. When I turned the checks off again and disabled optimization (I mean flag /od) it also worked. Does this seem like a compiler bug? I can't see why optimizatgion should produce this difference in behaviour.
@jimdempseyatthecove firstReg and lastReg are global parameters they have values 4, and 9.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
At the top of the code section of the subroutine insert
print *,"xreg2 is allocated?", allocated(xreg2)
if(allocated(xreg2)) print *,size(xreg2), lbound(xreg2)
and sprinkle these statements down towards the problem statement. Maybe you will discover something.
(include on both sides any CALL statements)
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So I tried this suggestion of putting the statements
print *,"xreg2 is allocated?", allocated(xreg2)
if(allocated(xreg2)) print *,size(xreg2), lbound(xreg2)
And everyone returned false including the ones immediatly before the allocate statement that fails with error allocatable already allocated. That is at least consitent as the statement
if (allocated(xreg2)) deallocate(xreg2)
Did not catch xreg2 as allocated either. However it makes this error even ore obscure to me.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anything that changes the sequence of code, or layout of data in memory, can change behavior. The chance that it is a compiler bug is very low. Indeed, your experiment increased the odds of it being memory corruption due to an error in the program.
It might be interesting to run under Intel Inspector's memory analysis, though I suspect the error will hide again if you do.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So finally got round to running the inspector on this. It identifies lots of memory leaks but none of them make sense to me and I would have put them down to false postive which I know the inspector can throw. The detected memory leaks falls mostly into two groups. Open file statements and allocate statements. They generally also occur outside of any paralell region. Below is a code snipiet that show the relevant code for a couple of representative examples.
program Console1
use Header
real (kind=rk), allocatable :: moments(:,:),weights(:,:)
integer :: ios
call setModel
allocate( moments(3,lastMoment),weights(3,lastMoment))
open (unit = 1001,file=trim(pathMoments) // 'LSMoments_60.txt', action='read', IOSTAT = ios)
end program Console1
module Header
implicit none
integer, parameter :: rk = selected_real_kind(15)
integer, parameter :: lastMoment = 21
character(len=250), protected :: path
contains
subroutine setModel
implicit none
path = "C:\Users\jamie_m\Dropbox\SourceCode\upgradeProject\VSProj - Copy\outBaseline\"
end subroutine
end module Header
Lines 6 and 7 of the main program are being flagged as memory leaks. Does anyone see something wrong in what I've done here?
Thanks
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As it happens, I am experimenting with the Intel Inspector myself. My interpretation of these messages in combination with the source code is that the file is open until the program stops and the memory is still allocated as well. The Inspector program can be told not to report that sort of things, as you know the file will be closed when the program stops and the memory will also be deallocated without you as a programmer having to take explicit action.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks. Do you know what combination of settings I should change for it not to report these as errors?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You should add "implicit none" to program Console1.
After doing this, you should see an error on line 7 relating to pathMoments.
Without the implicit none, the variable pathMoments will be implicitly be created, but the value will be undefined. Running the resultant program should result in an error, and that may complicate the matter of the memory leak reporting.
Jim Dempsey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Jim,
Sorry the code above was just a snippet as I was having trouble creating a minimal replicable example. The full program doesn't have these issue (i.e. has implicit none and path id defined). I have got to the bottom of this issue now anyway, although I would still like to know how I could do a better job of catching these errors in the future.
Thanks,
Jamie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I use the GUI and there you have the option, when setting up a new analysis to turn off "Report still-allocated memory at application exit". The command-line it produced for this setting was:
inspxe-cl -collect mi2 -knob detect-invalid-accesses=true -knob detect-leaks-on-exit=false -knob detect-resource-leaks=false -knob enable-memory-growth-detection=false -knob enable-on-demand-leak-detection=false -knob still-allocated-memory=false -knob stack-depth=16 -mrte-mode=auto -module-filter-mode=include -app-working-dir ...
I suspect "-knob still-allocated-memory=false" is meant for this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unticking the flag "Report still-allocated memory at application exit" did not cause the inspector to stop falgging these open and allocate statements as causing memory problems.
I have cleared up one or two little issue the inspector flagged and that did not fix the already allocated error I am getting. Apart from lots of allocate and open statements as innocous as the example above the only thing the inspector is finding now is an issue "Memory leak z_Windows_NT_util.cpp libiomp5md.dll".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So as predictied by a couple of people here it was caused by an overflow issue. Line 15 should read "real (kind=rk) :: beta(regPeriods-1+controls, 1)" rather than "real (kind=rk) :: beta(regPeriods+2, 1)" so that when parameter controls was increased from 3 to 5 the calls to doReg passed a smaller array than was anticiapted.
My remaining doubts is how could I have detected this more quickly as it took a couple of days to get to the bottom of this? I don't understand why neither the inspector nor the combintation of diagnoistic flags I showed above detected the misaligned between what I was passing to the subroutine and what it expected. Have I missed some compiler options that would have detected this for me?
Thanks,

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page