I am getting an error "forrtl: severe (151): allocatable array is already allocated" only in release configuration. This is not called within a loop and to try and get around this strange issue I am dellocating first but traceback is still throwing an error on the second of these two lines (but only in release not debug):
if (allocated(xreg2)) deallocate(xreg2) allocate(xreg2(regPeriods, dimAboveMed, regPeriods-1+controls))
I'm a bit at a loss to try to trouble shoot this any more. If I deallocate without testing for allocatation first it gives an access violation on the first line and if I comment out the first line it gives an access violation later when I try and use xreg2 (both of which seem like correct errors because I am not allocating xreg anywhere else).
Could this be masking a memory issue? What triggers this issue is increasing controls from 3 to 4 so it seems this might be a misleading error in relation to a memory issue. But not sure why it would only happen in release configuration. The other dimension variables have values regPeriods =24, dimAboveMed = 96940 and xreg is an arraoy of double precision reals.
It is quite difficult to give adequate advice about this. Have you got a small reproducer for the problem or is the complete program available?
You might want to try with the Intel Inspector tool from Intel oneAPI.
Alternatively, could it be that you first assign a value to xreg2 and then later do the explicit allocation? Though that error should have been solved by the deallocate statement.
It is multi-threaded (using omp) but this is not within a shared region which I thinks makes the question about whether shared void. I have also set the number of cores to run the paralllel code on to 1, so it wasn't really parallel and there could be no data race, and I got the same error.
Hard to create a minimal replicable example as I haven't figured out the conditions that it happens under but there is no prior assigment of xreg2. Here is the full subroutine it is contained in.
My hypothesis about it masking a memory error seems incorrect as I set dimAboveMed to 1947 and the reduction in that dimension didn't compensate for the increase in control. The strange thing is that if I comment out lines 46 and 47 xreg2 sucessfully allocates.
subroutine DIDreg(params,grids, modelObjects) implicit none !inputs type (structparamstype), intent(in) :: params !changing type(gridsType), intent(inout) :: grids type (modelObjectsType), intent(inout) :: modelObjects(numPointsType,Tperiods) !local real (kind=rk):: yreg(regPeriods, 3*numsims) real (kind=rk) :: xreg(regPeriods, 3*numsims, regPeriods-1+controls), assetstore( 3*numsims) logical :: mask(regPeriods,3*numsims) !, ols real (kind=rk) :: beta(regPeriods+2, 1) real (kind=rk), allocatable :: yreg2(:, :) real (kind=rk), allocatable :: xreg2(:, :, :) logical, allocatable :: mask2(:,:) logical:: recal = .true. real (kind=rk) :: ypath(Tperiods, numSims) !income real (kind=rk) :: cpath(Tperiods, numSims) !consumption integer :: lpath(Tperiods, numSims) !labour supply integer :: apath(Tperiods + 1,numSims) !this is the path at the start of each period, so we include the 'start' of death integer :: spa, i,j, dimAboveMed, counter, total real (kind=rk) :: AIME(Tperiods + 1,numSims), medianA xreg = 0.0 do SPA =1,3 if (rank == 0) write (*,*) ' SPA ', spa call setSPA(SPA) if (modelChoice==1 .OR. SPA==1) then if (recal) call solveValueFunction( params, grids, modelObjects, .TRUE., .TRUE. ) end if if (rank == 0) then call simWithUncer(params, grids, grids%Simy, cpath, apath, lpath, ypath, AIME, SPA, modelObjects, .false. ) do i = firstReg, lastReg !y variable is labour supply yreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims) = lpath(i,:) !Indicator below spa xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,1) = abs(i<Tretire+SPA-1) do j = 1,numsims !Wealth !xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,2) = grids%Agrid(1,1, apath(i,:) ) xreg(i-firstReg+1,j+(SPA-1)*numsims,2) = grids%AgridFine(1,1, apath(i,j) ) xreg(i-firstReg+1,j+(SPA-1)*numsims,3) = grids%AgridFine(1,1, apath(i,j) )**2 xreg(i-firstReg+1,j+(SPA-1)*numsims,4) = grids%AgridFine(1,1, apath(i,j) )**3 !!assetstore(1+(SPA-1)*numsims:SPA*numsims) = grids%AgridFine(1,1, apath(i,:) ) if (i==Tretire-1+SPA) then assetstore(j+(SPA-1)*numsims) = grids%AgridFine(1,1, apath(i,j) ) end if end do !Constant xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,controls) = 1.0 !Age dummy with first period excluded category if (i>firstReg) xreg(i-firstReg+1,1+(SPA-1)*numsims:SPA*numsims,i-firstReg+controls) = 1.0 end do end if if (modelChoice==1 ) then do i=1,Tperiods do j=1,4 if (allocated(modelObjects(j,i)%EV) ) deallocate(modelObjects(j,i)%EV ) if (allocated(modelObjects(j,i)%policy) ) deallocate(modelObjects(j,i)%policy) if (allocated(modelObjects(j,i)%V) ) deallocate(modelObjects(j,i)%V) end do end do end if end do if (rank == 0) then mask = .true. call doReg(yreg, xreg, 3*numsims, regPeriods, regPeriods-1+controls, mask, .false., beta) write (*,*) 'Treatment effect', beta(1,1) medianA = 34869.00 !3.0*34869.00 !7.0*34869.00 !34869.00 ! 24846.00 write (*,*) "median assets", medianA dimAboveMed = count(assetstore>medianA) Total = count(assetstore > 0.0) !count(xreg(3,:,2)>0.0) write (*,*) dimAboveMed, 'of', total, 'above med' if (dimAboveMed > 10) then counter = 0 allocate(yreg2(regPeriods, dimAboveMed)) if (allocated(xreg2)) deallocate(xreg2) allocate(xreg2(regPeriods, dimAboveMed, regPeriods-1+controls)) write (*,*) "aloocated xreg2!" if (allocated(mask2)) deallocate(mask2) allocate(mask2(regPeriods,dimAboveMed)) do j = 1,3*numsims if (assetstore(j) > medianA ) then counter = counter + 1 yreg2(:, counter) = yreg(:, j) xreg2(:, counter,
= xreg(:, j, :) mask2(:,counter) = mask(:,j) end if end do write (*,*) "Mean wealth of those above median: ", sum(xreg2(3, :, 2))/dimAboveMed call doReg(yreg2, xreg2, dimAboveMed, regPeriods, regPeriods-1+controls, mask2, .false., beta) write (*,*) 'Treatment effect above median', beta(1,1) deallocate(yreg2, xreg2, mask2) end if end if end subroutine
>> The strange thing is that if I comment out lines 46 and 47 xreg2 sucessfully allocates.
That is strange...
Might lines 46 and/or 47 result in an index out of range writing to xreg that then causes a trashing of the array descriptor of xreg2 in such a manner that it looks both deallocated and allocated?
BTW firstReg and lastReg are not defined.
Yes the debug configuration has nearly all of these set (details below). As I mentioned it does not produce this error in the debug configuration but it also does not throw any other error like out of bounds.
Perhaps use these extra options (but not the debugging option) in the Release configuration? I have noticed in the past that a debug version may hide problems such as an uninitialised variable because it is more likely to have a value zero than a random value like 234562323523 or some such.
Using the extra option on the release configuration fixed the issue. That seems to be because turning on these checks turns of optimization. When I turned the checks off again and disabled optimization (I mean flag /od) it also worked. Does this seem like a compiler bug? I can't see why optimizatgion should produce this difference in behaviour.
@jimdempseyatthecove firstReg and lastReg are global parameters they have values 4, and 9.
At the top of the code section of the subroutine insert
print *,"xreg2 is allocated?", allocated(xreg2)
if(allocated(xreg2)) print *,size(xreg2), lbound(xreg2)
and sprinkle these statements down towards the problem statement. Maybe you will discover something.
(include on both sides any CALL statements)
So I tried this suggestion of putting the statements
print *,"xreg2 is allocated?", allocated(xreg2)
if(allocated(xreg2)) print *,size(xreg2), lbound(xreg2)
And everyone returned false including the ones immediatly before the allocate statement that fails with error allocatable already allocated. That is at least consitent as the statement
if (allocated(xreg2)) deallocate(xreg2)
Did not catch xreg2 as allocated either. However it makes this error even ore obscure to me.
Anything that changes the sequence of code, or layout of data in memory, can change behavior. The chance that it is a compiler bug is very low. Indeed, your experiment increased the odds of it being memory corruption due to an error in the program.
It might be interesting to run under Intel Inspector's memory analysis, though I suspect the error will hide again if you do.
So finally got round to running the inspector on this. It identifies lots of memory leaks but none of them make sense to me and I would have put them down to false postive which I know the inspector can throw. The detected memory leaks falls mostly into two groups. Open file statements and allocate statements. They generally also occur outside of any paralell region. Below is a code snipiet that show the relevant code for a couple of representative examples.
program Console1 use Header real (kind=rk), allocatable :: moments(:,:),weights(:,:) integer :: ios call setModel allocate( moments(3,lastMoment),weights(3,lastMoment)) open (unit = 1001,file=trim(pathMoments) // 'LSMoments_60.txt', action='read', IOSTAT = ios) end program Console1
module Header implicit none integer, parameter :: rk = selected_real_kind(15) integer, parameter :: lastMoment = 21 character(len=250), protected :: path contains subroutine setModel implicit none path = "C:\Users\jamie_m\Dropbox\SourceCode\upgradeProject\VSProj - Copy\outBaseline\" end subroutine end module Header
Lines 6 and 7 of the main program are being flagged as memory leaks. Does anyone see something wrong in what I've done here?
As it happens, I am experimenting with the Intel Inspector myself. My interpretation of these messages in combination with the source code is that the file is open until the program stops and the memory is still allocated as well. The Inspector program can be told not to report that sort of things, as you know the file will be closed when the program stops and the memory will also be deallocated without you as a programmer having to take explicit action.
You should add "implicit none" to program Console1.
After doing this, you should see an error on line 7 relating to pathMoments.
Without the implicit none, the variable pathMoments will be implicitly be created, but the value will be undefined. Running the resultant program should result in an error, and that may complicate the matter of the memory leak reporting.
Sorry the code above was just a snippet as I was having trouble creating a minimal replicable example. The full program doesn't have these issue (i.e. has implicit none and path id defined). I have got to the bottom of this issue now anyway, although I would still like to know how I could do a better job of catching these errors in the future.
I use the GUI and there you have the option, when setting up a new analysis to turn off "Report still-allocated memory at application exit". The command-line it produced for this setting was:
inspxe-cl -collect mi2 -knob detect-invalid-accesses=true -knob detect-leaks-on-exit=false -knob detect-resource-leaks=false -knob enable-memory-growth-detection=false -knob enable-on-demand-leak-detection=false -knob still-allocated-memory=false -knob stack-depth=16 -mrte-mode=auto -module-filter-mode=include -app-working-dir ...
I suspect "-knob still-allocated-memory=false" is meant for this.
Unticking the flag "Report still-allocated memory at application exit" did not cause the inspector to stop falgging these open and allocate statements as causing memory problems.
I have cleared up one or two little issue the inspector flagged and that did not fix the already allocated error I am getting. Apart from lots of allocate and open statements as innocous as the example above the only thing the inspector is finding now is an issue "Memory leak z_Windows_NT_util.cpp libiomp5md.dll".
So as predictied by a couple of people here it was caused by an overflow issue. Line 15 should read "real (kind=rk) :: beta(regPeriods-1+controls, 1)" rather than "real (kind=rk) :: beta(regPeriods+2, 1)" so that when parameter controls was increased from 3 to 5 the calls to doReg passed a smaller array than was anticiapted.
My remaining doubts is how could I have detected this more quickly as it took a couple of days to get to the bottom of this? I don't understand why neither the inspector nor the combintation of diagnoistic flags I showed above detected the misaligned between what I was passing to the subroutine and what it expected. Have I missed some compiler options that would have detected this for me?