Slow deallocation in debug mode

[dannycat]

Has anyone experienced the problem where large arrays take a long time to deallocate in debug mode. It does not matter whether you step over the the deallocate line or not the program stops for almost a minute before moving on to the next line. This does not occur when running the same executable outside the debugger. One array in particular where this problems occurs is made up of a data structure with allocatable components. I deallocate the whole array (as I have read somewhere that allocatable array components are automatically deallocated). Would there be any advantage to doing this as opposed to explicitly deallocating all the components first and then deallocating the container array? I'd expect deallocating components would take longer.

Is this normal behavior for the debugger?

[Steven_L_Intel1]

There is a lot code that gets executed in such cases, and by doing a "step" this can require the debugger to keep interrupting execution to see if the next line is reached. I know we did some work in an earlier version to help with this - especially with array assignments - but had not heard of an issue with deallocation. Can you show a small example that demonstrates the problem?

[Antoon]

In IVF slow debugging is not typical for deallocation statements, I guess. I experienced something similar in debugging statements with whole array assignment, statements spread out over many continuation lines, etc.

If you step over (F10) the individual statements, it takes many, many seconds for the debugger to proceed to the next statement. At the other hand if you define a breakpoint just *after* those 'slow debugging' statements and then F5, it just takes a split-second for the debugger to get to the next breakpoint.

You need a sort of pro-active debugging, thus anticipating these kind of 'slow' statements and working around it by inserting an extra breakpoint. Nevertheless, most of the time you will betoo late ;-)

[dannycat]

I probably didn't explain clearly in my original post that the delay during deallocation occurs not only when stepping through the code but also when running the program without breakpoints. I put a write state just before the deallocate statement followed by another just after to confirm this. I'm putting together an example taken from my main project to try and illustrate this.

[Steven_L_Intel1]

Ok. Recognize that when you deallocate an array with allocatable components, the compiler has to loop through the array and deallocate each component. "Automatic" deallocation simply means that you don't have to explicitly deallocate it - the same work gets done.

[dannycat]

Quoting - Steve Lionel (Intel)

Ok. Recognize that when you deallocate an array with allocatable components, the compiler has to loop through the array and deallocate each component. "Automatic" deallocation simply means that you don't have to explicitly deallocate it - the same work gets done.

Yes I appreciate that"explicit" and "automatic"are probably the same in terms of the amount of work that the compiler has to dobut my question is why does the debugger take so long even when you don't step through the statement? What is the debugger doing during this time that is of benefit to the userwhen they don't actually step into the deallocate assembly code?

[Steven_L_Intel1]

Show me an example and I'll tell you. Are you sure it's a debugger issue? You don't see this delay when running the program outside the debugger?

[Andrew_Smith]

Automatic deallocation of allocatable components rarely works for me in IVF. If I fail to code explicit deallocations it usually results in a memory leak and degrading performance as used memory increases. Does your used memory all get returned with automatic deallocation? You can use task manager while testing allocate/dealocate in a loop. I am using V10.1 almost latest build. Not been able to try V11.0 in this area yet.

Andy

[Steven_L_Intel1]

I am not aware of any remaining bugs in this area as of 11.0.

[dannycat]

Quoting - Steve Lionel (Intel)

I am not aware of any remaining bugs in this area as of 11.0.

Steve,

In creating a simple example (Attached)for demonstrating the problem I have discovered that it is the operating system that is causing the problem. When I run the program on XP the deallocation takes about 5 seconds whereas it take 45 seconds on Vista (both run in debug). When the same executable is run outside the IDE the timings drop to0.14secs on Vista possibly indicating that the problem is thedebugger on Vsita. The "automatic deallocation is slightly faster than the explicit deallocation.

[Steven_L_Intel1]

I think the key is whether or not the debugger is active. On Vista, the program runs fast if I "Start without debugging" in the IDE. Here's the times I get:

Start with debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 17.504
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 33.633

Start without debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 0.125
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 0.094

I had thought that using the debug libraries or not would change things, but it doesn't. Darned if I know what Vista is doing here....

[dannycat]

Quoting - Steve Lionel (Intel)

I think the key is whether or not the debugger is active. On Vista, the program runs fast if I "Start without debugging" in the IDE. Here's the times I get:

Start with debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 17.504
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 33.633

Start without debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 0.125
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 0.094

I had thought that using the debug libraries or not would change things, but it doesn't. Darned if I know what Vista is doing here....

Steve,

At least you have manged to reproduce the problem. You can imagine how frustrating it is having to wait for the arrays to be deallocated during debugging sessions.

I suppose there is nothing morethat can be done regarding this from an Intel point of view.Am I right in thinkingthe problem lies in Microsoft'scourt as itis a debugger/operation system interaction issue both of which are MS (I believe). Will any further investigation be carried out at your end or should I try to pass the problem onto to Premier Support or even Microsoft though on the later I won't be holding my breath.

[Steven_L_Intel1]

Unless you can manage to reproduce this in a C++ program, I doubt you could get the interest of Microsoft. Let me play with this some more and talk to our developers.

[dannycat]

Quoting - Steve Lionel (Intel)

Unless you can manage to reproduce this in a C++ program, I doubt you could get the interest of Microsoft. Let me play with this some more and talk to our developers.

Steve, have you had chance to look at this yet?

[Steven_L_Intel1]

Some. What I found was that I can also see a difference on XP, though not as severe. On a different (slower) PC running XP, I get this:

Run without debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 0.453
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 0.320

Run with debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 10.375
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 7.434

I have sent this to the developers for comment, but given the holiday season, I don't expect a prompt reply. I will update you here if I get any news.

[jimdempseyatthecove]

Quoting - Steve Lionel (Intel)

Some. What I found was that I can also see a difference on XP, though not as severe. On a different (slower) PC running XP, I get this:

Run without debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 0.453
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 0.320

Run with debugging:

Generating Element List Table...
Explicit Deallocation 0.000
Deallocation Complete 10.375
Generating Element List Table...
Automatic Deallocation 0.000
Deallocation Complete 7.434

I have sent this to the developers for comment, but given the holiday season, I don't expect a prompt reply. I will update you here if I get any news.

If your allocation and deallocation routines are in seperate source files in your application I suggest, for those files,you disable the runtime check for out-of-bounds array subscripting (and uninitialized variable checking). You can leave these debug options on for the remaining files.

Jim Dempsey

[Steven_L_Intel1]

Interesting idea, Jim, but it doesn't help in this case. I see the same behavior in a Release build. Note that in a debug build, the same array check code is executed when the program is not run under debug control.

This has been reported as issue DPD200111163.

[Steven_L_Intel1]

Not an Intel problem. I can reproduce the same behavior in a C++ program using malloc and free, built with MSVC. The attached program runs in .1 seconds if I run without the debugger, 3.7 seconds with the debugger.

[jimdempseyatthecove]

Quoting - Steve Lionel (Intel)

Not an Intel problem. I can reproduce the same behavior in a C++ program using malloc and free, built with MSVC. The attached program runs in .1 seconds if I run without the debugger, 3.7 seconds with the debugger.

This then indicates the debug version of the runtime library is doing a rather lengthy heap walk. +3.6 seconds seems a bit absurd. (must be paging)

Jim

[Steven_L_Intel1]

Jim, as I indicated earlier, one does not need to use the debug libraries to see the difference. My guess is that free itself is somehow detecting that the debugger is active and doing something special.

[dannycat]

Hi Steve,

I see you came to the same conclusion as I suspected in my earlier post. Is it worth taking the matter further with Microsoft using your C++ example? If so would it best if Ifollow it up or do Intel have more "clout"when dealing with them (Microsoft).

All the best for 2009.