Fee.dll terribly slows down the debugger

[ovz]

Hi All,

My team works on a large code base a significant part of which is in Fortran. We recently upgraded to Intel(R) Visual Fortran Composer XE 2013 Update 3 Integration for Microsoft Visual Studio* 2012, 13.0.3624.11. After this we all are experiencing unbearable slowdowns when debugging the code. Whenever we try to debug Fortran or even just have a Fortran module on the call stack every debugger step through the code takes 10 or more seconds of full CPU load. Using Sysinternals Process Explorer I was able to see the call statck like one listed below. Note FEE.dll which is Intel(R) Fortran Expression Evaluator 13.0.2225.2. 

I need help with this urgently as the entire team cannot work normally because of this. 

msdia110.dll!DllGetClassObject+0x2c060
msdia110.dll!DllGetClassObject+0x2c1ff
msdia110.dll!DllGetClassObject+0x2c278
msdia110.dll!DllGetClassObject+0x2b3b7
msdia110.dll!DllGetClassObject+0x3536e
msdia110.dll!DllGetClassObject+0x1093a
FEE.dll!DllGetClassObject+0x205f7
FEE.dll!DllGetClassObject+0x20972
FEE.dll!DllGetClassObject+0xb18e
FEE.dll!DllGetClassObject+0xb4fe
FEE.dll!DllGetClassObject+0x9cc4
FEE.dll!DllGetClassObject+0x505f
FEE.dll!DllGetClassObject+0x655c
FEE.dll!DllGetClassObject+0x760a
FEE.dll!DllGetClassObject+0x7e14
FEE.dll!DllGetClassObject+0x8354
FEE.dll!DllGetClassObject+0x307e2
VSDebugEng.dll!DkmDllEnsureInitialized+0x8169c
VSDebugEng.dll!ProcDkmFree+0x1989
VSDebugEng.dll!ProcDkmFree+0x1776
VSDebugEng.dll!DkmDllEnsureInitialized+0x857e5
VSDebugEng.dll+0xc25d
VSDebugEng.dll!DkmDllUninitialize+0x24a
ntdll.dll!RtlInitializeExceptionChain+0x85
ntdll.dll!RtlInitializeExceptionChain+0x58

[SergeyKostrov]

>>...There was a 10 second slowdown after the breakpoint hit but before the enviroment is responsive... Could you try to press Break All from Debug menu during these 10 seconds? And then take a look at a Call Stack. >>...We recently upgraded to Intel(R) Visual Fortran Composer XE 2013 Update 3 Integration for Microsoft Visual Studio* 2012, >>13.0.3624.11... Does it mean that with Initial, Update 1 and Update 2 Releases everything was working without 10 secs delays?

[ovz]

Sergey,

I cannot break the execution during those 10 seconds. Visual Studio is totally unresponsive during this. The slowdown happend after our applications gets stopped for debugging.

We upgraded from Visual Studio 2010 and 

Intel(R) Visual Fortran Package ID: w_fcompxe_2011.9.300
Intel(R) Visual Fortran Composer XE 2011 Update 9 Integration for Microsoft Visual Studio* 2010, 12.1.3526.2010, Copyright (C) 2002-2012 Intel Corporation

To Visual Studio 2012 with full Intel Parallel Studio XE 

Microsoft Visual Studio Professional 2012
Version 11.0.60315.01 Update 2

Intel(R) Visual Fortran Package ID: w_fcompxe_2013.3.171

Intel(R) Visual Fortran Composer XE 2013 Update 3 Integration for Microsoft Visual Studio* 2012, 13.0.3624.11, Copyright (C) 2002-2013 Intel Corporation
* Other names and brands may be claimed as the property of others.

Intel® Advisor XE 2013 Update 2
Intel® Advisor XE 2013 Update 2, (build 270011), Copyright © 2009-2013 Intel Corporation. All rights reserved.

Intel® C++ Composer XE 2013 Package ID: w_ccompxe_2013.3.171
Intel® C++ Composer XE 2013 Update 3 Integration for Microsoft* Visual Studio* 2012, Version 13.0.1207.11, Copyright © 2002-2013 Intel Corporation. All rights reserved.
* Other names and brands may be claimed as the property of others

Intel® Inspector XE 2013 Update 5
Intel® Inspector XE 2013 Update 5, (build 278112), Copyright © 2009-2013 Intel Corporation. All rights reserved.

Intel® VTune™ Amplifier XE 2013 Update 5
Intel® VTune™ Amplifier XE 2013 Update 5, (build 274450), Copyright © 2009-2013 Intel Corporation. All rights reserved.

[Bernard]

You need to debug your program under windbg.I stated it earlier.First chance exception is excpected to be handled.Exception is thrown at code executing inside KernelBase32.dll at address  0x000007FBD76B811C and it seems that exception originated at code location or it is location of exception context 0xA1A01DB2.

My advise is to run your program under windbg and use automated analysis with the !analysis -v metacommand.You will not proceed further with VS debugger.Before setting up windbg ensure that you have connection to debug symbol server.

[Bernard]

While issueing !analyze-v command you will get a exception context record created by RaiseException function(can be obtained with .ecxr command also).Next step will be trying to understand the exact type of exception.Context contains usualy 3 parameters of exception one of them will be an exception address.Yo can use ln "one of exception parameters" command.If it is resolved successfully to exception address you can dump exception itself with dt command.

In your case it is first chance exception so I would advise to break with debugger on first chance.You can enable this with sxe -eh command.

[ovz]

Thanks for suggestions, Illya. I'm aware of those exceptions and they are not the concern I'm seeking help with. The slowdown of Visual Studio happens when applicaiton itself is already stopped in debugger.

[Bernard]

If your concern is perceived slowdown of VS debugger I gave you a two suggestion already.One is simply profile VS process with Xperf you will get nice breakdown of VS thread(s) activity with detailed call stack information and time spent in each functions.

[IanH]

iliyapolak wrote:

If your concern is perceived slowdown of VS debugger I gave you a two suggestion already.One is simply profile VS process with Xperf you will get nice breakdown of VS thread(s) activity with detailed call stack information and time spent in each functions.

And how is that going to help the OP?  Profiling/debugging/whatevering the Fortran integration in Visual Studio is a job for the Intel devs!

[Bernard]

So the best option is to wait for Intel devs to resolve the problem:)

[SergeyKostrov]

>>...You need to debug your program under windbg.I stated it earlier... No, this is totally not needed. That was told already many times that Visual Studio has integrated Debugger and it allows debugging at source codes level. Unfortunately, we see that you don't understand this and continue suggesting to everybody '...Use WinDbg..Use WinDbg...Use WinDbg....'. Are you OK? Are you doing any programming with Visual Studio and Fortran?

[SergeyKostrov]

Ovz, You have more than 100 First-chance exceptions in the application, like: ... First-chance exception at 0x000007FBD76B811C in SPM700_d_protected.exe: Microsoft C++ exception: GroveHandle::exception at memory location 0x0000000000137898. First-chance exception at 0x000007FBD76B811C in SPM700_d_protected.exe: Microsoft C++ exception: GroveHandle::exception at memory location 0x0000000000137898. First-chance exception at 0x000007FBD76B811C in SPM700_d_protected.exe: Microsoft C++ exception: GroveHandle::exception at memory location 0x0000000000137898. First-chance exception at 0x000007FBD76B811C in SPM700_d_protected.exe: Microsoft C++ exception: GroveHandle::exception at memory location 0x0000000000137898. First-chance exception at 0x000007FBD76B811C in SPM700_d_protected.exe: Microsoft C++ exception: GroveHandle::exception at memory location 0x0000000000137898. ... So, something really goes wrong and you need to try to understand what causes it. Another tip is, you could try to disable All exceptions in the Visual Studio when debugging and take a look at 'Debug' menu.

[SergeyKostrov]

And one more comment: ... 'SPM700_d_protected.exe' (Win32): Loaded 'C:\Windows\System32\SHCore.dll'. Cannot find or open the PDB file. First-chance exception at 0x000007FBD76B811C (KernelBase.dll) in SPM700_d_protected.exe: 0xA1A01DB1 (parameters: 0x00000000007658A6, 0x0000000000763420, 0x0000000000000000). 'SPM700_d_protected.exe' (Win32): Loaded 'C:\Windows\System32\uxtheme.dll'. Cannot find or open the PDB file. ... However, that exception happens only two times, one is before and another one is after the breakpoint was hit.

[Bernard]

.>>>Are you doing any programming with Visual Studio and Fortran>>>

All my C programming is done with Visual Studio, Java programming is done with Eclipse and assembly programming is done with MASM32

In one of the previous threads one of the users on Fortran forum has problem with stack overflow caused by an extensive recursive call I advised the user to use windbg to perform automated analysis and that forum user was able to locate and correct the error.  You are failling to understand that windbg is a lot more powerful than VS debugger and can also debug at source level and in case of user Ovz the problem is related to C++  exception and to slowdown of VS debugger during the program debugging and I do not know how the second problem can be solved when user cannot even continue debugging after breakpoint is hit because environment is not responsive. Moreover Ovz has stated that exceptions are not problem for him.

[ovz]

Illya, I agree that WinDbg is quite powerful. Not all the devs on the team are comfortable with it and we need to keep them productive. I could also do some sleuthing in collaboration with Intel developers, and in this I expect guidance from people who are actually in position to fix the issue.  

[Bernard]

Ovz completely agree with you.If you are interested I can provide a helping hand in usage of windbg if not to solve but maybe to understand what went wrong with VS debugger.

[ovz]

Sergey,

I'm well aware of the GroveHandle exception. It is thrown _and_ handled by my code. This solution is there for some time now and never caused a debugger issue. 

I've seend 0xA1A01DB1 for a long time too. I assume this is something being thrown and handled by Windows. 

I get the debugger slowdown right in the beginning of the first Fortran subroutine that I call before any of the exceptions listed in the output happen. I also tried a simple example doing the same thing, but that one debugs fine. 

I'm working on clearing the actual source code to be sent to Premier Support. In the meanwhile any suggestions how to resolve this are very welcome. 

[ovz]

Annalee, 

The issue number I would like you to take a look at is 696480. Could you please take a look and see if we can arrange source code transfer as soon as we can?

[Bernard]

Ovz,

you provided call stack of one of the VS thread hence my question are you sure that this thread  belongs to debugger?It seems so.As I stated in one of my previous posts the problem could be related to DllGetClassObject and to functions further in call chain.Perceived slowness of the debugger's thread can some kind of synchronous wait maybe hang while waiting on some processing beign done on behalf of caller.

Just my 0.02 cents:)

[jimdempseyatthecove]

As illyapolak mentions, VS Debugger will not complete a step until it has stopped all application threads. Check the Threads debug window to see if any threads are at highest priority. Note, if you are running OpenMP then Name with __kmp_launch_monitor will be Highest priority and this is expected. If any of your other threads are at Highest priority then this may be part of the issue.

Additionally, when array bounds checking is enabled, then sometimes what you have displayed in your Watch, Locals, and Autos may induce an excessivly large number of bounds checking. Experiment with turning off array index out of bounds testing.

Jim Dempsey

[Bernard]

Actually windbg should be able to locate quickly the stalled VS debugger thread mainly with !runaway command and setting thread context  and single stepping could  reveal what went wrong with debugger's thread.But this is a task for Intel devs.

[SergeyKostrov]

>>I'm well aware of the GroveHandle exception. It is thrown _and_ handled by my code. This solution is there for some time >>now and never caused a debugger issue. >> >>I've seend 0xA1A01DB1 for a long time too. I assume this is something being thrown and handled by Windows. Could you do a simple test? If Yes, just comment that piece of codes that throws exception and verify what happens after that. If it doesn't resolve that problem there are two possibilities: - This is some problem introduced by Intel Visual Fortran Update 3 - This is some problem with Debugger of Visual Studio 2012 ( you will need to contact to Microsoft / another long story, unfortunately ) If it does resolve that problem there is just one possibility: - This is a problem with application's codes ( You've already mentioned that a problem with the exception is old ) So, simply ask yourself "What if something is really wrong with our codes?"

[Bernard]

>>>- This is some problem introduced by Intel Visual Fortran Update 3
- This is some problem with Debugger of Visual Studio 2012 ( you will need to contact to Microsoft / another long story, unfortunately )>>>

The problem is mostly related to what you wrote in qouted sentences.