Fee.dll terribly slows down the debugger

[ovz]

Hi All,

My team works on a large code base a significant part of which is in Fortran. We recently upgraded to Intel(R) Visual Fortran Composer XE 2013 Update 3 Integration for Microsoft Visual Studio* 2012, 13.0.3624.11. After this we all are experiencing unbearable slowdowns when debugging the code. Whenever we try to debug Fortran or even just have a Fortran module on the call stack every debugger step through the code takes 10 or more seconds of full CPU load. Using Sysinternals Process Explorer I was able to see the call statck like one listed below. Note FEE.dll which is Intel(R) Fortran Expression Evaluator 13.0.2225.2. 

I need help with this urgently as the entire team cannot work normally because of this. 

msdia110.dll!DllGetClassObject+0x2c060
msdia110.dll!DllGetClassObject+0x2c1ff
msdia110.dll!DllGetClassObject+0x2c278
msdia110.dll!DllGetClassObject+0x2b3b7
msdia110.dll!DllGetClassObject+0x3536e
msdia110.dll!DllGetClassObject+0x1093a
FEE.dll!DllGetClassObject+0x205f7
FEE.dll!DllGetClassObject+0x20972
FEE.dll!DllGetClassObject+0xb18e
FEE.dll!DllGetClassObject+0xb4fe
FEE.dll!DllGetClassObject+0x9cc4
FEE.dll!DllGetClassObject+0x505f
FEE.dll!DllGetClassObject+0x655c
FEE.dll!DllGetClassObject+0x760a
FEE.dll!DllGetClassObject+0x7e14
FEE.dll!DllGetClassObject+0x8354
FEE.dll!DllGetClassObject+0x307e2
VSDebugEng.dll!DkmDllEnsureInitialized+0x8169c
VSDebugEng.dll!ProcDkmFree+0x1989
VSDebugEng.dll!ProcDkmFree+0x1776
VSDebugEng.dll!DkmDllEnsureInitialized+0x857e5
VSDebugEng.dll+0xc25d
VSDebugEng.dll!DkmDllUninitialize+0x24a
ntdll.dll!RtlInitializeExceptionChain+0x85
ntdll.dll!RtlInitializeExceptionChain+0x58

[SergeyKostrov]

>>...Whenever we try to debug Fortran or even just have a Fortran module on the call stack every debugger step through >>the code takes 10 or more seconds of full CPU load... By the way, it is a well know problem when debugging C/C++ applications which require lots of memory for processing. There are three possible reasons: - Application allocates lots of memory on the stack - Application allocates lots of memory from the system heap - Stack Reserve and Stack Commit values are too big

[ovz]

Sergey,

I appreciate the advise, but this is not the situation I described. The sluggishness in fee.dll happens _always_ whenver Fortran modules are on call stack, no matter how much memory is consumed etc. C/C++ with no Fortran debugs fine.

[SergeyKostrov]

I see. Here is another question: >> >>...msdia110.dll!DllGetClassObject+0x2c060 >>... Are you using DIA SDK ( Debug Interface Access SDK ) to get some information from program database files ( *.pdb )?

[ovz]

No, we don't. We don't have any funcitonality that talks to debugger.

[SergeyKostrov]

>>...Whenever we try to debug Fortran or even just have a Fortran module on the call stack every debugger step >>through the code takes 10 or more seconds of full CPU load... Is there any additional output in Output Window of Visual Studio when you're debugging? Please carefully verify.

[ovz]

No, just the usual stuff in Output Window. When I stepping through the code with 10s delay I can definitely see that nothing gets printed.

[SergeyKostrov]

I would also try commenting some blocks of codes. Sorry, it is really unclear what else could be wrong.

[Anonymous66]

Could you try setting "Tools > Options > Text Editor > Fortran > Advanced > Disable Database" to true?

[ovz]

Thanks for the suggestion Annalee. Disable Database has no effect on the phenomenon though. 

[Bernard]

Hi ovz,

did you try to debug with windbg?

[Bernard]

There is also possibility to debug a debugger.It can be done by attach invasively to VS process.I see that you were able with process explorer to identify and collect debugger's thread so it could not be a problem to do it with windbg.Next option after locating the VS debugger's thread will be execute !runaway metacommand in order to isolate most active threads in term of cpu time consumption.Next try to put breakpoint of functions that you suppose are responsible for hogging cpu.You can also be forced to disassemble those function to better understand what has happend.

[Bernard]

I would also advise before attaching windbg at least try to profile with Xperf VS process while debugging your application.Pay attention to FEE.dll

[IanH]

Debugging the Fortran integration to Visual Studio in the absence of any debug symbols or source is a completely absurd proposition for an Intel Visual Fortran end user.

In the absence of being able to create a simple reproducer I'd be zipping your entire solution up and uploading it as a Premier Support issue.

[ovz]

I did try debugging simple Fortran projects. There no slowdown stepping through it. Our codebase is indeed large and we have large number of lines in files that are in the call stack. It used to work fine in VS 2010 though. Hopefully there's a way to configure Fee.dll not to invoke things it does invoke on every step through the code in debugger. 

[Anonymous66]

Submitting your program to us as a test case would be very helpful. Since it is very large, I would suggest using Premier Support. If you post the issue number, I can assign it to myself.

Annalee

[SergeyKostrov]

>>>>...Whenever we try to debug Fortran or even just have a Fortran module on the call stack every debugger step >>>>through the code takes 10 or more seconds of full CPU load... >> >>Is there any additional output in Output Window of Visual Studio when you're debugging? Could you post a complete content of Output Window for the working cycle of your application? Thanks in advance.

[Bernard]

>>>Absence of any debug symbols>>>

Maybe FEE.dl has public debug symbols.Process explorer was able to resolve exact function name DllGetClassObject  called from FEE.dll I suppose that the problem is located further in call chain.At least you can inspect return values of callee.DllGetClassObject returns void pointer to interface on failure it will return null pointer.I think that this could be the case of extensive calls to DllGetClassObject.

[ovz]

Annalee, thanks for being ready to help. I need to get an approval to post the codebase to Premier support issue.

[ovz]

Sergey, 

I'm attaching the output in two files. the _BEFORE_ file has output captured before application hits the breakpoint in Fortran. There was a 10 second slowdown after the breakpoint hit but before the enviroment is responsive. Then I stepped through 3 lines, each step assoicated with a delay. _AFTER_ file contains the rest of the output. Again, the output looks totally normal for our applicaiton.

[SergeyKostrov]

Thanks for these Output Window files and I'll take a look tomorrow.