Customer software crashes when performing VTune 2013 basic hotspots analysis

[Hagen_G_]

I am using VTune Amplifier XE 2013 to find performance bottlenecks in the software our company develops.

Recently I encountered a crash of our own software during a VTune basic hotspots analysis. (The crash does not appear without having VTune attached.) I get a Windows dialog telling me that the software is not working anymore, giving the problem signature "BEX64".

I have conducted a few tests and found:

• The problem also appears when I disable my virus scanner. There is no 3rd party firewall installed (only the Windows-internal one).
• The problem also appears when I use only 1 display.
• I have tried VTune 2013 Update 12 and Update 14; the problem occurs in both.
• There is no crash when I use the "advanced hotspots analysis". However, this is no option for me, because I get no backtraces and therefore no hierarchical view in the top-down analysis when I use the advanced method.

My system:

• Windows 7 enterprise 64 bit, SP1
• Dell Workstation T7500: Xeon [email protected] GHz, 24 GB RAM

Can anyone help?

[Hagen_G_]

Sometimes the "problem signature" is not "BEX64", but instead "APPCRASH".

[Hagen_G_]

I just found out that when I use software rendering in my application (normally I use hardware rendering with OpenGL) the crashes do not appear.

If I find out which OpenGL feature causes the crash, I will post it here.

[Bernard]

Hi can you upload the dump file?

It is better idea to move your post to VTune forum.

[Hagen_G_]

I hadn't found the VTune forum at the first glance. If I am able to move the post there, can you tell me how?

Regarding the dump file: Our application has registered a callback so that we write out the minidumps inside a crash handler of our application. (Thus Windows does not create a minidump.) The minidump created by our application is empty (I guess the one Windows would have created would be empty, too), so it makes so sense posting it. Normally these minidumps are really full of information, but with this specific crash it's not. :-(  We have had others OpenGL-related crashes before which resulted in no backtraces / empty minidumps.

[Bernard]

Regarding the idea of moving your post to VTune forum , simply create new thread in that forum.

Can you enable the handling of minidump file creation?I mean remove the callback and let the Windows mechanism try to create a dump file?

[QIAOMIN_Q_]

Now your issue has been moved to this Vtune forum.

[Hagen_G_]

Thanx for moving the thread!

I have tried to remove our crash handler callback and let Windows handle the exception. Result: even in this setting, no minidump was created by Windows.

[Hagen_G_]

I have just seen such a crash although I was using software rendering. However, this one now is much more difficult to reproduce. => Possibly more than just one problem.

[Peter_W_Intel]

Can you verify if your software crashes if you work VTune with application in command line?

For example - "amplxe-cl -collect hotspots -- your_application" 

[Hagen_G_]

Hi Peter,

interesting, it seems as if with VTune command line mode my app does not crash (at least not as often as before - i.e. for now, I couldn't reproduce the crash in cmd line mode).

This could be an interesting workaround for me, but to better filter out the time slot which I really want to measure (and to not waste memory measuring application load time and test preparation) I would like to have VTune attached to a running process... it this also possible in command line mode?

Best regards,

Hagen

[David_A_Intel1]

Hello Hagen:

Also, don't overlook the fact that you *can* collect callstacks with Advanced Hotspots and, therefore, get your "backtraces".  This may be another option.

[Bernard]

Strange issue.Could that be a failed DLL injection or the other means of reading your process memory?

Regarding no minidump issue.You can set a windbg to be a default post mortem debugger.

[Hagen_G_]

@MrAnderson: Thanks for the hint. I have simply overlooked the radio buttons which enable me to collect stacks and even call counts. (Btw: I would suggest to enable "Hotspots, stacks and context switches" or even the call counts option by default instead of setting "Hotspots only" as default. Reason: with "Hotspots only", the advanced analysis is even less advanced than the basic analysis.) But most interesting for me: in the advanced mode incl. stack traces, my software does not crash, while it does crash in basic mode. Are internal VTune code paths different between basic analysis and "advanced analysis with stack trace"?

@Iliyapolak: I don't clearly understand. How do I check if there is something wrong with DLL injection or reading process memory? How do I "set a windbg"?

However, in the whole I can say at this moment that I'm happy, because I can simply use the advanced analysis now to prevent those crashes.

[Bernard]

@Hagen

Sorry for providing confusing information.Disregard that sentence which is related to dll injection.

You can set windbg for postmortem debugging by using command line switch.Simply navigate to your windbg folder and issue this command:

windbg -I

Relevant link: http://msdn.microsoft.com/en-us/library/windows/hardware/ff542967(v=vs.85).aspx

[Bernard]

@Hagen

You did not tell us what is the type of crash?

[Hagen_G_]

@iliyapolak: All I know about the "type" of the crash is the problem signature "BEX64" or "APPCRASH", as I have written before. Or what kind of "type" do you address?

[Bernard]

@Hagen

I meant type of error like access violation or stack overflow.

APPCRASH is a kind of general name for application failure(crash).

[Hagen_G_]

@iliyapolak: The thing is that the only information I get (which Windows tells me) is that "the application does not work anymore". So I cannot figure out what is really happening or if it's an access violation or sth else.

[Bernard]

Can you search for minidump files on your machine?You can look in Windows directory or in Windows\System32 directory.

Btw. I fear that you disabled or interferd with  a minidump generation mechanism.

[Hagen_G_]

Hi Iliyapolak,

that has been discussed more above in this thread: Yes, our application catches those exception and handles minidump generation by itself. I have as a test disabled this mechanism to let Windows generate the minidumps. The result was that there was no minidump created by Windows. We have experienced such strange behaviour before when dealing with NVidia driver crashes. It sounds bad to have no dump at all, but it seems as if this really can happen...

Regards, Hagen