x64 tachyon example fails to start with 'amplxe-cl -collect hotspots'

[Micah_Z_]

Hello

We are evaluating VTune Amplifier and are having trouble getting it to work with our x64 application.  When attempting a basic hotspot analysis, the application crashes as soon as it is launched (or attached to).  I've been able to get the 'advanced hotspot' analysis to work, but that's it.  I've also tried the tachyon example, and while the win32 version works ok, the x64 version has the same problem.  No matter how it is started (from VTune inside Visual Studio, amplxe-cl command line, or the standalone VTune), the find_hotspots.exe crashes immediately with no error messages beyond the standard windows error popups.

I dug into the windows event log and found this error for when the find_hotspots.exe fails:

"Faulting application name: find_hotspots.exe, version: 0.0.0.0, time stamp: 0x52570c83
Faulting module name: tpsstool.dll, version: 0.0.0.0, time stamp: 0x5241e2ee
Exception code: 0xc0000005
Fault offset: 0x000000000006f133
Faulting process id: 0x1bd0
Faulting application start time: 0x01cec5fb74065388
Faulting application path: C:\TEMP\vtune\tachyon\vc9\find_hotspots_x64_Debug\find_hotspots.exe
Faulting module path: C:\Program Files (x86)\Intel\VTune Amplifier XE 2013\bin64\tpsstool.dll"

Are there any special tricks/limitations to analysing x64 applications?

What works:

• An advanced hotspot analysis of our x64 application (vtunes standalone or inside visual studio)
• The win32 version of tachyon/find_hotspots.exe (visual studio or the command line amplxe-cl)

What Doesn't Work:

• Basic Hotspot, Concurrency, Lock and Wait of our x64 application (vtunes standalone or inside visual studio (launch or attach))
• The x64 version of tachyon/find_hotspots (visual studio)
• The x64 version of tachyon/find_hotspots using 'amplxe-cl' from the Intel 64 Visual Studio Mode command prompt ("amplxe-cl -collect hotspots -- ./find_hotspots.exe ..\..\dat\balls.dat")

Our setup:

• VTune Amplifier XE 2013 Update 13
• Windows 7 x64
• Visual Studio 2010
• CPU: Intel Xeon

Thanks in advance,

micah

[Bernard]

It is an access violation type error and in your case could be caused by calling unmapped tpsstool.dll exports.Fault offset is the virtual (rva) address of the instruction which caused this exception.

Have you tried to sample the other applications?Does tpsstool.dll crash still occur?

[Peter_W_Intel]

Interesting! Can you please attach find_hotspots.exe and its pdb file? I want to reproduce it on my side. Thanks!

[Micah_Z_]

Thanks for the quick resposes.

iliyapolak: I have the same problem with the analyze_locks example.

Peter: I've attached the exe and pdb as requested.

thanks,

micah

[Bernard]

@Micah

What is the error message when you try to run analyze_locks example.Does the fault offset stay in the same range?

[Peter_W_Intel]

Unfortunately I cannot reproduce this problem after I used your find-hotspots-xe64-debug version. Was it due to your system or VTune was not installed well (I suggest to uninstall/reinstall the product)? I don't think it was due to binary...

You may submit this specific issue to Intel Premier, with your binaries, command line with outputs, result directory AND run below cmd to attach logs when meeting the crash. That is all what I can help on forum.

amplxe-feedback -create-bug-report <report archive>

========================================================================

C:\temp\tmp>amplxe-cl -collect locksandwaits -- find_hotspots.exe balls.dat
amplxe: Warning: Highly accurate CPU time collection is disabled for this analys
is. To enable this feature, run the product with the administrative privileges.
amplxe: Using result path `C:\temp\tmp\r002lw'
amplxe: Executing actions 22 % Resolving information for `IMM32.DLL'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\GDI32.dll'.

amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\MSCTF.dll'.

amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\IMM32.DLL'.

amplxe: Executing actions 23 % Resolving information for `KERNELBASE.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\kernel32.dl
l'.
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\KERNELBASE.
dll'.
amplxe: Executing actions 23 % Resolving information for `uxtheme.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\uxtheme.dll
'.
amplxe: Executing actions 23 % Resolving information for `MSVCR100D.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\CRYPTBASE.d
ll'.
amplxe: Executing actions 24 % Resolving information for `LPK.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\LPK.dll'.
amplxe: Executing actions 24 % Resolving information for `gdiplus.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\PGPhk.dll'.

amplxe: Warning: Cannot locate symbols for file `C:\Windows\WinSxS\amd64_microso
ft.windows.gdiplus_6595b64144ccf1df_1.1.7601.18120_none_2b25b14c71ebf230\gdiplus
.dll'.
amplxe: Executing actions 24 % Resolving information for `ole32.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\ole32.dll'.

amplxe: Executing actions 25 % Resolving information for `USER32.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\USER32.dll'
.
amplxe: Executing actions 25 % Resolving information for `USP10.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\RPCRT4.dll'
.
amplxe: Warning: Cannot locate symbols for file `C:\Windows\SYSTEM32\ntdll.dll'.

amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\USP10.dll'.

amplxe: Executing actions 25 % Resolving information for `tpsstool.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Program Files (x86)\Intel\VT
une Amplifier XE 2013\bin64\tpsstool.dll'.
amplxe: Executing actions 25 % Resolving information for `msvcrt.dll'
amplxe: Warning: Cannot locate symbols for file `C:\Windows\system32\msvcrt.dll'
.
amplxe: Executing actions 50 % Generating a report

Collection and Platform Info
----------------------------
Parameter r002lw
------------------------ --------------------------------
Application Command Line find_hotspots.exe "balls.dat"
Operating System Microsoft Windows 7
Computer Name ZWANG14-MOBL2.ccr.corp.intel.com
Result Size 1643246

CPU
---
Parameter r002lw
----------------- ---------------------------------------
Name Intel(R) Core(TM) Processor 2xxx Series
Frequency 2500000000
Logical CPU Count 4

Summary
-------
Average Concurrency: 0.989
Elapsed Time: 107.690
CPU Time: 96.736
Wait Time: 214.882
CPU Usage: 0.895
amplxe: Executing actions 100 % done

[Bernard]

@Micah

any updates?

[Micah_Z_]

Thanks for the responses.

I'm away until Thursday. I'll  try what you've suggested then and post the results. 

I'd also wondered about trying on another machine on our corporate network. Is it possible that some anitivus app or system security/access setting could be interfering?

micah

[Bernard]

You can try to run VTune analysis on different machine(s) on your corporate network it will help us to isolate the problem maybe to one specific machine.

Regarding the error it is impossible to know what was the cause of the error without the collecting the dump file of the failed VTune process and performing at least simple heuristic analysis with windbg command !analyze -v

[Micah_Z_]

Solved!

It looks like an out-of-date installation of Symantec Endpoint Protection was interfering with VTune.  I've upgraded to SEP version 12.1.3001 and VTune is now working as expected.

Thanks for the help,

micah

[Bernard]

Glad that you found the solution for the problem.

Btw I suppose that export table of the tpsstool has been hooked by Symantec end point protection module.