A bug in vtss.sys

[Slava_I_]

There is a bug in vtss.sys - an attempt to close an invalid handle from the driver, the bug reveales itself only when the driver verifier is active. Mostly it is a nuisance as this bug should not have any impact on the system but the driver verifier must be disabled to use VTune 2013 as Microsoft considers this bug as a fatal error that should be fixed so the driver verifier crashes the system. The following is a crash analysis

0: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************

INVALID_KERNEL_HANDLE (93)
This message occurs if kernel code (server, redirector, other driver, etc.)
attempts to close a handle that is not a valid handle.
Arguments:
Arg1: 0000000000000000, The handle that NtClose was called with.
Arg2: fffff8a0000018b0,
Arg3: 0000000000000000
Arg4: 0000000000000001

Debugging Details:
------------------

DEFAULT_BUCKET_ID: VISTA_DRIVER_FAULT

BUGCHECK_STR: 0x93

PROCESS_NAME: System

CURRENT_IRQL: 2

LAST_CONTROL_TRANSFER: from fffff80003bbc3c2 to fffff80003abd620

STACK_TEXT:
fffff880`02fd2da8 fffff800`03bbc3c2 : 00000000`00000000 fffffa80`03d1e040 00000000`00000065 fffff800`03b03b10 : nt!RtlpBreakWithStatusInstruction
fffff880`02fd2db0 fffff800`03bbd1ae : 00000000`00000003 00000000`00000000 fffff800`03b006d0 00000000`00000093 : nt!KiBugCheckDebugBreak+0x12
fffff880`02fd2e10 fffff800`03ac56c4 : 00000000`0000001c fffff980`1288efe0 00000000`00000000 00000000`00000000 : nt!KeBugCheck2+0x71e
fffff880`02fd34e0 fffff800`03d2261b : 00000000`00000093 00000000`00000000 fffff8a0`000018b0 00000000`00000000 : nt!KeBugCheckEx+0x104
fffff880`02fd3520 fffff800`03ac4813 : fffff880`02fd3600 00000000`00000000 00000000`00000000 00000000`00000000 : nt! ?? ::NNGAKEGL::`string'+0x51ce4
fffff880`02fd3620 fffff800`03ac0db0 : fffff880`1fe0d3ff 00000000`00000000 fffff800`03c54880 00000000`00240024 : nt!KiSystemServiceCopyEnd+0x13
fffff880`02fd3828 fffff880`1fe0d3ff : 00000000`00000000 fffff800`03c54880 00000000`00240024 fffffa80`0509d4a0 : nt!KiServiceLinkage
fffff880`02fd3830 fffff880`1fe10502 : 00000000`00000000 fffffa80`05184db0 00000000`746c6600 fffff880`02fd3970 : vtss+0x73ff
fffff880`02fd38a0 fffff800`03eadeb7 : fffffa80`05184db0 ffffffff`80001bf0 fffff980`1288efe0 00000000`00000001 : vtss+0xa502
fffff880`02fd39a0 fffff800`03eae2b5 : 00000000`00000010 00000000`00000000 00000000`00000010 00000000`00010202 : nt!IopLoadDriver+0xa07
fffff880`02fd3c70 fffff800`03ad27e1 : fffff880`00000000 ffffffff`80001bf0 fffff800`03eae260 00000000`00000000 : nt!IopLoadUnloadDriver+0x55
fffff880`02fd3cb0 fffff800`03d656fa : ffffffff`ffffffff fffffa80`03d1e040 00000000`00000080 fffffa80`03d065a0 : nt!ExpWorkerThread+0x111
fffff880`02fd3d40 fffff800`03aa3b46 : fffff880`009e6180 fffffa80`03d1e040 fffff880`009f0f40 01e09a41`0c0a3590 : nt!PspSystemThreadStartup+0x5a
fffff880`02fd3d80 00000000`00000000 : fffff880`02fd4000 fffff880`02fce000 fffff880`02fd28b0 00000000`00000000 : nt!KiStartSystemThread+0x16

[Marián__VooDooMan__M]

iliyapolak wrote:

@Marian

If you need VTune analysis I can do it.My installation works perfectly.

IMO (tho I might be wrong) this depends on microarchitecture and various other things.

The bug is present in both core2 quad and laptop i7 with Haswell microarchittecture, i.e. 2 different machines. Clearly there is some bug. I don't want to blame Intel, since it is possible Microsoft drivers could be causing problem in association with Intel driver.

The last Microsoft's Tuesday (the second Tuesday in month bringing updates) didn't helped, though I always send BSOD dump to Microsoft.

[Marián__VooDooMan__M]

iliyapolak wrote:

@Marian

If you need VTune analysis I can do it.My installation works perfectly.

I really appreciate your offer, but it is a lot tricky, since I am coding VST plug-in for DAW software. Thank you very much, but it would be extremely painful for you to setup all needed SW's on your side (besides COM registration, etc...).

[Bernard]

Marián "VooDooMan" Meravý wrote:

Quote:

iliyapolak wrote:

@Marian

If you need VTune analysis I can do it.My installation works perfectly.

I really appreciate your offer, but it is a lot tricky, since I am coding VST plug-in for DAW software. Thank you very much, but it would be extremely painful for you to setup all needed SW's on your side (besides COM registration, etc...).

I thought that you have much simplier code or project to test.

[Bernard]

>>>Clearly there is some bug. I don't want to blame Intel, since it is possible Microsoft drivers could be causing problem in association with Intel driver>>>

One of the posted here BSOD 's was probably caused by vstss.sys because of forbidden by design of Win kernel operation performed by Intel driver.The second BSOD from the post #73 is much harder to understand.

[Marián__VooDooMan__M]

@ Intel

Same question again: Is there some time schedule for update VTune upgrading vtss.sys which fixes the bug?

[Peter_W_Intel]

I always track this issue and reported the status to our engineering. Unfortunately this problem is hard to be reproduced...but it should be exist because many users are ware of this BSOD. We are sorry for that and thanks to iliyapolak who provided a workaround for using VTune avoid vtss issue.

Here are resolution from our developer -"contains two different issues - (1) a crash on closing an invalid handle under Driver Verifier, and (2) a page fault at a DPC level in HalSendSoftwareInterrupt.".

The developer made an amendment to #1 - added explicit checks for bad args - see if that solves the problem. If not - that must be a crash caused by a subtler issue (like memory corruption), in which case we will need a fresh crash dump and a detailed description (step-by-step) of what was going on before the crash. As to #2 - that would be nice if the user tried update 16 (Peter: It will be ready soon), but I do not promise anything - that will just help us get a fresh crash dump to analyze. And again, more detailed description of what was going on before the crash is welcome. Like whether the Driver Verifier was active, whether it happened at the very beginning of profiling, what kind of workload was analyzed (multithreaded, single-threaded, etc.), which system it was, was any .vtss trace generated?

Peter: regarding that many BSOD errors occurred on Windows 8 with new microarchitecture, i.e. Haswell platform, it could be a workaround to work VTune on Windows 7 to avoid drivers conflicting.

Thanks for your patience if you can wait more time to try coming Update 16. Please provide info when BSOD happens again:

1. amplxe-feedback -create-bug-report <report archive> ; let us know your environment

2. Give us what you did as more as detail, with dump file; required by developer.

Again, sorry for any inconvenience caused.

Regards, Peter 

[Bernard]

>>>We are sorry for that and thanks to iliyapolak who provided a workaround for using VTune avoid vtss issue.>>>

@Peter

I am glad to hear that my workaround really was helpful.

[Peter_W_Intel]

You are always welcome, iliyapolak!

All, please note that VTune(TM) Amplifier XE 2013 Update 16 will be ready after 24 hours, if you find BSOD again - give me info as I posted on 03/14/2014 - 10:29

Thank you!

[Bernard]

@Peter

Thank you very much.

Fortunately I do not have BSOD more, but as I said it earlier VS debugger crashes when I try to debug my code.It is due to VTune integration.I plan to investigate that issue.

[Peter_W_Intel]

It sounds everything is OK after installing VTune(TM) Amplifier XE 2013 Update 16?

Then you never saw a BSOD, even when using the VS debugger?

[Bernard]

Hi Peter

I need to install Update 16 and check if I can use VS debugger.

I will let you know.

Yes VS debugger crashes when VTune VS integration.

[Marián__VooDooMan__M]

Peter,

I can confirm after like 1 week of using VTune, I had no BSOD.

Though, when I click "new analysis" I see no "haswell" node in the tree view. I have Haswell i7.

[Peter_W_Intel]

@ Marián "VooDooMan" Meravý

I'm glad there is no BSOD on your side. Yes, there is no node for "Haswell" - only Intel Core 2, Nehalem, SandBridge have CPU specific analysis types. Haswell & IvyBridge have no CPU specific analysis.

You can use General Exploration & Bandwidth analysis for Haswell platform.

[Robert_G_]

I am running the latest (update 16) and I am still getting the above BSOD. This is on a Win7-64bit install using a Core i7 processor. Happens everytime I attempt to install the sample driver. I just am trying to use the eval version to verify if this tool fits our needs.

[Marián__VooDooMan__M]

@Robert G.

can you post results from windbg tool? use "!analyze -v" command.

[Bernard]

@Robert

Please upload analyze -v output

[Marián__VooDooMan__M]

@iliyapolak

You don't need to verbatim copy my posts, to earn the points.

[Robert_G_]

here is the WinDBG output

*** Fatal System Error: 0x00000093
                       (0x0000000000000000,0xFFFFF8A000001910,0x0000000000000000,0x0000000000000001)

Break instruction exception - code 80000003 (first chance)

A fatal system error has occurred.
Debugger entered on first try; Bugcheck callbacks have not been invoked.

A fatal system error has occurred.

Connected to Windows 7 7601 x64 target at (Fri Apr 11 17:25:55.409 2014 (UTC - 5:00)), ptr64 TRUE
Loading Kernel Symbols
...............................................................
................................................................
...........................
Loading User Symbols

Loading unloaded module list
..................
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck 93, {0, fffff8a000001910, 0, 1}

*** ERROR: Module load completed but symbols could not be loaded for vtss.sys
Probably caused by : vtss.sys ( vtss+7ed3 )

Followup: MachineOwner
---------

nt!DbgBreakPointWithStatus:
fffff800`030759f0 cc              int     3
0: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

INVALID_KERNEL_HANDLE (93)
This message occurs if kernel code (server, redirector, other driver, etc.)
attempts to close a handle that is not a valid handle.
Arguments:
Arg1: 0000000000000000, The handle that NtClose was called with.
Arg2: fffff8a000001910, 
Arg3: 0000000000000000
Arg4: 0000000000000001

Debugging Details:
------------------

DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

BUGCHECK_STR:  0x93

PROCESS_NAME:  System

CURRENT_IRQL:  2

LAST_CONTROL_TRANSFER:  from fffff8000316ea12 to fffff800030759f0

STACK_TEXT:  
fffff880`033a0c78 fffff800`0316ea12 : 00000000`00000000 fffffa80`036c1b50 00000000`00000065 fffff800`030b8878 : nt!DbgBreakPointWithStatus
fffff880`033a0c80 fffff800`0316f7fe : 00000000`00000003 00000000`00000000 fffff800`030b90d0 00000000`00000093 : nt!KiBugCheckDebugBreak+0x12
fffff880`033a0ce0 fffff800`0307dcc4 : 00000000`0000001c fffff980`16cecfe0 00000000`00000000 00000000`00000000 : nt!KeBugCheck2+0x71e
fffff880`033a13b0 fffff800`032dbc9b : 00000000`00000093 00000000`00000000 fffff8a0`00001910 00000000`00000000 : nt!KeBugCheckEx+0x104
fffff880`033a13f0 fffff800`0307ce53 : fffff880`033a1400 00000000`00000000 00000000`00000000 00000000`00000000 : nt! ?? ::NNGAKEGL::`string'+0x4d9f4
fffff880`033a14f0 fffff800`03079410 : fffff880`1f2e9ed3 00000000`00000000 00000000`00000035 fffff800`00240024 : nt!KiSystemServiceCopyEnd+0x13
fffff880`033a16f8 fffff880`1f2e9ed3 : 00000000`00000000 00000000`00000035 fffff800`00240024 fffffa80`0871aa30 : nt!KiServiceLinkage
fffff880`033a1700 fffff880`1f2ed088 : 00000000`00000000 fffffa80`08c7eba0 fffff880`033a17d0 00000000`0000001c : vtss+0x7ed3
fffff880`033a1770 fffff800`03468467 : fffffa80`08c7eba0 00000000`00000000 fffff980`16cecfe0 00000000`00000001 : vtss+0xb088
fffff880`033a1860 fffff800`03468865 : 00000000`00000010 00000000`00000000 00000000`00000010 00000000`00010202 : nt!IopLoadDriver+0xa07
fffff880`033a1b30 fffff800`03087261 : 00000000`00000000 ffffffff`80000284 fffff800`03468810 fffffa80`036c1b50 : nt!IopLoadUnloadDriver+0x55
fffff880`033a1b70 fffff800`0331a2ea : 00000000`00000050 fffffa80`036c1b50 00000000`00000080 fffffa80`0363fb30 : nt!ExpWorkerThread+0x111
fffff880`033a1c00 fffff800`0306e8e6 : fffff880`031b4180 fffffa80`036c1b50 fffff880`031befc0 00000000`00000000 : nt!PspSystemThreadStartup+0x5a
fffff880`033a1c40 00000000`00000000 : fffff880`033a2000 fffff880`0339c000 fffff880`033a0760 00000000`00000000 : nt!KiStartSystemThread+0x16

STACK_COMMAND:  kb

FOLLOWUP_IP: 
vtss+7ed3
fffff880`1f2e9ed3 8b442458        mov     eax,dword ptr [rsp+58h]

SYMBOL_STACK_INDEX:  7

SYMBOL_NAME:  vtss+7ed3

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: vtss

IMAGE_NAME:  vtss.sys

DEBUG_FLR_IMAGE_TIMESTAMP:  531e3c70

FAILURE_BUCKET_ID:  X64_0x93_VRF_vtss+7ed3

BUCKET_ID:  X64_0x93_VRF_vtss+7ed3

Followup: MachineOwner
---------

0: kd> .reboot
Shutdown occurred at (Fri Apr 11 17:31:47.816 2014 (UTC - 5:00))...unloading all symbol tables.
Waiting to reconnect...
Connected to Windows 7 7601 x64 target at (Fri Apr 11 17:32:08.059 2014 (UTC - 5:00)), ptr64 TRUE

[Peter_W_Intel]

I have reported this to our developer, thank you. Is it possible that you can upload dump file for investigating?

[Bernard]

Marián "VooDooMan" Meravý wrote:

@iliyapolak

You don't need to verbatim copy my posts, to earn the points.

@Marian

I am quite satisfied with my total number of points so I  really do not need to copy your posts.

My intention is to help solving that dreaded BSOD  problem which plagues VTune users.

Is that so hard to understand?

[Bernard]

@Peter

If this is the same BSOD which was solved by lowering IRQL I suppose that some other kernel module could raise IRQL to level 0x2 right before that function call which caused kernel BugCheck.