Analyzers
Community support for Analyzers (Intel VTune™ Profiler, Intel Advisor, Intel Inspector)
Announcements
The Intel sign-in experience has changed to support enhanced security controls. If you sign in, click here for more information.
4828 Discussions

BSOD with Advanced Analysis

Marián__VooDooMan__M
New Contributor II
611 Views

I have 100% reproducible BugCheck with the latest VTune update to date. Though stack trace is not very meaningful.

1: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CLOCK_WATCHDOG_TIMEOUT (101)
An expected clock interrupt was not received on a secondary processor in an
MP system within the allocated interval. This indicates that the specified
processor is hung and not processing interrupts.
Arguments:
Arg1: 0000000000000018, Clock interrupt time out interval in nominal clock ticks.
Arg2: 0000000000000000, 0.
Arg3: ffffd0017f5a7180, The PRCB address of the hung processor.
Arg4: 0000000000000002, 0.

Debugging Details:
------------------


BUGCHECK_STR:  CLOCK_WATCHDOG_TIMEOUT_8_PROC

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

PROCESS_NAME:  Recorder7_Proc

CURRENT_IRQL:  d

ANALYSIS_VERSION: 6.3.9600.17298 (debuggers(dbg).141024-1500) amd64fre

STACK_TEXT:  
ffffd001`7f566c88 fffff803`e97f4ecf : 00000000`00000101 00000000`00000018 00000000`00000000 ffffd001`7f5a7180 : nt!KeBugCheckEx
ffffd001`7f566c90 fffff803`e96c5f67 : 00000000`00000000 00000000`00000000 00000000`00000001 00000000`00000000 : nt! ?? ::FNODOBFM::`string'+0xad0f
ffffd001`7f566d20 fffff803`e962167f : 00000000`00000002 00000000`00000000 fffff803`e966deb0 00000000`00000002 : nt!KeClockInterruptNotify+0x787
ffffd001`7f566f40 fffff803`e9760143 : fffff803`e966de00 fffff803`e9788057 ffffd001`00000000 00000000`00000000 : hal!HalpTimerClockInterrupt+0x4f
ffffd001`7f566f70 fffff803`e97db12a : fffff803`e966de00 00000000`00000000 00000000`00000000 00000000`00000002 : nt!KiCallInterruptServiceRoutine+0xa3
ffffd001`7f566fb0 fffff803`e97db50f : 00000000`00000000 00000000`67198f47 00000000`00000000 00000000`00000000 : nt!KiInterruptSubDispatchNoLockNoEtw+0xea
ffffd000`27ae0460 fffff803`e96e98a8 : 00000000`00000046 fffff800`7a06b8ed 00000000`80000005 ffffe001`051b7080 : nt!KiInterruptDispatchLBControl+0x11f
ffffd000`27ae05f0 fffff803`e96ec6d6 : 00000000`00000000 00000000`00000002 00000000`001f4604 00000000`00000001 : nt!MiFlushTbList+0x2d8
ffffd000`27ae0730 fffff803`e96c9b2a : 00000000`00000006 ffffd000`27ae0909 00000000`00000006 00000000`00000006 : nt!MiFlushTbAsNeeded+0xe6
ffffd000`27ae0870 fffff803`e96cb1df : fffff901`00000000 fffff960`0027973c 00000000`00000002 00000000`00000000 : nt!MiAllocatePagedPoolPages+0x17a
ffffd000`27ae0970 fffff803`e96ca590 : 00000000`00000000 00000000`00000021 00000000`000054a0 000000a1`8fb3ecc0 : nt!MiAllocatePoolPages+0xc3
ffffd000`27ae09c0 fffff803`e992d973 : 00000000`00000001 00007ff6`9ee8e9c0 00007ffe`00000021 fffff803`e973a226 : nt!ExpAllocateBigPool+0xd0
ffffd000`27ae0ab0 fffff960`0048bcd7 : ffffd000`27ae0c21 00000000`00000001 00000000`00000000 fffff960`0029ba0f : nt!ExAllocatePoolWithTag+0xa83
ffffd000`27ae0b80 fffff960`0048c436 : fffff901`45ba8000 00000000`00000000 00000000`00000001 00000000`00000000 : win32k!NSInstrumentation::CLeakTrackingAllocator::AllocateCommon<<lambda_16c8fa6c1365df5cd1a45d38bb6ccfe2>,<lambda_278f24766fae20cbf0c48ddfc336e8e8> >+0x2f
ffffd000`27ae0c50 fffff960`003d2f51 : ffffe001`00000001 00000000`00000021 00000000`00000000 ffffe001`6c777355 : win32k!NSInstrumentation::CLeakTrackingAllocator::Allocate+0x46
ffffd000`27ae0ca0 fffff960`001733e4 : fffff901`45ba8000 00000000`00000000 fffff901`45ba8000 00000000`00000000 : win32k!InternalRebuildHwndListForIMEClass+0x31
ffffd000`27ae0d20 fffff960`00295293 : 00000000`00000001 ffffd000`27ae0ec0 00000000`00000000 00000000`00000006 : win32k!BuildHwndList+0xb4
ffffd000`27ae0d50 fffff803`e97e54b3 : ffffe001`051b7080 00000000`00000000 000000a1`91467b48 00000083`00000000 : win32k!NtUserBuildHwndList+0x183
ffffd000`27ae0dd0 00007ffe`ac1dbcca : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x13
000000a1`91467b28 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ffe`ac1dbcca


STACK_COMMAND:  kb

SYMBOL_NAME:  ANALYSIS_INCONCLUSIVE

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: Unknown_Module

IMAGE_NAME:  Unknown_Image

DEBUG_FLR_IMAGE_TIMESTAMP:  0

IMAGE_VERSION:  

FAILURE_BUCKET_ID:  CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

BUCKET_ID:  CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:clock_watchdog_timeout_8_proc_analysis_inconclusive

FAILURE_ID_HASH:  {f60c102e-9b2c-017b-9309-e4af1999e360}

Followup: MachineOwner
---------

best, vdm.

0 Kudos
11 Replies
Peter_W_Intel
Employee
611 Views

I reported your data to development team. Thank you.

Bernard
Black Belt
611 Views

Yes I agree that call stack is not very informative in your case.

I would advise to begin investigating by issueing following command !prcb  ffffd0017f5a7180. You should search for TIB exception and definitaly have a look at currently running thread on CPU which was hung or inresponsive before BSOD.

Marián__VooDooMan__M
New Contributor II
611 Views

iliyapolak wrote:

Yes I agree that call stack is not very informative in your case.

I would advise to begin investigating by issueing following command !prcb  ffffd0017f5a7180. You should search for TIB exception and definitaly have a look at currently running thread on CPU which was hung or inresponsive before BSOD.

Too bad I have accidentally deleted MEMORY.DMP, so I will try later to get BSOD again, and save informations, and give you more data.

Marián__VooDooMan__M
New Contributor II
611 Views

I have reproduced BugCheck, but now I have in stack trace that kernel driver for my HW (USB device for home studio recording) was there. I will try to upgrade driver.

It seems that VTune and this HW don't like each other.

Marián__VooDooMan__M
New Contributor II
611 Views

I upgraded driver for audiosystem, but BugCheck is still reproducible.

It seems to me that there is some deadlock in VTune kernel-mode driver.

7: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CLOCK_WATCHDOG_TIMEOUT (101)
An expected clock interrupt was not received on a secondary processor in an
MP system within the allocated interval. This indicates that the specified
processor is hung and not processing interrupts.
Arguments:
Arg1: 0000000000000018, Clock interrupt time out interval in nominal clock ticks.
Arg2: 0000000000000000, 0.
Arg3: ffffd000cf6dc180, The PRCB address of the hung processor.
Arg4: 0000000000000006, 0.

Debugging Details:
------------------


BUGCHECK_STR:  CLOCK_WATCHDOG_TIMEOUT_8_PROC

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

PROCESS_NAME:  System

CURRENT_IRQL:  d

ANALYSIS_VERSION: 6.3.9600.17298 (debuggers(dbg).141024-1500) amd64fre

STACK_TEXT:  
ffffd000`cf794c88 fffff801`51d6decf : 00000000`00000101 00000000`00000018 00000000`00000000 ffffd000`cf6dc180 : nt!KeBugCheckEx
ffffd000`cf794c90 fffff801`51c3ef67 : 00000000`00000000 00000000`00000000 00000000`00000001 00000000`00000000 : nt! ?? ::FNODOBFM::`string'+0xad0f
ffffd000`cf794d20 fffff801`5239e67f : 00000000`00000000 00000000`00000000 ffffe000`ff201cb0 00000000`00000000 : nt!KeClockInterruptNotify+0x787
ffffd000`cf794f40 fffff801`51cd9143 : ffffe000`ff201c00 fffff801`51d01057 ffffd000`00000000 00000000`00000000 : hal!HalPerformEndOfInterrupt+0x6f
ffffd000`cf794f70 fffff801`51d5412a : ffffe000`ff201c00 00000000`00000000 00000000`00000000 00000000`00000001 : nt!KiCallInterruptServiceRoutine+0xa3
ffffd000`cf794fb0 fffff801`51d5450f : 00000000`00000000 00000000`6545cb88 00000000`00000000 ffffe001`051ffffa : nt!KiInterruptSubDispatchNoLockNoEtw+0xea
ffffd000`26d0d460 fffff801`51c628b0 : fffff801`b6caae40 fffff801`b6cabc80 00000000`80000005 00000000`00000000 : nt!KiInterruptDispatchLBControl+0x11f
ffffd000`26d0d5f0 fffff801`51ccaa83 : e9c00003`71edf921 00000000`00000001 ffffb001`b2b01000 00000000`00000000 : nt!MiFlushTbList+0x2e0
ffffd000`26d0d730 fffff801`51c6ad95 : 00000000`00000000 00000000`00000000 ffffd000`26d0d911 00000000`00001000 : nt!MmSetAddressRangeModified+0xc7
ffffd000`26d0d860 fffff801`51c6b57b : ffffe001`04b6f4c8 00000000`00000000 ffffe001`00000001 00000000`00000000 : nt!CcFlushCachePriv+0x3cd
ffffd000`26d0d970 fffff801`51c8c465 : ffffe001`00737760 fffff801`51c02000 ffffd000`00000000 ffffe001`00168040 : nt!CcWriteBehindInternal+0x17b
ffffd000`26d0da00 fffff801`51c8c81d : 00000000`00000000 ffffe000`fefc3bb0 ffffe001`02bcfdd0 00000000`00000000 : nt!CcWriteBehind+0x95
ffffd000`26d0daa0 fffff801`51caf6bc : fffff801`b38f3500 ffffe001`00168100 fffff801`51eeaa20 00000000`00000000 : nt!CcWorkerThread+0x22d
ffffd000`26d0db50 fffff801`51d0236c : ffffe000`ff0dd880 ffffe001`00168040 00000000`00000080 ffffe001`00168040 : nt!ExpWorkerThread+0x28c
ffffd000`26d0dc00 fffff801`51d592c6 : ffffd000`cf6dc180 ffffe001`00168040 ffffe000`ff0dd880 00000000`00000000 : nt!PspSystemThreadStartup+0x58
ffffd000`26d0dc60 00000000`00000000 : ffffd000`26d0e000 ffffd000`26d08000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16


STACK_COMMAND:  kb

SYMBOL_NAME:  ANALYSIS_INCONCLUSIVE

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: Unknown_Module

IMAGE_NAME:  Unknown_Image

DEBUG_FLR_IMAGE_TIMESTAMP:  0

IMAGE_VERSION:  

FAILURE_BUCKET_ID:  CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

BUCKET_ID:  CLOCK_WATCHDOG_TIMEOUT_8_PROC_ANALYSIS_INCONCLUSIVE

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:clock_watchdog_timeout_8_proc_analysis_inconclusive

FAILURE_ID_HASH:  {f60c102e-9b2c-017b-9309-e4af1999e360}

Followup: MachineOwner
---------

7: kd> !prcb ffffd000cf6dc180
Cannot get PRCB address
7: kd> !prcb 0xffffd000cf6dc180
Cannot get PRCB address

 

Bernard
Black Belt
611 Views

>>>It seems to me that there is some deadlock in VTune kernel-mode driver.>>>

Probably yes I think so, but without seeing prcb record is hard to tell what has happend.

Bernard
Black Belt
611 Views

Regarding that error where prcb cannot be read. Did you try to load full memory dump or at least kernel memory dump and then run !prcb command?

I wonder why this command failed.

Can you try and manually inspect prcb address?

Bernard
Black Belt
611 Views

You can read this excellent blog post related directly to debugging BSOD 0x101

http://blogs.msdn.com/b/ntdebugging/archive/2011/10/26/debugging-a-clock-watchdog-timeout-bugcheck.aspx

Bernard
Black Belt
611 Views

>>>I wonder why this command failed.

Can you try and manually inspect prcb address?>>>

@Marian

Please disregard my quoted post and run prcb command on each CPU core untill you find matching address of hung CPU prcb.

For example:

!prcb 0

!prcb 1

.....//.....//....

!prcb 7

Bernard
Black Belt
611 Views

@Marian

Any updates on your issues?

Marián__VooDooMan__M
New Contributor II
611 Views

I sent bug check dump to Microsoft for analysis (via "problems and solutions").

Meanwhile, the Microsoft's updates were installed, and I am not able to reproduce the bug check any more, so I guess problem was in Windows kernel.

I will update this topic in case another bug check I would encounter.

Reply