Windows 8.1 Hyper-V and Skylake CPUID Leaf 15 issue

[levicki]

When Hyper-V role is installed on Windows 8.1 Professional x64 the last CPUID leaf my Skylake i7-6700K reports is leaf 0xD. Any program using Leaf 0x15 to determine TSC ratio produces invalid time measurements because it cannot read the CPUID leaf 0x15. All leafs between 0xD and 0x16 are not reported for some reason with Hyper-V enabled.

Can someone from Intel please clarify:

1. Is this the expected behavior?

2. If yes, why?

3. If no, then whose bug is it? Microsoft? Intel? BIOS vendor

This is a serious issue, and should be addressed in some way -- at least tell developers how to work around it.

[Quoc-Thai_L_Intel]

I got some feedback from a colleague on this that may provide some hints for you:

( Microsoft could have virtualized the MSRs when Hyper-V role is installed. Even the root partition doesn’t see real hardware. Here it says:

https://msdn.microsoft.com/en-us/library/cc768520(v=bts.10).aspx

"Partitions do not have access to the physical processor, nor do they handle the processor interrupts. Instead, they have a virtual view of the processor and run in a virtual memory address region that is private to each guest partition. The hypervisor handles the interrupts to the processor, and redirects them to the respective partition.”

So this could be a Microsoft “feature”, not a bug. A reliable method is to install Linux on the same machine and compare it with Windows with the exact same BIOS configuration. If Linux doesn’t have the same issue then go to Microsoft for help.)

Regards, -Thai

[levicki]

Hello,

Thanks for the reply. If I am reading the article at the link you posted correctly it says:

 The virtualization stack runs in the parent partition and has direct access to the hardware devices.

I read that as "the hypervisor host has access to hardware".

I do not mind guests (child partitions) not seeing all CPU MSRs correctly if that was done to minimize chance of timing based side-channel attacks, but the host OS and applications should be able to have a full view of CPU MSRs including various multipliers related to TSC.

EDIT:

After checking with Skylake CPU and Hyper-V on Windows 10, CPUID leaves are available up to 0x15 (only 0x16 is missing) so definitely it seems to be a problem with Windows 8.1 Hyper-V implementation.

Is there anything Intel can do about this? For example, contact Microsoft Hyper-V team and ask them if they have any plans to fix this?

It is definitely causing application compatibility issues across different versions of their OS unless developers implement some sort of workaround which doesn't seem possible given that it's an artificial restriction imposed by the Hypervisor.

[Quoc-Thai_L_Intel]

Some more feedback from my colleague:

"
Yes, Hyper-V does not expose all CPUID features to Root or Guest partition. This is by design.  Root partition could observe more CPUID features than Guest Partition.

"

Per your questions, are you having problem running your workload without the TSC (timestamp counter) ratio?  If yes, can you share the type of workload that you are running. 

Regards,
-Thai

[levicki]

The workload is any benchmark/diagnostic program which uses TSC ratios to determine exact CPU frequency on Skylake. I observed this issue with AIDA64 on Windows 8.1 Hyper-V and after discussing it with developer and comparing MSR dumps with Hyper-V on and off we have determined that the problem is in Hyper-V is blocking access to said CPUID leafs in root partition.

It would be really nice if Microsoft could issue a patch so that on Windows 8.1 root partition could observe CPUID leafs at least up to and including 0x15 like in Windows 10 Hyper-V. As a matter of fact I would recommend all MSRs to be visible in root partition.

As for the design thing -- I can understand hiding MSRs from guest partitions to prevent timing based side-channel attacks, but messing with root partition is IMO poor design, because people do run applications which read MSRs in root partition as well, especially in non-Server OS editions and especially developers. Just because the CPU is virtualized doesn't mean software running in root partition shouldn't be able to accurately identify it.

Finally, if I am not mistaken, VTune Analyzer also won't work with Hyper-V enabled. The reason is probably the same (no access to performance counter MSRs).

In other words, please fix.

[Quoc-Thai_L_Intel]

Yes, you are correct that Intel(R) Vtune(TM) tools do not run in the root partition or guest partition of MS* Hyper-V* for the same reason.  I am passing your suggestion to the related teams. 

Regards, -Thai

[levicki]

Thanks Thai, I hope they will consider fixing it.

I am sure I am not the only developer who would like to be able to use Hyper-V and be able to run VTune in the root partition without having to uninstall the Hyper-V role first.

Please update the thread when you have more information.

[gaston-hillar]

Thai and Igor,

Igor was right. He was not the only developer who would like to be able to use Hyper-V and be able to run VTune in the root partition without having to uninstall the Hyper-V role first. Luckily, I found this thread with the discussion and I realized about the problem. It would be great to know whether this issue will be solved. I always search before starting a new thread. However, it would be great if there is any update on this, many months after the initial issue was reported.

[Quoc-Thai_L_Intel]

Hello,

I just checked again with my internal team but I have no new update on this issue.  Ultimately, the fix for TSC with Hyper-V enabled would have to come from Microsoft.  You can provide this feedback to the SW vendor as a product enhancement request.

-Thai

[levicki]

@Thai There is no point in complaining to Microsoft as a customer. @Gaston I stopped using Hyper-V and switched to VirtualBox. Not only it does not interfere with normal CPU operation (it doesn't affect MSR readout and it doesn't change the measured BCLK frequency from 100 to 97 MHz like Hyper-V does), but it can also handle iSCSI targets properly while Hyper-V randomly fails to connect to an iSCSI target, or outright mixes up two physical drives mapped as iSCSI targets making two VMs those drives belong to inoperable until problem is sorted out manually. There is also ability to re-route host USB devices to VM (such as USB to RS232 converters, smart card readers, cameras, dongles) and it supports more OS flavors and disk formats. As a side note I am in the process of switching away from all Microsoft products completely -- I am not going to endorse their outright anti-consumer policies. More on that in a separate thread I am about to compose soon.

[gaston-hillar]

@Thai, Thanks for the update.

@Igor, Thanks for your comment about VirtualBox. It is extremely helpful for me.