Odd issue with vmx

[Cody_D_]

So to explain the situation, what's happening is vmx works fine on the first four logical cores of the CPU without issue. Launch succeeds, they receive exits, vmresume is executed, all's good. However on cores 5,6,7, and 8 ( indices 4 to 7 ) vmlaunch succeeds but entry fails with "invalid first state" and immediately exits with that exit reason INVALID_GUEST_STATE (33) thus causing vmresume to fail. I ran all of the entry checks on said core and none of them come back as incorrect. Does anybody know what could be causing this issue? Thanks in advance.

[Quoc-Thai_L_Intel]

Hi,

What is the system configuration? And can you provide the specific processor that you are using? I found some information that is related to the VMRESUME that might provide you with some hints.

http://www.liteirc.net/mirrors/siyobik.info/instruction/VMLAUNCH%252FVMRESUME.html\

-Thai

[Cody_D_]

4 physical cores hyperthreaded. So there's 8 logical cores. Intel i7 and no that link doesn't provide any information I didn't already know. Like I stated in my original post there's an invalid guest state that causes VM guest ENTRY to fail. This happens before vmresume is ever executed and because the guest is in an invalid state vmresume obviously fails. Sorry if I wasn't clean :) Thanks for your response though :)

[Tiejun_C_Intel]

Its clear that actually vmlaunch doesn't be performed successfully 
because of "INVALID_GUEST_STATE (33)"

— Exit reason.
• Bits 15:0 of this field contain the basic exit reason. It is loaded 
with a number indicating the general cause of the VM-entry failure. The
following numbers are used:
33. VM-entry failure due to invalid guest state. A VM entry failed one 
of the checks identified in Section 23.3.1. [SDM]

So you have to check all items in Section 23.3.1, but Exit qualification 
can give you some references if Exit qualification != 0,

— Exit qualification. This field is set based on the exit reason.
• VM-entry failure due to invalid guest state. In most cases, the exit 
qualification is cleared to 0. The following non-zero values are used in 
the cases indicated:
1. Not used.
2. Failure was due to a problem loading the PDPTEs (see Section 23.3.1.6).
3. Failure was due to an attempt to inject a non-maskable interrupt
(NMI) into a guest that is blocking events through the STI blocking bit
in the interruptibility-state field. Such failures are implementation 
specific (see Section 23.3.1.5).
4. Failure was due to an invalid VMCS link pointer (see Section
23.3.1.5).
VM-entry checks on guest-state fields may be performed in any order.
Thus, an indication by exit qualification of one cause does not imply 
that there are not also other errors. Different processors may give 
different exit qualifications for the same VMCS.
• VM-entry failure due to MSR loading. The exit qualification is loaded 
to indicate which entry in the VM-entry MSR-load area caused the problem
(1 for the first entry, 2 for the second, etc.).

I guess you can check what value "Exit qualification" is, firstly.

Thanks Tiejun

[Cody_D_]

First off I already fixed everything. Second off the entire time vmlaunch did suceed but the guest didn't launch due to an invalid state. Third off as I said before all the checks passed It was an invalid encoding for a segment (the length was encoded wrong) That's why it was invalid but the checks passed Finally this is the least helpful forum I've ever been on. In the span of 7 days there was two replies that did nothing but regurgitate things from the manual which I was already aware of. Goodbye.

[Tiejun_C_Intel]

Sorry, I don't know this was fixed since I'm asked to take a look at this *just* today.  Next time you can post something to close your question if you already figure out the root cause.

Just talk about this problem, as you said "the checks passed". But this was clear just to you. We can't see how to check them by yourself so just in this normal case, we need more information to step next. Just that is it.

[Brett_S_]

I think I might be seeing something similar to the OP. The vmlaunch command succeeds (in the sense that ZF and CF are both 0), but exits after a few hundred instructions (executed in the guest) with "exit reason" set to 80000021 (i.e., "exit reason" 33 - "invalid guest-state"). Section 26.3 in the SDM states...

If all checks on the VMX controls and the host-state area pass (see Section 26.2), the following operations take place concurrently: (1) the guest-state area of the VMCS is checked to ensure that, after the VM entry completes, the state of the logical processor is consistent with IA-32 and Intel 64 architectures; (2) processor state is loaded from the guest-state area or as specified by the VM-entry control fields; and (3) address-range monitoring is cleared.

I think this explains why the ZF and CF flags are clear when vmlaunch returns: guest-state check failure is treated like a VM Exit rather than like failed vmlaunch. The strange thing is, I have verified that something on the order of a few hundred instructions are executed in the guest before the VM Exit occurs with INVALID_GUEST_STATE (33)! If the guest-state is invalid, why is the guest executing at all?! Is this by design?

Also, is there any way to narrow down the source of the guest-state invalidity? Section 23.3.1 contains a lot of conditions. Checking all of them one at a time would be very tedious and time-consuming. Surely there's a less brute-force way to determine the cause...

Thanks, Brett Stahlman

[Cody_D_]

Brett S. wrote:

I think I might be seeing something similar to the OP. The vmlaunch command succeeds (in the sense that ZF and CF are both 0), but exits after a few hundred instructions (executed in the guest) with "exit reason" set to 80000021 (i.e., "exit reason" 33 - "invalid guest-state"). Section 26.3 in the SDM states...

If all checks on the VMX controls and the host-state area pass (see Section 26.2), the following operations take place concurrently: (1) the guest-state area of the VMCS is checked to ensure that, after the VM entry completes, the state of the logical processor is consistent with IA-32 and Intel 64 architectures; (2) processor state is loaded from the guest-state area or as specified by the VM-entry control fields; and (3) address-range monitoring is cleared.

I think this explains why the ZF and CF flags are clear when vmlaunch returns: guest-state check failure is treated like a VM Exit rather than like failed vmlaunch. The strange thing is, I have verified that something on the order of a few hundred instructions are executed in the guest before the VM Exit occurs with INVALID_GUEST_STATE (33)! If the guest-state is invalid, why is the guest executing at all?! Is this by design?

Also, is there any way to narrow down the source of the guest-state invalidity? Section 23.3.1 contains a lot of conditions. Checking all of them one at a time would be very tedious and time-consuming. Surely there's a less brute-force way to determine the cause...

Thanks, Brett Stahlman

Sorry, I just noticed someone had replied to my original post.

If you are still having troubles I would be glad to help!