Correct LOCK CMPXCHG emulation when the destination operand is in separate pages

[Eugene_K_2]

How LOCK CMPXCHG instruction should be emulated if the destination operand crosses page boundary (and therefore can be in non-contiguous physical memory)?

I see, KVM gives up emulation in this case:

static int emulator_cmpxchg_emulated(struct x86_emulate_ctxt *ctxt,
				     unsigned long addr,
				     const void *old,
				     const void *new,
				     unsigned int bytes,
				     struct x86_exception *exception)
{
...

	if (((gpa + bytes - 1) & PAGE_MASK) != (gpa & PAGE_MASK))
		goto emul_write;

...

	return X86EMUL_CONTINUE;

emul_write:
	printk_once(KERN_WARNING "kvm: emulating exchange as write\n");

	return emulator_write_emulated(ctxt, addr, new, bytes, exception);
}

What is the approach for most correct emulation which is as close as possible to the native execution?

[Quoc-Thai_L_Intel]

Hi Eugene, I got some feedback from my colleague. 

" I'm not sure I'm understanding your question.

But KVM doesn't give up this sort of emulation.

emulator_cmpxchg_emulated()

     + if (((gpa + bytes - 1) & PAGE_MASK) != (gpa & PAGE_MASK)) goto emul_write;

                + emulator_write_emulated()

                    + emulator_read_write()

And emulator_read_write()

{

    ...

     /* Crossing a page boundary? */

     if (((addr + bytes - 1) ^ addr) & PAGE_MASK) {

         int now;

         now = -addr & ~PAGE_MASK;

         rc = emulator_read_write_onepage(addr, val, now, exception,

                          vcpu, ops);

         if (rc != X86EMUL_CONTINUE)

             return rc;

         addr += now;

         if (ctxt->mode != X86EMUL_MODE_PROT64)

             addr = (u32)addr;

         val += now;

         bytes -= now;

     }

     rc = emulator_read_write_onepage(addr, val, bytes, exception,

                      vcpu, ops);

     ...

So we just split this write on KVM side.

"

Regards, Thai

[Eugene_K_2]

Unlike CMPXCHG_TYPE and CMPXCHG64, emulator_read_write_onepage is not an atomic operation. It this correct for the instructions with LOCK? Does the real Intel CPU perform atomic exchange for such instructions when the destination is in separate pages?

[Quoc-Thai_L_Intel]

I got some feedback from my peer...

"The instruction with lock is still atomic even in the case of non-alignment.

- What we're talking is how we should emulate such a behavior under virtualization circumstance. So we just make sure mmu page is not wrote either by guest or by host at this moment.
- KVM emulates this kind of locked operations while we already hold mmu lock, and these pages should be write-protected. This means just one writer exists so actually it is safe here.
- But in some special cases, they may break this rule and then expose the page as write by guest. So CMPXCHG_TYPE and CMPXCHG64 are introduced to handle these things. Note: I don't go into details but this is true."

-Thai