Xeon Phi and Xen 4.2.1

[Justin_Bennett]

I am trying to use the Xeon Phi card with a Xen guest VM (HVM guest) with PCI passthrough (supported by Intel's VT-d).  The following is my setup:

SuperMicro SYS-1017GR-TF (has the X9SRG-F motherboard which is known to support the Phi with the latest BIOS)
Dom0: Debian Wheezy 64-bit
HVM Guest: CentOS 6.3 with stock kernel (2.6.32-279 x86_64), Xeon Phi is passed through with the Xen pciback driver

In this configuration, the VM will not boot.  I'm assuming it has something to do with the special BIOS requirements that the Phi card has for memory addressing, though I'm not sure how that translates to a Xen VM.

Any thoughts or advice are greatly appreciated.

[Xudong_H_Intel]

According to the 10GB of RAM log, bug may be the qemu does NOT update PCI memory hole(in this case is 0x400000000-0x5ffffffff), I think you also make another two patch in xen and qemu, then rebuild them.

http://marc.info/?l=xen-devel&m=136150317011027

http://marc.info/?l=qemu-devel&m=136177475215360&w=2

And you'd better to boot guest with RAM < 3.75GB, because there still a bug in qemu and Xen, which is in discussion in Xen community. http://qemu.11.n7.nabble.com/Re-Xen-devel-BUG-1747-Guest-could-t-find-bootable-device-with-memory-more-than-3600M-td207702.html

[Justin_Bennett]

Xudong,

Somehow I never got an email that you had posted, so I just saw it today.  I am going to give your suggestions a try today and I will let you know how it goes.  

Thanks again for your help.

Justin

[Justin_Bennett]

Just for clarification, should I be making those patches to the latest version of the xen and qemu source?  This is the process I'm going to try now, could you let me know if this is correct?

1. Grab the latest commits for Xen and qemu from the repository
2. Apply the patches originally suggested by Belinda
3. Apply the patches you have just provided
4. Build and install
5. Test VM with Xeon Phi card and <3.75GB RAM

Please let me know if you had something else in mind.

[Justin_Bennett]

So the patch you provided for qemu will not apply on the latest git commit, nor will it apply on the commit that Belinda specified in her post.  Could you let me know what commit of qemu I should use to test this?  Also might be good to use the same commit of Xen if you could provide that.

I will continue to try different things, but will be waiting for your reply.

Thanks again,

Justin

[Justin_Bennett]

Xudong,

Any thoughts on my last two posts? I've run out of things to try.  If you could get me the exact revisions of qemu and xen that you've been testing with, I can try applying the patches to those.

Thanks,

Justin

[Justin_Bennett]

Xudong and/or Belinda,

Any new news on this front?  My project has had to move away from the Xeon Phi due to not being able to get this functioning, but we are really hoping to be able to use it in the future.

Thanks,

Justin

[BelindaLiviero]

Hi Justin, I will see about tapping into my own systems to reproduce your problem

[TaylorIoTKidd]

For documentation purposes, an article on getting KVM to work with the coprocessor was just published (5/2014):

Getting Kernel-Based Virtual Machine (KVM) to Work with Intel® Xeon Phi™ Coprocessors" by Loc Nguyen.

(https://software.intel.com/en-us/articles/getting-kernel-based-virtual-m...) 

Regards
--
Taylor