Xeon Phi on Vanilla Kernel (3.13 and above)

[Mathias_R_]

Hi

I am currently programming for my master thesis in computer science.

My next step is to make the MIC runable on the vanilla kernel with MIC Modules in version 3.13 and above.

The Kernel documentations also delivers a micctrl and mpss/d implementation. And since the MPSS provided on the intel page is stricted to RHEL/SUSE with older Kernel versions I also need to make a Kernel for the MIC.

Is there any documentation how to compile the MIC Kernel? The mic_host module is included in the vanilla Linux Kernel so the host and card should be able to communicate. Are there any patches that need to be done so that the MIC will work on 3.13 and above?
I found the config for the 2.6.xx Kernel of the MIC but I am still a bit unsure if I need special customization so that the vanilla Kernel will run on the MIC

I want to use the vanilla kernel and not a ported version of the MPSS Modules and Deamon

[Mathias_R_]

Here's my log output from /var/log/mpssd

Sat Nov 15 13:37:08 2014: MIC Daemon start
Sat Nov 15 13:37:08 2014: MIC name mic0 id 0
Sat Nov 15 13:37:08 2014: MIC found 1 devices
Sat Nov 15 13:37:08 2014: mic0: mic_config 1432 state online
Sat Nov 15 13:37:08 2014: mic0: log_buf_addr: ffffffff8179f430
Sat Nov 15 13:37:08 2014: mic0: log_buf_len: ffffffff81605240
Sat Nov 15 13:37:08 2014: mic0: Command line: "clocksource=tsc highres=off nohz=off cpufreq_on;corec6_off;pc3_off;pc6_off ifcfg=static;address,172.31.0.1;netmask,255.255.255.0"
Sat Nov 15 13:37:08 2014: mic0: IPADDR: "172.31.0.1"
Sat Nov 15 13:37:08 2014: Added VIRTIO_ID_CONSOLE for mic0
Sat Nov 15 13:37:08 2014: Added VIRTIO_ID_NET for mic0
Sat Nov 15 13:37:08 2014: Created TAP mic0
Sat Nov 15 13:37:08 2014: mic0 console message goes to /dev/pts/2
Sat Nov 15 13:37:08 2014: Configuring mic0
Sat Nov 15 13:37:08 2014: init_vr mic0 vr0 0x7f324c4c9000 vr0->info 0x7f324c4ca406 vr_size 0x2000 vring 0x1406 Sat Nov 15 13:37:08 2014: magic 0xc0ffee03 expected 0xc0ffee03
Sat Nov 15 13:37:08 2014: init_vr mic0 vr1 0x7f324c4cb000 vr1->info 0x7f324c4cc406 vr_size 0x2000 vring 0x1406 Sat Nov 15 13:37:08 2014: magic 0xc0ffee04 expected 0xc0ffee04
Sat Nov 15 13:37:08 2014: mic0 get_device_desc d-> type 3 d 0x7f324c4c8010
Sat Nov 15 13:37:09 2014: mic0 execvp failed errno No such file or directory
Sat Nov 15 13:37:09 2014: Configuring mic0 ipaddr 172.31.0.254/24
Sat Nov 15 13:37:09 2014: mic0 execvp failed errno No such file or directory
Sat Nov 15 13:37:09 2014: MIC name mic0 tap_configure 226 DONE!
Sat Nov 15 13:37:09 2014: MIC name mic0 id 0
Sat Nov 15 13:37:09 2014: init_vr mic0 vr0 0x7f324c4c4000 vr0->info 0x7f324c4c5406 vr_size 0x2000 vring 0x1406 Sat Nov 15 13:37:09 2014: magic 0xc0ffee01 expected 0xc0ffee01
Sat Nov 15 13:37:09 2014: init_vr mic0 vr1 0x7f324c4c6000 vr1->info 0x7f324c4c7406 vr_size 0x2000 vring 0x1406 Sat Nov 15 13:37:09 2014: magic 0xc0ffee02 expected 0xc0ffee02
Sat Nov 15 13:37:09 2014: mic0 get_device_desc d-> type 3 d 0x7f324c4c3010
Sat Nov 15 13:37:09 2014: mic0 get_device_desc d-> type 1 d 0x7f324c4c3070
Sat Nov 15 13:38:30 2014: mic0 get_device_desc d-> type 3 d 0x7f324c4c8010
Sat Nov 15 13:38:30 2014: mic0 wait_for_card_driver Waiting .... desc-> type 3 status 0x0

[Ramakrishn_K_Intel]

 Hello Mathias,

This may not be exactly what you are looking for, installing a vanilla 3.13 kernel on Xeon PHI,

but it  incorporates a tickless patch from 3.13 on Xeon PHI and talks about installing 3.13 on the Xeon PHI.

This document may help you with some of the  installation issues.

https://software.intel.com/sites/default/files/managed/fd/e2/Tickless%20Kernel%20Support%20on%20Intel(R)%20Xeon%20Phi(TM)_0.pdf

Regards,

Rama K

[Frances_R_Intel]

Mathias,

Is the goal to run the same 3.13 or later kernel on both the host and the coprocessor? Moving the host to a later kernel is generally a simpler problem. You mostly need to worry about the mic.ko file (and other kernel modules, if you are using OFED or Lustre or the like). Moving the coprocessor to a later kernel is generally more work.

As far as porting a new kernel to the coprocessor is concerned, picking apart the rpm file from Rama's paper might not be a bad place for you to start. And as to your error messages, it looks like you might be missing a file system. In the current MPSS, when the coprocessor first boots, it creates an initial RAM file system and copies the contents of the initramfs cpio file to it. Then it either creates a second RAM file system (populated from a more complete cpio file) or NFS mounts a file system to be the real root file system, switches roots and removes the initial RAM file system. I could (easily) be wrong, but it looks like you are not creating your initial RAM file system.

Frances

[Mathias_R_]

Hi Ramakrishna
Hi Frances

I already tested the tickless package and I that the 3.13 Kernel used in this tickless package might be a patched kernel. I would like to use a Kernel which already as the MIC Implementation natively. Like the 3.13 and above Kernels have.

I cannot find any documentation how to build my own uos.img and mic0.image to customize the MIC OS to my needs.

Is there any kind of documentation for this? The only one I found was with a given uos.img source and not much space for customizations.

I am using the mic_host.ko as my MIC module on the host. I think that this might be the problem. I guess that the uos and rtamdisk images provided by Intel use the standard mic.ko module and therefore the Host cannot communicate with the card.

I would like to use the native Kernel implementation on Kernel 3.13 for host AND card and above but the more I search for it the more I am convinced that it is not possible without help and documentation of Intel.

[Evan_P_Intel]

Mathias R. wrote:

I cannot find any documentation how to build my own uos.img and mic0.image to customize the MIC OS to my needs.

If you mean customizing the userspace (that is, MPSS's Linux distribution for the card), this thread should be helpful. The root filesystem is built using infrastructure from the Yocto Project (v1.2); in combination with the Yocto Project's own documentation and the source itself, the above thread gives a "sufficiently motivated" person everything necessary to make arbitrary changes starting from a place that builds and boots. I can answer questions that come up.

Mathias R. wrote:

I would like to use the native Kernel implementation on Kernel 3.13 for host AND card and above but the more I search for it the more I am convinced that it is not possible without help and documentation of Intel.

Try asking (targeted) questions on LKML (and CC: Sudeep; you can lift his email address out of git), if you haven't yet.