Migrating from KNC to KNL

[Nick_W_]

We have an application running on Centos 6 and utilising a Phi native application (with communication between the two via the SCIF library)

Now that KNCs have been end-of-lifed, I'm looking at migrating it to the KNL platform. Just a little confused on a couple of details.

Does KNL have its own specialised O/S - or can you install Centos 6 on it as with any typical Xeon based server? Our application is somewhat tied to this O/S - porting it would be non trivial.

[TimP]

Apparently, CentOS 7.2 is preferred for KNL; if you have tied your application to CentOS 6, now would be the time to make it more portable, as it seems that using CentOS 6 with KNL mods might be equally non-trivial.

[Loc_N_Intel]

If you want to enable functionalities of the Intel Xeon Phi processor, you need the Intel(R) Xeon Phi(TM) processor software available at https://software.intel.com/en-us/articles/xeon-phi-software . The CentOS version that this package supports is CentOS 7.2 .

[JJK]

In theory, you could install EL6 on the box, but you would not get any support for the advanced features of the KNL processor (most notably, the kernel would not know about AVX512, nor would any of the compilers or libraries).

I've just tried booting an CentOS6 live USB stick on my KNL box and it kernel panicked on me - this was to be expected, as EL/CentSO6 won't boot from USB over USB3. Thus, actually *installing EL6 in reality* will be quite tricky - it'd require either a CD/DVD drive or a PXE install. My general advice would be: don't bother, as you'd end up with a crippled EL6 system anyways.

[Nick_W_]

Thanks guys, I expect a later version of Centos would not cause too many problems.

I couldn't find information like OS support when I first started looking at the upgrade. Took me a while to realise the Phi was now a standalone machine not a coprocessor.

Having said that, it might be useful to still have the host-coprocessor configuration. Is that possible? Do they still communicate by PCIe? How much rack space is needed?

[JJK]

nope, an Intel rep once told me that you cannot plug a KNC card into the PCIe slot of a KNL box . Needless to say, that doesn't mean I'm not going to try that ;)

The KNL box I have is a desktop machine (Colfax Ninja) but they also have rack-mountable stuff - 4 times as expensive, IIRC.

[jimdempseyatthecove]

>> also have rack-mountable stuff - 4 times as expensive,

It is a cluster of 4 KNL's

Jim Dempsey

[Nick_W_]

4 in one box sounds good.

Do they communicate using this:

http://www.intel.co.uk/content/www/uk/en/high-performance-computing-fabrics/omni-path-host-fabric-interface.html

So 100G throughput between them? That's better than PCIe if I remember correctly.

[JJK]

you can find specs for the currently available systems here:

  http://dap.xeonphi.com/#platformspecs

when I ordered my desktop box they only offered a desktop version or a 4-KNL 4U server; now they also offer a 1-KNL 1U server (which I'd have preferred to the desktop box).   The KNLs offered are NOT the Omnipath versions just yet - I've not seen any of those in production.

Also, 100 Gbps omnipath sounds very nice, but you'd get similar performance with a modern PCIe 3.0/4.0 16 lane card; I have a GPU box with dual Tesla K80 and they can communicate with each other at speeds > 15 GB/s (which is > 100 Gbps).