PHI cards inside single SMP instance of linux (centos6/7)

[Tymoteusz_O_Intel]

Hello

My customer is having a customized HPC system based on Xeon Phi coprocessors (5110P, 3120P). He is using intel paralel studio XE2015 for 5 paralel users.

Do you know what are the limitation for the use of many PHI cards inside single SMP instance of linux (centos6/7), is there some limit because of linux kernel or MPSS modules?
Is it possible to use like 64 PHI cards in single linux instance? Do you have some hints how to get it working? UDEV, lowmem allocations, modules params etc ...

I will highly appreciate any kind of help.

Regards,

Tymoteusz Olszewski
EMEA OSG Application Engineer.

[jimdempseyatthecove]

The typical workstation, capable of supporting Xeon Phi coprocessors, can support two coprocessors.

Some server motherboards, specifically designed for multiple Xeon Phi coprocessors, have sufficient PCIe x16 slots to support 4 or 8 coprocessors. These can be placed into a workstation. The system needs to have sufficient power supply capacity for such a configuration. If you build your own, *** you are responsible to provide sufficient cooling. This is a non-trivial process.

Also, for any single PCIe bus, I seem to recall an upper limit of 8 Xeon Phi coprocessors. As to if you can somehow use an expansion system, I do not know (beyond my experience). Otherwise, to utilize 64 Xeon Phi cards, you would require a cluster of 8, 16, or 32 nodes depending on motherboards.

I suggest you place a query on the HPC forum. Be explicit enough in your request.

Jim Dempsey

[Tymoteusz_O_Intel]

Thank you Jim for your response.