Is it possible to use the Xeon Phi coprocessor boards on a PCIe to Thunderbolt 3 chassis? Will it work? I have a PC laptop and would love to add a xeon phi board to the mix. I already have two SLI GTX 1080s on the laptop for CUDA development, but I also want to use the xeon phi coprocessor...
The MPSS driver on Windows expects to see a PCIe device on the PCIe bus. Additionally the "BIOS" must support PCIe devices with an aperture larger than 4GB. Your Windows Driver for your external chassis would have to fully emulate the PCIe bus with large base address and aperture. This usually is a BIOS setting not a device setting.
Thanks for the response. The thunderbold3 to PCIe interfaces are typically used to add an external graphics card to a laptop, aka. eGPU solutions. If you can drive a pretty hefty graphics card on these interfaces, I suspect it can drive the Xeon Phi, but I am not sure. Anything special that will prevent this? The interface vendor claims that the interface will work with most PCIe cards, but they don't know anything about the Xeon Phi...
Do you use the standard graphics driver especially built by the graphic card vendor (the same one you would use in a desktop with the card in the desktop?
Or, do you use a special driver supplied by a different vendor?
If the answer is yes to the first question, then you have three remaining issues:
a) Xeon Phi requires 64-bit O/S
b) Xeon Phi requires BIOS (or emulation) that supports Large Base Address (4GB) *** and Large Aperture (Xeon Phi KNC takes up to 16GB)
c) Your thunderbold3-to-PCIe, and optionally (some port on notebook)-to-thunderbolt3 would have to support the LBA and aperture requirements of the Xeon Phi KNC
Seeing that the determination of the support for Large Base and Aperture is typically performed in the BIOS at power on time, it may be unlikely that the driver for the thunderbolt 3 will synthesize this after power-on of the notebook. You could be the first to find out.
All the thunderbolt3-to-PCIe uses the native device drivers of the cards, so the answer to (1) is yes. I'll have to just try this out as you said and see if it will work. If it does, it is a huge game-changer for laptop users who want to use the Xeon Phi coprocessor boards. Stay tuned and thanks for the info...
A potential problem you may have is providing enough airflow. Check this forum for cooling solutions.
The 5100P units I have rather aggressive airflow requirements.
Checkout the fan selections and the ductwork.
Edit: it appears that some of the forum posts have been removed (as they no longer appear when searched)
I just placed an order for an AKiTio Node which is a Thunderbolt 3 to PCIe chassis. So, once it arrives, I am going to fire this puppy up and see if it works. I'll post the results here. Fingers crossed!
I haven't forgotten :-). The node will be arriving tomorrow, so once it arrives, I will be trying out the setup. It is in such high demand that vendors are having a tough time keeping the node in stock. I was one of the lucky ones. The node is going to be my setup for several board-level OpenCL development, including the Altera Stratix FPGA board, AMD FirePro, and hopefully, the Phi...
Well, my Akitio node arrived this morning and I rapidly tried it out. Even though the driver recognizes the Intel Phi, I am hit with the "Above 4GB Decoding" problem that seems to plague most enthusiasts. So, if I can somehow circumvent that problem, I am good to go...
Be careful in case your laptop is Lenovo. Some Lenovo laptops have issues after you enable the Above 4G Decoding option. However, the list is in the following link: https://support.lenovo.com/ar/en/solutions/ht072861
I had an issue with one the listed laptops in this link, not for working with Intel Xeon Phi. However, the issue caused me a big headache.
You mention your hardware in this link: https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/731325
I never worked with MSI laptops. I cannot provide any advice on how to unlock the BIOS to set this option.
I am currently working with MSI Engineers to see if they can enable the BIOS settings on my laptop so that the "Above 4GB Decoding" option becomes available via a custom BIOS. So, hopefully, it will work. Attached is my Device Manager output indicating the troublesome MMIO resources being exhausted. At this point, I am 90% of the way, but the remaining 10% is the critical part...
If I recall correctly, the Intel software stack for the first generation Xeon Phi coprocessor (Knights Corner) attempts to map the entire physical address space of the coprocessor (typically 8 GiB) in MMIO space. Most PC-type systems (and a great many servers) can put all of the required MMIO space in the "memory address hole" that typically occupies a portion of the bottom 4 GBytes of the system address space.
A common configuration is to use addresses from 0 to 3GiB for system memory, address from 3GiB to 4GiB for MMIO, and addresses from 4GiB to (InstalledRAM+1GiB) for system memory. The 8GiB physical memory for the Xeon Phi coprocessor is then mapped as MMIO using physical addresses from (InstalledRAM+1GiB) to (InstalledRAM+9GiB).
After getting a custom BIOS from MSI for my laptop, I was able to get the "Above 4GB Decoding" option enabled. The other issue I had was that I MUST turn on the external chassis PRIOR to turning on the laptop. Even though the chassis supports hotswapping, the BIOS needs to recognize the PCI cards. So, once I turned on the chassis and then turned on the laptop, everything just lined up. I am so happy :-). Now, its time to do some Phi coding on a laptop. Thanks guys for all the help! Couldn't have done this without you!!!
PS: The thunderbolt cables that comes with the external chassis is small, about a foot. Hence, it is really close to my laptop. This is because the thunderbolt spec allows passive cables at that length to transfer 40gbps. Larger passive cables only transfer at 20gbps. I will need to get an active thunderbolt cable in the future to move the chassis several feet away from my laptop, but that is for another day...