Software Archive
Read-only legacy content

5100 series compatible Motherboard.

ivan_V_3
Beginner
2,051 Views

I have a 5110P xeon phi coprocessor. I was wondering if the P9X79 WS motherboard will work with this. I emailed the company, and they seem to think it will, but it being a 300$ purchase, I would like a second opinion. 

I know P means passively cooled, but I am not worried about that part since I have plenty of fans, and it doesn't have to look pretty.

 

0 Kudos
1 Solution
jimdempseyatthecove
Honored Contributor III
2,002 Views

For KNC P means heat sink supplied, you supply the fan. I use dual 5100P's and built my own fan ductwork and selected appropriate fans.

See:

https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/537661
and
https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/498452

The fans were rated at 35 cfm *** at the required pressure differential of pushing that quantity of air through the canister

*** I've built different fan ducts since then, and used heftier fans than those listed.

Note, you will want to use a separate fan controller *** that has sufficient Watts capacity on each channel for the fans selected. Most have insufficient capacity, some do. Look at the specs, do not rely on "heavy duty".

From: https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/541647 post #13

On the ASUS P9X79-WS motherboard I had an issue with the motherboard BIOS firmware version. The version it shipped with had problems making it through POST. The latest version (at that time was 4404) locked up without recognizing the CPU when the Xeon Phi installed. It took a long time of searching around on the internet to find out BIOS version 4306 was the one that worked. A lot of time has passed since I updated the Firmware, as to if the 4306 is the one to use or not, I cannot say.

Jim Dempsey

View solution in original post

0 Kudos
30 Replies
jimdempseyatthecove
Honored Contributor III
2,003 Views

For KNC P means heat sink supplied, you supply the fan. I use dual 5100P's and built my own fan ductwork and selected appropriate fans.

See:

https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/537661
and
https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/498452

The fans were rated at 35 cfm *** at the required pressure differential of pushing that quantity of air through the canister

*** I've built different fan ducts since then, and used heftier fans than those listed.

Note, you will want to use a separate fan controller *** that has sufficient Watts capacity on each channel for the fans selected. Most have insufficient capacity, some do. Look at the specs, do not rely on "heavy duty".

From: https://software.intel.com/en-us/forums/intel-many-integrated-core/topic/541647 post #13

On the ASUS P9X79-WS motherboard I had an issue with the motherboard BIOS firmware version. The version it shipped with had problems making it through POST. The latest version (at that time was 4404) locked up without recognizing the CPU when the Xeon Phi installed. It took a long time of searching around on the internet to find out BIOS version 4306 was the one that worked. A lot of time has passed since I updated the Firmware, as to if the 4306 is the one to use or not, I cannot say.

Jim Dempsey

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

***

Although I am happy with this motherboard, I might suggest that you look for a board that has a V3 or later socket. I am unable to upgrade by E5-2620 v2 to v3 (now later versions).

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

thanks! I will try out your strategy.What is the point of a v3 socket? Honestly, as long as I can get some experience on the xeon phi, I am happy. Plus, I had enough trouble finding this affordable xeon phi compatible motherboard, I doubt Ill find another one with the v3 restriction.

If you do have some time, I have a couple extra questions. I have a haswell i5 64bit CPU. Is that compatible with that motherboard? Any particular ram I need to get?

IV

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

What is your processor number.

The P9X79 WS has an LGA 2011 socket.

LGA 2011 (subsequently called LGA 2011-0) is suitable for Sandy Bridge-E/EP and Ivy Bridge-E/EP
LGA 2011-1 is suitable for Ivy Bridge-EX and Haswell-EX
LGA 2011-v3 is suitable for Haswell-E and Haswell -EP

The X79 chipset handles Sandy Bridge-E and Ivy Bridge-E
The X99 chipset handles Haswell-E and Broadwell-E

It would appear that you would have issues using the Haswell in this board. Though you might check with the vendor.

I have a KNL system on order, but for the next Xeon system I'd like to consider dual Broadwell.

Jim Dempsey

0 Kudos
Bob_R_
Beginner
1,519 Views

I am running a i7-3820 on a p9x79-WS with a phi 31s1p without problems.  The memory is 32 GB (8x4GB) of G.Skill 1600 speed (F3-12800CL9Q-16GBXM).  I would suggest that you first check to see if the motherboard of your haswell i5 supports 64-bit base address register  settings.  If it does, there is a chance that you can enable the 64-bit BAR and use that board.

 

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

That is a Sandy Bridge LGA 2011-0 (LGA 2011 without added features of -1 or -v3)

The BIOS requires to support not only 64-bit Base Address, but also a device aperture of larger than 4GB (at least size of RAM on Xeon Phi).

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

Jim,
What operating system were you using for this? I tried ubuntu with instructions:
http://arrayfire.com/getting-started-with-the-intel-xeon-phi-on-ubuntu-14-04linux-kernel-3-13-0/

but it seems that I have to change the kernel in order to compile these. But that drops me into initramfs with no keyboard

IV

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

Initially CentOS 6.7, I have recently upgraded to CentOS 7.2 (server). Still performing updates and shakedown of system.

I haven't rigorously tested the system, but the mic utilities sees both MICs, I assume that they are working properly.

I haven't had Ubuntu on the system since ~2010 or so.

I upgraded to CentOS 7.2 due to having a KNL system on order which will have CentOS 7.2.

The project I am working on is a distributed MPI+OpenMP application. If I have time to experiment and do a writeup, I will experiment with distribution across Xeon, KNC and KNL.

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

So then which mpss release did you use?

also did you come across an ata comreset failure issue at linux bootup?

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

mpss 3.7.1

I haven't observed an ata com reset failure.

I had a BIOS version issue (see post #2), the symptoms of which the motherboard wouldn't make it trough POST. The diagnostic LED would usually indicate a CPU failure, but in fact it was the Large Base Address and aperture issue. It took me a while to figure out I had to retrograde the BIOS version. Some time has passed since then, I cannot say if the newest version of the BIOS is ok or not. The system is working - so I am reluctant to experiment with a BIOS update. In the process of the permutations of inserting, removing, relocating cards, different errors would show up on the diagnostic LEDs. Only by retrograding the BIOS did the problems go away.

Jim Dempsey

 

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

BTW I have 3 SATA disks:

Windows 7 x64 Pro
CentOS 6.7
CentOS 7.2

All have access the MICs (*** however I could only get one MIC to work on Windows 7 ***)

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

thanks!

how did you get your bios to 4306?
EZ flash utility wont flash to that because it is "outdated". Pressing the bios button with the flash drive in the white usb spot makes the light go blue after 3 seconds but then nothing happens. When I turn the computer on, It still does not have that bios version.

 

Ivan

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

I used a USB thumb drive in the rear (white) USB slot. Make sure the flash file is in the root directory.

My motherboard is P9X79 WS

Check the format of the USB stick. It may have to be FAT32 or FAT16/FAT16B. The ASUS website list NTFS as well for the X99, your/my board is X79, so I am unsure about NTFS. Many of the newer systems format USBs to ExFAT which may not work. The LED should flash during the flash process then stop flashing when complete. The flashing is to be performed with the system powered off but plugged in (iow running on standby power).

Make sure the file name is correct and it is residing in the root directory

P9X79WS.CAP  (or was it .ROM format ??)

If on Windows, use a CMD window. If you use Windows Explorer, you might end up with P9X79WS.CAP.CAP (or some other name).

The user manual does not say anything about the file name requirement.

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

Thanks! that was it. The instructions just say to remove dashes, not to rename it P9X79WS.CAP. Ive gotten through the mpss instructions and are now at service mpss start which fails just stating code=exited status=1. micctrl --config sees the card and tells me an IP for it, but when I ping it, it states "destination host unreachable". 

 

On a separate note, I purchased this card through ebay and the buyer told me it was a 5110P. But lspci is telling me its a 7120P. Is lspci wrong? is the dealer wrong? will my computer be compatible with the 7120P anyways?

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

On my system, the installation program did (does) not setup the system to automatically install the mic.ko module nor start the mpss service.

To compensate for this I created a bash shell script:   gomic.sh

#!/bin/sh
sudo insmod -f /lib/modules/3.10.0-327.el7.x86_64/extra/mic.ko
sudo micctrl --initdefaults
sudo service mpss start
sudo micsmc &

After boot, I log in to my user account with administrative privledges and issue:

  ./gomic.sh

or

  bash ./gomic.sh

*** Note, the path to mic.ko will vary depending on your installed version of MPSS.

As a test, of success, does micsmc run and show card?

Jim Dempsey

 

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

BTW

The 7120P is a better board

1.24 GHz vs 1.05 GHz
61 cores vs 60 cores
16GB RAM vs 8GB RAM

However

300 Watts vs 225 Watts

They are completely program compatible.

Jim Dempsey

0 Kudos
ivan_V_3
Beginner
1,519 Views

Jim,

Your script first said that it could not insert the mic.ko module because "file exists". It then fails to start mpss control service. Looking at the error logs, it may have something to do with "Error adding address mic0 for br0". Or "loading MIC module : [OK], starting INTEL MPSS [FAILED]".

Micsmc does start but it immediately gives me "error log alert" and "lost card alert"

where the error log tells me "device connection lost"

clicking on lost card alert gives me the option to restart all cards but it is lost as soon as it starts.

 

IV

0 Kudos
ivan_V_3
Beginner
1,519 Views

micctrl -s and micctrl -rw also seem to fail "mic0 reset failed"

when I try micflash -getversion it states : micflash:mic0 :failed to switch to maintenance mode: write :/sys/class/mic/mic0/state: input/output error

looking at the dmesg, mic gets to the ready state, but then fails at booting "fail booting MIC 0/ wait time exceed 180 seconds"

 

under lspci -vvv, it shows my xeon phi (apparently 7120) coprocessor, but then states "!!! unknown header type 7f" and "kernel driver in use: mic"

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,519 Views

>>Your script first said that it could not insert the mic.ko module because "file exists".

That means the file (module) has already been loaded.

Try removing it, then reinserting it by hand to see if any thing displays during insertion.

sudo rmmod mic.ko
sudo insmod mic.ko

You might need to do this from new boot. (I do not know the state of anything else loaded).

The rear of the Xeon Phi (at rear of case) has b blue LED. When the unit is running, the LED blinks fast (5 to 10 times per second).

Did you supply a cooling fan? (a really good one)

Jim Dempsey

0 Kudos
jimdempseyatthecove
Honored Contributor III
1,316 Views

The fans will typically require you to build some duct work to force the air thru the unit. Simply blowing at the unit is not sufficient.

Make sure your card is located in either Slot1 or Slot4 (blue slots). My two 5100Ps are in both. The video card is elsewhere.

Jim Dempsey

0 Kudos
Reply