Xeon PHI 31S1P

[Perri__Pierpaolo]

Hi all guys,can anyone suggest me an active cooler for a Xeon 31S1P1?Is it worth to buy the heat sink of the 3120A,and is it possibile to buy it from Intel?Thank you so much...

[jimdempseyatthecove]

Thanks for the fan link. That will be a last choice.

The fans I chose, StarTech FAN7X25TX3, though quiet (<28dBA), have sufficient cooling capacity until I get under real heavy loads.

Before replacing the fans with your 54dBA selection (note this isn't twice as loud, its ~10 times as loud), what I will experiment with is making a two-stage fan system, seeing that I also obtained the Evercool Fox-3 fans (SB-F3). This fan delivers 42CFM with <26dBA.

I could reconfigure to try this fan alone, but I suspect the pressure differential may be an issue as you experienced. The Fox3 has a higher flow capacity than the StarTech and therefore should supply a higher than ambient pressure on the inlet side to the StarTech - thus reducing the pressure differential the StarTech observes shoving air into the Xeon Phi, and therefore will increase the total airflow through the Xeon Phi. At least that is the idea.

Fortunately I have sufficient room in the case to make this configuration work. I will post my success or failure back here along with pictures.

The 2x<28dBA + 2x<26dBA should be approximately 1/5th the amount of noise of your 2x54dBA fans.

I see your heat sensor is attached to the Xeon Phi towards the back end of the case on the up side of the card. Did you find this was a better point for sensing than the area to the right (where sensor wire is also taped)?

Jim Dempsey

[Perri__Pierpaolo]

Two stage fan system,interesting idea,when you will carry out it,please post...and yes when my fans are at 6800 rpm are very,very loud but with 1 in.h2o of static pressure (and 100 cfm) I'm 100% sure that I can push the cards in heavy calculations ,typically operations on large matrices of double.Obviously during the development phase and with small instances of problems the temperatures are low, the fans are at 3400 rpm the noise is about 2 x 30db acceptable and however each fan offers a static pressure of 0.5 in H2O with 50 cfm values,enough for a normal level of workload.Again,I think that the best thing to do in the cooling of these cards is to respect above all the value of static pressure required as showed in the datasheet,for the 31S1P is 0.54 in.H2O (also if one has a well cooled case).

I attached the heat sensor in that point because it's the most distant spot from the fan and also the hottest spot on the card...I tested it with my finger :)

[Robert_F_2]

Has anyone had succuss mounting generic water cooling blocks?

[Perri__Pierpaolo]

Robert F. wrote:

Has anyone had succuss mounting generic water cooling blocks?

No because,unfortunately, doesn't exist a certified waterblock for Xeon Phi...

[jimdempseyatthecove]

The two stage fan setup did not work. I created a common plenum for the two Xeon Phi's. Due to physical constraints imposed by the case, I had one fan feeding from the bottom, below the bottom Xeon Phi, and the second fan feeding directly into the bottom of the top fan set back by the depth of the plenum. The depth of the plenum may have been the issue. I had about 1" depth to the plenum due to the hard drive bays. This seemed to cause the air to flow crosswise past the lower Xeon Phi fan inlet. The top Xeon Phi may have had asymmetric pressure and/or turbulence. Whatever the cause, the bottom Xeon Phi rest temperature went up by 15C or possibly a tad more. The top one was up slightly too. I did not run the full power test due to the at-rest temperatures going up. This it too bad, since the noise hardly went up at all.

I am not using RAID and only have two HD in the drive bay that is in the way. I will consider removing the drive bay and replacing it with different one that will permit a different plenum design (larger cavity and to permit the input fans to be mounted further away with strait through airflow). I may also look higher performing squirrel cage feeder fan(s).

I will post back when I have a better solution.

Jim Dempsey 

[Perri__Pierpaolo]

I had the same problems with my drive bay,so I had to remove it and mount my 3 disks on the side of my tower as you can see in this photo. But in my case the solution was simple,because they are SSD...Mr Dempsey,maybe it's the time to replace your 3,5 mechanical hard drives :)

[Perri__Pierpaolo]

Well,some tests in the morning:simulating the worst case scenario! Power of matrices
using Matlab and the latest version of MKL!In particular with these four lines of code it's possible to calculate A^100,where A is a 10000x10000 matrix of double.Note, in the posted photos,that the each Phi card,after 3-4 minutes of calculus, has a power consumption of about 200 W...the temperatures were low ,the cards worked safely,but I confess that the noise was terrible!...I had to leave the room,it was making me crazy!

[Perri__Pierpaolo]

Sorry the first snapshot is not clear...

[jimdempseyatthecove]

Pierpaolo,

Thanks for the posts. I tried SSD about 2 years ago (Corsair F60). It ran nice, size was too small but today you can get much larger. My issue with the SSD, at least on the Windows box, Windows was constantly updating a sector on the drive, and this eventually cause a disk failure (bad sector in critical sector). I know that drivers have been updated since then, as well as firmware inside the SSD's have been fixed (or secondary static memory added as a cache to buffer those sectors that get overwritten too frequently). So maybe I should re-look at this.

A secondary concern is the time it takes to rebuild (reinstall) everything to get the system back working as it was. To install both Windows 7 with Visual Studio, Office, updates, and install Linux (Cent O/S), updates. Then install Intel Parallel Studio on both O/S, this is likely 2 days of time. Then if something broke, the time to fix it. In particular, I do not want to install the newest version of everything, because too often this leads to unstable or un-working system. My systems tend to stay running until power failure. I can replace the 6-bay rack with a 3-bay rack (2 will be sufficient), I will be able to construct a two-stage strait through system. The top Xeon Phi can be fed through the front load drive bays with th bottom three full (1/2 height) bays empty. And the bottom Xeon Phi with the shorter internal drive bay. This will give me an additional 6" of depth for the extra in-line fan and ductwork.

In looking at your temperature output you could reduce your fan speed to the point where the temperature under full load nears 80C. I think at 81C or 82C the temperature enters the yellow zone. And at about 100C it enters red zone.

Jim Dempsey

[Perri__Pierpaolo]

I agree,reinstalling  OS and programs it's a very tedious activity,but now SSDs are mature enough. My PC server was turned on 3 years ago with an OCZ VERTEX 3 and I never turned it off.For example,in the snapshot posted ,Hard Disk Sentinel after a usage period of 1219 days and 34 TB of total writes estimates a duration greater than 605 days.

However the two-stage fan remains an intriguing idea and I'm very curious about how you will realize it...

[jimdempseyatthecove]

I just ordered 3 bgears b-Blaster 90 90mm Case Fans (one for spare). http://www.newegg.com/Product/Product.aspx?Item=N82E16835132021&Tpk=N82E16835132021

2 Ball bearing, 3500RPM, 78CFM, 39dBA, 3 pin, 12V, 0.5A/6W, 5.50mmH2O

Data sheet for 5110P states 24.3cuft at 0.21"H2O. or 5.334mmH2O.

The specifications for the fan appear in excess of what is required. Will have to see how well they work.

What I plan on doing is modifying the top Xeon Phi cooling system by removing the 70mm fan, leaving the existing enclosure as-is.

Then construct a reducing duct work (to a squarish looking funnel) such that the small end, snuggly slips into the space formerly occupied by the 70mm fan. The large end will hold the 90mm fan. The surprise is, everything should fit just right for the case. At least for the top Xeon Phi. If the test works out well on that Xeon Phi, then I will rework the case to replace the internal drive bay with a shorter unit.

Should have the fans towards the end of the week.

Hopefully I can configure the fan controller for satisfactory noise control.

Jim Dempsey

[Perri__Pierpaolo]

The static air pressure offered by that fan is on the same level of that required in the datasheet,so my only perplexity is: the original duct + the new reducing duct can negative influence that parameter?In other words,which will be the effective value of the static air pressure at the card inlet?Mr. Dempsey which is the ambient temperature during the summer season in your room?it could be also a future problem...when I had the precedeng Delta fans with 0.59 in.H2O working at their maximum,and in my house was active the central heating,the temperatures of the cards colud rise to around 78-80° during heavy warokloads,also for this reason I decided to choose fans with 1.01 in.H2O...I live in the south of Italy and the summer can be very hot! :)

[jimdempseyatthecove]

I have two options which I will explore.

a) reduce the ~90x90 to 80x80 and insert into the existing reducer from 80x80 to ~80x30. Or
b) remove the existing reducer and make one from 90x90 to ~80x30

Tthen determine how to affix that to the MIC after installing the MIC. The first route (using existing reducer) eliminates the mounting issue since the additional reducer inserts as a sleeve. This should be relatively easy for the top MIC due to the floor of the upper exterior drive bay aligns with the bracket where the current fan/reducer is located. The lower MIC new reducer and fan I anticipate will sit atop the replacement internal drive bay. I will send pictures when complete.

Yesterday outside temperatures was -10F (-23C) office ambient is quite cool. In summer I have air-conditioning, ambient office temperature likely <25C at worst point in the summer.

Jim Dempsey

[jimdempseyatthecove]

Did you name your system Roma... as it wasn't built in a day...;)

[Perri__Pierpaolo]

jimdempseyatthecove wrote:

Did you name your system Roma... as it wasn't built in a day...;)

Pc-Roma is a good choice..and maybe one day,during some heavy calculation,when the fans will be at 6300 rpm,I'll become mad like Nero and I'll burn it! :)

-23C???,"Great Scott!" as Doc Brown said,Mr.Dempsey,during winter the problem is solved,put your pc outside the window ,the weather will do the job! :)

[jimdempseyatthecove]

You won't have to burn your PC, it will take care of itself. Spontaneous combustion.

Yesterday, it got a balmy 0C. The large variation is due to the movement of the jet stream bringing in cold air from Siberia via Canada.

Jim Dempsey

[Perri__Pierpaolo]

jimdempseyatthecove wrote:

You won't have to burn your PC, it will take care of itself. Spontaneous combustion.

Yesterday, it got a balmy 0C. The large variation is due to the movement of the jet stream bringing in cold air from Siberia via Canada.

Jim Dempsey

Mr Dempsey the lowest temperature in my town is +10° (and only during winter), 0° is a "catastrophic" event,-23°...my mother probably would say me :"Pray God,the end of the world is coming"! :) :)

[Dake_F_]

I used the same fan and fan controller to Pierpaolo's work. 

Just made the bracket by myself on my bridgeport milling machine, and printed the wind shield on solidoodle 3d printer.

Works good so far, die temperature is 49-51 celsius with no heavy loading. Will try work more.

[kecoro]

i'm sorry, i don't know this..

[Dake_F_]

Just got some hints on reducing the noise from the 6800rpm 80x80x38mm fan.

A thin rubber gasket between the fan and wind shield helps. The structure should be rigid, nothing loose and rattle. I made the bracket following the stand ATX long card dimension, so that it fits into the long card holder inside chassis (the cyan part with fingers).

The power cable is kind of tough, difficult to bent. But after removing the skin of that cable a little, job is a lot easy to do.

3D printer seems to be a must for attach the fan to provide a tunnel. At the beginning I was trying to mill the whole thing, but a thin shell is almost impossible to do.

At last, I had major in ME, so I love to make gadgets!  

[jimdempseyatthecove]

I re-did my fan setup and am now using bgears b-Blaster 90 90mm Case Fan with good success. This fan is not as powerful as the one you have, but it is sufficient to get the job done while keeping the noise at an acceptable level.

The initial fan controller I used with the earlier 70mm fans did not have sufficient output to get the newer fans up to speed. I replaced the prior fan controller NZXT Sentry LX, with a different model: NZXT Sentry 2. I would have preferred to use the LX because that had tachometer display. The Sentry 2 list % of power. I had considered using the LX and using a Y-joiner to gang power, and two temperature sensors. Though without experimentation, I am not sure if it would have worked.

My duct is hand crafted out of vinyl 'no parking" sign and vinyl tape. It looks crappy, but it works fine and with the case cover on you do not see it.

Jim Dempsey