S5520SC Retrospective

[idata]

I have had my Intel S5520SC Workstation Board for two years now and I thought I would remenisce about the experience.

I hope some Intel system board designers read this - there are some lessons to be learned.

In 2006 I decided I was going to build my own computer. I started by making it a hobby to learn everything I could about building a computer. After about a year I had decided on the AMD dual processor workstation boards because they had a superior architecture with HyperTransport and NUMA, but I was waiting for AMD's next generation of processor the K10, such as the Phenom FX. However, once it came out the reviews revealed disappointing performance, significantly less that AMD had been implying in development.

I had heard that Intel was working on something similar to HyperTransport, but the details were elusive. Eventually Intel announced QuickPath so I became interested in what they might come up with. When Nehalem was announced it was so similar to the K10 that it was easy for me understand the technology, so I waited for a dual processor motherboards to emerge on the market.

There were a lot of Xeon 5500 based motherboards appearing in 2009, but most of them were server boards, and few were workstation boards like I wanted. Of the workstation boards available, some of implementations just looked silly, but most just did not fit my goals, they were always missing one small detail or another. Eventually I stumbled across the Intel S5520SC which was a very close match to my goals. The main drawback was that it was not an enthusiast class board with overclocking capabilities, but all the similar boards with overclocking were missing too many critical features.

Finally I purchaced an S5520SC and built my system. It was relatively straightforward to get it working, albeit my first time assembling a system from scratch. After a few weeks the boad just stopped working and I sent it back to Intel for a replacement. For the person who decided to implement that stupid RAID 5 key - there is no good way to remove it - you make my life a personal hell for about a day.

Once I got my replacement board back and running, then it kept powering down after running for an hour or so. Eventually after some telephone support from Intel I found in the System Event Log (SEL) that the IOH was overheating and causing the system to shutdown. It turns out the first S5520SC I bought had older firmware that did not sense the IOH overheating so it burned out. The second S5520SC had newer firmware that was able to detect the problem and shut down the system before any serious damage was done.

I have a custom SilverStone Temjin TJ07 chassis that was modified by CoolIT in Calgary Canada for their peltier cooling system. I had an extra hole cut in the pexliglass window and installed a 120 mm fan right above the IOH. That seemed to solve the problem as the SEL reveals the IOH temperature is running at nominal levels, and I have had no similar problems since.

Basically the IOH needs a better heat sink, but because it sits right under the slots where the graphics boards go, cooling it is problematic. A better design would be to use a heat pipe like most of the enthusiast boards and dissipate the heat somewhere more convenient.

The next big problem I had was trying to get the RAID 5 working. I was use to working with the Intel Matrix Storage Technology that is found in conventional desktop computers - it is simple to set up and flexible. However the S5520SC uses the Intel Embedded Server RAID Technology that is a software application in the BMC. This is increadibly clumsy to set up from either the BIOS or the Deployment Toolkit CD. It was also very frustrating to find out that the ESRT cannot support RAID 0 on one virtual disk and RAID 5 on another virtual disk the way the MST can. Since the S5520SC has an ICH10 which implements hardware RAID and uses Matrix Storage Technology, I still cannot fathom why Intel chose to use ESRT, an no-one has been able to explain any particular advantage it has over using the hardware RAID in the ICH10.

Another frustration is that the S5520SC only has 6 SATA ports, while most server/workstation chassis have room for six 3.5" disks. This mean that I have to give up one SATA port for my optical drive, and cannot build a RAID system with 6 drives. Many boards from other vendors (who also use the ICH10) add one or more SATA ports for optical and other drives. One extra internal SATA and one extra external SATA is a very popular configuration.

The next big frustration was EFI support. It turns out that if you are using ESRT, you cannot enable EFI optimized boot, so you cannot boot an operating system in native EFI mode. This forced me to create two virtual disks because I had 8 TB of RAID 5, and non-EFI Windows cannot boot from a GPT formatted disk. Consequently I had to create a smaller virtual disk with MBR, and the other virtual disk with GPT.

At this point I was beginning to feel like the S5520SC was an experiment or prototype that was never finished proprerly and sent to market prematurely - there were just too many good intentions with incomplete implementations. This is still my feeling today.

My more recent problem has been that the S5520SC cannot support graphics cards that require more than 300 watts. This is not a physical limitation of the motherboard, it is a design constraint implemented in firmware by the design team - most likely for chassis cooling considerations and not power utilization considerations. It is expecially frustrating that it is not possible to remedy this problem with a firmeware update.

Given all the problems I have encountered I would like to remind people that in most other aspects the S5520SC is a very competent piece of engineering which is why I choose it over about a dozen other similar system boards.

Now what I really want...

Given two years of experience with the S5520SC here is a list of some of the things I really want in a workstation board:

1. Get rid of all the stupid SATA and replace it with ThunderBolt. Connect disks, or even better, SSD devices via ThunderBolt. There should be at least 7 or 8 internal ThunderBolt ports, and 2 to 4 (or more) external ThunderBolt ports. You probably don't need 8 external ports the way most systems offer with USB as you can daisy-chain ThunderBolt. You probably don't need as many internal ports either as current SSD devices are not likely to saturate ThunderBolt bandwidth, so daisy-chaning is pragmatic. This is really a long term solution as there are no appropriate storage devices on the market yet - but maybe Intel will build some :-)
2. I really have no use for USB or FireWire anymore given ThunderBolt makes more sense. It should not be hard to design a ThunderBolt cable that supports a number of USB devices such as keyboard, mouse, printer, etc. However, we really need to be able to boot the system from a ThunderBolt device, and the EFI shell needs to be able to read ThunderBolt storage devices (i.e. thumb drives).
3. Replace the ESRT with Matrix Storage Manager, or do some serious work to improve ESRT (it really sucks).
4. Support EFI properly - no exceptions. While you are at it, please write some better EFI utilities that are more consistent and easy to use, with better UIs. Or document enough of the system specs so third parties can do it themselves.
5. Don't build any serious decisions - like limiting the graphics card(s) to 300 watts - into firmware that is not field upgradable. Put more options in the FRUSDR so the system is more configurable and flexible.
6. Overclocking. Please put some options in the BIOS so that enthusiast builders can have some fun with the system.

Cheers, Eric

[DSilv11]

Thanks for the feedback Eric.

I will see that the design teams gets a chance to see it.

I personally have little impact and no knowledge on what is in planning cycle so I am just speculating also.

1) I don't see Thunderbolt replacing SATA in the next generation. It is just too new. I suspect we will see both on a board before SATA fades away.

2) Yea, from what I have read on ThunderBolt has the potential to sweep the I/O market place, but do you know how long it taken to get legacy I/O and PS2 off the boards? I expect to see ThunderBolt ramp up in popularly and in number of devices over the next couple years, but until then Thunderbotl will likely be one of several choices for I/O on the system.

3) Forwarded to designer

4) Anything in particular your looking for here? Intel developed EFI so the available specification are out there (if somewhat hard to find at times) System specifications are also available. The product TPS lists a lot of the basic information, but if more is needed, your Intel Field team can go much deeper.

5) I need to look into this one. My understanding was the 300W limit is a system / power supply supply / thermal limit also. The limitation does not show up in the mother board TPS , just the system TPS. I don't know of anything in the code that would prevent a higher power card from functioning. As I recall the PCIe configuration space power draw is for power from the MB connector and is only 7 bits for wattage (127W max) that it is capable of reporting. I need to check this one as I agree, List the specification and if the customer want to exceed them, let them do so at their own risk.

6) Currently a processor limitation. XEON's don't support over clocking since they are designed as server processors where reliability is foremost, but the overclocking community have been pushing on the Intel Processor guys for a while now and OC has really taken off in the desktop world. It would not suprise me to see a unlocked Xeon in the pipeline.

On on one last note. My first comment on all issue is "Update the code stack". Bug fixes & feature adds roll out every quater.

[idata]

1. I know it will be a while before ThunderBolt will become more widely accepted. This is just my long term wish that I hope Intel can see the wisdom in going down this path. This is a great opportunity for Intel to put ThunderBolt in their discrete SSD drives to encourage third parties and system builders to try it out. Because size is not relevant to performance as in rotating media, the 3.5" form factor is still relevant and may offer economies of scale over the 2.5" form factor. Technically you could build devices with a 5.25" form factor for further economies of scale, but I doubt that is likely to happen. Also, being able to daisy-chain can improve cable management in the chassis.
2. I think Intel has a great opportunity here of starting out by putting some ThunderBolt on their system boards. If it's there, then third parties, system builders, and users are more likely to use it. I don't think Intel should wait for Apple to convince the marketplace that ThunderBolt is a good idea - to some of us it is ovbious ThunderBold is a great idea.
3. Thanks.
4. I found it really frustrating that I could not use EFI Optimized Boot just because I was using software RAID. There is no good excuse for this other than "we didn't have enough time to finish designing and implementing the S5520SC." I'm also a designer and developer and we rarely get enough time to finish things the way we want to :-) For all I know it may be possible to fix this with a firmware update, but I could not get enough information from technical support to know if this was possible. It certainly did not seem to be a priority. The point is, a product should operate in reasonable ways without such arbitrary exceptions leading to astonishing surprises.I also found some of the EFI utilities frustrating in that they assume they are the only utility on your thumbdrive, and you have to over write them in the root directory (or their supporting files) when you want to use another utility. It should be possible to just creat a directory like /s5520sc/utils/ and run everything out of there (which was the first thing I tried), but you cannot do that because the utilities were not thought out well as a collection.The last utilitiy I used was the Platform Confidence Test, which has a rudimentary GUI. This seems promising and it would be nice if more utilities could go this route. In particular, it would be nice if there could be a comprehsive application that you could just launch from the EFI shell, and run all the Intel (or other) utilties.When you are troubleshooting and diagnosing problems it is stressful enough, but having to read through complicated and inconsistent documentaiton and readme files, search the web for other utilities, etc. makes things all the more stressful.If I had an SDK on writing EFI applications, and enough specs on the S5520SC I might be tempted to write some of my own applications or utilities. I would certainly pay someone for higher quality diagnostic tools as money is a good trade-off for stress. On the other hand, the open-source approach might work equally well.All in all, it is really awesome to be able to run diagnosic software on a system with no O/S. In the future I hope someone develops a little EFI Operating System with a GUI that is independent of Windows, Linux or any other O/S and is portable to all systems running UEFI. If someone could port the Java runtime to the UEFI environment I would be tempted to start such a project myself. Since all UEFI systems run the same byte-codes, this would not depend on whether the main processor was x64, Itanium, Power, SPARC, or any other architecture. For example, it would better to run the Intel Server Deployment Toolkit as an EFI application from a USB or ThunderBolt thumbdrive, rather than a native application from a CD. In the future there really will be no need for optical drives any more as anyone can download anything to a thumbdrive.
5. As far as I can tell this cannot be a power supply problem because the 6990 only needs 75 watts from the PCI Express connector. I can only reason it out that the designers were concerned about thermal issues in the chassis and built in this limitation because they did not have an opportunity to test with such powerful graphics cards - better safe than sorry.
6. I can understand that Xeons are not intended for enthusiasts, but we are out there and it can only bring Intel goodwill to give some small nod to the enthusiast community.

By "update the code stack" do you mean the firmware? I try to keep my firmware up-to-date, and I always update it when I am in a troubleshooting situation as I don't want to report problems against out-of-date firmware.

Thanks for the response - I'm glad there is interest in such things at Intel.

Cheers, Eric