Intel X520-2 Strange Speed Setting

[idata]

Hey, been working with these card for a few months now. Every one of our servers seems to be working great but 1 server, Dell 2900 server. We Just installed one card with DAC cables to an Extreme x670 Switch. Now I know Intel doent seem to support cables over 10m, but we have been running just fine with the exception of the speed related issue.

On to the Specs of the Server

Dell 2900

x520-da2 card installed in the x8 pcie slot 3.

32 GB Ram

10tb disk

Windows Server 2003 x64 SP2

Set jumbo frames (9000) up on both switch and Interface to the Storage SFP. one of the copper sfp goes to one Vlan and the other to our Storage Network Vlan, The storage device is Bluarc Mercury with 10gb fiber SFPs in a bond, so 20gb bonded connection.

We have the Extreme # 10307 10gb 10m copper DAC cables and no issue except for speed.

When we swap out the cable for an Extreme # 10306 5m cable speed stays the same.

Now we havent rebooted since the cable swap, and noticed a small revision on the driver which we plan on upgrading as soon as we can reboot the server. The fun part, in Windows task manager the link never gets above 10% utilization on the link ever!

In Dell open Manage the Speed of the link show 1410.1 Mbps. This is where it gets interesting. When I check a different server, Same nic and network it shows 10000 Mbps. I have pictures to post too. But wont let me off hand. I have verified that it stays the same with the 5m cable and the 10m cable. All of our other servers burst above 60% in task manager.

Any ideas? Thanks Andrew

[Mark_H_Intel]

Hi Andrew,

This is a bit puzzling. Have you tried swapping the adapter with one of the adapters that you know is working in one of the other servers? If you have not already tried a swap, the swap would quickly let you know if the adapter was defective.If it is the adapter, you can contact Intel Customer support for an RMA. If it is not the adapter, then we will need to look for other causes. If the server is a different model or configured differently, then maybe there might be a clue in what is different.

Also, the PCIe slot can make a difference. If the adapter is good, do you have a different slot you can try the adapter in?

If you Intel(R) PROSet installed, you can look in the link speed tab under identify adapter to check the PCIe slot width negotiation to see if we are really running at x8 instead of x4. (Screen shots below)

[idata]

I cant seem to post pictures yet, but we changed to a new card. Changed the links to 5m cables, and verified that it is in a 8x slot. We also installed the I/O OAT since it was not enabled in the bios. Installed the Updated drivers and rebooted. Same result. So we switched back to the 10m cables.

[Mark_H_Intel]

Thanks for sending those photos to me in email. Can you tell us more about the type of traffic on the ports? For example, is the storage traffic using iSCSI, NFS, both? I thought I might be able to find a case study where they did something similar that we could work from if their is some configuration tweaks that might help. I think most of the case studies use a newer OS and are often on newer platforms, but we might still find something helpful.

Also, what tuning have you done on the connection? Is everything using the default settings right now?

Mark H

[Mark_H_Intel]

One of my colleagues thought that this hotfix might help: http://support.microsoft.com/kb/972071 http://support.microsoft.com/kb/972071. My understanding is that the hotfix will change the priority of the process that handles the packets. Let us know if this hotfix helps.

Mark H