Community
cancel
Showing results for 
Search instead for 
Did you mean: 
idata
Community Manager
893 Views

Packet Loss on 82579LM in Fedora 14 and 17

0eHello,

 

I've been trying to solve a problem I'm having with an onboard 82579LM (on a Supermicro X9SCV-Q board). Whenever the NIC is connected to a gigabit switch and negotiates to 1000/full, I see packet loss when running UDP traffic. This packet loss occurs at any bit rate of traffic. I've seen the problem on Fedora 14 (2.6.35.14-106.fc14.i686) and Fedora 17 (3.3.4-5.fc17.x86_64) as well as RHEL5 (2.6.32-279.9.1.e16.x86_64). When I use ethtool -s em1 speed 100 duplex full, I can run up to 95Mbps with 0 lost packets. Set the unit to speed 1000 duplex full and I can't run the same test without regular loss. I've tried several versions of the e1000e drivers available from intel built several ways:

e1000e-1.2.20

e1000e-2.0.0

e1000e-2.0.0.1

e1000e-2.1.4

I've tried building with and without NAPI and I've played with the crc stripping, interrupt throttle rate, and interrupt mode when choosing the module. Some settings made problems worse but nothing helped.

I've used ethtool to run at speed 100 duplex full and that allows for 0 loss up to 95 Mbps. I have also tried the following other ethtool commands with no success:

ethtool -G em1 rx 4096

ethtool -G em1 tx 4096

ethtool -S em1 speed 1000 duplex full autoneg off

ethtool -K em1 rx off

ethtool -K em1 rx on

ethtool -K em1 tso off

ethtool -K em1 gso off

ethtool -K em1 gro off

ethtool -K em1 rxhash off

Any help would be appreciated. I have 16 computers running this NIC and am hoping I can solve the problem.

My actual application for this computer is streaming UDP multicast video, but to see the loss I've been testing with iperf. My setup is as follows:

On box A:

IP - 192.168.188.126

Subnet 255.255.255.0

iperf -s -u -i1 -l1358

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

On box B:

IP - 192.168.188.127

Subnet 255.255.255.0

iperf -c 192.168.188.126 -b 95000000 -i1 -l1358 -t10 -d

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

Results will look something like this on both sides when running at 1000 full:

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

[ 4] local 192.168.188.127 port 46518 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 48961

[ ID] Interval Transfer Bandwidth

[ 4] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8773 (0.011%)

[ 4] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8772 (0.011%)

[ 4] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8772 (0.011%)

[ 4] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 3.0- 4.0 sec 11.4 MBytes 95.2 Mbits/sec 0.005 ms 6/ 8771 (0.068%)

[ 4] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec 0.001 ms 0/ 8772 (0%)

[ 4] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 5.0- 6.0 sec 11.4 MBytes 95.2 Mbits/sec 0.002 ms 5/ 8772 (0.057%)

[ 4] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec 0.002 ms 2/ 8772 (0.023%)

[ 4] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec 0.003 ms 2/ 8772 (0.023%)

[ 4] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec 0.002 ms 4/ 8772 (0.046%)

[ 4] 9.0-10.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 4] 0.0-10.0 sec 114 MBytes 95.3 Mbits/sec

[ 4] Sent 87720 datagrams

[ 3] 0.0-10.0 sec 114 MBytes 95.3 Mbits/sec 0.003 ms 24/87719 (0.027%)

[ 3] 0.0-10.0 sec 1 datagrams received out-of-order

[ 4] Server Report:

[ 4] 0.0-10.0 sec 113 MBytes 95.2 Mbits/sec 0.001 ms 100/87719 (0.11%)

[ 4] 0.0-10.0 sec 1 datagrams received out-of-order

 

When running at 100 full

WARNING: option -b implies udp testing

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

[ 4] local 192.168.188.127 port 45827 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 57617

[ ID] Interval Transfer Bandwidth

[ 4] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8773 (0%)

[ 4] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec 0.122 ms 0/ 8770 (0%)

[ 4] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec 0.120 ms 0/ 8773 (0%)

[ 4] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec 0.122 ms 0/ 8772 (0%)

[ 4] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8773 (0%)

[ 4] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8771 (0%)

[ 4] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 9.0-10.0 sec 11.4 MByt...

0 Kudos
5 Replies
Mark_H_Intel
Employee
60 Views

This issue might be fixed by a BIOS update. I notice that Super Micro has a recent BIOS update for your motherboard. I recommend giving that a try.

Mark H

idata
Community Manager
60 Views

I updated to the most recent Supermicro BIOS (X9SCVQ2.829) and the behavior is the same. Other thoughts? I know my hardware is all capable outside the box as I've run the same tests just moving to the second NIC (82574) and it's clean as a whistle.

As another data point, I ran iperf as TCP and I can get 742 Mbps across the link. The problem seems to be limited to UDP or masked by the acks in TCP.

On box A:

IP - 192.168.188.126

Subnet 255.255.255.0

iperf -s -i1 -l1358

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

On box B:

IP - 192.168.188.127

Subnet 255.255.255.0

iperf -c 192.168.188.126 -i1 -l1358 -t10

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

idata
Community Manager
60 Views

As another observation, I loaded Windows 7 Professional on the same hardware and ran the same test (using driver 17.3). The results are actually worse. In this case iperf pushed ~50Mbps despite running on a 1000/Full connection. It also still had packet loss even at 50Mbps. This is a clean install of Windows 7 Professional no other software loaded just the driver and iperf.

iperf -c 192.168.188.126 -b 100000000 -t10 -i1 -l1358 -d

WARNING: option -b implies udp testing

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1021 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1021 KByte (default)

------------------------------------------------------------

[ 5] local 192.168.188.127 port 60908 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 64112

[ ID] Interval Transfer Bandwidth

[ 5] 0.0- 1.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 0.0- 1.0 sec 5.95 MBytes 49.9 Mbits/sec 0.110 ms 0/ 4597 (0%)

[ 5] 1.0- 2.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 1.0- 2.0 sec 5.86 MBytes 49.1 Mbits/sec 0.108 ms 0/ 4524 (0%)

[ 5] 2.0- 3.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 2.0- 3.0 sec 6.08 MBytes 51.0 Mbits/sec 0.645 ms 0/ 4693 (0%)

[ 5] 3.0- 4.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 3.0- 4.0 sec 6.58 MBytes 55.2 Mbits/sec 0.108 ms 1/ 5078 (0.02%)

[ 5] 4.0- 5.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 4.0- 5.0 sec 6.24 MBytes 52.3 Mbits/sec 0.636 ms 2/ 4817 (0.042%)

[ 5] 5.0- 6.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 5.0- 6.0 sec 6.22 MBytes 52.2 Mbits/sec 0.340 ms 1/ 4806 (0.021%)

[ 5] 6.0- 7.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 6.0- 7.0 sec 5.87 MBytes 49.2 Mbits/sec 0.183 ms 1/ 4532 (0.022%)

[ 5] 7.0- 8.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 7.0- 8.0 sec 6.04 MBytes 50.7 Mbits/sec 0.146 ms 0/ 4665 (0%)

[ 5] 8.0- 9.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 8.0- 9.0 sec 6.23 MBytes 52.2 Mbits/sec 0.120 ms 3/ 4812 (0.062%)

[ 5] 9.0-10.0 sec 12.0 MBytes 101 Mbits/sec

[ 5] 0.0-10.0 sec 120 MBytes 101 Mbits/sec

[ 5] Sent 92593 datagrams

[ 3] 9.0-10.0 sec 6.05 MBytes 50.7 Mbits/sec 0.092 ms 1/ 4671 (0.021%)

[ 5] Server Report:

[ 5] 0.0-10.0 sec 120 MBytes 101 Mbits/sec 0.125 ms 0/92592 (0%)

[ 5] 0.0-10.0 sec 1 datagrams received out-of-order

[ 3] 0.0-10.1 sec 61.2 MBytes 51.1 Mbits/sec 0.899 ms 8/47261 (0.017%)

[ 3] 0.0-10.1 sec 1 datagrams received out-of-order

Mark_H_Intel
Employee
60 Views

Erratum 23 in the http://www.intel.com/content/www/us/en/chipsets/6-and-c200-chipset-specification-update.html Intel® 6 Series Chipset, Intel® C200 Series Chipset: Spec Update lists an issue titled Packet Loss on Intel® 82579 Gigabit Ethernet Controller. The workaround says, "A BIOS code change has been identified and may be implemented as a workaround for this erratum." That was why I suggested the BIOS update. However, this specification update was published in August, so Super Micro's posted BIOS update might not include this workaround. I recommend that you check with Super Micro concerning the availability of a BIOS update that contains the workaround for the erratum.

Mark H

idata
Community Manager
60 Views

Mark,

Thanks for the link to the spec document. I have a parallel conversation going with Supermicro and have asked them whether this change is reflected in the August BIOS I flashed to. I will update if there is a change.

Reply