Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4865 Discussions

Packet Loss on 82579LM in Fedora 14 and 17

idata
Employee
‎10-02-2012 11:13 AM
1,409 Views

0eHello,

 

I've been trying to solve a problem I'm having with an onboard 82579LM (on a Supermicro X9SCV-Q board). Whenever the NIC is connected to a gigabit switch and negotiates to 1000/full, I see packet loss when running UDP traffic. This packet loss occurs at any bit rate of traffic. I've seen the problem on Fedora 14 (2.6.35.14-106.fc14.i686) and Fedora 17 (3.3.4-5.fc17.x86_64) as well as RHEL5 (2.6.32-279.9.1.e16.x86_64). When I use ethtool -s em1 speed 100 duplex full, I can run up to 95Mbps with 0 lost packets. Set the unit to speed 1000 duplex full and I can't run the same test without regular loss. I've tried several versions of the e1000e drivers available from intel built several ways:

e1000e-1.2.20

e1000e-2.0.0

e1000e-2.0.0.1

e1000e-2.1.4

I've tried building with and without NAPI and I've played with the crc stripping, interrupt throttle rate, and interrupt mode when choosing the module. Some settings made problems worse but nothing helped.

I've used ethtool to run at speed 100 duplex full and that allows for 0 loss up to 95 Mbps. I have also tried the following other ethtool commands with no success:

ethtool -G em1 rx 4096

ethtool -G em1 tx 4096

ethtool -S em1 speed 1000 duplex full autoneg off

ethtool -K em1 rx off

ethtool -K em1 rx on

ethtool -K em1 tso off

ethtool -K em1 gso off

ethtool -K em1 gro off

ethtool -K em1 rxhash off

Any help would be appreciated. I have 16 computers running this NIC and am hoping I can solve the problem.

My actual application for this computer is streaming UDP multicast video, but to see the loss I've been testing with iperf. My setup is as follows:

On box A:

IP - 192.168.188.126

Subnet 255.255.255.0

iperf -s -u -i1 -l1358

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

On box B:

IP - 192.168.188.127

Subnet 255.255.255.0

iperf -c 192.168.188.126 -b 95000000 -i1 -l1358 -t10 -d

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

Results will look something like this on both sides when running at 1000 full:

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

[ 4] local 192.168.188.127 port 46518 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 48961

[ ID] Interval Transfer Bandwidth

[ 4] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8773 (0.011%)

[ 4] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8772 (0.011%)

[ 4] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec 0.004 ms 1/ 8772 (0.011%)

[ 4] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 3.0- 4.0 sec 11.4 MBytes 95.2 Mbits/sec 0.005 ms 6/ 8771 (0.068%)

[ 4] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec 0.001 ms 0/ 8772 (0%)

[ 4] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 5.0- 6.0 sec 11.4 MBytes 95.2 Mbits/sec 0.002 ms 5/ 8772 (0.057%)

[ 4] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec 0.002 ms 2/ 8772 (0.023%)

[ 4] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec 0.003 ms 2/ 8772 (0.023%)

[ 4] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec 0.002 ms 4/ 8772 (0.046%)

[ 4] 9.0-10.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 4] 0.0-10.0 sec 114 MBytes 95.3 Mbits/sec

[ 4] Sent 87720 datagrams

[ 3] 0.0-10.0 sec 114 MBytes 95.3 Mbits/sec 0.003 ms 24/87719 (0.027%)

[ 3] 0.0-10.0 sec 1 datagrams received out-of-order

[ 4] Server Report:

[ 4] 0.0-10.0 sec 113 MBytes 95.2 Mbits/sec 0.001 ms 100/87719 (0.11%)

[ 4] 0.0-10.0 sec 1 datagrams received out-of-order

 

When running at 100 full

WARNING: option -b implies udp testing

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1023 KByte (default)

------------------------------------------------------------

[ 4] local 192.168.188.127 port 45827 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 57617

[ ID] Interval Transfer Bandwidth

[ 4] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 0.0- 1.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8773 (0%)

[ 4] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 1.0- 2.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 2.0- 3.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 3.0- 4.0 sec 11.4 MBytes 95.3 Mbits/sec 0.122 ms 0/ 8770 (0%)

[ 4] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 4.0- 5.0 sec 11.4 MBytes 95.3 Mbits/sec 0.120 ms 0/ 8773 (0%)

[ 4] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 5.0- 6.0 sec 11.4 MBytes 95.3 Mbits/sec 0.122 ms 0/ 8772 (0%)

[ 4] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 6.0- 7.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8773 (0%)

[ 4] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 7.0- 8.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8771 (0%)

[ 4] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec

[ 3] 8.0- 9.0 sec 11.4 MBytes 95.3 Mbits/sec 0.121 ms 0/ 8772 (0%)

[ 4] 9.0-10.0 sec 11.4 MByt...

0 Kudos

Link Copied

5 Replies
Mark_H_Intel
Employee
‎10-02-2012 12:52 PM
576 Views

This issue might be fixed by a BIOS update. I notice that Super Micro has a recent BIOS update for your motherboard. I recommend giving that a try.

Mark H

0 Kudos
Copy link
idata
Employee
‎10-02-2012 02:40 PM
576 Views
In response to Mark_H_Intel

I updated to the most recent Supermicro BIOS (X9SCVQ2.829) and the behavior is the same. Other thoughts? I know my hardware is all capable outside the box as I've run the same tests just moving to the second NIC (82574) and it's clean as a whistle.

As another data point, I ran iperf as TCP and I can get 742 Mbps across the link. The problem seems to be limited to UDP or masked by the acks in TCP.

On box A:

IP - 192.168.188.126

Subnet 255.255.255.0

iperf -s -i1 -l1358

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

On box B:

IP - 192.168.188.127

Subnet 255.255.255.0

iperf -c 192.168.188.126 -i1 -l1358 -t10

sysctl -w net.core.rmem_max=1047856

sysctl -w net.core.rmem_default=1047856

sysctl -w net.core.wmem_default=1047856

sysctl -w net.core.wmem_max=1047856

In response to Mark_H_Intel
0 Kudos
Copy link
idata
Employee
‎10-03-2012 11:02 AM
576 Views
In response to Mark_H_Intel

As another observation, I loaded Windows 7 Professional on the same hardware and ran the same test (using driver 17.3). The results are actually worse. In this case iperf pushed ~50Mbps despite running on a 1000/Full connection. It also still had packet loss even at 50Mbps. This is a clean install of Windows 7 Professional no other software loaded just the driver and iperf.

iperf -c 192.168.188.126 -b 100000000 -t10 -i1 -l1358 -d

WARNING: option -b implies udp testing

------------------------------------------------------------

Server listening on UDP port 5001

Receiving 1358 byte datagrams

UDP buffer size: 1021 KByte (default)

------------------------------------------------------------

------------------------------------------------------------

Client connecting to 192.168.188.126, UDP port 5001

Sending 1358 byte datagrams

UDP buffer size: 1021 KByte (default)

------------------------------------------------------------

[ 5] local 192.168.188.127 port 60908 connected with 192.168.188.126 port 5001

[ 3] local 192.168.188.127 port 5001 connected with 192.168.188.126 port 64112

[ ID] Interval Transfer Bandwidth

[ 5] 0.0- 1.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 0.0- 1.0 sec 5.95 MBytes 49.9 Mbits/sec 0.110 ms 0/ 4597 (0%)

[ 5] 1.0- 2.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 1.0- 2.0 sec 5.86 MBytes 49.1 Mbits/sec 0.108 ms 0/ 4524 (0%)

[ 5] 2.0- 3.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 2.0- 3.0 sec 6.08 MBytes 51.0 Mbits/sec 0.645 ms 0/ 4693 (0%)

[ 5] 3.0- 4.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 3.0- 4.0 sec 6.58 MBytes 55.2 Mbits/sec 0.108 ms 1/ 5078 (0.02%)

[ 5] 4.0- 5.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 4.0- 5.0 sec 6.24 MBytes 52.3 Mbits/sec 0.636 ms 2/ 4817 (0.042%)

[ 5] 5.0- 6.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 5.0- 6.0 sec 6.22 MBytes 52.2 Mbits/sec 0.340 ms 1/ 4806 (0.021%)

[ 5] 6.0- 7.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 6.0- 7.0 sec 5.87 MBytes 49.2 Mbits/sec 0.183 ms 1/ 4532 (0.022%)

[ 5] 7.0- 8.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 7.0- 8.0 sec 6.04 MBytes 50.7 Mbits/sec 0.146 ms 0/ 4665 (0%)

[ 5] 8.0- 9.0 sec 12.0 MBytes 101 Mbits/sec

[ 3] 8.0- 9.0 sec 6.23 MBytes 52.2 Mbits/sec 0.120 ms 3/ 4812 (0.062%)

[ 5] 9.0-10.0 sec 12.0 MBytes 101 Mbits/sec

[ 5] 0.0-10.0 sec 120 MBytes 101 Mbits/sec

[ 5] Sent 92593 datagrams

[ 3] 9.0-10.0 sec 6.05 MBytes 50.7 Mbits/sec 0.092 ms 1/ 4671 (0.021%)

[ 5] Server Report:

[ 5] 0.0-10.0 sec 120 MBytes 101 Mbits/sec 0.125 ms 0/92592 (0%)

[ 5] 0.0-10.0 sec 1 datagrams received out-of-order

[ 3] 0.0-10.1 sec 61.2 MBytes 51.1 Mbits/sec 0.899 ms 8/47261 (0.017%)

[ 3] 0.0-10.1 sec 1 datagrams received out-of-order

In response to Mark_H_Intel
0 Kudos
Copy link
Mark_H_Intel
Employee
‎10-03-2012 04:50 PM
576 Views
In response to idata

Erratum 23 in the http://www.intel.com/content/www/us/en/chipsets/6-and-c200-chipset-specification-update.html Intel® 6 Series Chipset, Intel® C200 Series Chipset: Spec Update lists an issue titled Packet Loss on Intel® 82579 Gigabit Ethernet Controller. The workaround says, "A BIOS code change has been identified and may be implemented as a workaround for this erratum." That was why I suggested the BIOS update. However, this specification update was published in August, so Super Micro's posted BIOS update might not include this workaround. I recommend that you check with Super Micro concerning the availability of a BIOS update that contains the workaround for the erratum.

Mark H

In response to idata
0 Kudos
Copy link
idata
Employee
‎10-04-2012 07:51 AM
576 Views
In response to idata

Mark,

Thanks for the link to the spec document. I have a parallel conversation going with Supermicro and have asked them whether this change is reflected in the August BIOS I flashed to. I will update if there is a change.

In response to idata
0 Kudos
Copy link
Reply

Community support is provided during standard business hours (Monday to Friday 7AM - 5PM PST). Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.