Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
5212 Discussions

Intel 82579LM Gigabit Ethernet - UDP Packet Loss

idata
Employee
14,584 Views

I have a software application that runs on a laptop running Windows 7 SP1 which receives data at around 100Mbps via a UDP stream over a 1GbE connection. I've run the software on multiple laptops successfully in the past (older network adapters - e.g. 82577LM) . Recently I installed the software on a newer model laptop which has a 82579LM network adapter. When running the software on the laptop with the 82579LM adapter. With the 82579LM adapter I'm seeing approximately one lost packet every 25K-60K packets.

1. I'm running the version 16.7 (latest) of the Intel Network Drivers

2. Windows 7 - SP1 + latest updates

3. I've observed the same behavior on multiple laptop models by different vendors which contain the 82579LM adapter.

Searching the Internet I've seen some postings of issues under Linux complaining about similar 82579LM adapter issues:

http://www.mail-archive.com/e1000-devel@lists.sourceforge.net/msg04765.html http://www.mail-archive.com/e1000-devel@lists.sourceforge.net/msg04765.html

And some more here:

https://bugs.launchpad.net/ubuntu/natty/+source/linux/+bug/870127 https://bugs.launchpad.net/ubuntu/natty/+source/linux/+bug/870127

Since this is running under Windows 7 SP1 we obviously have different drivers but I'm wondering if the issue is related. Any ideas for things I could try to fix the issue?

Thanks,

Ryan

0 Kudos
32 Replies
Mark_H_Intel
Employee
5,641 Views

Hi Ryan,

I do not know if you're report is related to the Linux report or not. However, I do know that an updated Windows driver will be released very soon, and the new driver might help.

Since you have a built-in network connection, there is a chance that your packet loss issue does not exist in whatever driver version your laptop manufacturer might have posted. They probably posted whatever driver version they tested with, and that might be a different version than what you are using now. If that does not do it, try the new driver posted by Intel when it comes out.

You should also make sure that a BIOS or firmware update is not available for your laptop, because that could affect built-in network connections.

I hope this helps.

Mark H

0 Kudos
idata
Employee
5,641 Views

Found something similar using UDP/RTP for video transmission on our Windows 7 machines with the 82579 PHY. We recently received driver: 11.15.12.0, date: (11/30/11). Our test shows that this new driver did not resolve the packet loss problem. It's the first time we've seen this kind of loss on a PHY.

0 Kudos
idata
Employee
5,641 Views

Hi all,

I see the same problem as Ryan using both 16.7 and 16.8 drivers on a Dell Latitude E6520. In our setup, an external hardware device is producing a stream of UDP multicast data, with a constant packet size of approximately 800 bytes. I've seen packets being dropped with as little as 7 MB/s (~9400 packets/s). They are dropped one at a time, not in large groups. I've tried maxing out the number of receive buffers, change the interrupt moderation settings, and disabling all of the checksum offload options.

Any ideas?

-Andy

0 Kudos
Mark_H_Intel
Employee
5,641 Views

Are you using default speed and duplex settings or are you setting the speed / duplex to one of the 100 Mbps settings? I am asking, because I want to rule out the issue from the discussion at /message/148594# 148594 http://communities.intel.com/message/148594# 148594.

Mark H

0 Kudos
idata
Employee
5,641 Views

I am using the default options - the negotiated link speed is 1.0 Gbps/Full Duplex (which is all our external device supports).

0 Kudos
Mark_H_Intel
Employee
5,641 Views

The latest driver that was included with software version 16.8 has all the bug fixes from previously known packet drop issues. However, I just found out about another bug fix for packet drops that will be in the driver refresh planned for late Q1. I will be sending private messages to participants in this thread to get more information.

Mark H

0 Kudos
idata
Employee
5,641 Views

I am also experiencing a similar problem. I also have a Dell Lattidude E6420 that shows significant packet loss when receiving UDP packets. I have a hardware device that streams UDP with configurable packet size and packet rate, and I see missing packets at pretty much all packet sizes (including Jumbo 4K and 8K Ethernet frames) and rates. Bizarrely, if I generate data at around 100MBytes/sec (ie getting close to maxing out the 1GigE link) I see no packet loss. If I drop the rate down to 80MBytes/sec, the packet loss occurs. I see just single missing packets, the same as the earlier post.

I have tried installing an ExpressCard Ethernet adaptor in the same laptop (Startech EC1000S, Realtek chipset) and experience no problems using the same software and UDP packet source on this alternative Ethernet interface with the same hardware and OS setup etc.

I am currently running CentOS Linux 5.7, with the latest 1.9.5 e1000e driver. I have also tried the same thing under Windows 7 and get the same results.

0 Kudos
Mark_H_Intel
Employee
5,641 Views

Thanks for the details about the amount of data seeming to affect the packet drops. I do not know for sure if this is related to some of the power saving features in the chipset or not. I do not have experience with any chipset support; however, I figure that some power saving mode that partially shuts things down to extend the battery power might behind the issue. I think this because when you are processing lots of packets, you are not seeing the packet loss. The packet loss becomes noticeable with a small amount of packets being processed. So my theory is that the power savings features would be in use when processing the smaller number of packets. I do not know this for sure, but the theory is a potential way of explaining what you are seeing.

I was looking at the datasheet for the Intel® 6 Series Chipset used in your PC.

I see that the PCH supports several power management features:

 

— Magic Packet* wake-up enable with unique MAC address

 

— ACPI register set and power down functionality supporting D0 and D3 states

 

— Full wake up support (APM, ACPI)

 

— MAC power down at Sx, DMoff with and without WoL

The datasheet is at http://intel.ly/zRDm8y http://intel.ly/zRDm8y for anyone who is interested in details about the chipset.

There are some power management options that can be configured in Windows Device Manager. I assume Linux supports some of the same options, but I am not familiar with the Linux configurations.

The BIOS probably has several power management options that you could try disabling to test the theory. You might find a particular power management setting that will resolve the dropped packet issue. If you find one, please post here and let everyone know.

Depending on the cause, a fix might require a BIOS update from Dell. Make sure you are already running the latest BIOS in case a fix was already made.

On the other hand, there might be some unknown driver bug that is causing the issue or a driver tweak that works around the issue. The driver refresh for your network connection is planned for late Q1. That refresh is still weeks away, so I would check out the power settings.

I hope this helps.

Mark H

0 Kudos
idata
Employee
5,641 Views

Hi Mark,

Thanks for the advice. I have the latest BIOS from Dell (version A08) and have tried this morning making mods to the various power management settings as suggested. Unfortunately, none of the settings seems to make any difference to the rate of packet loss I am seeing.

An easy way to reproduce this problem is to simply perform a flood ping test from an external linux machine. e.g the command "ping -f -c 100000 " under linux results in:

-- ping statistics --

100000 packets transmitted. 99996 received. 0% packet loss. time 13859 ms

On other machines on my network, the transmitted packets are the same as the received.

Not sure whether you can do a flood ping under Windows........

I'm not very familiar with the level of interaction that is required by the device driver or the kernel, so not sure whether this is relevant, but I have also observed the missing packets within a Wireshark capture of the recieved network traffic.

I believe my laptop has a QM67 chipset. Do you know if I'm likely to see the same problem in other 6 series chipsets? ie if I bought another computer with a HM65, is the ethernet section the same? The website suggests that this has a 82579V. How different is this from a 82579LM? It would be useful to know as I need to build a large number of systems that use this capability and I am keen to avoid using a AMD based system.

Many thanks

Dan

0 Kudos
Mark_H_Intel
Employee
5,641 Views

As far as I know, the chipset does not really have any issues that are causing your drops. If disabling power management features did not help, then my theory on items going to sleep doesn't make much sense. So now I am trying to think of other possibilities.

What kind of switch or router are you connected to? What category of Ethernet cable are you using and how long? I have seen cases where the switch on the other end was putting out a signal that was not quite up to standards or if the cable is missing pairs you can have connection issues. However, in those cases I have seen with cable or switch problems, the connection issues were usually much more severe than what you are seeing.

You might be able to tweak the interupt moderation or other driver setting to get better performance out of the network connection for the UDP drops.I am not sure where the bottleneck happens, but sometimes settings other than the default settings can help. I was looking in the e1000e driver readme for some configuration items that you could try.

InterruptThrottleRate 

---------------------

 

Valid Range: 0,1,3,4, 100-100000 (0=off, 1=dynamic, 3=dynamic conservative, 4=simplified balancing)

 

Default Value: 3

Since the system seems to handle a large number of packets well and has trouble with smaller numbers of packets, you could try configuring the InterruptThrottleRate to 0 (zero). Or maybe one of the other settings would help. I am not sure exactly why you are having the drops, so I cannot say for sure how to set this parameter. Check out the driver readme for details on this setting.

SmartPowerDownEnable 

--------------------

 

Valid Range: 0-1

 

Default Value: 0 (disabled)

Allows Phy to turn off in lower power states. The user can turn off this parameter in supported chipsets.

I assume that SmartPowerDownEnable is at the default of zero, but if it iset to 1, try 0.

I hope this helps.

Mark H

0 Kudos
Mark_H_Intel
Employee
5,641 Views

Another driver refresh is planned for the next software release. Watch for the next software release to be posted before the end of the quarter. I am not sure about the number yet, but I am guessing that the release will have number 17.0.

I plan to post another update in this thread when the software is available for download.

Mark H

0 Kudos
TMeye5
Beginner
5,641 Views

The Intel 82579 chips are not capable of sustained high bandwidth transfers. Their internal connection to the chipset is PCI-Express like, but only half the speed. This means they have less than 1 GBit/s from the 82579 to the chipset --> packets may drop anytime there is too much traffic during a short time. We are using Gigabit Ethernet for full 1GBit/s transfers to FPGA hardware using 100% of the bandwidth --> these chips never worked for that because of the above reason.

Use another network card, that will do (if it is PCI-Express based).

0 Kudos
Mark_H_Intel
Employee
5,641 Views

You are correct that the PCI Express* speed is half of the full speed. And Also we are aware of an issue with the Intel® 82579 Gigabit Ethernet Controllers where some customers are experiencing packet drops like those described in this thread.

However, the speed on the bus is unrelated to the packet drop issue. Intel® 82579 Gigabit Ethernet Controllers meet the specifications of a gigabit network connection and are capable of passing packets at the full 1 Gbps speed. The reduced PCI Express bus speed does not reduce the speed of the connection to less than 1 Gbps.

We have also posted a driver update as part of the version 17.0 release that eliminates the packet drop for many users. You can download the latest Windows* drivers from http://downloadcenter.intel.com/Detail_Desc.aspx?DwnldID=18713 http://downloadcenter.intel.com/Detail_Desc.aspx?DwnldID=18713. We recommend using the updated driver.

Mark H

0 Kudos
TMeye5
Beginner
5,641 Views

I agree, for most people, this network card will do its job.

As long as the internal fifo buffers of the network chip aren't flooded with sustained reception of full speed 1 Gbit network traffic.

I assume that the used 1.25 Gbit serial transmission to the chipset is reduced by 8/10 bit coding to 1 Gbit. But PCI-Express (or something similar) has a much higher data overhead than Ethernet, so it is just impossible to sustain the full bandwidth of 1 Gbit/s from Ethernet to the chipset.

Ethernet with Jumbo frames may transfer over 99% useful payload of the 1Gbit/s theoretical bandwidth. PCI-Express is far away from this, with usually 64 or 128 bytes payload using at least 20 bytes or so of header (data link layer + transaction layer).

We were just disappointed by Intel HW, as these 82579 network chips are built into new (expensive!) hardware like laptops. So we got customer complaints that were like this "we bought new hardware, it doesn't work, although the older one worked".

I agree that our application is a bit special in that it utilizes the Ethernet bandwidth up to over 99%, but that should be possible nowadays, especially considering that Intel has a lot of very capable network chips based on PCI-Express (for instance 82574L).

Unfortunately, these chips aren't built into laptops or express cards, only desktop sized cards.

Thomas

0 Kudos
idata
Employee
5,641 Views

Hi Mark,

I tried the 17.0 version of the driver, but I'm still seeing the same packet drops as before. The actual driver version for e1c62x64.sys is 11.16.96.0. Do I have the right version?

Have any other issues been reported? Is anyone else still having problems with this?

Thanks,

Andy

0 Kudos
Mark_H_Intel
Employee
5,641 Views

Hi Andy,

Thank you for trying the latest driver and providing feedback. Yes, driver version 11.16.96.0 dated 03/15/2012 is the latest driver.

I am sorry to hear that the updated driver did not help you. The information I have is that the latest driver will help many instances of packet drops. So anyone else who is experiencing the drops should still try the updated driver. I don't have any other ideas on what to try, but if I come up with anything, I will be sure to post the latest information.

Mark H

0 Kudos
MThom23
Beginner
5,641 Views

Mark,

I want to reiterate what Andrew said. I too have loaded the latest driver but am seeing UDP packets being discarded. I have a Panasonic H2 Toughbook communicating with a SiLabs CP2201 embeded ethernet chip. I have no problems with packets being dropped if I am connected to other laptops or if I connect through a router, or a Ethernet to USB device.

Using the windows 7 netstat -e -u -p udp I can see that the packets are tagged with errors by the Intel 82579LM card and are thus not passed to the upper layers. The Ethernet sniffer, EtherShark, between the devices reports the packets as valid, but for whatever reason the 82579LM is discarding them.

Please continue the investigation on this so these issues can be resolved.

Thanks,

Matt

0 Kudos
idata
Employee
5,641 Views

Hi Mark,

Have their been any developments on this? This is a rather big issue for me personally as I am unable to access any iSCSI target if Jumbo Frames is enabled on the NIC.

Chhers

0 Kudos
Mark_H_Intel
Employee
5,354 Views

The developers are aware of the dropped packets issue and are working on it. Unfortunately, I do not know of a solution. I will do my best to get the word out when an update is available.

Mark H

0 Kudos
idata
Employee
5,354 Views

Mark,

I couldn't find a way to create a new post, but this issue seems close enough. I recently got a HP 8560W laptop running Win7 64bit enterprise, with the Intel 82579 Gigabit adapter in it. The symptom was audio noise, but updating all the BIOS/Firmware and drivers from HP and Intel did nothing to resolve the issue. Found a couple threads on the web that pointed to the NIC and used the DPC latency checker to confirm the issue was the 82579. The resolution was ultimately to use old drivers posted here:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=US&swItem=ob-96025-1&mode=4&idx=2&prodTypeId=321957&prodSeriesId=5071180 http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=US&swItem=ob-96025-1&mode=4&idx=2&prodTypeId=321957&prodSeriesId=5071180

For more details you can see the post that outlines the fix here:

http://forums.lenovo.com/t5/T400-T500-and-newer-T-series/DPC-latency-audio-mouse-skip-and-stutter-on-t420s/td-p/602481 DPC latency + audio/mouse skip and stutter on t420... - Lenovo Community

I spent many hours attempting to fix the static and skipping audio issue that I'm hoping Intel will fix the latest drivers (based on a diff of the new and old drivers). The driver version that works (and is only available from HP) is version 11.12.36.0 (17 Jun 2011). Considering this issue is several years old, please get the word out and light a fire under the NIC driver team! I can once again experience the full value of my Intel system!

0 Kudos
Reply