Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4810 Discussions

Dual port 10Gbe performing at half speed

NElmo
Beginner
1,366 Views

I have two servers running OpenSUSE both with Intel dual port 82599EB 10-Gigabit SFI/SFP+ using the ixgbe drivers. And I'm not getting the bandwidth I expect with iperf. Both ports are connected directly to the the other two ports of the second server.

On one machine I run

iperf -s

On the other machine I run these two commands in separate terminals.

iperf -c 192.168.1.10 -t 20 -B 192.168.1.20

iperf -c 192.168.1.11 -t 20 -B 192.168.1.21

And I get

[ 4] 0.0-20.0 sec 7.63 GBytes 3.28 Gbits/sec

[ 5] 0.0-20.0 sec 14.7 GBytes 6.30 Gbits/sec

If I run only one port, I get

[ 6] 0.0-20.0 sec 22.8 GBytes 9.80 Gbits/sec

Shouldn't I be able to expect roughly 10Gbe on each port simultaneously? Do I have the wrong hardware for that requirement? Or an invalid test?

0 Kudos
4 Replies
Patrick_K_Intel1
Employee
511 Views

Hi,

Performance testing is a challenging topic with many, many variables. Ranging from the # of threads you specify for iperf to how much memory is on the system and in what slots they are in and what slot your NIC is in (a x1 vs a x8 for example).

Having done a lot of tests with iperf and netperf, I can tell you that all of these factors and many more can come into play. I'd recommend doing some research, the 1st thing you might try is specify the number of cores and threads to use.

Lots of articles out there on different apsects of this topic. There is no single answer, lots of variables.

best of luck,

Patrick

0 Kudos
NElmo
Beginner
511 Views

Thanks for the info. That's kinda what I've run into. I've already fiddled with the rmem buffer sizes and MTU. Didn't think to check the PCI speeds. They are both in PCIe x16 slots so that shouldn't be a problem.

Found a few decent articles to that have shed some light, but I haven't fixed it yet:

http://www.intel.com/support/network/sb/cs-025829.htm Network Connectivity — Tuning Intel® Ethernet Adapter throughput performance

https://www.kernel.org/doc/Documentation/networking/ixgbe.txt https://www.kernel.org/doc/Documentation/networking/ixgbe.txt

http://www.redhat.com/promo/summit/2008/downloads/pdf/Thursday/Mark_Wagner.pdf http://www.redhat.com/promo/summit/2008/downloads/pdf/Thursday/Mark_Wagner.pdf (PDF)

Still running at max of about 9.5Gbe when using a single port, and 4.8Gbe on each port when using both ports though . Will keep trying and post here if I find an answer.

0 Kudos
IIRCA2
Beginner
511 Views

It sure seems that you are going out the same port for both tests. Have you verified that you really are sending packets out both ports (say using dstat).

0 Kudos
TLipp1
Beginner
511 Views

What about running

iperf -s -B 192.168.1.10

and

iperf -s -B 192.168.1.11

on the server side?

0 Kudos
Reply