I have two servers running OpenSUSE both with Intel dual port 82599EB 10-Gigabit SFI/SFP+ using the ixgbe drivers. And I'm not getting the bandwidth I expect with iperf. Both ports are connected directly to the the other two ports of the second server.
On one machine I run
On the other machine I run these two commands in separate terminals.
iperf -c 192.168.1.10 -t 20 -B 192.168.1.20
iperf -c 192.168.1.11 -t 20 -B 192.168.1.21
And I get
[ 4] 0.0-20.0 sec 7.63 GBytes 3.28 Gbits/sec
[ 5] 0.0-20.0 sec 14.7 GBytes 6.30 Gbits/sec
If I run only one port, I get
[ 6] 0.0-20.0 sec 22.8 GBytes 9.80 Gbits/sec
Shouldn't I be able to expect roughly 10Gbe on each port simultaneously? Do I have the wrong hardware for that requirement? Or an invalid test?
Performance testing is a challenging topic with many, many variables. Ranging from the # of threads you specify for iperf to how much memory is on the system and in what slots they are in and what slot your NIC is in (a x1 vs a x8 for example).
Having done a lot of tests with iperf and netperf, I can tell you that all of these factors and many more can come into play. I'd recommend doing some research, the 1st thing you might try is specify the number of cores and threads to use.
Lots of articles out there on different apsects of this topic. There is no single answer, lots of variables.
best of luck,
Thanks for the info. That's kinda what I've run into. I've already fiddled with the rmem buffer sizes and MTU. Didn't think to check the PCI speeds. They are both in PCIe x16 slots so that shouldn't be a problem.
Found a few decent articles to that have shed some light, but I haven't fixed it yet:
http://www.intel.com/support/network/sb/cs-025829.htm Network Connectivity — Tuning Intel® Ethernet Adapter throughput performance
Still running at max of about 9.5Gbe when using a single port, and 4.8Gbe on each port when using both ports though . Will keep trying and post here if I find an answer.