i40e nics are down when we use them on Dual CPU

[WL]

Hi All,

When we use netperf to generate traffic, i40e nics are down very soon(

the throughput is about 76Gbps).

CPU: "Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz" X2

TOPO:

port5, port6, port11, port12 are i40e interfaces.

port6 and port12 are in a net namespace.

port5<--->port6: port5 is connected port6 directly.

port11<--->port12: port11 is connected port12 directly.

nics interrupt bind cpu:

port5: 0, 1, 2, 3, 4 (CPU0)

port6: 10, 11, 12, 13, 14 (CPU1)

port11: 5, 6, 7, 8, 9(CPU0)

port12: 15, 16, 17, 18, 19 (CPU1)

kernel: 3.13.11

driver: i40e stable

1.0.15(http://sourceforge.net/projects/e1000/files/i40e%20stable/1.0.15/ http://sourceforge.net/projects/e1000/files/i40e%20stable/1.0.15/):

version: 1.0.15

firmware-version: f4.1 a1.1 n04.10 e800010e0

bus-info: 0000:09:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

We also tried latest kernel 3.16.3 (with its own driver), it has the same issue.

netperf cmd:

netperf -T 14,19 -L 15.3.2.1 -H 15.3.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 13,18 -L 15.5.2.1 -H 15.5.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 12,17 -L 15.2.2.1 -H 15.2.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 11,16 -L 15.1.2.1 -H 15.1.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 10,15 -L 15.4.2.1 -H 15.4.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 4,9 -L 14.4.2.1 -H 14.4.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 3,8 -L 14.1.2.1 -H 14.1.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 2,7 -L 14.5.2.1 -H 14.5.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 1,6 -L 14.2.2.1 -H 14.2.1.100 -f m -D 1 -l 600 >/dev/null &

netperf -T 0,5 -L 14.3.2.1 -H 14.3.1.100 -f m -D 1 -l 600 >/dev/null &

dmesg:

...

i40e 0000:09:00.1 port6: NIC Link is Up 40 Gbps Full Duplex, Flow Control: None

i40e 0000:09:00.0 port5: NIC Link is Up 40 Gbps Full Duplex, Flow Control: None

IPv6: ADDRCONF(NETDEV_CHANGE): port6: link becomes ready

IPv6: ADDRCONF(NETDEV_CHANGE): port5: link becomes ready

i40e 0000:8a:00.0 port11: NIC Link is Up 40 Gbps Full Duplex, Flow Control: None

i40e 0000:8a:00.1 port12: NIC Link is Up 40 Gbps Full Duplex, Flow Control: None

IPv6: ADDRCONF(NETDEV_CHANGE): port11: link becomes ready

IPv6: ADDRCONF(NETDEV_CHANGE): port12: link becomes ready

------------[ cut here ]------------

WARNING: at net/sched/sch_generic.c:254 dev_watchdog+0x174/0x1da()

Hardware name: To be filled by O.E.M.

NETDEV WATCHDOG: port5 (i40e): transmit queue 3 timed out

<span style="color: # 2222...