Link state flap with traffic does not recover for SR-IOV VFs

[CDech1]

Using DPDK 2.3 and ixgbe driver 4.13.15. When there is no traffic on the link and it is flapped, the VFs recover the link state properly. When there is too much traffic though, the link state can't be recovered for VFs. It seems to be caused by the PF driver putting the VF in a not clear to send state. Since the VF is not clear to send, it can't retrieve the link state via the mailbox. It looks like this can be fixed by restoring the clear to send state in the PF after the cleanup on state down:

*** ixgbe_main.c 2016-05-09 11:07:44.365431345 -0400

--- ixgbe_main.c.bak 2016-05-09 11:07:22.309430641 -0400

***************

*** 6128,6140 ****

ixgbe_clean_all_tx_rings(adapter);

ixgbe_clean_all_rx_rings(adapter);

-

- if (adapter->num_vfs) {

-

- /* Mark all the VFs as clear to send again */

- for (i = 0 ; i < adapter->num_vfs; i++)

- adapter->vfinfo[i].clear_to_send = 1;

- }

}

/**

--- 6128,6133 ----

Thoughts? Not sure why it works without above fix when traffic is light. The DPDK does resets here and there for the VF so perhaps timing makes a difference at restoring the clear to send state.

Thanks.

[idata · Accepted solution]

Hi cdechamplain,

Thank you for contacting Intel. For inquiry on Intel® DPDK, kindly contact https://kaveri.intel.com/ibl/help/support.htm Intel® Business Link Support.Sincerely,Sandy

[CDech1]

Understood, but this fix I posted above is not in DPDK, it is in the ixgbe driver. In other words, the DPDK side where the VFs are used seems fine to me. It is the ixgbe side where the PF sits that needs the above fix. Sorry for the confusion.

[idata]

Hi cdechamplain,

Thanks for replying. I'll further check on this. Sincerely,Sandy

[idata]

Hi cdechamplain,

I would like to request your system configuration. Please provide the output of ethtool -i.Thanks again. Sincerely,Sandy

[CDech1]

It looks like this:

> ethtool -i p1p1

driver: ixgbe

version: 4.3.15

firmware-version: 0x61b50001, 0.324.97

bus-info: 0000:03:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

[idata]

Hi cdechamplain,

Thank you for the information. While we are checking on this, we would like to know your setup in case we need to dig deeper:1. your network card model2. SFP+ module3. OS and kernel version. Thank you and have a great day.Sincerely,Sandy

[CDech1]

Sandy,

1. 03:00.0Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ NetworkConnection (rev 01)
2. [579084.108425]ixgbe 0000:87:00.1 eth0: MAC: 2, PHY: 18, SFP+: 6, PBA No: E96137-005
3. Linuxott-pd-vs-020 3.13.0-32-generic # 57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014x86_64 x86_64 x86_64 GNU/Linux
4. Thanks.

[idata]

Hi cdechamplain,

Thank you for your information. We'll keep you posted on our findings. Sincerely,Sandy

[idata]

Hi cdechamplain,

Will you be able to uninstall the DPDK and see if the problem will occur?Sincerely,Sandy