ixgbe: link flapping on two directly connected servers with 82599ES adapter

[idata]

Good time.

We experince link flapping while connecting two supermicro servers with 82599ES adapters:

[Wed Jan 6 07:10:14 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down

[Wed Jan 6 07:10:15 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

[Wed Jan 6 07:10:15 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down

[Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

[Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down

[Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

[Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down

[Wed Jan 6 07:10:16 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

[Wed Jan 6 07:10:17 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Down

[Wed Jan 6 07:10:17 2016] ixgbe 0000:81:00.1 enp129s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

So we are trying to connect two supermicro servers with E10G42BTDA X520-DA2 controller in each server:

81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

In one controller there is following SFP module:

# ethtool --module-info enp129s0f1

Identifier : 0x03 (SFP)

Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)

Connector : 0x07 (LC)

Transceiver codes : 0x20 0x00 0x00 0x00 0x00 0x00 0x00 0x00

Transceiver type : 10G Ethernet: 10G Base-LR

Encoding : 0x06 (64B/66B)

BR, Nominal : 10300MBd

Rate identifier : 0x00 (unspecified)

Length (SMF,km) : 20km

Length (SMF) : 20000m

Length (50um) : 0m

Length (62.5um) : 0m

Length (Copper) : 0m

Length (OM3) : 0m

Laser wavelength : 1330nm

Vendor name : GIGALINK

Vendor OUI : 00:90:65

Vendor PN : GL-OT-ST12LC1-13

Vendor rev : A

Optical diagnostics support : Yes

Laser bias current : 37.020 mA

Laser output power : 0.9654 mW / -0.15 dBm

Receiver signal average optical power : 0.4904 mW / -3.09 dBm

Module temperature : 44.23 degrees C / 111.61 degrees F

Module voltage : 3.2836 V

Alarm/warning flags implemented : Yes

Laser bias current high alarm : Off

Laser bias current low alarm : Off

Laser bias current high warning : Off

Laser bias current low warning : Off

Laser output power high alarm : Off

Laser output power low alarm : Off

Laser output power high warning : Off

Laser output power low warning : Off

Module temperature high alarm : Off

Module temperature low alarm : Off

Module temperature high warning : Off

Module temperature low war...

[st4]

Hi pva,

Thank you for the post. You are right we recommend to use supported fiber module such as

the one posted on this websitehttp://www.intel.ph/content/dam/doc/product-brief/ethernet-sfp-optics-brief.pdf http://www.intel.ph/content/dam/doc/product-brief/ethernet-sfp-optics-brief.pdf. Currently,I do not have information about WDM fiber module for this NIC. If the 82599 NIC is an onboard NIC on your Supermicro* system, you may check with themto better assist you about onboard NIC. Thank you.

rgds,

wb