i40e x710 4x10G card not working anymore with DAC

[JGuy]

Using an updated driver, I cannot seem to get my interfaces up. Looks like the driver is not allowing the NIC to work due to the SFP+ module type, which worked previously on my Debian Stretch host. Clearly the NIC is detecting and reading the DAC, but I don't understand why it is not working. This is a 40G-breakout cable with 40-to-4x10G SFP+ DAC (TwinAx). I have tried adding the following to work around it, but it still does not work:

root@lab4:~# cat /etc/modprobe.d/i40e.conf

options i40e allow_unsupported_sfp=1

This was also discussed here, but there was no followup:

https://sourceforge.net/p/e1000/mailman/message/36029185/ Intel Ethernet Drivers and Utilities / Mailing Lists

Output from dmesg:

[ 4.439190] i40e 0000:08:00.3: PCI-Express: Speed 8.0GT/s Width x8

[ 4.447918] i40e 0000:08:00.3: Features: PF-id[3] VFs: 32 VSIs: 34 QP: 8 RSS FD_ATR FD_SB NTUPLE CloudF DCB VxLAN Geneve NVGRE PTP VEPA

[ 4.453414] i40e 0000:08:00.2: Rx/Tx is disabled on this device because an unsupported SFP+ module type was detected.

[ 4.465288] i40e 0000:08:00.2: Refer to the Intel(R) Ethernet Adapters and Devices User Guide for a list of supported modules.

Other pertinent output:

root@lab4:~# ethtool -m eth6

Identifier : 0x03 (SFP)

Extended identifier : 0x04 (GBIC/SFP defined by 2-wire interface ID)

Connector : 0x21 (Copper pigtail)

Transceiver codes : 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00

Transceiver type : FC: Copper Passive

Encoding : 0x00 (unspecified)

BR, Nominal : 10300MBd

Rate identifier : 0x00 (unspecified)

Length (SMF,km) : 0km

Length (SMF) : 0m

Length (50um) : 0m

Length (62.5um) : 0m

Length (Copper) : 3m

Length (OM3) : 0m

Passive Cu cmplnce. : 0x01 (SFF-8431 appendix E) [SFF-8472 rev10.4 only]

Vendor name : Amphenol

Vendor OUI : 78:a7:14

Vendor PN : 624400003

Vendor rev : A

Option values : 0x00 0x00

BR margin, max : 0%

BR margin, min : 0%

Vendor SN : APF13470035GDT

Date code : 131122

root@lab4:~# ethtool -i eth6

driver: i40e

version: 2.4.3

firmware-version: 6.01 0x800034af 1.1747.0

expansion-rom-version:

bus-info: 0000:08:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

root@lab4:~# ethtool eth6

Settings for eth6:

Supported ports: [ ]

Supported link modes: Not reported

Supported pause frame use: Symmetric

Supports auto-negotiation: No

Advertised link modes: Not reported

Advertised pause frame use: No

Advertised auto-negotiation: No

Speed: Unknown!

Duplex: Unknown! (255)

Port: Other

PHYAD: 0

<span ...

[idata]

HI Ninwardspiral,

Thank you for taking time to perform additional tests and provide the information. We are still checking and will update you.Thanks,Sharon T

[idata]

Hi Ninwardspiral,

I sent a PM to you about some request, please check and reply from there. As an additional information, you may refer to page 10 and 15 of below XL710 controller matrix for the changes from firmware 5.05 to 6.01, there are some enhancement such as SR-IOV (Intel® Ethernet Adaptive Virtual Function) and support of 1GbE SX\LX Optical modules. https://www.intel.com/content/dam/www/public/us/en/documents/release-notes/xl710-ethernet-controller-feature-matrix.pdf Looking forward to your reply.Thanks,Sharon T

[JGuy]

Hi Sharon,

I have looked through the source code for the driver, and this is the function in the file i40e_main.c that is creating the error message found in the logs. As I understand this, the firmware is setting the values for the struct, and the driver is reading them. What I cannot figure out is why the firmware is flagging a 10G SFP+ DAC as 'I40E_AQ_QUALIFIED_MODULE'. Clearly this function is not the one suppressing the module, but it is reporting the problem based on the firmware. Clearly the firmware is determining this module is unqualified for some reason. However, everything I have compared, between the Dell Amphenol breakout DAC, the Generic Amphenol breakout DAC, and the straight Amphenol SFP+ DAC, there is absolutely no functional difference in the eeprom information. Are you sure there is no "whitelist" or "blacklist" in the Firmware?

Thanks,

Jason

/**

* i40e_handle_link_event - Handle link event

* @pf: board private structure

* @e: event info posted on ARQ

**/

static void i40e_handle_link_event(struct i40e_pf *pf,

struct i40e_arq_event_info *e)

{

struct i40e_aqc_get_link_status *status =

(struct i40e_aqc_get_link_status *)&e->desc.params.raw;

/* Do a new status request to re-enable LSE reporting

* and load new status information into the hw struct

* This completely ignores any state information

* in the ARQ event info, instead choosing to always

* issue the AQ update link status command.

*/

i40e_link_event(pf);

/* Check if module meets thermal requirements */

if (status->phy_type == I40E_PHY_TYPE_NOT_SUPPORTED_HIGH_TEMP) {

dev_err(&pf->pdev->dev,

"Rx/Tx is disabled on this device because the module does not meet thermal requirements.\n");

dev_err(&pf->pdev->dev,

"Refer to the Intel(R) Ethernet Adapters and Devices User Guide for a list of supported modules.\n");

} else {

/* check for unqualified module, if link is down, suppress

* the message if link was forced to be down.

*/

if ((status->link_info & I40E_AQ_MEDIA_AVAILABLE) &&

(!(status->an_info & I40E_AQ_QUALIFIED_MODULE)) &&

(!(status->link_info & I40E_AQ_LINK_UP)) &&

(!(pf->flags & I40E_FLAG_LINK_DOWN_ON_CLOSE_ENABLED))) {

dev_err(&pf->pdev->dev,

"Rx/Tx is disabled on this device because an unsupported SFP+ module type was detected.\n");

dev_err(&pf->pdev->dev,

"Refer to the Intel(R) Ethernet Adapters and Devices User Guide for a list of supported modules.\n");

}

}

}

[idata]

Hi Ninwardspiral,

Thank you for sharing the information, I will further check with our engineering team.Regards,Sharon T

[idata]

Hi Swedebay,

Thank you for posting in Wired Communities. Can you share what is the exact network adapter model? (e.g. XL710-QDA1 or XL710-QDA2?) and what is the fiber module model attached to the network adapter? Looking forward to your reply. Thanks.Regards,Sharon T

[idata]

Hi Ninwardspiral,

To provide an update, there is no "whitelist" or "blacklist" in the firmware. Thank you. Regards,Sharon T

[idata]

Hi Ninwardspiral,

I sent a PM to you, please feel free to reply me from there.Thanks,Sharon T

[idata]

Hi Ninwardspiral,

I sent a PM to you about some request, please feel free to update me. Thanks,Sharon T

[idata]

Hi Ninwardspiral,

I sent a PM to you. Thanks.Regards,Sharon TIntel Customer SupportAgent under contract to Intel

[ANieś]

ninwardspiral did you manage to run the module on this card?

[JGuy]

Hi Adam, I am not sure what you mean by "run the module" .

Some breakout DACs work, while others do not. Intel gave me a diagnostics tool to try, and there are no issues found.

Currently I can only use Dell branded breakout DACs, which are identical to my other amphenol DACs. I don't know what Intel is doing with all of the data I have collected for them, but it does not seem like there is a fix anytime soon.

[idata]

Hi Ninwardspiral,

We've sent you a private message. Looking forward to your response.Regards,Vince T.Intel Customer Support