XL710QDA2 support 10GBASE-KX4

[Scott129]

Does the XL710QDA2 support 10GBASE-KX4 using Ubuntu 22.04?   

For our setup, I'm getting the following messages in 'dmesg' log as the Link is coming up.  Note: were using QSPF+ copper cables.  

[ 4806.234280] i40e 0000:81:00.1 enp129s0f1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None

[4806.241135] i40e 0000:81:00.1 enp129s0f1: WARNING: Link is up but PHY type 0x25 is not recognized, or incorrect cable is in use

'ifconfig' also reports the link is UP, RUNNING at 10000Mbps.    But the link is not working -- we're not getting any Rx or Tx packets.   We also installed the latest 'i40e' driver, 2.23.17 (i40e-2.23.17.tar.gz) from Intel and got the same warning message and behavior.

Note: the PHY Type should be 0x2 for 10GBASE-KX4.   

How can we set the PHY type?   Or set it up for 10GBASE-KX4 operation?

thanks,

Scott

[Yogaeasvaran]

Greetings Mr. Scott,

Thank you for contacting Intel Customer Support.

We have checked the specifications which you are using it correctly, but the error is showing incorrect cable.

[ 4806.234280] i40e 0000:81:00.1 enp129s0f1: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None

[4806.241135] i40e 0000:81:00.1 enp129s0f1: WARNING: Link is up but PHY type 0x25 is not recognized, or incorrect cable is in use.

Therefore, I have attached the link below for you to download and update the driver version as below:

https://www.intel.com/content/www/us/en/products/sku/83967/intel-ethernet-converged-network-adapter-xl710qda2/downloads.html

Please do provide feedback once updat eht driver.

Thank You & Regards,

Yogaeasvaran

Intel Customer Support

[Scott129]

Hello,

Thank you for your quick reply.

Before my initial posting, we had already downloaded and installed the latest i40e driver for linux --  i40e-2.23.17.tar.gz file from here https://www.intel.com/content/www/us/en/download/18026/intel-network-adapter-driver-for-pcie-40-gigabit-ethernet-network-connections-under-linux.html .      The results from my initial post are using this latest i40e driver (2.23.17).

As an separate experiment, we connected the A port to the B port of the XL710QDA2 using one of our cables (QSFP+), and it worked fine (no warning/error messages) and connected at 40Gbps.  So that says to me the cable is not the problem.   I also tried to get it working using 10GBase-KX4 from Port A to Port B (using ethtool -s dev speed 10000, and ethtool -s dev advertise 0x40000) but it said wasn't supported.    I wonder why this is, since the XL710 datasheet says its supported.

Does Intel have or recommend any Linux software utility for debugging or communicating with the i40e driver?   The standard Linux tool I used (ethtool) doesn't provide much detail as to what's going on.

Thanks,

Scott

message when trying to set the ethtool advertise parameter to 0x40000 (10GBase-KX4):

azure@ubuntu2204-sm1u:~$ sudo ethtool -s enp129s0f1 advertise 0x40000
netlink error: link settings update failed
netlink error: Invalid argument

[Scott129]

Hello,

We get the same behavior after installing the latest i40e driver (2.23.17).   PHY type = 0x25, which is not recognized.   It does say the link is UP and at the correct speed of 10000Mbps.   But it doesn't say whether its at 10000BaseKX4 or 10000BaseKR4.

How do we configure the XL710QDA2 to force it work at 10000BaseKX4 or at least advertise as 10000BaseKX4?    The only advertised or supported PHY type seems to be 40000BaseKR4 (which we proved works by connecting the NIC's Port A & B with a QSFP+ Cable).    But we need it to operate at 10000BaseKX4.    We're not sure of the auto negotiation capability of our card, so being able to force it to 10000BaseKX4 would be preferred.

Thanks,

Scott

[Yogaeasvaran]

Greetings Mr. Scott,

Thank you for the feedback.

Could you provide me the current firmware version for me to check?

Also, have you update the firmware version as well?

Thank You & Regards,

Yogaeasvaran

Intel Customer Support

[Scott129]

We are now using firmware version 9.30, which we downloaded from Intel's website and installed using the 'nvmupdate64e' tool.   We originally had firmware version 6.01 before updating.    Note: after updating the i40e driver (2.23.17) and after updating the firmware the behavior did not change -- the LINK is UP, but its not working.    The 'dmesg' Warning message did change after we installed 9.30 to 'PHY type 0x32 is not recognized, or incorrect cable is in use'.

Note: we have tried different QSPF+ cables, and connected Port A to B with one of our QSFP+ cables to verify that the cables aren't the problem.

What's next?    How can we force the XL710QDA2 to use 10GBase-KX4?

Thanks.

Scott

[Irwan_Intel]

Hi Scott129,

Based on the Feature Support Matrix for the Intel® Ethernet Controller X710/XXV710/XL710, https://cdrdv2.intel.com/v1/dl/getContent/332191

KX4 is not supported on the cards in question. However, it is supported for other configurations as specified in the matrix.

Another important note is that the use of Active Copper Cable is not supported based on the same document.

To further investigate and provide appropriate assistance, could you kindly gather information regarding the cables specifically,such as the make, model, and any other relevant information regarding the cables in use.

Feel free to reach out if you require any additional clarification or assistance.

Regards,

Irwan_Intel

[Scott129]

We are using passive copper QSFP+ cables (p/n C9797-1M-CX4).    We have proven this cable works with the XL710QDA2, by connecting Port A to Port B and verifying that the Link is UP and we can ping from one port to the other.    The link comes up as 40GBase-CR4.

Does Intel offer any Ethernet Network Adapters for 10GBase-CX4 that are in production?     We're using an older one by Intel, p/n EXPX9502CX4, but its no longer available.

Does Intel offer any NVM (firmware) updates for the XL710QDA2 that would support 10GBase-CX4 over QSFP+ (passive copper).    The CFG_ID we need for our application would be similar to 1.0, per Table 3-78 of the XL710 datasheet.    Note: 10GBase-CX4 uses the same physical layer coding as 10GBase-KX4.

thanks,

Scott

[Yogaeasvaran]

Greetings Mr. Scott,

We need to know if you tried changing the port configuration on the card from 2 ports at 40Gb to 4 10Gb ports?

This can be done with EPCT https://www.intel.com/content/www/us/en/download/19435/ethernet-port-configuration-tool-all-supported-oss.html

You also can configure:

1 X 10 (port 1 has 4 10G connections) or

2 x 2 x 10 (port 1 has 2 10G connections and port 2 has 2 10G)

May I also know what you are connecting this to?

Is it to Make/model of switch?

Thank You & Regards,

Yogaeasvaran

Intel Customer Support

[Yogaeasvaran]

Greetings Mr. Scott,

Regarding your case number 05999960, we are following up to find out if you were able to complete the actions, we previously recommended.

Please reply to this thread to confirm, so we can continue helping with a resolution. Looking forward to receiving your reply!

Thank You & Regards,

Yogaeasvaran

Intel Customer Support

[Yogaeasvaran]

Greetings Mr. Scott,

We are following up to find out if you were able to complete the actions, we previously recommended.

Please reply to this thread to confirm, so we can continue helping with a resolution. Looking forward to receiving your reply!

Thank You & Regards,

Yogaeasvaran

Intel Customer Support

[Yogaeasvaran]

Greetings Mr. Scott,

Kindly note that as we have not received any response from our previous follow-ups, we will be closing this request.

If you have any more questions in the future, please don't hesitate to post a new question, as this thread will no longer be monitored.

Thank You & Regards,

Yogaeasvaran

Intel Customer Support