Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4811 Discussions

Intel x550 not negotiating to 10Gb on x299 chipset

KVask
Beginner
4,495 Views

Using Ubuntu 18.04.

Asus X299 with Intel X550-T2 onboard

 

I am having an issue with this particular machine auto negotiating 10Gb. The onboard LOM will only negotiate to 1Gb. If I run `ethtool -s <device> advertise 0x1000`, the machine will delink and then negotiate to 10Gb and work as expecting. Rebooting the machine (or unplugging network cable) will cause it to negotiate to 1G again resulting in having to run the ethtool command again.

This is what I have tried:

Tried different cable from confirmed working machine that auto negotiates to 10Gb.

Tried upgrading ixgbe driver to latest (5.5.3)

Tried upgrading X550 firmware.

I put an Intel x540-T1 card in the machine and that card works properly and negotiates to 10Gb automatically. Move the cable to the LOM x550, and it still negotiates to 1Gb.

I also tried loading a live version of Fedora 29 (Kernel 4.18) with same exact symptoms.

We have several other machine negotiating fine on the same exact switch. I have tried those cables to test this problematic machine with same result as above.

We have a machine that is pretty similar in hardware that is not exhibiting this issue. It is an ASUS x99 motherboard, with the same LOM Intel x550 card, running on 16.04.

The machine that is causing problems is an Asus x299 motherboard with the Intel x550 LOM card running 18.04 (16.04 won't work on the x299 chipset for some reason). I have a suspicion that it could potentially be related to the x299?

The only way I have been able to get this problematic machine to negotiate to 10Gbps automatically is by plugging a cable directly from the LOM x550 of the problematic machine to another machines 10Gb network card. Doing that, the problematic machine will negotiate properly to 10Gb.

I'm trying to find another 10Gb switch to test with to see if it could be unique to this combination of switch and machine. However, I'm reluctant to blame it on the switch since we have about several other machine working perfectly fine on the switch (some with the same exact x550 LOM).

Other notes:

https://serverfault.com/questions/949006/intel-x550-not-negotiating-to-10gb?noredirect=1#comment1233436_949006

 

0 Kudos
19 Replies
Daniel_D_Intel1
Employee
2,927 Views
Hello KVask, Thank you for posting in Intel Ethernet Communities. Please let us know what model of Cisco switch you are connecting to, and type of cable used. Please provide "ethtool -k <interface>" and "ethtool -i <interface>" outputs. When the ixgbe was loaded were there any error outputs? Did you also try the in-kernel driver, and was 5.5.3 also used on Fedora 29? In the serverfault thread you mentioned the transfer speed was 1Gbps when the link speed was 10Gbps with a point to point connection. Were the LAN LEDs displaying green for 10Gbps or orange for 1Gbps on the link speed LED? Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,928 Views

The switch is a Cisco Nexus 3172TQ. We are using CAT6 cable.

 

I didn't see any errors. Yes I have tried the in-kernel driver that came with 18.04, I upgraded the driver to 5.5.3 in 18.04 (just to test). I also use the Fedora 29 in-kernel driver (don't remember which version is included though). So in the serverfault thread I mentioned the transfer speed was 1Gbps, but that was when I forced the speed to be 10Gb and not letting the card autonegotiate to 10Gbps with the advertise command.

 

The point-to-point will negotiate automatically to 10Gbps. The link will be green in this case.

 

In all other cases when it negotiates to 1Gbps it will be amber in color.

 

 

ethtool -k <interface> and ethtool -i <interface> are below.

driver: ixgbe version: 5.5.3 firmware-version: 0x80000d04 expansion-rom-version: bus-info: 0000:b3:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yesFeatures for enp179s0f0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: on [fixed] tx-checksum-sctp: on scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: on highdma: on [fixed] rx-vlan-filter: on vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: on [fixed] tx-gre-segmentation: on tx-gre-csum-segmentation: on tx-ipxip4-segmentation: on tx-ipxip6-segmentation: on tx-udp_tnl-segmentation: on tx-udp_tnl-csum-segmentation: on tx-gso-partial: on tx-sctp-segmentation: off [fixed] tx-esp-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: on

 

 

 

 

0 Kudos
Daniel_D_Intel1
Employee
2,928 Views
Hello KVask, Thank you for the reply. We will check on this information, and update you as soon as possible. Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
Daniel_D_Intel1
Employee
2,928 Views
Hello KVask, Have you checked with Asus if this is a known issue with the onboard Ethernet? Please also provide the "ethtool -i <interface>" and "ethtool -k <interface>" for the working X99 Asus board for comparison. A work around while we check on this issue would be to add ethtool -s <interface> advertise 0x1000 to the startup file rc.local. Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,928 Views

My next step was to get with ASUS support in parallel with this thread.

 

Working x99 motherboard output.

 

This system is running Ubuntu 16.04 (which is why you see older versions of the driver). I'm hesitant to make any changes to this x99 box as people are using it.

ethtool -i enp2s0f0 driver: ixgbe version: 4.2.1-k firmware-version: 0x80000573 expansion-rom-version: bus-info: 0000:02:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: no~$ ethtool -k enp2s0f0 Features for enp2s0f0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: on tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: on tx-checksum-fcoe-crc: on [fixed] tx-checksum-sctp: on scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp6-segmentation: on udp-fragmentation-offload: off [fixed] generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: on highdma: on [fixed] rx-vlan-filter: on [fixed] vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: on [fixed] tx-gre-segmentation: off [fixed] tx-ipip-segmentation: off [fixed] tx-sit-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off busy-poll: on [fixed] hw-tc-offload: off

 

0 Kudos
Daniel_D_Intel1
Employee
2,927 Views
Hello KVask, Thank you for the outputs. We will check on this and update you soon. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
Daniel_D_Intel1
Employee
2,927 Views
Hello KVask, Thank you for your patience. We would recommend continuing support with ASUS. The Etrack ID and ethtool -k outputs are different on the two boards, so ASUS may heavily customize the controllers. ASUS may be able to isolate the issue with a board replacement. Let us know if you still have questions after contacting ASUS. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
Daniel_D_Intel1
Employee
2,927 Views
Hello KVask, Please let us know if you have been able to resolve this issue with help from ASUS. If you have any questions please ask. Best regards, Daniel D Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,927 Views

Thanks. I’m trying to get ASUS support to answer my question. I’m linking them here with all of the content and debugging information in this thread but they haven’t gotten back with me.

 

Im sending another email today to try and get them.

0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, Thank you for the update. If you have any questions please ask. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, Checking if you tried contacting Asus and if the assisted you on the issue. If you have any questions please ask. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,927 Views

I have and they are looking into the issue. They told me they would reply in 24-48 hours with an update.

0 Kudos
KVask
Beginner
2,927 Views

Mike,

 

I heard back from ASUS and this is their response

 

ASUS Support Response:

I appreciate your patience and for giving us time to come up with a resolution for your case. After collaborating with our engineers we are now able to provide you with a recommendation on how to address your concern.

 

The motherboard only support WIndows 10 64 bit. Other OS, we do not have any software support. We do not have driver for Linux OS.

Please do contact Intel technical support to get the driver working.”

 

 

So based on their response, they do not support anything but Windows 10 (which I feel is a complete joke) these motherboards are targeted at machine learning people (supporting 4x gpus) yet the motherboard doesn’t support Linux, which is where a vast majority of the development is in the ML space.

 

But I digress, so at this point ASUS is pointing the finger back to Intel saying Intel should fix your driver on Linux. Unfortunately, I’m in the middle of it.

 

What is your recommendation at this point?

0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, Thank you for the update and I am glad that you are now talking to Asus support. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, Thank you for the update and I am very to sorry to hear that. As much as we want to further help you, upon checking the Etrack ID and ethtool -k outputs, they are different. Let me also further explain that the board has the control on which OS they validate on their board and if they validate an OS, they should also provide a driver on their own download site specially if they altered some specification of the controller. In your case, the Etrack ID and ethtool -k outputs are different so they may have alter some of its specifications. That is why on cases like this, we advise the customer's to contact the the board manufacturer for the appropriate driver. Please let us know if you have questions. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,927 Views

Thsts completely understandable.

 

Just curious, where are you finding the ETrackID information? Did I miss it?

 

0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, I am sorry, we just ask for the ethtool -k and ethtool -i then counter check it here in our database. And if the product comes up as OEM, it will only say OEM without a 3rd party brand or manufacturer. Please let us know if you have questions. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
Mike_Intel
Moderator
2,927 Views
Hello KVask, I just want to check if you have other questions or clarifications. Please let us know if you have questions. Best regards, Michael L. Intel Customer Support Under contract to Intel Corporation
0 Kudos
KVask
Beginner
2,927 Views

Nope! I appreciate it. You can close the ticket if you need.

0 Kudos
Reply