Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4809 Discussions

E810-C does not forward lldp

StefanMajer
Beginner
6,271 Views

Hi, 

I have a Server running ubuntu-20.04 with two E810-C cards connected to two Edgecore AS7732-12. This is the first server with the 810 series card, the farm we have is equipped with xxv710 25G models. 

We run a small lldpd https://github.com/metal-stack/go-lldpd on all machines to send custom TLVs via lldp. This is required for our bare metal provisioning.

This works fine for XXV710 and X550 type ethernet cards, but not with the e800 family.

The only lldpd which is able to send lldp frames is lldpad, but this daemon does not allow to send SystemName/SystemDescription with own content.

All other lldpd implementation does not work. We are pretty sure this is due the fact that this card has DCB as a hard requirement. 

Any hints howto make vanilla lldp daemons work here ?

Greetings

Stefan Majer

Details:

lspci

3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)
d8:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02)

 

root@100gtest:# ethtool --show-priv-flags lan0
Private flags for lan0:
link-down-on-close : off
fw-lldp-agent : off
vf-true-promisc-support: off
mdd-auto-reset-vf : off
legacy-rx : off

 

root@100gtest:# ethtool -k lan0
Features for lan0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: on
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: on
receive-hashing: on
highdma: on
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: on
tx-ipxip6-segmentation: on
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
tx-gso-list: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off

 

 

 

root@100gtest:# ethtool -i lan0
driver: ice
version: 0.8.2-k
firmware-version: 2.30 0x80005d1e 1.2877.0
expansion-rom-version:
bus-info: 0000:3b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

 

0 Kudos
28 Replies
StefanMajer
Beginner
4,960 Views

Interesting side node: Disable DCB with dcbtool does not have any effect:

 

root@100gtest:~/go-lldpd# dcbtool gc lan0 dcb
Command: Get Config
Feature: DCB State
Port: lan0
Status: Successful
DCB State: on
root@100gtest:~/go-lldpd# dcbtool sc lan0 dcb off
Command: Set Config
Feature: DCB State
Port: lan0
Status: Successful
root@100gtest:~/go-lldpd# dcbtool gc lan0 dcb
Command: Get Config
Feature: DCB State
Port: lan0
Status: Successful
DCB State: on

0 Kudos
StefanMajer
Beginner
4,869 Views

Trying to update the firmware with the most recent (2.42) version leads to:

with

./nvmupdate64e -a .
Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.35.62.7
Copyright(C) 2013 - 2021 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [+.........]
Tool execution completed with the following status: The configuration file could not be opened/read, or a syntax error was discovered in the file.
Press any key to exit.

 

 

 

and with the next older one (2.32) gives me after ~20mins:

Num Description Ver.(hex) DevId S:B Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Network Adapter n/a(n/a) 1592 00:059 Access error
E810-C-Q1
02) Intel(R) Ethernet Network Adapter n/a(n/a) 1592 00:216 Access error
E810-C-Q1


Tool execution completed with the following status: An error occurred accessing the device.
Press any key to exit. 

 

So im even not able to update to firmware

0 Kudos
Caguicla_Intel
Moderator
4,943 Views

Hello StefanMajer,


Thank you for posting in Intel Ethernet Communities. 


Please provide the following information that would help in checking your request.

1. What is the exact Operating System used?

2. Where was the driver version: 0.8.2-k and firmware 2.30 0x80005d1e 1.2877.0 downloaded from? 

3. Have you tried using the Intel ice driver that can be downloaded from link below? 

Intel® Network Adapter Driver for E810 Series Devices under Linux*

https://downloadcenter.intel.com/download/29746/Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-

4. Will you be able to provide photos of the adapter focusing on the white sticker markings? This will help us identify if you are using an Original Equipment Manufacturer (OEM) or retail version of Intel Ethernet Adapter. We will send an email for you to send the photos. Please be reminded not to post the photos on this thread to avoid exposing confidential information.


We hope to hear from you soon. 


Should there be no response from you, I’ll make sure to reach out after 3 business days.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
StefanMajer
Beginner
4,917 Views

Operating System:

- ubuntu 20.04 with HWE Kernel

- The ice driver is from the HWE kernel package ubuntu provides and matches the most recent 5.8.x vanilla kernel and the ice driver version included there

- The firmware is unmodified and the card is provided directly from Intel ! 

 

Same behavior with a stock vanilla 5.10.26 kernel and the matching ice driver.

This is a quite serious issue for us because we are going to order hundreds of servers equipped which this type of cards as a update to our existing fleet of servers with XXV710 cards deployed where all LLDPd operations work. 

I also tried some other lldpd implementation like https://github.com/lldpd/lldpd which also works without any issues on the XXV710 but does not work on the e810.

 

 

0 Kudos
Caguicla_Intel
Moderator
4,930 Views

Hello StefanMajer,


Good day!


We'd like to follow up the information requested for us to proceed checking your query. Should you need more time in gathering the details, feel free to let us know and we will be glad to adjust our follow up date.


Awaiting to your reply.


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,917 Views

Hello StefanMajer,


Thank you for providing the requested information.


We understand the urgency of your query and rest assured we will closely work on this together with our engineers.


Please allow us escalate this request. We will get back to you as soon as we heard from our engineers but no later than 2-3 business days. 


Hoping for your kind patience.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,898 Views

Hello StefanMajer,


I hope this message finds you well!


Our engineer is requesting for the following information. We hope you don't mind sharing this with us. 

1.Adapter's markings. We need the PBA markings for validation. You may refer to the link below on where to find the PBA number. Providing photos of the adapter focusing on the markings (white sticker) found on the physical card will be highly appreciated for us to double check on it. The PBA consists 6-3 digit number located at the last part of the serial number. We will be sending an email for you to attach the photos of the adapter. Please be reminded not to post the photos on this thread to avoid exposing confidential information. If providing the photos is not possible, the PBA number will do. 

Identify Your Intel® Network Adapter Model Using PBA Number

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products.html

2. Brand and model of the system where the adapter is installed.


Awaiting to your reply.


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,871 Views

Hello StefanMajer,


Thank you for the prompt reply on this thread and via email.


Please allow me to re-escalate this request to our engineers and forward the information you provided. Rest assured that we will get back to you as soon as we heard from them but no later than 2-3 business days.


We would also like to apologize for any inconvenience caused on this matter. 


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,839 Views

Hello StefanMajer,


Good day and I hope you will be having great weekend!


Please be informed that we are consulting with our Columbiaville (E810 engineers/developers regarding your request/issue for vanilla LLDP). We will also forward the information you provided via email. We will get back to you as soon as we heard an update from the concerned team but no later than 2-3 business days.


Best regards,

Crisselle C.

Intel® Customer Support 


0 Kudos
StefanMajer
Beginner
4,820 Views

Hi Intel Support,

 

any updates here ?

 

kind regards

Stefan

0 Kudos
Caguicla_Intel
Moderator
4,809 Views

Hello StefanMajer,


I hope this message finds you well!


Please accept our sincerest apologies for the delay on this matter as we are thoroughly checking on this. Rest assured that we are actively working on this together with our engineers and we will give you an update as soon as there is any findings but not later than 3 business days.


Hoping for your kind understanding.


Best regards,

Crisselle C.

Intel® Customer Support 


0 Kudos
Caguicla_Intel
Moderator
4,787 Views

Hello Stefan,


Thank you for the patience on this matter.


We already received an update from our engineers and they found out that you are using driver version 0.8.2-k. In order to update the Firmware, you need to use first the latest Linux driver which is 1.4.11.

https://sourceforge.net/projects/e1000/files/ice%20stable/1.4.11/


Once the driver is updated, update the Firmware to version 2.42

https://downloadcenter.intel.com/download/29738/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapters-E810-Series-Linux-


In regard to your concern about LLDP and DCB, we have found the readme file that has an entire section detailing LLDP and DCBX in both Firmware and Software mode. Kindly use the recommended driver, update the Firmware and consult the readme.

https://www.kernel.org/doc/html/latest/networking/device_drivers/ethernet/intel/ice.html


In firmware-based mode, firmware intercepts all LLDP traffic and handles DCBX

negotiation transparently for the user. In this mode, the adapter operates in

"willing" DCBX mode, receiving DCB settings from the link partner (typically a

switch). The local user can only query the negotiated DCB configuration. For

information on configuring DCBX parameters on a switch, please consult the

switch manufacturer's documentation.


In software-based mode, LLDP traffic is forwarded to the network stack and user

space, where a software agent can handle it. In this mode, the adapter can

operate in either "willing" or "nonwilling" DCBX mode and DCB configuration can

be both queried and set locally. This mode requires the FW-based LLDP Agent to

be disabled.


Feel free to let us know if you have questions or clarifications.


Awaiting to your reply.


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C.

Intel® Customer Support 


0 Kudos
StefanMajer
Beginner
4,734 Views

Hi Support,

thanks for the pointer to the ice kernel driver update, this is not so easy to do if you must install a Distribution which does not have the ability to deploy self deployed kernel drivers. In our case we need to support centos and debian.

But i will check this to be able to at least update the firmware to the latest version.

 

Your explanation regarding the support of "vanilla" lldp, this is wrong. I got from another source the hint that your card actually requires that the LLDP Ethernet frames are sent to a socket which socket priority ist set to at least TC_PRIO_CONTROL (7). Otherwise the ice driver will not forward this Ethernet Frame out of the card itself. I was able to test this and send LLDP packets through your e810 card without any kernel driver or firmware updates.

I also checked the source of lldpad (your lldpd implementation) and other lldpd implementations out there. The only lldpd server implementation is lldpad which actually set the socket priority to TC_PRIO_CONTROL.

 

Please update your documentation in this aspect, because other users out there should not spend 3 weeks or more on this topic as i did.

 

I am also not very pleased with the support i got from this channel tbh. 

 

Greetings

Stefan Majer

 

 

 

0 Kudos
Caguicla_Intel
Moderator
4,717 Views

Hello Stefan,


Thank you for the update.


We totally understand the importance of this request and how this has impacted your workflow. With this, we sincerely apologize for the delay and inconvenience that this have caused. Rest assured that we will communicate this update to our engineers and take all the necessary measures to ensure that this does not happen again in the future. Thank you for bringing this matter to our attention.


We will get back to you as soon as there is additional information from our engineering team. In case you have further questions and clarifications, feel free to let us know and we will be glad to be of help.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,681 Views

Hello Stefan,


I hope this message finds you well!


Please be informed that we already relayed your update to our engineers and take note of your concern regarding the documentation. Thank you again for the effort in sharing the information. We will now proceed closing this request. Just feel free to post a new question if you may have any other inquiry in the future as this thread will no longer be monitored.


May you have an amazing day!


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
StefanMajer
Beginner
4,677 Views

Hi,

please add the pointer to the updated documentation here as well.

 

Greetings

Stefan

0 Kudos
Caguicla_Intel
Moderator
4,673 Views

Hello Stefan,


Good day!


Thank you for the reply. Rest assured that we will communicate this to our engineers.


May you have an amazing weekend ahead!


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,613 Views

Hello Stefan,


Thank you for the patience on this matter. Please see below information for the update from our Engineering team.


Your statement regarding LLDP is accurate for driver versions 1.4.11 and older. This issue (PF LLDP bug) was found and fixed in the upcoming 26.3 release (driver ice-1.5.4).


Issue: The driver was incorrectly flagging egressing packets which causes LLDP packets to be dropped from interfaces. 


Fix: The newer driver adds better checking to determine if the egressing packets are LLDP packets or not.


Release 26.3 should be available in the next couple of weeks. The 26.3 release notes might have a better description text on the issue and fix.


Feel free to let us know if you have additional questions and clarifications.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,576 Views

Hello Stefan,


Good day!


Please be informed that this request is still on our engineering's end. Rest assured that we will give you an update as soon as we heard from them.


Thank you for your kind understanding.


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,474 Views

Hello Stefan,


Good day!


We are glad to inform you that 26.3 is now released.


Complete Driver Pack: https://downloadcenter.intel.com/download/22283/Intel-Ethernet-Adapter-Complete-Driver-Pack


Release Notes Link: https://downloadcenter.intel.com/download/29706/Intel-Ethernet-Product-Software-Release-Notes 


Entry into 26.3 Release Notes:

Section 2.0 Fixed Issues

Sub-Section 2.1.2 Linux Driver

The driver is incorrectly flagging egressing packets which causes LLDP packets to be dropped from interfaces.


Feel free to let us know if you have additional questions and clarifications.


We look forward to your reply.


We will follow up after 3 business days in case we don't hear from you. 


Best regards,

Crisselle C.

Intel® Customer Support


0 Kudos
Reply