e810 ice: lldp packets dropped after changing number of channels

[petonagy]

Hello,

we are facing an issue with Intel 100G E810-C card. If we change

number of channels using ethtool -L, LLDP packets stop to be forwarded

to the network stack and user space.

Firmware-based lldp agent is disabled:

# ethtool --show-priv-flags hge11

Private flags for hge11:

link-down-on-close : off

fw-lldp-agent : off

vf-true-promisc-support: off

mdd-auto-reset-vf : off

vf-vlan-pruning : off

legacy-rx : off

[ 363.664628] ice 0000:41:00.0: The DDP package was successfully

loaded: ICE COMMS Package version 1.3.45.0

[ 363.983822] ice 0000:41:00.0: 252.048 Gb/s available PCIe bandwidth

(16.0 GT/s PCIe x16 link)

[ 364.003040] ice 0000:41:00.0: PTP init successful

[ 364.202873] ice 0000:41:00.0: DCB is enabled in the hardware, max

number of TCs supported on this port are 8

[ 364.202880] ice 0000:41:00.0: FW LLDP is disabled, DCBx/LLDP in SW mode.

[ 364.203045] ice 0000:41:00.0: Commit DCB Configuration to the hardware

The channels are set to the default values:

ethtool -l hge11

Channel parameters for hge11:

Pre-set maximums:

RX: 64

TX: 64

Other: n/a

Combined: 64

Current hardware settings:

RX: 0

TX: 0

Other: n/a

Combined: 64

We are able to see the LLDP frames:

tcpdump -i hge11 -nn 'ether proto 0x88cc'

16:21:08.528229 4c:ae:a3:6a:80:ee > 01:80:c2:00:00:0e, ethertype LLDP

(0x88cc), length 347: LLDP, length 333: test-hp5945

16:21:09.528501 4c:ae:a3:6a:80:ee > 01:80:c2:00:00:0e, ethertype LLDP

(0x88cc), length 347: LLDP, length 333: test-hp5945

16:21:10.528325 4c:ae:a3:6a:80:ee > 01:80:c2:00:00:0e, ethertype LLDP

(0x88cc), length 347: LLDP, length 333: test-hp5945

16:21:11.528217 4c:ae:a3:6a:80:ee > 01:80:c2:00:00:0e, ethertype LLDP

(0x88cc), length 347: LLDP, length 333: test-hp5945

However, if we changed the number of channels, e.g.:

ethtool -L hge11 combined 63

no LLDP frames are seen using tcpdump.

There is a workaround - enable and disable fw-lldp-agent using ethool.

However enabling/disabling fw-lldp-agent using the ethtool flaps the

link which is a problem as a connectivity is lost for a short time period.

Kernel version is 6.8.2, but we see the issue with previous versions as

well. Firmware version is

ethtool -i hge11

driver: ice

version: 6.8.2

firmware-version: 4.40 0x8001c967 1.3534.0

lspci info:

41:00.1 Ethernet controller: Intel Corporation Ethernet Controller

E810-C for QSFP (rev 02)

Subsystem: Intel Corporation Ethernet Network Adapter E810-C-Q2

Product Name: Intel(R) Ethernet Network Adapter E810-CQDA2

Read-only fields:

[V1] Vendor specific: Intel(R) Ethernet Network Adapter E810-CQDA2

[PN] Part number: K91258-011

[SN] Serial number: 6CFE545A0C20

[V2] Vendor specific: 4022

Thank you for your answer.

[Pintu]

Hello petonagy,

Thank you for posting in Intel Ethernet Communities. 

Please provide the following information that would help in checking your request.

1. What is the exact Operating System used?

2. Where was the driver version: 6.8.2,firmware-version: 4.40 0x8001c967 1.3534.0?\

3. Have you tried using the Intel ice driver that can be downloaded from link below? 

Intel® Network Adapter Driver for E810 Series Devices under Linux*

https://downloadcenter.intel.com/download/29746/Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-

4. Will you be able to provide photos of the adapter focusing on the white sticker markings? This will help us identify if you are using an Original Equipment Manufacturer (OEM) or retail version of Intel Ethernet Adapter. We will send an email for you to send the photos. Please be reminded not to post the photos on this thread to avoid exposing confidential information.

Thank you for using Intel products and services.

Best regards,

Manoranjan.

Intel® Customer Support

[petonagy]

Hi,

1. CentOS 7.9

2. It's a driver inside the kernel.

3. Yes, we also tied the latest driver from intel (same result).

4. It's not a OEM, it's a retail version for sure. You can send me an email and I can send you the photos if it is necessary.

Thank you.

[Pintu]

Hello petonagy,

Greetings for the day!

As per your reply, We sent an email; kindly share the picture to proceed further.

Thank you for choosing Intel products and services.

Best regards,

Manoranjan.

Intel® Customer Support 

[petonagy]

Hi,

I sent you the pictures via email.

Thank you.

Best regards,

Peter

[Sachinks]

Hello Petonagy,

Greetings!

Thank you for sending the picture. In the picture we see 2 E810-C cards. 

May I know if you are experiencing this issue with both the cards or only with 1 card?

Regards,

Sachin KS

[petonagy]

We have even more cards (more than 10). Experience is the same on each of them.

[Sachinks]

Hello petonagy,

Greetings!

Thank you for the confirmation. 

May I know which server/system you are using with the card?

Also please send us a screenshot of the packet drop error that you are receiving so that we can get the next plan of action for this issue.

You can send the screenshot on the earlier email thread.

Regards,

Sachin KS

[petonagy]

It's a Supermicro Motherboard X13 (intel xeon scalable 4th gen) running CentOS 7.9. I'm not receiving any error packets or drops. 

I don't know what screenshot should I send you. There is no packet drop error.

[Pintu]

Hello Peter Nagy,

Greeting for the day!

1. Kindly help with the clear picture of Adapter, as the BradyID is not clear.

2. Kindly confirm why you are changing the channels.

3. Kindly confirm where LLDP packets dropped after changing the number of channels.

Thank you for choosing Intel products and services.

Best regards,

Manoranjan.

Intel® Customer Support 

[petonagy]

1. Ok, I will send you new pictures next week.

2. We are changing the number of channels because we need to reserve some cores for other jobs.

3. We don't know where are the packets dropped. Error counters are not incremented and packets are not seen on the interfaces using tcpdump.

[Pintu]

Hello petonagy,

Greetings for the day!

As per your reply, we are waiting for a response with BradyID clear picture.

Thank you for choosing Intel products and services.

Best regards,

Manoranjan.

Intel® Customer Support 

[petonagy]

Hi,

I sent you the pictures (I hope everything is clear and readable this time).

Thank you.

Peter

[Pintu]

Hello petonagy,

Greetings for the day!

We appreciate your response to the case. We are actively working on this case, and we will update you on the status shortly.

Thank you for choosing Intel products and services.

Best Regards,

Manoranjan Das.

[Pintu]

Hello petonagy,

Greetings for the day!

We sincerely apologize for the delay in our response. Please rest assured that our team is actively checking with our internal resources to gather the necessary information.

We understand the importance of your inquiry and are committed to providing you with a prompt and accurate update. We will ensure to keep you informed of the status shortly.

Once again, we appreciate your cooperation and apologize for any inconvenience caused. If you have any further questions or concerns in the meantime, please feel free to reach out.

Thank you for choosing Intel products and services.

Best Regards,

Manoranjan Das.