Packet loss on NIC connected to the trunk port on the switch.

[Surfin]

We have implemented submariner using HELM charts into an Openshift 4.12.28 cluster running on Dell hardware.
Worker nodes have one of the following configurations :
2 X Intel X710 NIC's
2 X Mellanox MT27800
The 2 NIC's are bonded together with LACP (802.3ad) using 1 port from each NIC.

We have configured multiple tagged vLAN's as sub-interfaces on top of this bonded interface.
The 2 ports of each NIC are connected to a pair of switches in stack (CISCO Nexus)
We want submariner to use one of the tagged vLAN's for communication between the 2 clusters.

We have 3 gateway nodes and are using VXLAN as the cable driver.
The problem was initially seen as latency of communication between pods on the 2 clusters.
We see packets loss (using ping command) on all sub-interfaces only on the worker node that is running the active gateway. This packet loss is to all other workers in the cluster.

This packet loss only happens on the workers with Intel NIC's. There is no issue with Mellanox MT27800 cards.

We haven't seen any anomaly related to configuration or events on the CISCO Nexus switches.

If we enable promiscuous mode on the port (by taking a tcpdump), then the problem is not observed.

If the switch-port mode on the switch is changed to access from trunk, and the worker node reconfigured to use physical interface without sub-interface, then we do not observe the packet loss.

We can provide a TCP dump of the interaction between the worker nodes and the switches.
the firmware for the Intel NIC's is currently in the process of being upgraded to the latest version as it is 9 months old.

[Fariz_Intel]

Hi Surfin,

Thank you for posting in Intel Ethernet Communities.

Could you try if Intel System Support Utility (Intel SSU) can be used with your system? Intel SSU for the Linux operating system is a standalone utility that performs a detailed scan and report of the computer system information to assist with the troubleshooting. Intel SSU for Linux produces an output file that can be saved, viewed, and shared. You may refer link below for more details.

Intel System Support Utility for the Linux Operating System

https://www.intel.com/content/www/us/en/download/18895/intel-system-support-utility-for-the-linux-operating-system.html

Best regards,

Fariz_Intel

[Sazirah]

Hi Surfin,

Good day.

I am following up on behalf of Fariz_Intel regarding your case.

This is related to your enquiries above, I would like to check if the issue has resolved?

If you have any questions or clarifications, please let me know.

Regards,

Sazzy_Intel

[Sazirah]

Hi Surfin,

Good day.

This is regarding your enquiry above.

Since we have not seen an update, we will be closing this forum case from our end. If you need any additional information, please submit a new question as this thread will no longer be monitored.

Regards,

Sazzy_Intel