Re: Intel I210-IT Gigabit Ethernet Controller Stripping VLAN Tags under Centos

Mike1111111 · ‎01-11-2021

We are using a Single Board Computer which has a Intel I210-IT Ethernet Controller on it. We are finding that the I210-IT is stripping off the VLAN Tag as it processes packets.

Here is a diagram of my setup;

I have a SBC running Centos 7 with a physical Ethernet Adapter (ens35u1) which uses the Intel I210-IT. It is tied to a trunk from a Layer 2 switch. The trunk has two VLANs on it and the native VLAN.

Off of ens35u1 I have two Virtual Interfaces ens35u1.101 and ens35u1.102. ens35u1.101 will handle VLAN 101 traffic and ens35u1.102 will handle VLAN 102 traffic.

I verified that the egress packets from each VLAN interface are tagged with the appropriate VLAN tags and sent on the Trunk to the Layer 2 Switch:

ens35u1.101 is setup to tag egress packets with a VLAN tag = 101
ens35u1.102 is setup to tag egress packets with a VLAN tag = 102

The defined operation for ingress packets to the VLAN interfaces is as follows;

ens35u1.101 will filter ingress packets for a VLAN tag = 101 and ignore all other packets different VLAN tags
ens35u1.102 will filter ingress packets for a VLAN tag = 102 and ignore all other packets different VLAN tags.

However, instead of the ingress packets being filtered and handled by individual Virtual interface they are receives and handled by both interfaces regardless of the VLAN Tag. They do not appear to filter out VLAN tags that don’t match their particular configuration. In other words:

If the Centos machine receives a packet with a VLAN tag = 101 from the Switch it is being handled by both VLAN interfaces (ens35u1.101 and ens35u1.102). It should only be processed by ens35u1.101, and ens35u1.102 should ignore it since it doesn’t match **bleep** configuration.
The same happens if the Centos machine receives a packet with a VLAN tag = 102 from the Switch, it is also being handled by both VLAN interfaces instead of just ens35u1.102...

After looking into it using tcpdump I can see that the physical interface ens35u1 is stripping off the VLAN Tag before passing it on to the Virtual Interfaces.

I looked at the datasheet for the INTEL I210-IT and it states that the device support different modes of operation for handling VLAN Tags. I believe we need to be able to set the device to MODE #3 below in order to pass the VLAN Tags (don't strip them off) onto the Virtual interfaces.

There are other post where other folks had this same issue, however it provides a solution for Windows not Linux. Here is the link.

https://community.intel.com/t5/Ethernet-Products/How-to-configure-Vlan-tagging-for-I210-IT-amp-I219-LM/td-p/1214721

I really need the instructions that are geared for enabling this mode in a Centos 7 Linux environment. Any help or guidance would be greatly appreciated.

Thanks - mike

Mike1111111 · ‎01-11-2021

More information to add:

Looking at Intel's Driver site it is using the igb driver, see below:

I looked at the Datasheet for the I219-IT and it supports not stripping the VLAN Tag . It is called Mode =3, see excerpt from the datasheet below (page 751):

file:///C:/Users/P009271d/Downloads/333016_I210_Datasheet_v_3_6.pdf

Here is a link to the igb driver but not sure what I need to do to keep it from stripping the VLAN Tag.

I would like to know what I need to do to read those settings and then put it into Mode =3 in a Linux environment. Any help would e greatly appreciated.

Thanks - mike

AlfredoS_Intel · ‎01-11-2021

Hi Mike1111111,

Thank you for posting in our Intel® Ethernet Communities Page.

We are sorry to hear about the issue that you are experiencing with your network adapter. We appreciate the additional information that you have provided.

So we would have a better understanding of your issue, please provide the following information:

1. Please download and run our Intel® System Support Utility from this page, https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System#:~:text=Intel%20SSU%20for%20the%20Linux,and%20shared%20by%20the%20user. After running it, you will be given an option to save the logs to a text file, please do so and attach the file on your reply.

2. Kindly provide us the results of this command: ethtool -i ethx where ethx is the Ethernet port.

We look forward to hearing from you. If we do not get your reply, we will follow up after 3 business days.

Best Regards,

Alfred S

Intel® Customer Support

Mike1111111 · ‎01-12-2021

Hi AlfredoS,

Thank you, I appreciate your help.

When I ran ssu.sh it indicated that it was not supported on this OS and prompted me to load hdparm which I did. I included the console output below just in case.

[eio@localhost Downloads]$ sudo ./ssu.sh
This product is not supported on this operating system.
Would you like to try to scan? (y/n)
y
Cannot get driver information: Operation not supported
The hdparm package is recommended to retrieve hdparm details.
Would you like to try and install it now? (y/n)
y

I attached the following files as you requested:

localhost.localdomain.txt - output of running your tools
ethtoolOutput1.txt - contains the "ethtool -i" for the physical ens35u1 and both virtual adapters (ens35u1.101 and ens35u1.102) that are children of ens35u1.

In addition, I also added the following file which contains the output of "ethtool -k" for the physical ens35u1 and both virtual

ethtoolOutput2.txt - contains the "ethtool -k" for the physical ens35u1 and both virtual

I added ethtoolOutput2.txt because I had read some blogs that said one of the parameters called rx-vlan-offload: may cause VLAN tag Stripping?

Please let me know if there is anything else you need.

Regards, mike

Mike1111111 · ‎01-12-2021

Hi AlfredoS,

I tried running your program again using a different option and it seemed to work so I am attaching the file All_Information.txt to this post..

[eio@localhost Downloads]$ sudo ./ssu.sh -o=All_Information.txt
[sudo] password for eio:
This product is not supported on this operating system.
Would you like to try to scan? (y/n)
y
Cannot get driver information: Operation not supported

Thanks - mike

AlfredoS_Intel · ‎01-12-2021

Hi Mike1111111,

Thank you providing those information.

Please allow us some time to check on this.

We will get back to you no later than 3 business days from now.

Best Regards,

Alfred S

Intel® Customer Support

AlfredoS_Intel · ‎01-13-2021

Hi Mike1111111,

Thank you for waiting for our update.

While we were closely checking your concern, we find the need to confirm some information:

1. May we know the complete model of your ethernet card which is based on the I210-IT ethernet controller?

2. Was this issue not happening before and it only happened after a recent change on your system?

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel Customer Support

Mike1111111 · ‎01-14-2021

Hi

Thank you, we are using the Osprey VL-EPU-3311 Single Board Computer from Versalogic. Here is the link to the board.

https://www.versalogic.com/product/Osprey/

This is a new SW Feature.

Here is a detailed description of my test setup. But it all breaks down to my original description.

DESCRIPTION:

I have setup two Virtual LAN (VLAN) interfaces (ens35u1.101 and ens35u1.102) off of a parent Physical Ethernet Interface (ens35u1). The physical Interface ens35u1 is connected to a Trunk port of a Layer 2 switch, see diagram below.

I verified that the egress packets from each VLAN interface are tagged with the appropriate VLAN tags and are sent on the Trunk to the Layer 2 Switch:

ens35u1.101 is setup to tag egress packets with a VLAN tag = 101
ens35u1.102 is setup to tag egress packets with a VLAN tag = 102

I would expect the following operation for ingress packets to the VLAN interfaces.

ens35u1.101 will filter ingress packets for a VLAN tag = 101 and ignore all other packets different VLAN tags
ens35u1.102 will filter ingress packets for a VLAN tag = 102 and ignore all other packets different VLAN tags.

PROBLEM:

The actual operation for ingress packets us they are handled by both interfaces because the VLAN Tag is being stripped off by ens35u1.

FULL TEST SETUP DETAILS:

Here is my full test setup, I have two Centos Machines (A and B), they connected through two layer 2 switches (A and B) as shown below:

Each Centos Machine has a single physical Ethernet interface called ens35u1 and eth0, respectively. Off of each Physical interface I set up two "child" VLAN interfaces using VLAN ID's 101, and 102 on both machines.

Centos Machine 'A' VLANS (ens35u1.101 and ens35u1.102)

Centos Machine 'B' VLANS (eth0.101 and eth0.102)

Here is what they look like.

[eio@localhost ~]$ ifconfig

ens35u1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.173 netmask 255.255.255.0 broadcast 192.168.0.255

inet6 fe80::176a:2296:f962:c68c prefixlen 64 scopeid 0x20<link>

ether 50:3e:aa:dc:4f:a6 txqueuelen 1000 (Ethernet)

RX packets 458 bytes 300810 (293.7 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 782 bytes 443914 (433.5 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens35u1.101: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.121 netmask 255.255.255.0 broadcast 192.168.0.255

inet6 fe80::8fd7:549a:c21b:29ed prefixlen 64 scopeid 0x20<link>

ether 50:3e:aa:dc:4f:a6 txqueuelen 1000 (Ethernet)

RX packets 33 bytes 20048 (19.5 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 64 bytes 26646 (26.0 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens35u1.102: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.122 netmask 255.255.255.0 broadcast 192.168.0.255

inet6 fe80::a632:8e05:45a3:524e prefixlen 64 scopeid 0x20<link>

ether 50:3e:aa:dc:4f:a6 txqueuelen 1000 (Ethernet)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 63 bytes 26516 (25.8 KiB)

TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

CONFIGURATION

Centos Machine 'A' is connected to Layer 2 switch 'A' using a TRUNK that allows VLAN ID's 1, 101,and 102 on it.
Switch 'A' splits the VLANS into their own individual TRUNKs using the VLAN ID before sending it to Switch 'B'.
Switch 'B' then combines the individual VLANs into a single TRUNK before sending it to Machine 'B'.
This configuration is mirrored on both A and B sides of the network

OPERATION:

Sending Multicast packets:

Each Machine has two Multicast Transmit threads (one for each VLAN interface 101 and 102) that periodically multicasts packets out each VLAN interface to port 12345.
I can see the individual packets leaving Centos Machine 'A' onto TRUNK 'A' with the appropriate VLAN ID Tag on the packet.
Switch 'A' steers the packets to the INTER-TRUNK dedicated to that particular VLAN ID and is forwarded to Switch 'B'.
Switch 'B' combines the VLAN Packets back onto TRUNK 'B' and onto Centos Machine 'B'

Receiving Multicast packets:

Each Machine has two Multicast Receiver threads (one for each VLAN interface) that joins a multicast group on port 12345 and listens for packets.
When I look over the logs I can see that the receiver threads are capturing packets that have a VLAN ID TAG which do not match the VLAN ID of the interface. In other words eth0.101 is capturing packets with 101 and 102 tag and eth0.102 is capturing packets with 101 and 102 tags. It seems like the interfaces are not filtering based on VLAN ID TAG.

=========================================================================

SIMPLIFIED SCENARIO

To further simplify and explain the test scenario, I removed the Ethernet Cable that connects VLAN ID 102 TRUNK between switches. This way I would only receive packets from VLAN ID 101 at both Centos Machines since there is no longer a path for VLAN ID 102 packets. Everything else is the same. Here is the simplified diagram.

SIMPLIFIED SCENARIO BLOCK DIAGRAM:

NOTES:

Ignoring the operation of VLAN ID 1 (Native VLAN) for now.
Under this setup Machine 'A' will receive only packets for VLAN ID 101 via TRUNK 'A'.

OPERATION:

Using tcpdump, I verified that only packets are tagged with a VLAN ID = 101 are received by ens35u1 interface on Centos Machine 'A'. Those packets are multicast from Centos 7 Machine 'B'. Here is the tcpdump command and a sample packet captured with a VLAN ID Tag = 101.
- sudo tcpdump -nnei ens35u1 -vvv
- 14:53:47.515030 00:04:bf:c4:0b:48 > 33:33:00:00:00:05, ethertype 802.1Q (0x8100), length 361: vlan 101, p 0, ethertype IPv6, (hlim 1, next-header UDP (17) payload length: 303) fe80::204:bfff:fec4:b48.54505 > ff05::5.12345: [udp sum ok] UDP, length 295
  
  192.168.10.201.52203 > 224.0.0.2.32123: [udp sum ok] UDP, length 295
The Multicast Receiver Thread listening on the ens35u1.101 interface processes the packets. Here is the tcpdump command and a sample packet captured without a VLAN ID Tag = 101.
- sudo tcpdump -nnei ens35u1.101 -vvv
- 14:58:17.066621 00:04:bf:c4:0b:48 > 33:33:00:00:00:05, ethertype IPv6 (0x86dd), length 357: (hlim 1, next-header UDP (17) payload length: 303) fe80::204:bfff:fec4:b48.54505 > ff05::5.32123: [udp sum ok] UDP, length 295
  192.168.10.201.52203 > 224.0.0.2.32123: [udp sum ok] UDP, length 295
Unfortunately the Multicast Receiver Thread listening on the ens35u1.102 interface also processes the packets which is incorrect.

Hopefully this information helps!

Thanks - mike

AlfredoS_Intel · ‎01-17-2021

Hi Mike1111111,

Thank you for waiting for our update.

Since your issue is regarding an embedded design, we need to route this thread to correct channel so you will be better assisted by the proper team.

Please wait for their reply within 1 to 2 business days.

Best Regards,

Alfred S

Intel® Customer Support

CarlosAM_INTEL · ‎01-20-2021

Hello, @Mike1111111:

Thank you for contacting Intel Embedded Community.

You should address a reference the networking issues of the cited Operating System (OS) in the channel stated on the following website:

https://forums.centos.org/viewforum.php?f=50

Best regards,

@CarlosAM_INTEL.