X710 strips incoming vlan tag with SRIOV

SLitk · ‎02-07-2018

Hi,

I have a VM using SRIOV interfaces based on X710 interfaces. This VM is connected to a physical router through the VF.

I would like to enable vlan tagging on the VM.

I can ping the physical router from the VM if I'm not using VLAN tagging on the interface.

When I add VLAN tagging on the interface (on the VM and on the physical router), the network connectivity is broken.

The physical router correctly receives the packets/frames from the VM.

The VM receives the packets/frames but without a VLAN tag, it seems that the VLAN tag has been stripped.

Here are some informations on my setup:

[root@SLI-SRV3 sysadmin]# ethtool -i eth1

driver: i40e

version: 1.6.27-k

firmware-version: 6.00 0x800034e6 18.3.6

expansion-rom-version:

bus-info: 0000:19:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

[root@SLI-SRV3 sysadmin]# ip link show eth1

3: eth1: mtu 1500 qdisc mq portid 246e967eaabe state UP mode DEFAULT qlen 1000

link/ether 24:6e:96:7e:aa:be brd ff:ff:ff:ff:ff:ff

vf 0 MAC 52:54:00:5d:6a:60, spoof checking on, link-state auto, trust off

VIRSH network config:

Any idea about what could be the issue ?

Thanks,

Stephane

idata · ‎02-07-2018

Hi Slitkowski,

Thank you for posting in Wired Communities. What is the exact Linux operating system used here? Please try run the SSU utility and send me the log file.

https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System

Thanks,

Sharon T

SLitk · ‎02-08-2018

I'm running a RHEL 7.4.

# uname -a

Linux SLI-SRV3 3.10.0-693.5.2.el7.x86_64 # 1 SMP Fri Oct 13 10:46:25 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

SSU result is attached.

Thanks

idata · ‎02-08-2018

Hi Slitkowski,

Thank you for the information. Let me further investigate on this.

Regards,

Sharon T

idata · ‎02-16-2018

Hi Slitkowski,

Please be informed I am still investigating on this, will update the post once there is any finding.

Thanks,

Sharon T

idata · ‎02-18-2018

Hi Slitkowski,

Please try check on the information regarding VLAN tag stripping in our READMe file at https://downloadmirror.intel.com/24693/eng/readme.txt https://downloadmirror.intel.com/24693/eng/readme.txt (extracted here for your reference). Try turning off the rxvlan and check if it works.

Setting VLAN Tag Stripping

--------------------------

If you have applications that require Virtual Functions (VFs) to receive

packets with VLAN tags, you can disable VLAN tag stripping for the VF. The

Physical Function (PF) processes requests issued from the VF to enable or

disable VLAN tag stripping. Note that if the PF has assigned a VLAN to a VF,

then requests from that VF to set VLAN tag stripping will be ignored.

To enable/disable VLAN tag stripping for a VF, issue the following command

from inside the VM in which you are running the VF:

ethtool -K rxvlan on/off

or alternatively:

ethtool --offload rxvlan on/off

The MAC address shown is from Dell, can you confirm if the NIC is an embedded NIC on the system? or a standalone NIC comes with Dell system?

Please feel free to update me.

Thanks,

Sharon T

SLitk · ‎02-19-2018

I tried, but it does work. VLAN is still stripped:

[root@SLI-SRV3 sysadmin]# ethtool -k eno1

Features for eno1:

rx-checksumming: on

tx-checksumming: on

tx-checksum-ipv4: on

tx-checksum-ip-generic: off [fixed]

tx-checksum-ipv6: on

tx-checksum-fcoe-crc: off [fixed]

tx-checksum-sctp: on

scatter-gather: on

tx-scatter-gather: on

tx-scatter-gather-fraglist: off [fixed]

tcp-segmentation-offload: on

tx-tcp-segmentation: on

tx-tcp-ecn-segmentation: on

tx-tcp6-segmentation: on

tx-tcp-mangleid-segmentation: off

udp-fragmentation-offload: off [fixed]

generic-segmentation-offload: on

generic-receive-offload: on

large-receive-offload: off [fixed]

rx-vlan-offload: off

tx-vlan-offload: on

ntuple-filters: on

receive-hashing: on

highdma: on

rx-vlan-filter: on [fixed]

vlan-challenged: off [fixed]

tx-lockless: off [fixed]

netns-local: off [fixed]

tx-gso-robust: off [fixed]

tx-fcoe-segmentation: off [fixed]

tx-gre-segmentation: on

tx-ipip-segmentation: on

tx-sit-segmentation: on

tx-udp_tnl-segmentation: on

tx-mpls-segmentation: off [fixed]

fcoe-mtu: off [fixed]

tx-nocache-copy: off

loopback: off [fixed]

rx-fcs: off [fixed]

rx-all: off [fixed]

tx-vlan-stag-hw-insert: off [fixed]

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: off [fixed]

busy-poll: off [fixed]

tx-gre-csum-segmentation: on

tx-udp_tnl-csum-segmentation: on

tx-gso-partial: on

tx-sctp-segmentation: off [fixed]

l2-fwd-offload: off [fixed]

hw-tc-offload: off [fixed]

11:43:12.537863 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 0, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 0, length 80

11:43:14.539047 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 1, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 1, length 80

11:43:16.540645 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 2, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 2, length 80

11:43:18.542226 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 3, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 3, length 80

11:43:20.544015 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 4, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 4, length 80

I have a PCIE X710 Dual port 10G card and also a combo i350 DP+X710 DP as a LOM card.

Both have the same issue.

SLitk · ‎02-19-2018

I found a way to get the VLAN tag.

I need to set a VLAN on the VF and remove it:

[root@SLI-SRV3 images]# ip link set eno1 vf 1 vlan 10

[root@SLI-SRV3 images]# ip link set eno1 vf 1 vlan 0

[root@SLI-SRV3 images]#

15:18:26.214811 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 27, length 80

15:18:28.216687 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 28, length 80

15:18:30.218106 In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 29, length 80

It looks like a bug in the driver.

idata · ‎02-19-2018

Hi Slitkowski,

Thank you for the update and sharing the information, I am glad the issue is fixed as well. Just to double check for the i40e driver, where did you download the driver from? Thank you.

Regards,

Sharon T

SLitk · ‎02-20-2018

I consider this as a workaround rather than a fix.

There is no reason to strip the VLAN by default as the "ip link set eno1 vf 1 vlan 0" normally sets back the vf to its default vlan config.

The driver is the one provided by Redhat. I did not change it.

idata · ‎02-20-2018

Hi Slitkowski,

Thank you for the information. Let me check in case there is information to share.

Regards,

Sharon T

idata · ‎03-01-2018

Hi Slitkowski,

I have forwarded your feedback to our engineering team. Just would like to double check

if there is switch between the router and the X710 NIC?

Regards

Sharon T

SLitk · ‎03-02-2018

No there is no switch, my router that initiates the PING and the server which hosts the VM are connected using a back to back cable.

idata · ‎03-02-2018

Hi Slitkowski,

Thank you for the confirmation.

Regards,

Sharon T

idata · ‎03-06-2018

Hi Slitkowski,

Please be informed that we already share your information with our engineering team and they will determine if a driver update is necessary.

Regards,

Sharon T