SR-IOV with IXGBE - Vlan packets getting spoofed

[Pmaru]

Hi All,

I am using RHEL7.3 with Intel-82599ES nic cards to launch VMs with SRIOV enabled nic cards. I am using configuring only one VF per PF. I am configuring this VF with vlan, trust mode on and disabling spoof chk.

But, when I am sending vlan tagged packets from Guest VM, I can see the "spoofed packet detected" message in dmesg for this PF card.

We have also disabled the rx/tx vlan offload using ethtool command.

Here are setup details:

Kernel version

# uname -r

3.10.0-514.el7.x86_64

PF/VF configuration:

# ip link show eth2

4: eth2: mtu 9192 qdisc mq state UP mode DEFAULT qlen 1000

link/ether 90:e2:ba:a5:98:7c brd ff:ff:ff:ff:ff:ff

vf 0 MAC fa:16:3e:73:12:6c, vlan 1500, spoof checking off, link-state auto, trust on

IXGBE version

# ethtool -i eth2

driver: ixgbe

version: 4.4.0-k-rh7.3

firmware-version: 0x61bd0001

expansion-rom-version:

bus-info: 0000:81:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: no

Messages from dmesg

[441100.018278] ixgbe 0000:81:00.0 eth2: 3 Spoofed packets detected

[441102.022383] ixgbe 0000:81:00.0 eth2: 2 Spoofed packets detected

[441104.026460] ixgbe 0000:81:00.0 eth2: 3 Spoofed packets detected

[441106.030516] ixgbe 0000:81:00.0 eth2: 2 Spoofed packets detected

LSPCI output

# lspci -nn | grep Ether | grep 82599

81:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

81:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)

81:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)

Ethtool -k output

# ethtool -k eth2 | grep vlan

rx-vlan-offload: off

tx-vlan-offload: off

rx-vlan-filter: on

vlan-challenged: off [fixed]

tx-vlan-stag-hw-insert: off [fixed]

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: off [fixed]

Please let me know, if you any need any other information.

Regards

Pratik

[idata]

Hi Pratik,

Thank you for posting in Wired Ethernet Community.Kindly refer to the thread below for suggestions related to your concern.https://communities.intel.com/thread/73179?start=15&tstart=0 https://communities.intel.com/thread/73179?start=15&tstart=0regards,Vince

[Pmaru]

Hello Vince,

Thanks for the reference thread. Though the thread discusses about my use case, but eventually deviates from it without proposing any solution.

Can you please check and let me know, if there is any workaround to send tagged packets from VM ?

Regards

Pratik

[idata]

Hi Pratik,

Thank you for the update. Just to double check are you referring that you tried already the suggestion provided by RaviKB dated July but this did not work for you. If that is the case, can you share more information after you applied the suggestion. Thank you.

Rgds,

sharon

[Pmaru]

Hello Sharon,

In my understanding, RaviKB is not using any vlan tags for VF and hence he was able to send traffic. Please correct me, if my understanding is incorrect.

But, I have a requirement where I need my VF to have vlan tag.

Thanks

Pratik

[idata]

Hi Pratik,

Thank you for the clarification. Based on the post from Ravi dated July 7, 2015 at 1:16 AM. He was able to receive packets w/ Vlan tag in his virtual machine after he modified the code. In case this is not applicable to your case, please share more information below: • Host kernel and dmesg logs.• Guest kernel and dmesg logs.• Detailed setup instructions for reproducing the issue in house. Looking forward to your update. Thanks. rgds,sharon

[Pmaru]

Hello Sharon,

I am attaching below message from Ravi where he describes his solution, there you see, he is working with VF's which doesn't have vlan tags.

====

Shaham,

As I mentioned in my earlier mails, my requirement was to get the VLAN tagged packets in the VM and VM sending out fully tagged packet. For sending out VLAN tagged packet, I disabled the spoof check using ip link command and did not add any VLAN tag to interface (VF). However, for receiving the VLAN tagged packet into the VM, I had to change a little in ixgbe driver and forcefully disable VLAN filtering.

In ixgbe_main.c in the function: "void ixgbe_set_rx_mode(struct net_device *netdev)" before writing to VLANCTRL register at the end of function, I have added:

/* Forcefully Disable VLAN Filtering in VLANCTRL */

vlnctrl &= ~(IXGBE_VLNCTRL_VFE | IXGBE_VLNCTRL_CFIEN);

Hope it helps.

regards,

Ravi

======

Coming back to my requirement, what I am looking for is sending out tagged packets from Guest OS, whereas at the same time my VF also has vlan tag configured.

This could either behave as q-in-q where vlan tag from guest will be seen as inner tag and tag from VF will be seen as outer tag, when packet comes out of nic card, or it can have the both the guest and host tag can be same and VF doesn't add any extra tag, so that packet coming out of nic will have one tag sent from guest.

To reproduce this internally, you can just configure the VF with vlan tag, and send tagged packets from Guest. Let me know, if that works for you. For me, as I send tagged packets from guest (when VF is also tagged), I can see dmesg logs for detected spoofed packets, and no packets comes out of nic card.

Let me know, if you need any more info in addition to what I have already provided in problem description.

Thanks

Pratik

[idata]

Hi Pratik,

Thank you for the information. I will further check on this.rgds,sharon

[idata]

Hi Pratik

Please try update the following driver:1) ixgbe version 5.1 https://downloadcenter.intel.com/download/14687/Intel-Network-Adapter-Driver-for-PCIe-Intel-10-Gigabit-Ethernet-Network-Connections-Under-Linux-2) ixgbevf version 4.1 https://downloadcenter.intel.com/download/18700/Intel-Network-Adapter-Virtual-Function-Driver-for-Intel-10-Gigabit-Ethernet-Network-ConnectionsFeel free to update me.rgds,sharonThanks,sharon

[idata]

Hi Pratik,

Please feel free to update me if the driver update help resolve the issue. Looking forward to your reply. Thank you.regards,sharon

[Pmaru]

Hello Sharon,

Yes, I am looking into testing this. Wis update you, once I am done testing. This might take some time, as I have to rebuild my setup.

Thanks

Pratik

[idata]

Hi Pratik,

Thank you for the the update. I will follow up with you next week in case there is any progress. Thank you.regards,sharon

[idata]

Hi Pratik,

Is there any update for your test result? Please feel free to update me. Thanks.regards,sharon