STAG support on XL710?

[NKnut]

Does the Intel XL710 4x10GE card support the processing of VLAN frames that have are tagged with STAGs (TPID == 0x88a8) instead of CTAGs (TPID == 0x8100)? In my lab, I have tried a number of things to enable the processing of STAG frames on the XL710 board, including setting up the ethernet interface with commands such as the one below, but no luck, the interface will still not see the VLAN frames with a TPID of 0x88a8. Frames tagged with CTAGs are processed just fine however.

$ sudo ip link add link enp1s0f0 name enp1s0f0.4004 type vlan protocol 8021.ad id 4004

Also, I have tried to setup the board in both promiscuous mode and true promiscuous mode. Again, no joy, the board still refuse to detect/receive a VLAN tagged frame if the TPID is 0x88a8 instead of 0x8100. By the way, for true promiscuous mode, I am using this command.

$ sudo ethtool --set-priv-flags enp1s0f0 vf-true-promisc-support on

I have the latest Intel firmware installed on this board and am using the latest Intel kernel module for this board.

$ sudo ethtool -i enp1s0f0

[sudo] password for nknuth:

driver: i40e

version: 1.5.16

firmware-version: 5.04 0x800024ca 0.0.0

expansion-rom-version:

bus-info: 0000:01:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

Alas, when I query the features of this ethernet board, it seems clear that STAG frames are not supported.

$ sudo ethtool -k enp1s0f0

...

tx-vlan-stag-hw-insert: off [fixed]

rx-vlan-stag-hw-parse: off [fixed]

rx-vlan-stag-filter: off [fixed]

...

Any help is appreciated. It really is a show stopper for my company to not be able to process STAGs on the XL710 boards. We were using the 2 port 10GE boards that use the ixgbe kernel module and STAG frames were processed on those boards without issue.

Thanks in advance for any clarity here.

Nathan

[idata]

Hi Nathan,

Thank you for the detail information. I need to verify this first. I will update you once there is any finding.rgds,wb

[NKnut]

Thanks, am very interested to know if STAGs can be received.

Nathan

[idata]

HI Nathan,

This is noted. rgds,wb

[VIgna1]

Hi, Intel.

We have the same issue with DPDK. We cannot receive any packets which are tagged with 0x88a8. Any other is ok. So it seems that they are filtrated inside your firmware? Furthermore, we can see that they are received but then disappear somewhere; For example, this statistics from registers:

rx_good_packets = 127rx_good_bytes = 98087rx_unicast_packets = 127rx_unknown_protocol_packets = 127rx_size_65_to_127_packets = 6rx_size_128_to_255_packets = 12rx_size_256_to_511_packets = 24rx_size_512_to_1023_packets = 43rx_size_1024_to_1522_packets = 42

But DPDK don't received any packets, so I can't get them to analyse.

The same APP is working perfectly now with 82599 cards. Please clarify this situation and possible fix dates cause we have our project stalling due to this issue.

Regards,

Vitaly

[idata]

Hi Vitaly,

Currently we are investigating on this. will update once there is any finding. rgds,wb

[NKnut]

Hi Intel, any update at all on this? Is it addressable with a firmware update? My management is starting to get nervous.

Thanks,

Nathan

[idata]

Hi Nathan,

Apologize for any delay. We are still checking on this, rest assured will update you once there is any finding. rgds,wb

[idata]

Hi Nathan,

Here is an option you can try if you want to make changes to the device registers: 1) It is recommended you try this first with J3 Jumper installed on Intel NIC as GL_SWT_L2TAGCTRL is RO to PF driver. 2) Then perform the following: -Clear bit 1 of PRT_L2TAGSEN.ENABLE. -Set GL_SWT_L2TAGCTRL[2].ETHERTYPE to 0x88A8. Note that this is a global setting for the device. All ports must use the same TPID. 3) you may refer to the registers in v2.5 of the datasheet below: http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-10-40-controller-datasheet.htmlHope this could be of help.rgds,wb

[VIgna1]

Hello, Intel.

We tried your advice but nothing changed.

First of all, we tested without Jumper installed cause we could rewrite register. Additionally, we tested with installed Jumper.

We cleared bit 1 (which represents STAG (?) in register PRT_L2TAGSEN.ENABLE), so there bit 3 only was set (Inner VLAN).

As you mentioned we changed Ethertype value for Outer Vlan (?) from 0x8100 to 0x88a8. And that didn't help again.

As we see, bytes increase on each packet but amount of packets is 0:

0 packets input, 23824395988 bytes, 3 input errors

0 64 bytes size packets, 10145870 65-127 bytes size packets

0 128-255 bytes size packets, 0 256-511 bytes size packets

10016066 512-1023 bytes size packets, 0 1024-1522 bytes size packets

0 local MAC error, 1 remote MAC errors

0 no buffer (RX queue is full), 0 no buffer alloc (RX buffer allocation failure)

Should notice, we have Intel X710DA4 card. But I think there is no difference. And we use DPDK 16.07.

And one more question, can we completely disable any firmware handling of any tags?

Regards,

Vitaly.

[idata]

Hi Vitaly,

Thank you for the update for us to understand the situation. Let me further check.rgds,wb

[VIgna1]

Hi Intel,

We have played a little with registers and found the correct way to enable handling. It is neccessary to clear bit 1 in PRT_L2TAGSEN.ENABLE (for STag handling). It was completely dependent on the function of register write call. We used incorrect function for pci register writing using admin queue instead of direct register writing. So now it's working as you noticed.

Thanks for your help!

Regards,

Vitaly.

[idata]

Hi Vitaly,

Thank you for sharing the good news and information.rgds,wb

[idata]

Hi Nathan,

Good day. We are still in the process of checking the solution, for the meantime, can you provide us your expectation about the fix. IF there would be a fix that enabled this via FW admin queue, will that be sufficientor are you expecting it to be default operation of the device after the driver is loaded.Thankswb

[idata]

Hi Nathan,

Good day. Please feel free to update me, we would like to get your expectation about the fix. rgds,wb

[NKnut]

Hi Vitaly,

Do you have a simple script that you can share with the recommended settings (from Intel) that I can run on my linux host to confirm that STAGs are working with my XL710 board? Thanks for your help driving this issue forward.

Nathan

[NKnut]

wb, (from Intel)

I just received a customer survey email from Intel regarding this issue and the fact that this issue has been resolved? What?! I have a hard time believing that this issue has been resolved. At best, it has been recreated, verified, and a workaround has been confirmed. The ability to process VLANs that have an S-TAG TPID is fundamental. If the only way to process these TPIDs is to use some register hacks that you provided, then we have a very different opinion of a functional device.

Please provide us the status on the real fix for this issue and the timeline for such a fix.

Thanks,

Nathan

[idata]

Hi Nathan,

Thank you for the update. Here is our suggestion about the registration and I understand the registration work well with Vitaly:GL_SWT_L2TAGCTRL[2] 88A82882 PRT_L2TAGSEN 0000000C But I will further check for you about the fix. May I know what is the NVM version you are using? My apologies for any inconvenience this might have caused.Thankswb

[VIgna1]

Hi Nathan,

Sorry to say, but I haven't any scripts to change registers of NIC. As I said before, we used DPDK and we just added few lines to the dev_init function of DPDK i40e driver sources.

Regards,

Vitaly.