Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
255 Views

SR-IOV E810 sending VLAN tagged traffic

Is it possible to send VLAN (802.1Q) tagged traffic on out an SR-IOV virtual function that is itself assigned a VLAN?

For example I have set up 4 virtual functions on a Ubuntu host with a E810-CAM1/2 NIC. These virtual functions are assigned a vlan id so that traffic can be directed to a specific VM to which it is connected based on the VLAN tag. The VLAN tag gets stripped before arriving at the guest OS. When sending a packet from the guest OS a VLAN tag gets pushed onto the packet before being sent out. So far so good.

However, when I try to send a packet from the guest OS that already contains a 802.1Q VLAN tag it does not seem to get sent out. Even the packet statistics for the VF as seen from the host do not seem to increase.

The virtual functions all have spoof checking turned off and trust turned on. Example output for `ip link show`:

 

11: enp59s0f7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:90:0b:86:99:cd brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 52:54:00:d4:c1:95, vlan 301, spoof checking off, link-state auto, trust on
    vf 1 MAC 52:40:63:7e:bc:8e, vlan 302, spoof checking off, link-state auto, trust on
    vf 2 MAC 0e:7f:a5:09:b2:69, vlan 401, spoof checking off, link-state auto, trust on
    vf 3 MAC fa:1b:41:93:2c:7d, vlan 402, spoof checking off, link-state auto, trust on

 

To make promiscuous on the guest OS work correctly, I also had to turn on the 'vf-true-promisc-support' flag using the ethtool interface. Current flags for the interface are:

 

Private flags for enp59s0f7:
link-down-on-close      : off
fw-lldp-agent           : off
vf-channel-filter-as-udp: off
vf-true-promisc-support : on
legacy-rx               : off

 

 

From what I've gathered, maybe the solution would be to use 802.1ad instead of 802.1Q, however the ip-route2 command returns an error when I try this:

 

# ip link set dev enp59s0f7 vf 2 vlan 401 proto 802.1AD
RTNETLINK answers: Protocol not supported

 

I have tried with both the default installed version of the 'ice' driver (0.12.34) and the latest (1.0.4) compiled from source but with the same results.

Full lspci output for the physical function:

 

3b:00.7 Ethernet controller: Intel Corporation Device 1591 (rev 01)
	Subsystem: Intel Corporation Device 0000
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 50
	NUMA node: 0
	Region 0: Memory at 58000000 (64-bit, prefetchable) [size=128M]
	Region 3: Memory at 9c000000 (64-bit, prefetchable) [size=64K]
	Expansion ROM at 9cb00000 [disabled] [size=1M]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
	Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] MSI-X: Enable+ Count=256 Masked-
		Vector table: BAR=3 offset=00000000
		PBA: BAR=3 offset=00008000
	Capabilities: [a0] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 16GT/s, Width x16, ASPM not supported, Exit Latency L0s unlimited, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [e0] Vital Product Data
		Product Name: Intel(r) Ethernet Controller E810-CAM1/2
		Read-only fields:
			[V0] Vendor specific: Intel(r) Ethernet Controller E810-CAM1/2
			[RV] Reserved: checksum good, 0 byte(s) reserved
		End
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [150 v1] Device Serial Number 00-01-00-ff-ff-00-00-00
	Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable+ Migration- Interrupt- MSE+ ARIHierarchy-
		IOVSta:	Migration-
		Initial VFs: 32, Total VFs: 32, Number of VFs: 4, Function Dependency Link: 07
		VF offset: 225, stride: 1, Device ID: 1889
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 0000000098000000 (64-bit, prefetchable)
		Region 3: Memory at 000000009c080000 (64-bit, prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [1a0 v1] Transaction Processing Hints
		Device specific mode supported
		No steering table available
	Capabilities: [1b0 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Capabilities: [200 v1] #25
	Kernel driver in use: ice
	Kernel modules: ice

 

 

OS (host) info:

 

Linux host 5.0.0-36-generic #39~18.04.1-Ubuntu SMP Tue Nov 12 11:09:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 

 

Driver (host) info:

 

filename:       /lib/modules/5.0.0-36-generic/updates/drivers/net/ethernet/intel/ice/ice.ko
firmware:       updates/intel/ice/ddp/ice.pkg
version:        0.12.34
license:        GPL v2
description:    Intel(R) Ethernet Connection E800 Series Linux Driver
author:         Intel Corporation, <linux.nics@intel.com>
srcversion:     74C71245162C3B9484C36B2
alias:          pci:v00008086d00001892sv*sd*bc*sc*i*
alias:          pci:v00008086d00001891sv*sd*bc*sc*i*
alias:          pci:v00008086d00001890sv*sd*bc*sc*i*
alias:          pci:v00008086d00001593sv*sd*bc*sc*i*
alias:          pci:v00008086d00001592sv*sd*bc*sc*i*
alias:          pci:v00008086d00001591sv*sd*bc*sc*i*
depends:        
retpoline:      Y
name:           ice
vermagic:       5.0.0-36-generic SMP mod_unload 
parm:           debug:netif level (0=none,...,16=all) (int)

 

 

Please let me know if I am missing any relevant info to answer my question.

Thanks

 

0 Kudos
10 Replies
Highlighted
196 Views

Hello geenid,


Thank you for posting in Intel Ethernet Communities. 


First of all, we apologize for the delay in our response in this thread.

And before we proceed with, can you confirm if you are using an embedded network controller or a stand alone PCI network card?



If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.


Best regards,

Michael L.

Intel Customer Support Technicians


0 Kudos
Highlighted
Beginner
191 Views

Hello Michael,

That would be an embedded controller.

0 Kudos
Highlighted
186 Views

Hello geenid,


Thank you for the quick response.


Can you also share the motherboard model of the system that you are using or did you design your our system?


If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.


Best regards,

Michael L.

Intel Customer Support Technicians


0 Kudos
Highlighted
Beginner
183 Views

Hello Michael,

It is a custom board indeed, so I can not share too much details about it.

0 Kudos
Highlighted
177 Views

Hello geenid,


Base on your inquiry, we have specific forum for these issues and I will be transferring this thread for faster response.


Best regards,

Michael L.

Intel Customer Support Technicians


0 Kudos
Highlighted
Moderator
153 Views

Hello, @geenid:

Thank you for contacting Intel Embedded Community.

The Intel(R) Ethernet Inspector document # 626451 is the suggested tool to test the affected units. You need to be logged into your Resource & Design Center (RDC) privileged account to find the user guide and this tool on the following website:

http://www.intel.com/cd/edesign/library/asmo-na/eng/626451.htm

The RDC Account Support form is the channel to process your update account or report any problems with the provided site. You should fill it out on the following website:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best Regards,

@Mæcenas_INTEL.

0 Kudos
Highlighted
Beginner
140 Views

Hello Mæcenas,

Thank you for the response.

I don't suspect a problem with the controllers so I'm not sure a test with a diagnostic tool is warranted at this time at least. I would only like to know if what I'm trying to achieve is possible with this particular controller and if so, how it could be achieved with this particular host OS (Ubuntu 18.04)

0 Kudos
Highlighted
Moderator
134 Views

Hello, @geenid:

Thanks for your clarification.

Based on your last communication, we suggest addressing your question to the channel mentioned as a reference on the following website:

https://askubuntu.com/

Best regards,

@Mæcenas_INTEL.

0 Kudos
Highlighted
Beginner
118 Views

Hello @Mæcenas_INTEL ,

Indeed the second part of my question might be better suited somewhere else, however, I think the first part still stands. That is, is it possible to do what I'm trying to do in my original question.

I have recently found this section in the README file for the ice driver:

IEEE 802.1ad (QinQ) Support
---------------------------
The IEEE 802.1ad standard, informally known as QinQ, allows for multiple VLAN
IDs within a single Ethernet frame. VLAN IDs are sometimes referred to as
"tags," and multiple VLAN IDs are thus referred to as a "tag stack." Tag stacks
allow L2 tunneling and the ability to segregate traffic within a particular
VLAN ID, among other uses.

NOTES:
- 802.1ad (QinQ) is supported in 3.19 and later kernels.
- 802.1ad (QinQ) and RDMA are not compatible.
- Receive checksum offloads and VLAN acceleration are not supported for 802.1ad
(QinQ) packets.
- 0x88A8 traffic will not be received unless VLAN stripping is disabled with
the following command:
# ethool -K <ethX> rxvlan off
- 0x88A8/0x8100 double VLANs cannot be used with 0x8100 or 0x8100/0x8100 VLANS
configured on the same port. 0x88a8/0x8100 traffic will not be received if
0x8100 VLANs are configured.
- The VF can only transmit 0x88A8/0x8100 (i.e., 802.1ad/802.1Q) traffic if:
1) The VF is not assigned a port VLAN.
2) spoofchk is disabled from the PF. If you enable spoofchk, the VF will not
transmit 0x88A8/0x8100 traffic.
- The VF may not receive all network traffic based on the Inner VLAN header
when VF true promiscuous mode (vf-true-promisc-support) and double VLANs are
enabled in SR-IOV mode.

It looks like in particular this line:

- The VF can only transmit 0x88A8/0x8100 (i.e., 802.1ad/802.1Q) traffic if:
1) The VF is not assigned a port VLAN.

Implies that I can not send out VLAN tagged traffic if the VF uses VLAN tag packet steering and that what I am trying to do might not be possible, but I am not sure.

Can someone clarify this part for me to see if I am interpreting this correctly?

Thanks,

@geenid 

0 Kudos
Highlighted
Moderator
109 Views

Hello, @geenid:

Thanks for your reply.

The information stated on the following website may answer your question:

https://www.intel.com/content/www/us/en/support/articles/000005722/network-and-i-o/ethernet-products...

Best Regards,

@Mæcenas_INTEL.

0 Kudos