- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is it possible to send VLAN (802.1Q) tagged traffic on out an SR-IOV virtual function that is itself assigned a VLAN?
For example I have set up 4 virtual functions on a Ubuntu host with a E810-CAM1/2 NIC. These virtual functions are assigned a vlan id so that traffic can be directed to a specific VM to which it is connected based on the VLAN tag. The VLAN tag gets stripped before arriving at the guest OS. When sending a packet from the guest OS a VLAN tag gets pushed onto the packet before being sent out. So far so good.
However, when I try to send a packet from the guest OS that already contains a 802.1Q VLAN tag it does not seem to get sent out. Even the packet statistics for the VF as seen from the host do not seem to increase.
The virtual functions all have spoof checking turned off and trust turned on. Example output for `ip link show`:
11: enp59s0f7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 00:90:0b:86:99:cd brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:d4:c1:95, vlan 301, spoof checking off, link-state auto, trust on
vf 1 MAC 52:40:63:7e:bc:8e, vlan 302, spoof checking off, link-state auto, trust on
vf 2 MAC 0e:7f:a5:09:b2:69, vlan 401, spoof checking off, link-state auto, trust on
vf 3 MAC fa:1b:41:93:2c:7d, vlan 402, spoof checking off, link-state auto, trust on
To make promiscuous on the guest OS work correctly, I also had to turn on the 'vf-true-promisc-support' flag using the ethtool interface. Current flags for the interface are:
Private flags for enp59s0f7:
link-down-on-close : off
fw-lldp-agent : off
vf-channel-filter-as-udp: off
vf-true-promisc-support : on
legacy-rx : off
From what I've gathered, maybe the solution would be to use 802.1ad instead of 802.1Q, however the ip-route2 command returns an error when I try this:
# ip link set dev enp59s0f7 vf 2 vlan 401 proto 802.1AD
RTNETLINK answers: Protocol not supported
I have tried with both the default installed version of the 'ice' driver (0.12.34) and the latest (1.0.4) compiled from source but with the same results.
Full lspci output for the physical function:
3b:00.7 Ethernet controller: Intel Corporation Device 1591 (rev 01)
Subsystem: Intel Corporation Device 0000
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 50
NUMA node: 0
Region 0: Memory at 58000000 (64-bit, prefetchable) [size=128M]
Region 3: Memory at 9c000000 (64-bit, prefetchable) [size=64K]
Expansion ROM at 9cb00000 [disabled] [size=1M]
Capabilities: [40] Power Management version 3
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Address: 0000000000000000 Data: 0000
Masking: 00000000 Pending: 00000000
Capabilities: [70] MSI-X: Enable+ Count=256 Masked-
Vector table: BAR=3 offset=00000000
PBA: BAR=3 offset=00008000
Capabilities: [a0] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- FLReset-
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM not supported, Exit Latency L0s unlimited, L1 <4us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR-, OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
Capabilities: [e0] Vital Product Data
Product Name: Intel(r) Ethernet Controller E810-CAM1/2
Read-only fields:
[V0] Vendor specific: Intel(r) Ethernet Controller E810-CAM1/2
[RV] Reserved: checksum good, 0 byte(s) reserved
End
Capabilities: [100 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
Capabilities: [148 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [150 v1] Device Serial Number 00-01-00-ff-ff-00-00-00
Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable+ Migration- Interrupt- MSE+ ARIHierarchy-
IOVSta: Migration-
Initial VFs: 32, Total VFs: 32, Number of VFs: 4, Function Dependency Link: 07
VF offset: 225, stride: 1, Device ID: 1889
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 0000000098000000 (64-bit, prefetchable)
Region 3: Memory at 000000009c080000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [1a0 v1] Transaction Processing Hints
Device specific mode supported
No steering table available
Capabilities: [1b0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
Capabilities: [200 v1] #25
Kernel driver in use: ice
Kernel modules: ice
OS (host) info:
Linux host 5.0.0-36-generic #39~18.04.1-Ubuntu SMP Tue Nov 12 11:09:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Driver (host) info:
filename: /lib/modules/5.0.0-36-generic/updates/drivers/net/ethernet/intel/ice/ice.ko
firmware: updates/intel/ice/ddp/ice.pkg
version: 0.12.34
license: GPL v2
description: Intel(R) Ethernet Connection E800 Series Linux Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: 74C71245162C3B9484C36B2
alias: pci:v00008086d00001892sv*sd*bc*sc*i*
alias: pci:v00008086d00001891sv*sd*bc*sc*i*
alias: pci:v00008086d00001890sv*sd*bc*sc*i*
alias: pci:v00008086d00001593sv*sd*bc*sc*i*
alias: pci:v00008086d00001592sv*sd*bc*sc*i*
alias: pci:v00008086d00001591sv*sd*bc*sc*i*
depends:
retpoline: Y
name: ice
vermagic: 5.0.0-36-generic SMP mod_unload
parm: debug:netif level (0=none,...,16=all) (int)
Please let me know if I am missing any relevant info to answer my question.
Thanks
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello geenid,
Thank you for posting in Intel Ethernet Communities.
First of all, we apologize for the delay in our response in this thread.
And before we proceed with, can you confirm if you are using an embedded network controller or a stand alone PCI network card?
If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.
Best regards,
Michael L.
Intel Customer Support Technicians
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Michael,
That would be an embedded controller.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello geenid,
Thank you for the quick response.
Can you also share the motherboard model of the system that you are using or did you design your our system?
If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.
Best regards,
Michael L.
Intel Customer Support Technicians
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Michael,
It is a custom board indeed, so I can not share too much details about it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello geenid,
Base on your inquiry, we have specific forum for these issues and I will be transferring this thread for faster response.
Best regards,
Michael L.
Intel Customer Support Technicians
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, @geenid:
Thank you for contacting Intel Embedded Community.
The Intel(R) Ethernet Inspector document # 626451 is the suggested tool to test the affected units. You need to be logged into your Resource & Design Center (RDC) privileged account to find the user guide and this tool on the following website:
http://www.intel.com/cd/edesign/library/asmo-na/eng/626451.htm
The RDC Account Support form is the channel to process your update account or report any problems with the provided site. You should fill it out on the following website:
https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html
Best Regards,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello Mæcenas,
Thank you for the response.
I don't suspect a problem with the controllers so I'm not sure a test with a diagnostic tool is warranted at this time at least. I would only like to know if what I'm trying to achieve is possible with this particular controller and if so, how it could be achieved with this particular host OS (Ubuntu 18.04)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, @geenid:
Thanks for your clarification.
Based on your last communication, we suggest addressing your question to the channel mentioned as a reference on the following website:
Best regards,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello @CarlosAM_INTEL ,
Indeed the second part of my question might be better suited somewhere else, however, I think the first part still stands. That is, is it possible to do what I'm trying to do in my original question.
I have recently found this section in the README file for the ice driver:
IEEE 802.1ad (QinQ) Support
---------------------------
The IEEE 802.1ad standard, informally known as QinQ, allows for multiple VLAN
IDs within a single Ethernet frame. VLAN IDs are sometimes referred to as
"tags," and multiple VLAN IDs are thus referred to as a "tag stack." Tag stacks
allow L2 tunneling and the ability to segregate traffic within a particular
VLAN ID, among other uses.
NOTES:
- 802.1ad (QinQ) is supported in 3.19 and later kernels.
- 802.1ad (QinQ) and RDMA are not compatible.
- Receive checksum offloads and VLAN acceleration are not supported for 802.1ad
(QinQ) packets.
- 0x88A8 traffic will not be received unless VLAN stripping is disabled with
the following command:
# ethool -K <ethX> rxvlan off
- 0x88A8/0x8100 double VLANs cannot be used with 0x8100 or 0x8100/0x8100 VLANS
configured on the same port. 0x88a8/0x8100 traffic will not be received if
0x8100 VLANs are configured.
- The VF can only transmit 0x88A8/0x8100 (i.e., 802.1ad/802.1Q) traffic if:
1) The VF is not assigned a port VLAN.
2) spoofchk is disabled from the PF. If you enable spoofchk, the VF will not
transmit 0x88A8/0x8100 traffic.
- The VF may not receive all network traffic based on the Inner VLAN header
when VF true promiscuous mode (vf-true-promisc-support) and double VLANs are
enabled in SR-IOV mode.
It looks like in particular this line:
- The VF can only transmit 0x88A8/0x8100 (i.e., 802.1ad/802.1Q) traffic if:
1) The VF is not assigned a port VLAN.
Implies that I can not send out VLAN tagged traffic if the VF uses VLAN tag packet steering and that what I am trying to do might not be possible, but I am not sure.
Can someone clarify this part for me to see if I am interpreting this correctly?
Thanks,
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello, @geenid:
Thanks for your reply.
The information stated on the following website may answer your question:
Best Regards,
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page