- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm trying out mutlicast streaming of uncompressed HD and UHD video uisng VMs and SRIOV with an Intel X520-DA2 NIC (82599 Etherent Control VF).
I can send a mutlicast stream from a VM OK and receive a mutlicast stream on a VM using a SRIOV VF. I'm not using VLANs, just separate MAC/IP address on the same LAN.
If I have 2 VMs on the same host, using separate SRIOV VFs from the same Intel NIC, I can see multicast traffic on both VM interfaces (when only 1 VM has joined the stream).
I have tried turning allmutlicast & promisc mode on/off, but it seems to make no difference.
I can see the traffic using the network stats and using tcpdump in the unjoined VM.
Is this a feature of the NIC, or is there a way to stop traffic going to both VFs/VMs.
For the purpose of testing, I'm using KVM (Virtual Machine Manager) on Ubuntu 16.04, my setup is as follows:
$ lsb_release -a
Ubuntu 16.04.3 LTS, xenial
$ uname -a
Linux ap-z420-0 4.4.0-104-generic # 127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
$ lspci -nn | grep -i 'Ethernet Controller'
06:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
06:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
07:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
07:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
07:10.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
07:10.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
# modinfo ixgbevf
filename: /lib/modules/4.4.0-104-generic/updates/dkms/ixgbevf.ko
version: 4.2.2
license: GPL
description: Intel(R) 10 Gigabit Virtual Function Network Driver
author: Intel Corporation, <</span>mailto:linux.nics@intel.com linux.nics@intel.com>
srcversion: 0F89E29947DD944E62FA118
$ ip link show ens3f0
5: ens3f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:72:70:e0 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:1d:e1:aa, spoof checking on, link-state auto
vf 1 MAC 52:54:00:1d:e1:ab, spoof checking on, link-state auto
Any suggestions gratefully received.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Thank you for posting in Wired Communities. I will further check on this.
Regards,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Please try use below driver to see if this will work for your setup.
https://sourceforge.net/projects/e1000/files/ixgbevf%20stable/ https://sourceforge.net/projects/e1000/files/ixgbevf%20stable/
Feel free to update me.
Thanks,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Sharon,
Thanks for the reply.
How related are the ixgbe and ixgbevf drivers?
My colleagues are uisng netmap (kernel stack bypass) to achieve UHD bit rates on a Linux server. The ixgbe driver is patched with netmap specifc code:
https://github.com/luigirizzo/netmap/tree/master/LINUX/final-patches https://github.com/luigirizzo/netmap/tree/master/LINUX/final-patches
Currently we are using ixgbe-5.2.4 and ixgebvf-4.2.2. It is my understanding that ixgbevf-4.2.2 is unmodified.
Previously I had updated to ixgbevf-4.3.2, but had compatability issues.
Does ixgbevf-4.3.3 require ixgbe-5.3.5 or can it be used with an older ixgbe driver?
Thanks,
Dave B.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Thank you for the reply. The ixgbevf driver is for the virtual function (VF) for the virtual NIC. The virtual function driver rely on the services of the physical driver to handle the operation with global impact. It can only be activated on kernels that support SR-IOV.
It is recommended to upgrade to latest driver version.
Please feel free to update me the result if you proceed to upgrade the driver.
Thanks,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sharon,
i've update to ixgbevf 4.3.3:
$ lsmod | grep ixg
ixgbevf 65536 0
ixgbe_bbcrd 323584 0
vxlan 49152 1 ixgbe_bbcrd
dca 16384 2 ixgbe_bbcrd,ioatdma
netmap 151552 1 ixgbe_bbcrd
ptp 20480 5 ixgbe_bbcrd,e1000e,userclock
$ lsmod | grep ixg
ipstudio-admin@ap-z420-0:~$ modinfo ixgbevf
filename: /lib/modules/4.4.0-104-generic/updates/dkms/ixgbevf.ko
version: 4.3.3
license: GPL
description: Intel(R) 10 Gigabit Virtual Function Network Driver
author: Intel Corporation, <</span>mailto:linux.nics@intel.com linux.nics@intel.com>
srcversion: 2054440C35DA63B66C62480
Again I am running 2 Ubuntu 16.04 VMs, using VMM (KVM) on an Ubuntu 16.04 host with sriov, where both VF are on same physical interface (ens3f0).
$ ip link show ens3f0
5: ens3f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:72:70:e0 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:1d:e1:aa, spoof checking on, link-state auto
vf 1 MAC 52:54:00:1d:e1:ab, spoof checking on, link-state auto
When 1 VM joins the uncompressed HD multicast stream, again I see 1.09Gb/s on the sriov VF network interfaces in both VMs.
Dave B.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Thank you for the update. Let me further check.
Regards,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Further checking, the multicast in SR-IOV is working as designed. X520 has a limited number of Multicast filters in SR-IOV. When the virtual function multicast is enabled in SR-IOV mode, all the VF created will see the multicast traffic. This can be avoided if you can enable the VLAN tags for the virtual functions.
Thanks,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sharon,
I tried adding VLAN tags to each VF, e.g.
$ ip link show ens3f0
5: ens3f0: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 90:e2:ba:72:70:e0 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 52:54:00:1d:e1:aa, vlan 100, spoof checking on, link-state auto
vf 1 MAC 52:54:00:1d:e1:ab, vlan 200, spoof checking on, link-state auto
However, this broke the network connectivity. The VM interface coudl not get an IP address via DHCP
My VMs need to be on the same LAN / subnet and the connection to the host server is not a trunk connection.
Is this what you meant by enabling VLAN tags on the VFs?
I only need 2 to 4 VMs, Is there a way to use whatever mutlicast filtering is available in SRIOV mode?
I have not found any info on doing this.
Thanks,
Dave.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Thank you for the update. Let me further check.
Regards,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
The VLAN configuration should be done before the VF driver
is loaded or the VM is booted. Please try to check the sample command below:
$ ip link set dev vf vlan
For example, the following instructions will configure PF eth0 and the first VF
on VLAN 10.
$ ip link set dev eth0 vf 0 vlan 10
Thanks,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sharon,
Those are the commands I used to set a vlan for each VF.
My understanding is that using separate VLANs for each VF on the same PF also requires the VLANs to be configured in the rest of the network.
This will make the network routing of the mutlicast streams significantly more complex and could cause duplication of streams.
This approach is not really compatible with how our network is set up.
Is there another way of stopping the mutlicast stream going to un-joined VMs?
There are non-Intel NICs that handle mutlicast streams correctly, but I also want rate control per VF.
If the X520 cannot do this , are there other 10G/25G Intel NICs that support netmap, multicast, PF rate control and VF rate control?
Thanks,
Dave.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
Thank you for the reply. Let me further check.
Regards,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Davidpb_RD,
As mentioned earlier this is working as designed. In case you are interested, you may look at the Fortville adapters:
https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/ethernet-x710-brief.pdf
As stated in the product brief " VMDq for Emulated Path: VMDq, enables a hypervisor to represent a single network port as multiple network ports that can be assigned to the individual VMs. Traffic handling is offloaded to the network controller, delivering the benefits of port partitioning with little to no administrative overhead by the IT staff."
"SR-IOV for Direct Assignment: Adapter-based isolation and switching for various virtual station instances enables optimal CPU usage in virtualized environments.
• With up to 128 Virtual Functions (VF)s, each VF can support a unique and separate data path for I/Orelated functions within the PCI Express* hierarchy.
• Use of SR-IOV with a networking device, for example, allows the bandwidth of a single port (function) to be partitioned into smaller slices that can be allocated to specific VMs or guests, via a standard interface.
Hope the above information help.
Regards,
Sharon T
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi David_RD,
Please feel free to update me if there are other inquiries. Thank you.
Regards,
Sharon T
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page