SR-IOV with 802.3ad and Redhat KVM

[NSiou]

Hello,

We are planning on installing single host server with 2 x Intel X520 DP 10Gb adapters and installing 20 guest VM's. We have two VLANS and were planning on using one 802.3ad bonded interface for one VLAN and another 802.3ad bonded interface for the other VLAN.

If you take two PF's and bond them in Redhat at the host level, does the Guest OS see that as one VF?

We are trying to understand, would it make sense to bond at the host level or guest level using 802.3ad in order to achieve the best performance and redundancy and be able to assign a dedicated VF to each VM?

Thanks,

Nick

[Mark_H_Intel]

Hi Nick,

I asked one of our local experts on what he would advise. Here is the response:

If you take two PF's and bond them in Redhat at the host level, does the Guest OS see that as one VF?

Answer: PF and VF are independent. Bonding two PF doesn't mean that VFs are bonded also. VFs are virtual PCI-Express devices. VFs show up in host server PCI-Express hierarchy as a device.

We are trying to understand, would it make sense to bond at the host level or guest level using 802.3ad in order to achieve the best performance and redundancy and be able to assign a dedicated VF to each VM?

Answer: I would recommend assigning a VF from each X520 port to a Guest VM and then bond the VFs from with Guest VF. See example below:

Server has one X520 dual port adapter installed. For this example I will use the following naming convention.

• Adapter 1 Port 0 = A1P0

• Adapter 1 Port 1 = A1P1

1. Create desired number of VFs on A1P0 and A1P1.
2. Assign one VF from each A1P0 and A1P1 to a Guest VM.
3. Start Guest VM.
4. Create a channel bond using Linux Channel bonding driver.
5. Create a VLAN tag for the bonded interface.

Now all the traffic flowing over the bonded driver will have the assigned VLAN tag.

You might find the paper at http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/config-qos-with-flexible-port-partitioning.html http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/config-qos-with-flexible-port-partitioning.html useful in comparing the different bonding types used with virtual machines.

I hope this helps.

Mark H

[NSiou]

Thanks Mark, this definitely helps.

In the whitepaper it says the 802.3ad test failed due to anti-spoofing capabilities, is this still the case or has this issue been resolved and can we use 802.3ad?

Also, just to confirm, for tagging the bonded interface is this done by executing the ip link set command on each VF at the kernel level? And is this required even if only one VLAN will be on the bond?

Thanks,

Nick

[Patrick_K_Intel1]

Hi Nick,

Glad the paper helped, always good to know folks actually read stuff you write

The anti-spoofing is on by default as a security measure. You need to use iproute2 (ip) to disable it on each VF you intend to bond with the "spoofchk off" option.

To add VLAN tagging to bonded VF's, you must yest add the tag to each of the bonded VF's. This is because VLAN tagging is checked in hardware of our controllers, while the bonding is a software entity.

Hope this helps, and have fun with SR-IOV!

- Patrick