82599 VF to Linux host bridge

[JHall6]

We are configuring VFs for our high-volume network traffic, but want to continue to use Linux bridging for other VMs. Is there some configuration we need to do to enable packets from a VF to be switched to a bridge off the eth0 PF?

We create several VMs. One of them has a high volume of traffic, as it serves as a load balancer. It takes external traffic on one VLAN, and distributes it to other VMs on a different VM. If the different VM is on the same host, the packets are never switched in the 82599 back to the eth0 for feeding into the Linux bridge. We are using Red Hat EL 6.4. We have been able to get VM to VM traffic working on the same host, if both VMs use VFs, but are unable to get VM to VM traffic if one is using VF and the other is using bridges.

Suggestions?

[Patrick_K_Intel1]

Hi John,

Thanx for posting to our site.

You are experiencing this issue because the VF's are connected to a kind of small, light-weight 'switch' within the Intel(r) 82599. When the Hypervisor creates a VF and a MAC address is assigned to it (either automatically or manually) this 'switch' is notified via the driver.

With this internal switch, VF's can talk to VF's because when one VF transmits a packet and the destination MAC address matches another VF the 'switch' will automatically send it to the VF rather than out to the wire.

However VM's and bridges that are using emulated ethernet devices do not have their MAC addresses added to this 'switch'. That is the source of your issue.

I'm doing some digging - I believe there have been, or is soon to be, some updates that will allow this kind of traffic to be passed between a VF and an emulated path.

- Patrick

[Patrick_K_Intel1]

Found some more data:

The functionality you are after may not be included in RHEL 6.4. Our development team is not

sure which patches from upstream kernel RedHat team has back ported to RHEL 6.4 kernel.

This functionality is included in upstream kernel. I'd suggest updating your kernel to the 3.8.5

available at kernel.org. You will also need to update IPROUTE2 utility to latest version. You will need to add the VF and PF mac addresses to bridge forwarding database using IPROUTE2 utility's bridgecommands.

Example steps below:

• Create 10 VF on eth0 (assuming eth0 is a Intel 82599port). Each VF all will have a macaddress. You can find the VF MAC address by executing "ip link showeth0".
• Create a Linux bridge using brctl command.
• Add eth0 interface to the newly created bridge using "brctl addif switchname eth0"
• Use "bridge fdb add" command to add VF mac addresses and eth0 mac address to bridge forwardingdatabase.

Latest ixgbe & ixgbevf drivers are included in stable 3.8.5 upstream kernel.

Hope this helps!

- Patrick

[JHall6]

Thanks for the information, Patrick. We will try just updating the drivers and iproute2 first, and try updating the kernel second.

[Patrick_K_Intel1]

Sure thing. If you can come back and let us know how it went, would be appreciated.

- Patrick

[JHall6]

We verified that the proper support is not in Red Hat EL 6.4. We were able to get a newer version of the driver, which helped.

[Patrick_K_Intel1]

Great! I love success stories. Thanx for posting the update!

- Patrick