Best way to handle mismatched ixgbe & ixgbevf drivers for SR-IOV

[JLi90]

We run into some issues when setting up a special VM (customized Linux) with Intel SR-IOV support enabled. The issues are:

1. We have to manually set MAC addresses for VFs

2. We have to manually change PF or VF MTU to make PF's is greater than or equal to VF's

Or else no traffic gets through VF interfaces in VM.

- For issue # 1:

We found that the matched ixgbevf for our host Linux's ixgbe driver (version 3.17.3) is newer than the shipped ixgbevf in VM. It sounds like Intel moved the feature of setting VF MAC addresses from ixgbe to ixgbevf (VF MACs are set If I install the matched ixgbevf (version 2.10.3) to the VM).

Can someone confirm my guess or detail why this happens? What is the best way to deal with it?

- For issue # 2:

We found that, for Intel 2p X520 card, I have to make sure PF MTU is greater than or equal to VF MTU.

Is this the limitation added to the newer ixgbe driver? Is this only for 82599-based adapters?

[Patrick_K_Intel1 · Accepted solution]

Thanx for posting to our site.

You must keep your PF and VF drivers aligned, which you have. For the benefit of others, at this time the current driver versions are:

PF – ixgbe driver – version 3.17.3

VF – ixgbevf driver – version 2.10.3

We used to have the PF driver assign a random MAC address to the VF at creation time. Rational was that an application planning to use the VF will assign a valid MAC address. For example – Libvirt will assign a valid MAC address when VF was assigned to the VM using Virt-Manager. This worked well for a while, however as people started to use SR-IOV frequently random MAC address assignment caused udev device naming convention to malfunction during kernel boot time. Starting in ixgbe version 3.14.5 driver assigns MAC address 00:00:00:00:00:00 to VF during VF creation time. This change requires VF driver version 2.8.7 or newer for proper functionality.

Regarding MTU - the MTU of the VF must be less than or equal to the PF when using the Intel X520. This limitation was addressed in the Intel X540 device.

thanx,

Patrick

[JLi90]

Hi Patrick,

Are there any Intel 10Gbps SFP+ adapters existing that do not have the MTU limitation as Intel X540 provides? We cannot test Intel X540 (RJ45 only?) unless we can get a compliant switch (which is not possible for us).

Thanks,

u6006

[Patrick_K_Intel1]

I am afraid that is the only one available at this time.

You can set the MTU of the PF to be larger. Many deployment won't use the PF for traffic anyhow, and only use VF's. You can set the MTU in the PF and any specific VF's you want, while leaving the MTU for other VF's as is.

thanx,

Patrick