SR-IOV Virtual Function spoof check not persistent across VM reboots

[Dudeson]

Configuration Details:

1. Intel x710-T4 NIC running NVM firmware 9.4
   NOTE: the same behavior also existed on NVM 7.10
2. vSphere 8.0 U2 running the matched i40en driver (2.7.2.0-1OEM.800.1.0.20613240) as per the VMware hardware compatibility list
3. Each of the 4 physical ports is configured with a single virtual function
4. pfSense 2.7.2 CE clean install on a brand new VM
5. pfSense is using the 3.0.26-k in-box iavf kernel driver for x710 SR-IOV virtual network devices

While I can disable "Spoof Check" and enable "Trusted" mode in ESXi using the following command, the setting does not persist across reboots.

esxcli intnet sriovnic vf set -n vmnic3 -v 0 -s false -t true

This is a bug in the Intel i40en driver documented here: https://kb.vmware.com/s/article/74909

As per that VMware KB article the bug was fixed by Intel in the v1.9.5 driver, and the linked release notes agree that it was fixed.

Can somebody from Intel Support please tell me why is this behavior still occurring 5 years after it was supposedly fixed?

[Azeem_Intel]

Hi Dudeson,

Greetings for the day!

I hope this message finds you well. Could you please share a snapshot of the part with the serial number visible? Additionally, please let us know where you obtained this driver update.

Please do not hesitate to contact us; we are more than glad to assist you.

Best Regards,

Azeem_Intel

[Dudeson]

The ESXi driver is linked on the VMware hardware compatibility website and downloaded from VMware Customer Connect.  As far as I am aware the driver is provided by Intel directly and is simply hosted by VMware for convenience.

I'm not able to get a picture of the card at this time.

[Azeem_Intel]

Hi Dudeson,

Greetings for the day!

Hope you are doing well. If you visit the Intel site, you will find the updated driver for the X710-T4 NIC.

Please do not hesitate to contact us; we are more than glad to assist you.

Best Regards,

Azeem_Intel

[Dudeson]

@Azeem_Intelwith all due respect I don't think you're familiar with how drivers for VMware ESXi work...

Here's what happens when you attempt to download the driver for VMware ESXi from www.intel.com:

1. Go to Intel® Ethernet Adapter Complete Driver Pack
2. Read the release notes (Intel® Ethernet Controller Products Release Notes)
3. You'll find the text below within the release notes at the bottom of page #11:

1.4.3 ESXi Drivers
Note: Intel® ESXi drivers are available from VMware.
• VMWare ESXi 8.0
• VMware ESXi 7.0
Refer to VMWare's download site for the latest ESXi drivers for Intel ® Ethernet® devices

I have provided to you in my previous post a link to the VMware download site.

In short: I am already running the latest available driver and the driver was downloaded as per formal instructions provided directly by Intel.

Thus I once again ask you to please investigate what is wrong with the NVM (firmware) or driver that is causing this behavior.

[Sachinks]

Hello Dudeson,

Greetings!

We understand that "Spoof Check" and enable "Trusted" mode in ESXi is not persisting after reboots and We see that you are in 2.7.2.0 NIC driver which is the latest. 

We will check this with our internal team and we will get back to you with an update at the earliest. 

Regards,
Sachin KS

[Dudeson]

Thank you!

[Azeem_Intel]

Hi Dudeson,

Greetings for the day!

Could you please let us know if this product was purchased separately or if it was shipped with the system?

Please do not hesitate to contact us; we are more than glad to assist you.

Best Regards,

Azeem_Intel

[Dudeson · Accepted solution]

You can close this thread.  I discovered the fix...unfortunately this is much better documented for Palo Alto, Juniper, Cisco, etc. than it is for pfSense.

What is needed to enable persistent trusted mode on virtual functions in ESXi is the following command:

esxcli system module parameters set -a -m i40en -p trust_all_vfs=1

As per release notes for later versions of the i40en driver, it is by design that trusted mode is not persistent across reboots because the host may assign a different VF to a VM when it is rebooted--the only way to ensure that any VF assigned to the VM is trusted is to trust all VFs.

[Azeem_Intel]

Hi Dudeson,

Greetings for the day!

Thank you for your response. We will proceed to close the case for now. If you require any further assistance, please do not hesitate to let us know. Thank you!

Best Regards,

Azeem_Intel