On SRIOV interface (I350) unable to change the VF state from auto to enable.

[Swapnil]

We are having Dell hardware with

{01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)} type NIC cards.

With reference to Virtualization settings in BIOS

and openstack config and RedHat link, we are able to configure the setup.

We are not able to enable vf state using ip link command.

[root@overcloud-novacompute-0 heat-admin]# modinfo igb

filename:      /lib/modules/3.10.0-957.27.2.el7.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko.xz

version:       5.4.0-k

license:       GPL

description:   Intel(R) Gigabit Ethernet Network Driver

author:        Intel Corporation, <e1000-devel@lists.sourceforge.net>

retpoline:     Y

rhelversion:   7.6

[root@overcloud-novacompute-0 heat-admin]# lsmod | grep igb

igbvf                 46563 0

igb                  214975 0

i2c_algo_bit          13413 2 igb,mgag200

dca                   15130 1 igb

ptp                   19231 1 igb

[root@overcloud-novacompute-0 heat-admin]# ip li sh em3

4: em3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1618 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000

   link/ether 24:6e:96:36:f6:c6 brd ff:ff:ff:ff:ff:ff

   vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off

   vf 1 MAC de:46:05:1b:c8:2a, spoof checking off, link-state auto, trust off

   vf 2 MAC 32:fd:0e:bf:0b:cc, spoof checking off, link-state auto, trust off

   vf 3 MAC 4a:4f:0a:1b:a6:04, spoof checking off, link-state auto, trust off

   vf 4 MAC ae:d9:27:89:a9:50, spoof checking off, link-state auto, trust off

   vf 5 MAC 86:6f:0a:d4:f6:27, spoof checking off, link-state auto, trust off

I'm not sure is anything else is missed out in this configuration or what.

Simple steps I did for configuration are as below:

echo '6' > /sys/class/net/em3/device/sriov_numvfs vi /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on" grub2-mkconfig -o /boot/grub2/grub.cfg rebootecho '6' > /sys/class/net/em3/device/sriov_numvfs vi /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on" grub2-mkconfig -o /boot/grub2/grub.cfg reboot

Let me know if anything else need that help you to identify the problem or i missed out on anything.

[root@overcloud-novacompute-0 heat-admin]# ip link set em3 vf 0 state enable RTNETLINK answers: Operation not supported [root@overcloud-novacompute-0 heat-admin]#

Thanks,

Swapnil

[Caguicla_Intel]

Hello Swapnil,

Thank you for posting in Intel Ethernet Communities. Please provide the following information for us to check on your request.

1.) Exact model of the I350. You may also provide the PBA of the adapter for us to double check on it. It consist of 6-digit number located at the last part of the serial number. Please refer to the link on where to find the PBA number.

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-io/ethernet-products.html

2.) What is the exact igb and igbvf driver version installed on your system? Can you share where is the driver downloaded from?

3.) Have you tried the latest igb and igbvf driver from the link below?

igb Version: 5.3.5.39

https://downloadcenter.intel.com/download/13663/Intel-Network-Adapter-Driver-for-82575-6-82580-I350-and-I210-211-Based-Gigabit-Network-Connections-for-Linux-?product=59679

igbvf Version: 2.3.9.6 

https://downloadcenter.intel.com/download/29140/Network-Adapter-Virtual-Function-Driver-for-Intel-Gigabit-Ethernet-Network-Connections-for-Linux-?product=59679

Looking forward to your response.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Swapnil]

Hello Crisselle,

Thank you for the quick response. Please find the required details inline.

1.) Exact model of the I350. You may also provide the PBA of the adapter for us to double check on it. It consist of 6-digit number located at the last part of the serial number. Please refer to the link on where to find the PBA number.

Exact model of the I350 => I350-T4   PBA Number is : H81148-003 {Whole string is : A0369FD58EE0 436AD H81148-003 }   [root@overcloud-novacompute-0 heat-admin]# lspci -v | grep Ether 01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 01:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 01:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 04:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter I350-T4 04:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter I350-T4 04:00.2 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter I350-T4 04:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter I350-T4 04:10.0 Ethernet controller: Intel Corporation I350 Ethernet Controller Virtual Function (rev 01)     Check if the command output can help

2.) What is the exact igb and igbvf driver version installed on your system? Can you share where is the driver downloaded from?

IGB Version: [root@overcloud-novacompute-0 heat-admin]# modinfo igb filename: /lib/modules/3.10.0-957.27.2.el7.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko.xz version: 5.4.0-k license: GPL description: Intel(R) Gigabit Ethernet Network Driver author: Intel Corporation, <e1000-devel@lists.sourceforge.net>   IGBVF Version: [root@overcloud-novacompute-0 heat-admin]# modinfo igbvf filename: /lib/modules/3.10.0-957.27.2.el7.x86_64/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko.xz version: 2.4.0-k license: GPL description: Intel(R) Gigabit Virtual Function Network Driver author: Intel Corporation, <e1000-devel@lists.sourceforge.net> retpoline: Y rhelversion: 7.6     Driver Source: These are default drivers we got installed with Centos:   [root@overcloud-novacompute-0 heat-admin]# uname -a Linux overcloud-novacompute-0 3.10.0-957.27.2.el7.x86_64 #1 SMP Mon Jul 29 17:46:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

3.) Have you tried the latest igb and igbvf driver from the link below?

igb Version: 5.3.5.39

igbvf Version: 2.3.9.6

I think I'm using the higher versions of drivers than what you suggested.   modinfo igb filename: /lib/modules/3.10.0-957.27.2.el7.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko.xz version: 5.4.0-k     modinfo igbvf filename: /lib/modules/3.10.0-957.27.2.el7.x86_64/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko.xz version: 2.4.0-k   But still Ill give it a try with the versions you specified and see if it solves the problem.

Thanks,

Swapnil

[Caguicla_Intel]

Hello Swapnil,

We appreciate your reply.

Please allow us to check on this. Rest assured that we will get back to you within 1-3 business days. We will also wait for your update once you've tried the driver versions mentioned on our above post.

Hoping for your patience.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Swapnil]

Hello Crisselle,

I tried with the versions mentioned but it is not helping us.

Thanks,

Swapnil

[Caguicla_Intel]

Hello Swapnil,

Thank you for the update.

We are sorry to hear that the latest versions didn't help to fix the issue. We will continue to check on this and get back to you within 1-3 business days.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello Swapnil,

Thank you for the patience on this matter.

Based on the result of the command, it is not supported. Please try to post a query in sourceforge link is down below to double check the issue.

https://sourceforge.net/p/e1000/feature-requests/

Let us know if there is any additional assistance needed form Intel Ethernet team.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Swapnil]

It is working even though ' ip link set em3 vf 0 state enable' command is failing.

with auto mode we can attach interface to the guest and receive packets in the expected manner.

I'm using openstack pike for VM and Network creation.

openstack SRIOV-agent service is detecting failure on this configuration command execution.

Bypassing this configuration in openstack can allow VM spawn and network attach (But it is out of sync port attachment, you need to find ports based on MACs you assigned to them and configure network file(s) accordingly.

[Caguicla_Intel]

Hello Swapnil,

Thank you for the reply.

Please allow us to further check on this. We will send an update as soon as possible but not later than 3 business days.

Hoping for your patience.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello Swapnil,

Apologies for the delay.

Since the command is working and it appeared to be not supported, please submit a bug issue to sourceforge link below.

https://sourceforge.net/p/e1000/bugs/

Let us know if you have additional questions and clarifications on this matter.

Awaiting to your response.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello Swapnil,

Good day!

We'd like to check if you were able to submit a bug issue to sourceforge? In case you have additional questions and clarifications on this matter, please do not hesitate to ask.

Hoping to hear from you soon.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Swapnil]

Hi Crisselle,

I have created issue as you suggested. ID: https://sourceforge.net/p/e1000/bugs/653/

Thanks,

Swapnil

[Mike_Intel]

Hello Swapnil,

Thank you for the confirmation.

Please continue coordinating the issue with them and as for this thread, we will be closing this one now.

Thank you and have a good day.

Best regards,

Michael L.

Intel Customer Support Technicians

A Contingent Worker at Intel