- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We are attempting to use XL710 together with Openstack, and acceleration using SR-IOV.
With the base drivers provided by Ubuntu 14, VM's can be created but during boot cloud-init fails to create any interfaces with the following error message:
[ 7.562357] cloud-init[395]: ret = functor(name, args)
[ 7.563806] cloud-init[395]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 364, in main_init
[ 7.568756] cloud-init[395]: init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
[ 7.626863] cloud-init[395]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in apply_network_config
[ 7.627928] cloud-init[395]: netcfg, src = self._find_networking_config()
[ 7.628708] cloud-init[395]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 622, in _find_networking_config
[ 7.632059] cloud-init[395]: if self.datasource and hasattr(self.datasource, 'network_config'):
[ 7.636062] cloud-init[395]: File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", line 150, in network_config
[ 7.644072] cloud-init[395]: self.network_json, known_macs=self.known_macs)
[ 7.705553] cloud-init[395]: File "/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 652, in convert_net_json
[ 7.706582] cloud-init[395]: raise ValueError("Unable to find a system nic for %s" % d)
[ 7.707395] cloud-init[395]: ValueError: Unable to find a system nic for {'mtu': 1500, 'type': 'physical', 'mac_address': 'fa:16:3e:c8:81:d9', 'subnets': [{'routes': [], 'type': 'static', 'address': '10.0.20.6', 'netmask': '255.255.255.0', 'ipv4': True}]}
When installing the latest drivers available from Intel, VM's can not be created at all. Looking at the OpenStack logs, this error message can be found: "internal error: couldn't find IFLA_VF_INFO for VF 7 in netlink response". Searching for this error on google it seems like there could be problems when creating more than 30 VF's in some older versions of libvirt, but we are only testing with 8 VF's right now.
We are running Ubuntu 14 and Openstack Mitaka. We have tried a later Ubuntu/Openstack version (Ubuntu 16 and OpenStack Newton) but there were problems there too -- unsure if it was the same issue, since the logs are long gone.
We have the latest drivers and firmware updates installed. We do -not- have the i40evf driver installed since that would conflict with the VM claiming VF's according to this document https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/xl710-sr-iov-config-guide-gbe-linux-brief.pdf https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/xl710-sr-iov-config-guide-gbe-linux-brief.pdf .
When configuring SR-IOV we are following the guide on https://docs.openstack.org/mitaka/networking-guide/config-sriov.html https://docs.openstack.org/mitaka/networking-guide/config-sriov.html
We have some other Intel cards (I can't remember the model) which this works fine for.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Thank you for posting in Wired Communities. Please provide below information:
1) What is the firmware of your XL710?
2) Please provide the serial number, you may refer to this website for the serial number location on the physical NIC
https://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000007022.html
The format is 15 digits + 6 digits + 6 digits - 3 digits
Thanks,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
Thanks for the reply.
More specifically we have XL710-QDA2 cards.
The firmware is the latest from the Intel website, from ethtool in Linux:
root # ethtool -i ens2f0
driver: i40e
version: 1.5.19
firmware-version: 6.01 0x800034a4 1.1747.0
The serial numbers for the cards are:
3CFDFEAFDB48
008 3317AD
J11363-003
3CFDFEAFE2F0
008 3317AD
J11363-003
3CFDFEAFECF0
008 3317AD
J11363-003
/Tomas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Thank you for the information. I will further check on this.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Please help verify if you updated to the latest driver version at website below:
https://sourceforge.net/projects/e1000/files/i40e%20stable/
Looking forward to your reply. Thank you.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes we were running the latest drivers (2.1.26). (the ethtool output above was taken after an OS reinstall, before drivers were installed)
/Tomas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Thank you for the confirmation. Will further check for you.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
Any further progress?
I can mention that our working 10Gb cards uses the "ixgbe" drivers, and in the VM the virtual functions get the "ixgbevf" driver. They are otherwise configured in the same way as the XL710-QDA2 cards
EDIT: I just did some more testing and noticed that the i40evf driver was actually loaded, which probably was why we got the "internal error: couldn't find IFLA_VF_INFO for VF 7 in netlink response" error. We had missed to blacklist it on the server it seems. But with the module blacklisted, we still get this error from cloud-init (same as the original post with the older drivers):
[ 7.818035] cloud-init[791]: Traceback (most recent call last):
[ 7.818915] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 650, in status_wrapper
[ 7.820120] cloud-init[791]: ret = functor(name, args)
[ 7.820960] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 364, in main_init
[ 7.822095] cloud-init[791]: init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
[ 7.823158] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 635, in apply_network_config
[ 7.824420] cloud-init[791]: netcfg, src = self._find_networking_config()
[ 7.825430] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 622, in _find_networking_config
[ 7.826657] cloud-init[791]: if self.datasource and hasattr(self.datasource, 'network_config'):
[ 7.827754] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", line 150, in network_config
[ 7.829097] cloud-init[791]: self.network_json, known_macs=self.known_macs)
[ 7.830092] cloud-init[791]: File "/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py", line 652, in convert_net_json
[ 7.831434] cloud-init[791]: raise ValueError("Unable to find a system nic for %s" % d)
[ 7.832494] cloud-init[791]: ValueError: Unable to find a system nic for {'mac_address': 'fa:16:3e:3e:d1:dd', 'mtu': 1500, 'subnets': [{'ipv4': True, 'routes': [], 'netmask': '255.255.255.0', 'address': '10.0.30.100', 'type': 'static'}], 'type': 'physical'}
EDIT2: Actually, I realized that the document https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/xl710-sr-iov-config-guide-gbe-linux-brief.pdf https://www.intel.com/content/dam/www/public/us/en/documents/technology-briefs/xl710-sr-iov-config-guide-gbe-linux-brief… mentions that RHEL is a prerequisite. So I tried to create a centos based VM, and it can successfully boot and seems to have a working interface! I have not verified connectivity but it has successfully loaded the drivers etc.
So I guess my question changes a bit: Is there actually no support for SR-IOV for this card for Ubuntu based cloud virtual machines?
/Tomas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Thank you for the information, I was about to ask you to provide more information such as
Linux Kernel log from Ubuntu host that is having the issue. With the new setup you have, I will further check for you about your new inquiry.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Please provide below information for better investigation:
1) Host OS and version
2) VM OS and version
3) Is the Ubuntu problem happening only on the VM?
4) What combinations fail and what work?
Thanks,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Please feel free to provide the information requested on my previous post. Thank you.
Thanks,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Sorry, I missed the last post
* The host OS is Ubuntu 14.04.5 server version
* The VM OS is Ubuntu 16.04 server based cloud image https://cloud-images.ubuntu.com/xenial/current/ Ubuntu 16.04 LTS (Xenial Xerus) Daily Build [20171011]
* Yes, we only see it in the VM. So I suspect the cloud image does not have the drivers.
* What we tried that worked was using CentOS based cloud image instead.
/Tomas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI TomasBrn,
Thank you for the information, I will further check and update you once there is any finding.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
Understand that there is no issue using Centos cloud image. Further checking, we are not able to comment on why Ubuntu cloud image fails since there are various possibilities such as image issue, driver problem or other factors.
Regards,
Sharon
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK. Since it works on centos we will have to use that for now. Thank you for your efforts
/Tomas
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi TomasBrn,
You are welcome and appreciate your time on this matter.
Regards,
Sharon
![](/skins/images/D3C0B914909A6564BBB97F4AD1ED1973/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page