- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm using Ubuntu 16.04.3 LTS with KVM virtualization. Also i'm using
Intel XL710 card with SR-IOV feature.
Here are the outputs:
/lib/modprobe.d$ modinfo i40e
filename: /lib/modules/4.4.0-96-generic/updates/drivers/net/ethernet/intel/i40e/i40e.ko
version: 2.4.10
license: GPL
description: Intel(R) 40-10 Gigabit Ethernet Connection Network Driver
author: Intel Corporation, <e1000-devel@lists.sourceforge.net>
srcversion: 3977C21019A3C4865FF253A
alias: pci:v00008086d0000158Bsv*sd*bc*sc*i*
alias: pci:v00008086d0000158Asv*sd*bc*sc*i*
alias: pci:v00008086d000037D3sv*sd*bc*sc*i*
alias: pci:v00008086d000037D2sv*sd*bc*sc*i*
alias: pci:v00008086d000037D1sv*sd*bc*sc*i*
alias: pci:v00008086d000037D0sv*sd*bc*sc*i*
alias: pci:v00008086d000037CFsv*sd*bc*sc*i*
alias: pci:v00008086d000037CEsv*sd*bc*sc*i*
alias: pci:v00008086d00001588sv*sd*bc*sc*i*
alias: pci:v00008086d00001587sv*sd*bc*sc*i*
alias: pci:v00008086d00001589sv*sd*bc*sc*i*
alias: pci:v00008086d00001586sv*sd*bc*sc*i*
alias: pci:v00008086d00001585sv*sd*bc*sc*i*
alias: pci:v00008086d00001584sv*sd*bc*sc*i*
alias: pci:v00008086d00001583sv*sd*bc*sc*i*
alias: pci:v00008086d00001581sv*sd*bc*sc*i*
alias: pci:v00008086d00001580sv*sd*bc*sc*i*
alias: pci:v00008086d00001574sv*sd*bc*sc*i*
alias: pci:v00008086d00001572sv*sd*bc*sc*i*
depends: ptp,vxlan
vermagic: 4.4.0-96-generic SMP mod_unload modversions
parm: debug:Debug level (0=none,...,16=all) (int)
cat /etc/rc.local
#!/bin/sh -e
#
# rc.local
#
#
# By default this script does nothing.
echo 8 > /sys/class/net/enp6s0f1/device/sriov_numvfs
echo 8 > /sys/class/net/enp6s0f0/device/sriov_numvfs
ip link set enp6s0f0 vf 0 mac aa:bb:cc:aa:00:02 vlan 12
ip link set enp6s0f0 vf 1 mac aa:bb:cc:aa:01:02 vlan 32
ip link set enp6s0f0 vf 2 mac aa:ff:ff:aa:00:12 vlan 22
ip link set enp6s0f0 vf 3 mac aa:ff:ff:aa:01:12 vlan 42
ip link set enp6s0f1 vf 0 mac aa:bb:cc:bb:00:02 vlan 52
ip link set enp6s0f1 vf 1 mac aa:bb:cc:bb:01:02 vlan 72
ip link set enp6s0f1 vf 2 mac aa:ff:ff:bb:00:12 vlan 62
ip link set enp6s0f1 vf 3 mac aa:ff:ff:bb:01:12 vlan 82
exit 0
Driver i40e is also blacklisted as suggested in Intel SR-IOV configuration guide.
Everything works ok, but in case i reboot the VM, some of the virtual functions won't work. I need to reboot VM a few times (sometimes i need just one reboot, sometimes like three reboots) and virtual functions start working properly.
I was able only to find this in logs on KVM host (/var/log/kern.log) when VF doesn't work:
Jan 8 09:56:23 ri-cgn-kvm-2 kernel: [15787510.840948] i40e 0000:06:00.0: VF 1 failed opcode 3, retval: -5
Jan 8 09:56:23 ri-cgn-kvm-2 kernel: [15787510.841010] i40e 0000:06:00.0: Invalid message from VF 1, opcode 3, len 4
Jan 8 09:57:08 ri-cgn-kvm-2 kernel: [15787555.936159] i40e 0000:06:00.1: VF 0 failed opcode 10, retval: -5
Jan 8 09:57:08 ri-cgn-kvm-2 kernel: [15787555.936222] i40e 0000:06:00.1: Invalid message from VF 0, opcode 10, len 44
Jan 8 09:57:08 ri-cgn-kvm-2 kernel: [15787555.976198] i40e 0000:06:00.1: VF 0 failed opcode 11, retval: -10
Jan 8 10:05:14 ri-cgn-kvm-2 kernel: [15788042.047074] i40e 0000:06:00.0: VF 0 failed opcode 3, retval: -5
Jan 8 10:05:14 ri-cgn-kvm-2 kernel: [15788042.047138] i40e 0000:06:00.0: Invalid message from VF 0, opcode 3, len 4
Jan 8 10:21:14 ri-cgn-kvm-2 kernel: [15789001.950375] i40e 0000:06:00.0: VF is not trusted, switch the VF to trusted to add more functionality
Jan 8 10:21:14 ri-cgn-kvm-2 kernel: [15789001.951291] i40e 0000:06:00.1: VF is not trusted, switch the VF to trusted to add more functionality
Jan 8 10:21:14 ri-cgn-kvm-2 kernel: [15789001.951300] i40e 0000:06:00.1: VF 1 failed opcode 10, retval: -1
Jan 8 10:21:14 ri-cgn-kvm-2 kernel: [15789001.977115] i40e 0000:06:00.0: VF 1 failed opcode 10, retval: -1
Jan 8 10:37:25 ri-cgn-kvm-2 kernel: [15789972.835652] i40e 0000:06:00.0: VF 1 failed opcode 11, retval: -10
Jan 8 10:37:27 ri-cgn-kvm-2 kernel: [15789975.074702] i40e 0000:06:00.1: VF 1 failed opcode 11, retval: -10
What is causing this issue?
Regards,
Kresimir
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Daniel,
here are outputs of ethtool command:
ethtool -i enp1s0f0
driver: igb
version: 5.3.0-k
firmware-version: 1.63, 0x80000e74, 0.385.8
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
ethtool -k enp1s0f0
Features for enp1s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-tc-offload: off [fixed]
ethtool -i enp1s0f1
driver: igb
version: 5.3.0-k
firmware-version: 1.63, 0x80000e74, 0.385.8
expansion-rom-version:
bus-info: 0000:01:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no
ethtool -k enp1s0f1
Features for enp1s0f1:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: on
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp6-segmentation: on
udp-fragmentation-offload: off [fixed]
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-ipip-segmentation: off [fixed]
tx-sit-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
busy-poll: off [fixed]
hw-tc-offload: off [fixed]
This Intel cards were ordered and provided with Cisco. I'm running CISCO UCS C220M4S server.
Regards,
Kresimir
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Daneil,
i've installed latest drivers, v.2.7.26, one host and didn't experience issue with VF.
I'll be testing for next couple of days to see will issue appear.
Regards,
Kresimir
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Michael,
for now i didn't have any issues with latest drivers, so i can say that issue is solved.
Thank you for the assistance.
Regards,
Kresimir

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page