Embedded Connectivity
Intel network controllers, Firmware, and drivers support systems
845 Discussions

TC flower hardware offload for ice driver

geenid
Beginner
4,871 Views

I am using the E810 controller with ice driver version 1.2.1 (more detailed information below). I would like to use tc flower commands to offload to the embedded switch for use with SR-IOV. I can see that this interface is implemented in the latest driver but the flows seem to have no effect.

The steps I follow to test this are as follows:

First I create the VFs:

# echo 4 > /sys/class/net/enp59s0f0/device/sriov_numvfs


This creates the interfaces 'enp59s0f0v0' through 'enp59s0f0v3'

I set the eswitch to 'switchdev' mode like this:

# devlink dev eswitch set pci/0000:3b:00.0 mode switchdev


which creates the representor interfaces in this case named 'eth0' through 'eth3'.

I turn on 'hw-tc-offload':

# ethtool -K enp59s0f0 hw-tc-offload on

 

And I make sure the PF, VFs and VF representor interfaces are all up.

I create the ingress qdisc:

# tc qdisc add dev enp59s0f0 ingress

Then I create a flow using the PF (enp59s0f0) and a VF representor, e.g. eth0 for VF0 using the `skip_sw` flag like this:

# tc filter add dev enp59s0f0 protocol 802.1Q parent ffff: prio 1 flower skip_sw vlan_id 300 action mirred egress redirect dev eth0

The debug message then shows:

[ 1348.256269] Mirror/redirect action on

 

This filter shows as being 'in hardware'

# tc filter show dev enp59s0f0 ingress
filter parent ffff: protocol 802.1Q pref 1 flower chain 0
filter parent ffff: protocol 802.1Q pref 1 flower chain 0 handle 0x1
vlan_id 300
skip_sw
in_hw in_hw_count 1
action order 1: mirred (Egress Redirect to device eth0) stolen
index 1 ref 1 bind 1

However, when I send packets to this interface (enp59s0f0) that match the flow the packets show up at the PF and not at the VF as expected.

When I make a software rule however, like this:

# tc filter show dev enp59s0f0 ingress
filter parent ffff: protocol 802.1Q pref 1 flower chain 0
filter parent ffff: protocol 802.1Q pref 1 flower chain 0 handle 0x1
vlan_id 300
skip_sw
in_hw in_hw_count 1
action order 1: mirred (Egress Redirect to device enp59s0f0v0) stolen
index 1 ref 1 bind 1

It works as expected, although of course it doesn't give the performance it would when offloading to hardware.

Did I miss a step in the configuration for offloading these flows correctly or is this feature maybe not fully supported in the current driver yet?

Driver information:

filename: /lib/modules/5.8.0-25-generic/updates/drivers/net/ethernet/intel/ice/ice.ko
firmware: updates/intel/ice/ddp/ice.pkg
version: 1.2.1
license: GPL v2
description: Intel(R) Ethernet Connection E800 Series Linux Driver
author: Intel Corporation, <linux.nics@intel.com>
srcversion: 129D78830ABE5A3876785EF
alias: pci:v00008086d0000151Dsv*sd*bc*sc*i*
alias: pci:v00008086d0000124Fsv*sd*bc*sc*i*
alias: pci:v00008086d0000124Esv*sd*bc*sc*i*
alias: pci:v00008086d0000124Dsv*sd*bc*sc*i*
alias: pci:v00008086d0000124Csv*sd*bc*sc*i*
alias: pci:v00008086d0000189Asv*sd*bc*sc*i*
alias: pci:v00008086d00001899sv*sd*bc*sc*i*
alias: pci:v00008086d00001898sv*sd*bc*sc*i*
alias: pci:v00008086d00001897sv*sd*bc*sc*i*
alias: pci:v00008086d00001894sv*sd*bc*sc*i*
alias: pci:v00008086d00001893sv*sd*bc*sc*i*
alias: pci:v00008086d00001892sv*sd*bc*sc*i*
alias: pci:v00008086d00001891sv*sd*bc*sc*i*
alias: pci:v00008086d00001890sv*sd*bc*sc*i*
alias: pci:v00008086d0000188Esv*sd*bc*sc*i*
alias: pci:v00008086d0000188Dsv*sd*bc*sc*i*
alias: pci:v00008086d0000188Csv*sd*bc*sc*i*
alias: pci:v00008086d0000188Bsv*sd*bc*sc*i*
alias: pci:v00008086d0000188Asv*sd*bc*sc*i*
alias: pci:v00008086d0000159Bsv*sd*bc*sc*i*
alias: pci:v00008086d0000159Asv*sd*bc*sc*i*
alias: pci:v00008086d00001599sv*sd*bc*sc*i*
alias: pci:v00008086d00001593sv*sd*bc*sc*i*
alias: pci:v00008086d00001592sv*sd*bc*sc*i*
alias: pci:v00008086d00001591sv*sd*bc*sc*i*
depends:
retpoline: Y
name: ice
vermagic: 5.8.0-25-generic SMP mod_unload
parm: debug:netif level (0=none,...,16=all) (int)



Host OS : Ubuntu 20.10 server, kernel 5.8.0-25-generic

 

Thanks!

Labels (1)
0 Kudos
18 Replies
Caguicla_Intel
Moderator
4,846 Views

Hello geenid,


Thank you for posting in Intel Ethernet Communities. 


Can you share the following information that would help in checking your request.

1.) Link where the driver 1.2.1 was downloaded from?

2.) What is the current version of your VF driver and share the link where it was downloaded from? 

3.) Please share the PBA and serial number of the adapter. You may refer to the link below on where to find the PBA number. Providing photos of the adapter focusing on the markings (white sticker) found on the physical card will be highly appreciated for us to double check on it. The PBA consists 6-3 digit number located at the last part of the serial number.

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products.html


Looking forward to your reply. 


Should there be no response from you, I’ll make sure to reach out after 3 business days.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
geenid
Beginner
4,844 Views

Hello Criselle,

Thank you for the quick response. Please find my answers below:

1) The ice driver was downloaded from http://e1000.sourceforge.net. I Have also tried with the 1.1.4 version of the driver which is available on the same site and on https://downloadcenter.intel.com/ with the same results.

2) The VF driver is currently the default one from the Ubuntu installation, which modinfo displays as version 3.2.3-k , src version F00DC8F961227FFC4C81E21 .

3) I currently don't have access to the hardware directly, I could check later should it be necessary. Detailed lspci output shows:

3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for backplane (rev 01)
Subsystem: Intel Corporation Ethernet Controller E810-C for backplane

 

I noticed a mistake in my post above as well, the software rule which did work but was not offloaded to hardware should have read:

filter parent ffff: protocol 802.1Q pref 1 flower chain 0 
filter parent ffff: protocol 802.1Q pref 1 flower chain 0 handle 0x1
vlan_id 300
skip_hw
not_in_hw
action order 1: mirred (Egress Redirect to device enp59s0f0v0) stolen
index 1 ref 1 bind 1

 

Thanks!

0 Kudos
Caguicla_Intel
Moderator
4,828 Views

Hello geenid,


No problem at all, appreciate your swift reply.


Can you confirm if you already tried the latest VF from our download site link below? This would help us in further checking the issue.

Intel® Network Adapter Linux* Virtual Function Driver for Intel® Ethernet Controller 700 and E810 Series

https://downloadcenter.intel.com/download/24693/Intel-Network-Adapter-Linux-Virtual-Function-Driver-for-Intel-Ethernet-Controller-700-and-E810-Series?product=184846


Please also let us know if you checked the readme file of the VF driver link above under "To create traffic classes (TCs) on the interface"?

https://downloadmirror.intel.com/24693/eng/readme.txt


Awaiting to your reply.


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
geenid
Beginner
4,822 Views

Hello Crisselle,

I have just tried with the latest version of the VF driver downloaded from your link (4.0.1), unfortunately with the same results.

The text in the README you mentioned reads:

NOTE: Run all TC commands from the ../iproute2/tc/ directory

Since this directory is not there in the driver sources I take it this means the latest version of iproute2? I have just tried with the latest version as downloaded from the official source here https://mirrors.edge.kernel.org/pub/linux/utils/net/iproute2/ , also with the same results as before.

One thing I noticed just now that may or may not be a clue:

Although I have turned on the hw-tc-offload feature in both the PF and in the VFs, in the VF representor , which in my case is just called eth0 for VF0, this feature appears off:

# ethtool -k eth0 | grep tc-offload
hw-tc-offload: off [fixed]


# ip -d link show eth0
19: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
link/ether fe:9e:d3:75:ce:fa brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535 portname pf0vfr0

 

Should this be on perhaps? Or is there some other reason why the hardware offload does not seem to work?

Kind regards,

geenid

0 Kudos
Caguicla_Intel
Moderator
4,808 Views

Hello geenid,


Thank you for the response. 


Can you please provide the System Support Utility log from your system? This will allow us to check your Adapter details and configuration. We will also check if the SN and PBA number of the adapter will be captured. Refer to the link below for Help Guide for the Intel® System Support Utility.

https://www.intel.com/content/www/us/en/support/articles/000008563/software/software-applications.html


Looking forward to your reply. 


Should there be no response from you, I’ll make sure to reach out after 3 business days. 


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
geenid
Beginner
4,797 Views

Hello Crisselle,

Please find the output from the utility for the network adapter below:

 

 - "Intel Corporation Ethernet Controller E810-C for backplane (rev 01)"
Supports auto-negotiation:"Yes"
Advertised auto-negotiation:"Yes"
Link partner advertised auto-negotiation:"Yes"
Auto-negotiation:"on"
Availability:"enp59s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500"
Capabilities:"[40] Power Management version 3"
Capabilities:"[50] MSI: Enable- Count=1/1 Maskable+ 64bit+"
Capabilities:"[70] MSI-X: Enable+ Count=1024 Masked-"
Capabilities:"[a0] Express Endpoint, MSI 00"
Capabilities:"[e0] Vital Product Data"
Capabilities:"[100] Advanced Error Reporting"
Capabilities:"[148] Alternative Routing-ID Interpretation (ARI)"
Capabilities:"[150] Device Serial Number 00-01-00-ff-ff-00-00-00"
Capabilities:"[160] Single Root I/O Virtualization (SR-IOV)"
Capabilities:"[1a0] Transaction Processing Hints"
Capabilities:"[1b0] Access Control Services"
Capabilities:"[1d0] Secondary PCI Express"
Capabilities:"[200] Data Link Feature <?>"
Capabilities:"[210] Physical Layer 16.0 GT/s <?>"
Capabilities:"[250] Lane Margining at the Receiver <?>"
Caption:"Not Available"
Default IP Gateway:"Not Available"
DHCP Enabled:"Yes"
DHCP Lease Expires:"Not Available"
DHCP Lease Obtained:"Not Available"
DHCP Server:"Not Available"
Driver:"ice"
Driver Path:"/lib/modules/5.8.0-28-generic/kernel/drivers/net/ethernet/intel/ice/ice.ko"
Driver Provider:"Intel Corporation, <linux.nics@intel.com>"
Driver Version:""
Duplex:"Full"
Firmware Version:"2.00 0x80003e3f 1.2751.0"
Interface:"enp59s0f0"
I/O Ports:"Not Available"
IP Address:"Not Available"
IP Subnet:"Not Available"
MAC Address:"00:90:0b:8b:83:02"
Location:"3b:00.0"
Manufacturer:"Intel Corporation

 

King regards,

geenid

0 Kudos
geenid
Beginner
4,792 Views

P.S.:

Just noticed that this output was with the default Ubuntu driver loaded, I made sure to load the 1.2.1 driver again. The "Driver Path" line should have read:

/lib/modules/5.8.0-28-generic/updates/drivers/net/ethernet/intel/ice/ice.ko

 

Kind regards,

geenid

0 Kudos
Caguicla_Intel
Moderator
4,766 Views

Hello geenid,


Thank you for providing the requested information. 


Please allow us to further investigate on this. We will get back to you as soon as there is any findings but no later than 2-3 business days. 


Hoping for your kind patience.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,742 Views

Hello geenid,


We sincerely apologize for the delay on this matter.


Please be informed that this request is still under investigation. Rest assured that we will send an update as soon as there is any but no later than 2-3 business days.


Thank you for your kind understanding. 


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
Caguicla_Intel
Moderator
4,720 Views

Hello geenid,


Thank you for the patience on this matter. Please see below details for the response from our higher level. 


Can you confirm if you are using an Ethernet controller or adapter? If yes, can you please provide complete Intel SSU logs and the complete markings of the adapter?


Refer to this link on where to find the markings of the adapter. Providing photos of the adapter focusing on the markings (white sticker) found on the physical card will be highly appreciated for us to double check on it.

Identify Your Intel® Network Adapter Model Using PBA Number

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-i-o/ethernet-products.html


Refer to the link below for Help Guide for the Intel® System Support Utility. Scroll down to Intel® System Support Utility Process>Linux Process

https://www.intel.com/content/www/us/en/support/articles/000008563/software/software-applications.html


Download the file needed here:

https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System


Looking forward to your reply.


To give you enough time on this, we will follow up after 4 business days. 


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
geenid
Beginner
4,713 Views

Hello Crisselle,

That would be a controller on a custom board, not an adapter . I don't think it should matter in this case however as my question relates mostly to the Linux driver I would think.

Let me know if you need any other information.

Kind regards,

geenid

0 Kudos
Caguicla_Intel
Moderator
4,705 Views

Hello geenid,


Appreciate your swift response.


When you mentioned 'custom board', can you share if this is your own designed board? If yes, we may need to refer you to a different team as this would be best handled by them. If no, we hope you don't mind sharing the brand and model of your board for us to further investigate. 


Looking forward to your reply. 


We will follow up after 3 business days in case we don't hear from you.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
geenid
Beginner
4,700 Views

Hello Crisselle,

I haven't personally designed the board but it is a new system under development so there is no (known) model name.

What other team would be better suited to answer the question? I had an earlier question which got referred to the 'Embedded Connectivity' board, where I got directed to the Ubuntu forum but of course they will not be able to help me with a question about an out of tree kernel driver.

However, if there is another board better suited for linux driver questions that would be helpful indeed.

Thanks!

Regards,

geenid

0 Kudos
AlfredoS_Intel
Moderator
4,696 Views

Hi Geenid,

Thank you providing those information.

Please allow us some time to check on this. 

We will get back to you no later than 3 business days from now.



Best Regards,

Alfred S

Intel® Customer Support 


0 Kudos
Caguicla_Intel
Moderator
4,670 Views

Hello geenid,


Thank you for the patience on this matter. 


After checking with our engineers, please be informed that your query will be best answered by our Embedded Connectivity Support team. We will help you moving this post to the designated team. Please feel free to contact us if you need assistance from Ethernet support team.


Best regards,

Crisselle C

Intel® Customer Support


0 Kudos
CarlosAM_INTEL
Moderator
4,643 Views

Hello, @geenid:

Thank you for contacting Intel Embedded Community.

You should address your consultations of this type through the channels listed as a reference on the following website:

https://bugs.dpdk.org/

Best regards,

@CarlosAM_INTEL

0 Kudos
geenid
Beginner
4,641 Views

Hello Carlos,

Thank you for your suggestion, however in this case I am not using DPDK, so I don't think posting on the DPDK bug tracker would be of much help unfortunately.

 

Kind regards,

geenid

0 Kudos
CarlosAM_INTEL
Moderator
4,636 Views

Hello, @geenid:

Thanks for your reply.

You need to be sure that your implementation is using the Intel® Ethernet Controller E810 (Columbiaville) Alpha 3 NVM document #611561. 

You can find it when you are logged into your Resource & Design Center (RDC) privileged account on the following website:

https://cdrdv2.intel.com/v1/dl/getContent/611561

The RDC Account Support form is the channel to process your account update request or report any inconveniences with the provided site. It can be found at:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best regards,

@CarlosAM_INTEL.

0 Kudos
Reply