Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
5227 Discussions

Rate limiting on Virtual Function

srang3
Beginner
3,486 Views

Hi,

I am using Intel 10 Gig Dual-Port Ethernet PCI Card (BN8110470) ixgbe driver on Ubuntu 14.04 Host.

I was able to create VF's which are listed below from lspci command:

02:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

02:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

03:10.0 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.1 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.2 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.3 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.4 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.5 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.6 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:10.7 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.0 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.1 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.2 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.3 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.4 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

03:11.5 Ethernet controller: Intel Corporation X540 Ethernet Controller Virtual Function (rev 01)

The VF's are also listed from ip link command:

20: rename20: mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000

link/ether a0:36:9f:11:b8:10 brd ff:ff:ff:ff:ff:ff

vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

28: rename28: mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000

link/ether a0:36:9f:11:b8:12 brd ff:ff:ff:ff:ff:ff

vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

vf 6 MAC 00:00:00:00:00:00, vlan 10, spoof checking on, link-state auto

As visible from dev rename28, vf 6, assigning vlan is working as expected,

ip link set rename28 vf 6 vlan 10

But setting rate limit on same VF is working,

sudo ip link set rename28 vf 6 rate 250

RTNETLINK answers: Invalid argument

The guide used to realize this configuration is here, http://www.intel.com/content/www/us/en/network-adapters/10-gigabit-network-adapters/config-qos-with-flexible-port-partitioning.html Configure QoS with Intel® Flexible Port Partitioning

I know that I should be using the latest iproute2, I am using iproute2-ss131122 which is never version.

10 Replies
TKnau
Beginner
2,431 Views

Hi Vince,

we have a similar problem. lspci reports our card as

 

03:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)

We use the following drivers on the machine

thomas@sep1:~$ modinfo ixgbe

filename: /lib/modules/3.16.0-36-generic/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko

version: 3.23.2.1

thomas@sep1:~$ modinfo ixgbevf

filename: /lib/modules/3.16.0-36-generic/kernel/drivers/net/ixgbevf/ixgbevf.ko

version: 2.16.1

 

Both of which I believe to be the most recent version downloaded from Intel's website.

 

Setting a VLAN with

 

thomas@sep1:~$ sudo ip link set em1 vf 0 vlan 0

 

works. Setting a rate limit with

thomas@sep1:~$ sudo ip link set em1 vf 0 rate 500

RTNETLINK answers: Invalid argument

 

does not. We are a bit clueless how to proceed from here.

 

Thanks for you time,

Thomas.

 

0 Kudos
VincentT_T_Intel
Employee
2,431 Views

Hi Thomas, let me further check this one.

thanks,

Vince

0 Kudos
TKnau
Beginner
2,431 Views

We figured out the "problem" by instrumenting the kernel driver with printf()s. It seems the adapter only allows rate limiting if it is in 10Gbps mode.

I'm curious what is the reason for that?

We will test again, once we have an appropriate 10Gbps cable.

0 Kudos
VincentT_T_Intel
Employee
2,431 Views

Hi Thomas, glad to know it works, currently checking on your inquiry. will update this thread as soon as possible.

regards,

Vince

0 Kudos
VincentT_T_Intel
Employee
2,431 Views

Hi Thomas, based from the internal testing - using Ubuntu 14.04 LTS with ixgbe driver 3.15.1-k, ixgbevf version 2.11.3-k and same iproute2 utility. rate limit was successful on 1Gbps and 10Gbps. Please share the Ubuntu version used if it's 14.04 or 14.04.1, 14.04.2 and are you using stock kernel or custom modified kernel. thanks

regards,

Vince

0 Kudos
TKnau
Beginner
2,431 Views

Not that it really matters anymore, but we are using ixgbevf 2.16.1 and ixgbe 3.23.2.1 on Ubuntu 14.10 (kernel 3.16.0-36-generic).

If I look at the ixgbe code, specifically in ixgbe_srio.c for function ixgbe_ndo_set_vf_bw(), I can see that it tests for 10 Gbps mode before allowing me to set the rate limit. We've instrumented the code, i.e. added printfs, to see that this is actually the place where setting the rate limit fails.

Now that we have the adapter in 10 Gbps mode, setting the rate limit works. Previously we only had a CAT5 cable, which is why the adapter was only operating in 1Gbps mode.

0 Kudos
st4
New Contributor III
2,431 Views

Hi Thomas,

Thank you for the reply. We will check on this.

rgds,

wb

0 Kudos
VincentT_T_Intel
Employee
2,431 Views

Hi Thomas, this is to confirm your findings is accurate about rate limit would test the link on 10Gbps

regards,

Vince

0 Kudos
RB_
Beginner
2,431 Views

Hi,

I have the similar problem on a 1 Gbps link.

Do you confirm that rate limit is only working on 10 Gbps link ?

Thanks for your help

Ronan

0 Kudos
Reply