How to make N3000 support jumbo frames?

[ahaa]

We want to change the configuration of N3000-2/N3000-1 to make the card support jumbo frames（larger than the default size 1518B，1519B-9000B）. Now we have tried to modify application of dpdk such as skeleton basic forward or l2fwd，and it makes sense.

When tested the dpdk application on other NIC cards，it could receive and transmit jumbo frames. However，in the card of N3000-2（2x2x25G），it could only receive jumbo frames without transmitting .

We also did another test to connect ing_in* signal with egr_out* signal to make an internal lookback. The result shows it can’t transmit jumbo frames.

According to our analysis, it is most likely that there are some configuration changes in Arria10 , IP or some relevant registers.

Anyone knows how to do or any advice?

[JonWay_C_Intel]

Have you looked into https://www.intel.com/content/www/us/en/programmable/documentation/xgz1560360700260.html#tne1573761134640

on setting/checking the MTU size?

[ahaa]

Thanks for your reply!

Actually we have checked the size of MTU of wrapper and modified it to 9600.  The MTU of XL710 would be returned to the default value 1500 even though we set it to 9600 after bounding to vfio-pci driver in the mode of DPDK . But we could receive jumbo frames.

The problem is the card can not forward jumbo frames. 

Using signal-tap we captured some signals and found that egr_out* signals are correct. Based on this, we can deduce XL710 and DPDK have no problem. So we think that there may exist some configurations in the path of egr_out* to QSFP, which limits the forwarding of jumbo frames. 

In the other experiment, we connected the ing_in*----egr_out* directly, and the forwarding of jumbo frames failed too. I think this is further confirmation that our consideration is right. The bottleneck is between egr_out and QSFP,  and that's exactly what corresponds to nfv_eth_wrapper, mux, demux and some other modules in  RTL project.

[JonWay_C_Intel]

I believe if you are in DPDK, you need to revert back to OPAE to set the MTU. See  below link to revert back to OPAE.

https://www.intel.com/content/www/us/en/programmable/documentation/xgz1560360700260.html#jzj1565282560626

[ahaa]

The matter is that if I revert back to OPAE and set the MTU size again, the MTU of XL710 would be returned to the default value 1500 even though we set it to 9600 after bounding to vfio-pci driver in the mode of DPDK 

[JonWay_C_Intel]

How do you set the MTU of the XL710? What is the command? 

[ahaa]

ls -la /sys/class/net

sudo ip link set dev enp61s0f0 mtu 9600
sudo ip link set dev enp61s0f1 mtu 9600
sudo ip link set dev enp64s0f0 mtu 9600
sudo ip link set dev enp64s0f1 mtu 9600
ip link show enp61s0f0
ip link show enp61s0f1
ip link show enp64s0f0
ip link show enp64s0f1
sudo fpgadiag -b 0b3f -m fpgamac --side=host --mtu

then bound to dpdk

[ahaa]

another problem is that I cannot check mtu size after bounding to dpdk

[ahaa]

Thank you for your continued attention.

We have found that some dpdk application can fully support giant frame and we ported the related functions to other applications. So far the issue with giant frame support has been resolved.

[JonWay_C_Intel]

Thank you for the update. Would you share with us what is the dpdk application and how you used it to support jumbo frames? I think it will be a good learning for all other forum users as well. 

[ahaa]

well, It's my pleasure to help others.

1. DPDK L3fwd supports jumbo frames originally when you run it with '--enable-jumbo --max-pkt-len 9600 ';

2.the default mtu size of N3000-1(10G) is 1500,  one should change it to 9600, and  you can get the command by looking in the manual of ug-ias-n3000(chapter 8.1); the default mtu size of N3000-2(25G) is 9600, it have no need to be changed.

3. you can modify the mode of rx and tx to support jumbo frames by reference to l3fwd  CMD_LINE_OPT_ENABLE_JUMBO_NUM module

For example, in the l2fwd application, you need to make the following changes

[JonWay_C_Intel]

If you are using the 8x10G card, make sure to use the Production version card. See release note about jumbo frames: https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/rn/rn-ias-n3000.pdf