FPGA, SoC, And CPLD Boards And Kits
FPGA Evaluation and Development Kits
Announcements
Intel Support hours are Monday-Fridays, 8am-5pm PST, except Holidays. Thanks to our community members who provide support during our down time or before we get to your questions. We appreciate you!

Need Forum Guidance? Click here
Search our FPGA Knowledge Articles here.
5203 Discussions

How to make N3000 support jumbo frames?

ahaa
Beginner
1,090 Views

We want to change the configuration of N3000-2/N3000-1 to make the card support jumbo frameslarger than the default size 1518B1519B-9000B. Now we have tried to modify application of dpdk such as skeleton basic forward or l2fwdand it makes sense.

When tested the dpdk application on other NIC cardsit could receive and transmit jumbo frames. Howeverin the card of N3000-22x2x25G),it could only receive jumbo frames without transmitting .

We also did another test to connect ing_in* signal with egr_out* signal to make an internal lookback. The result shows it can’t transmit jumbo frames.

According to our analysis, it is most likely that there are some configuration changes in Arria10 , IP or some relevant registers.

Anyone knows how to do or any advice?

0 Kudos
11 Replies
JonWay_C_Intel
Employee
1,070 Views
ahaa
Beginner
1,058 Views

Thanks for your reply!

Actually we have checked the size of MTU of wrapper and modified it to 9600.  The MTU of XL710 would be returned to the default value 1500 even though we set it to 9600 after bounding to vfio-pci driver in the mode of DPDK . But we could receive jumbo frames.

The problem is the card can not forward jumbo frames. 

Using signal-tap we captured some signals and found that egr_out* signals are correct. Based on this, we can deduce XL710 and DPDK have no problem. So we think that there may exist some configurations in the path of egr_out* to QSFP, which limits the forwarding of jumbo frames. 

In the other experiment, we connected the ing_in*----egr_out* directly, and the forwarding of jumbo frames failed too. I think this is further confirmation that our consideration is right. The bottleneck is between egr_out and QSFP,  and that's exactly what corresponds to nfv_eth_wrapper, mux, demux and some other modules in  RTL project.

JonWay_C_Intel
Employee
1,055 Views

I believe if you are in DPDK, you need to revert back to OPAE to set the MTU. See  below link to revert back to OPAE.

https://www.intel.com/content/www/us/en/programmable/documentation/xgz1560360700260.html#jzj15652825...

 

ahaa
Beginner
1,048 Views

The matter is that if I revert back to OPAE and set the MTU size again, the MTU of XL710 would be returned to the default value 1500 even though we set it to 9600 after bounding to vfio-pci driver in the mode of DPDK 

JonWay_C_Intel
Employee
1,043 Views

How do you set the MTU of the XL710? What is the command? 

ahaa
Beginner
1,042 Views

ls -la /sys/class/net

ahaa_0-1624271114833.png

sudo ip link set dev enp61s0f0 mtu 9600
sudo ip link set dev enp61s0f1 mtu 9600
sudo ip link set dev enp64s0f0 mtu 9600
sudo ip link set dev enp64s0f1 mtu 9600
ip link show enp61s0f0
ip link show enp61s0f1
ip link show enp64s0f0
ip link show enp64s0f1
sudo fpgadiag -b 0b3f -m fpgamac --side=host --mtu

ahaa_1-1624271215119.png

then bound to dpdk

 

ahaa
Beginner
1,035 Views

another problem is that I cannot check mtu size after bounding to dpdk

ahaa
Beginner
785 Views

Thank you for your continued attention.

We have found that some dpdk application can fully support giant frame and we ported the related functions to other applications. So far the issue with giant frame support has been resolved.

JonWay_C_Intel
Employee
779 Views

Thank you for the update. Would you share with us what is the dpdk application and how you used it to support jumbo frames? I think it will be a good learning for all other forum users as well. 

ahaa
Beginner
773 Views

well, It's my pleasure to help others.

1. DPDK L3fwd supports jumbo frames originally when you run it with '--enable-jumbo --max-pkt-len 9600 ';

2.the default mtu size of N3000-1(10G) is 1500,  one should change it to 9600, and  you can get the command by looking in the manual of ug-ias-n3000(chapter 8.1); the default mtu size of N3000-2(25G) is 9600, it have no need to be changed.

3. you can modify the mode of rx and tx to support jumbo frames by reference to l3fwd  CMD_LINE_OPT_ENABLE_JUMBO_NUM module

For example, in the l2fwd application, you need to make the following changes

ahaa_0-1624448611058.png

 

JonWay_C_Intel
Employee
1,068 Views

If you are using the 8x10G card, make sure to use the Production version card. See release note about jumbo frames: https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/rn/rn-ias-n3000.pdf

 

Reply