X710: very slow rx speed (a few MiB/second) when using non-default VSI.

[YYANG110]

I experienced very strange performance issue when sending data from a client to a server with X710 card. Details about the investigation can be found here:

https://github.com/youzhongyang/i40e_investigation

The server runs illumos (derived from open solaris), its i40e driver can be found here:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/i40e/

So for each i40e instance, we create 32 VSIs (including the default one created by the firmware), each VSI is assigned 8 trq pairs. When a VIF/VNIC is created, the OS assigns a VSI to it, data receiving through the VNIC's IP will be done by the relevant VSI's 8 RX rings. However, data sending will always go through the default VSI's 8 TX rings.

The function for creating non-default VSI is here:

http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/i40e/i40e_main.c#2111

I've upgraded the firmware version to the latest, still the same issue. Is this a known issue? or is there anything missing when setting up the non-default VSI?

Your help/advice would be highly appreciated.

[Caguicla_Intel]

Hello YYANG110,

Thank you for posting in Intel Ethernet Communities. 

Please provide the markings of your adapter. This would help us identify if you are using an OEM or retail version of Intel Ethernet Adapter. Kindly refer to the link below on where to find the PBA of the adapter. It is a 6-digit number located at the last part of the serial number.

https://www.intel.com/content/www/us/en/support/articles/000007022/network-and-io/ethernet-products.html

Please also share if you have tried the latest i40e driver from the link below. 

https://sourceforge.net/projects/e1000/files/i40e%20stable/2.10.19.30/

Looking forward to your response.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[YYANG110]

Hello Crisselle,

Thanks for your attention. The PBA of our card is J11365.

`We purchased a bunch of them for our servers recently, hoping the card can deliver the best networking performance. The link you recommended has the driver for Linux operating system, but unfortunately we are not using Linux, we use SmartOS (an illumos-based OS, similar to Solaris).

I tested the same card on Linux using i40e's VFs, it works, no issue, but the i40e driver in our illumos does not create VF VSIs. If this is not the right place to ask such driver related question, would you please recommend the right mailing list, or technical forum?

Thanks very much,

-Youzhong Yang

MathWorks

[Caguicla_Intel]

Hello Youzhong,

Thank you for the prompt reply.

Upon checking, SmartOS is not on the list of supported OS for your adapter. You may visit this link for the list Supported Operating Systems for Retail Intel® Ethernet Adapters

https://www.intel.com/content/www/us/en/support/articles/000025890/network-and-io/ethernet-products.html

Can you share if you have tried to check this issue on where you have obtained the driver? They might be of help in resolving this issue.

Awaiting to your response.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[YYANG110]

Thanks. The illumos/SmartOS community is small, I tried but there's no response.

[YYANG110]

I am now trying to resolve the issue by myself, that's why I need someone from Intel who has deep understanding of how the X710 card works to take a look. I suspect there may be a firmware bug as the "Intel® Ethernet Controller X710/ XXV710/XL710 Datasheet" suggests there's nothing wrong by creating 32 PF VSIs. Please refer to its page 20:

"VSI Support

The X710/XXV710/XL710 supports a total of 384 VSIs. VSI assignment is flexible, but the choice to support 384 VSIs is motivated by the following usage example:

• 256 VSIs for VFs or VMDq2

• 32 VSIs for PFs"

Thanks for your help.

[Caguicla_Intel]

Hello Youzhong,

Appreciate your response.

We will try to check on this internally. Rest assured that we will get back to you within 1-3 business days.

Hoping for your patience. 

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Caguicla_Intel]

Hello Youzhong,

Apologies for the delay.

Please try to submit a request through sourceforge link below.

https://sourceforge.net/projects/e1000/

Looking forward to hear an update from you.

Best regards,

Crisselle C

Intel Customer Support

A Contingent Worker at Intel

[Mike_Intel]

Hello Youzhong,

We just want to make a follow up to check if you already logged your inquiry in sourceforge?

If you have questions, please let us know.

Best regards,

Michael L.

Intel Customer Support Technicians

A Contingent Worker at Intel

[YYANG110]

Thanks for asking. I sent an e-mail to e1000-devel@lists.sourceforge.net, so far no response yet.

[Mike_Intel]

Hello Youzhong,

Thank you for the update. Please continue to coordinate with them regarding your concern.

As for this thread, do you still have other questions?

Best regards,

Michael L.

Intel Customer Support Technicians

A Contingent Worker at Intel

[YYANG110]

No, I don't have other question. Thank you very much for your support.

[Caguicla_Intel]

Hello Youzhong,

Thank you for the reply.

If that's the case then, please be informed that we will now proceed in closing this request. Should you have any other assistance needed from Intel Ethernet Team, please do not hesitate to post a new question.

Best regards,

Crisselle C.

Intel Customer Support

A Contingent Worker at Intel

[YYANG110]

No problem, this request can be closed for now.