XL710 configure physical port ETS

[Matevz]

Hi,

we are trying to program XL710 physical port to allow 8 strict priority TCs, Linux opensource driver or Intel driver, does not matter, same behviour. NVM 6.01

The following code:

        memset(&ets_data, 0, sizeof(ets_data));

        ets_data.tc_valid_bits = 0xff;

        ets_data.tc_strict_priority_flags = 0xff;

        ets_data.tc_bw_share_credits[0] = 127;

        ets_data.tc_bw_share_credits[1] = 127;

        ets_data.tc_bw_share_credits[2] = 127;

        ets_data.tc_bw_share_credits[3] = 127;

        ets_data.tc_bw_share_credits[4] = 127;

        ets_data.tc_bw_share_credits[5] = 127;

        ets_data.tc_bw_share_credits[6] = 127;

        ets_data.tc_bw_share_credits[7] = 127;

        ret = i40e_aq_config_switch_comp_ets(hw,

              pf->mac_seid,

              &ets_data,

              i40e_aqc_opc_enable_switching_comp_ets,

              NULL);

works and returns no error, but TC Strict priority filed stays at 0 and TCs are not getting scheduled in SP fashion, but Round robin.

Does XL710 need to operate in any special mode to get SP scheduling working? Or any other idea what might be wrong with it.

Matevz

[Matevz]

Hi Carlos,

thanks, I checked the site, but it is not helpful. We actually need somebody to discuss the topic on configuring scheduler in NVM firmware.

We configure it correctly (as per datasheet), but scheduler behaves incorrectly. However during configuration it does not report any errors.

An example how to configure our requirement would be really nice. It seems firmware is not configuring hardware underneath correctly.

regards,

Matevz

[Matevz]

Dear Intel,

is there any progress on resolving this issue?

regards,

Matevz

[CarlosAM_INTEL]

Hello, @Matevz:

Thanks for your reply.

You should confirm the affected implementations fulfill the requirements stated in sections 7.7.1.1, 7.7.2.1, 7.8.1.4, 7.12.1, and figure 7-70; on pages 848, 849, 850, 863, 887, 892, and 992 of the Intel® Ethernet Controller X710/XXV710/XL710: Datasheet document # 332464. You can find this document when you are logged into your Resource & Design Center (RDC) privileged account on the following website:

https://cdrdv2.intel.com/v1/dl/getContent/332464

You should fill out the RDC Account Support form to process your account update request or report any inconveniences with the provided site. You can be found on the following website:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best regards,

@CarlosAM_INTEL.

[Matevz]

Hi,

we have checked many times that documents and exactly that chapters in particular. Our implementation is compliant with it. But XL710 does not work according to settings.

Can we get configuration steps for i40e driver for Linux to configure XL710 in mode where XL710 has 8 TCs per PF and is performing strict priority among all 8 TCs on egress?

Just send us the list of Linux commands to achieve that configuration.

best regards,

Matevz

[CarlosAM_INTEL]

Hello, @Matevz :

Thanks for your reply.

We have suggested a specialized channel on the Operating System (OS) that you mentioned in our communication of this thread of the past January 13th, 2021. ‎

Due to this fact, could you please follow our suggestion to address your request to the recommended channel? 

Best regards,

@CarlosAM_INTEL.

[Matevz]

Hi,

OK, let me rephrase. We do not need Linux configuration, we need XL710 working in strict priority mode. I was thinking it might be easier for you to give us working mode setup using Linux i40e driver.

But we actually need list of register settings/AdminQ commands to get this mode working. We have used 

Intel® Ethernet Controller X710/ XXV710/XL710 Datasheet

Order No.: 332464-023 Revision: 3.8 December 2020

in order to get XL710 working in required mode, but we can not get it working in full strict priority. With NVM 6.10 it was not behaving any differently after calling relevant AdminQ commands for configuring scheduler, but after upgrading to NVM 8.15 it started to work somewhat similar to strict priority, however not fully.

The issue apparently is in NVM firmware not correctly configuring scheduler. Although at all times it says OK to all our admin commands issued to XL710.

Attached also diagram of how we want to configure XL710.

Please point us towards the solution not around to various useless tasks. What we are asking is I believe pretty straight forward. 8 TCs, 8 queues and strict priority among them. No rocket science.

Please show us any way you like how to configure xl710 for this operation. This is very basic stuff. Or is XL710 having a bug and this mode is not supported? If so, please say so immediately so we can stop throwing precious resources into it.

Many thanks,

Matevz

[CarlosAM_INTEL]

Hello, @Matevz:

Thanks for your reply.

Based on the provided information, we need to address the following questions to have a better comprehension of this situation from the hardware perspective:

Could you please clarify if there are some non-affected devices and the differences with the faulty ones? 

Could you please provide pictures of the top side markings of the affected Intel Ethernet controllers?

Could you please list the sources used from your side to design the affected units? By way, could you please let us know if it has been verified by Intel?

Could you please let us know the part number, model, and where we can find the information associated with the affected design if it is a third-party design?

We are waiting for your answer to these questions.

Best regards,

@CarlosAM_INTEL.

[Matevz]

Hi,

my answers inline:

comprehension of this situation from the hardware perspective:

ML> all devices behave the same way. Devices running NVM 6.10 are not obeying any priority, NVM 8.15 is obeying some strict priority.

Could you please provide pictures of the top side markings of the affected Intel Ethernet controllers?

ML> it is chip based design, part number was indicated earlier in this post. We can provide picture only tomorrow.

Could you please list the sources used from your side to design the affected units? By way, could you please let us know if it has been verified by Intel?

ML> do you mean hardware design or driver design?

Could you please let us know the part number, model, and where we can find the information associated with the affected design if it is a third-party design?

ML> part number ITLFTXL710BM1SLLK9

The controller is working otherwise stable, RX & TX at full line rate works without any problems. The only issue is TX scheduling is not working as configured.

thanks,

Matevz

[CarlosAM_INTEL]

Hello, @Matevz:

Thanks for your reply.

It is confused your answer to our last questions because it is unclear if the design has been developed by you or by a third-party company.

However, you answered our last consultation with characters that seem to be the Intel part number  FTXL710-BM1 SLLK9. In case that our assumption is correct, it is related to the B1 stepping of the Intel® Ethernet Controller XL710-BM1. You can confirm this information on the ordering section stated on the following website:

https://ark.intel.com/content/www/us/en/ark/products/93104/intel-ethernet-controller-xl710-bm1.html

Based on this information, we suggest reviewing the erratum 11 information stated in Tables 1-1, 2-4, and section 2.4, on pages 9, 15, and 34 of the Intel® Ethernet Controller X710/ XXV710/XL710 Specification Update document # 331430. You can find this document when you are logged into your Resource & Design Center (RDC) privileged account on the following website:

https://cdrdv2.intel.com/v1/dl/getContent/331430

You should fill out the RDC Account Support form to process your account update request or report any inconveniences with the provided site. You can be found on the following website:

https://www.intel.com/content/www/us/en/forms/support/my-intel-sign-on-support.html

Best regards,

@CarlosAM_INTEL.

[Matevz]

Hi,

design has been developed by us.

We are aware of that erratum, however the erratum is talking about RX TC. This is not what we are experiencing.

We are having an issue with TX strict priority scheduling. 

We are sending traffic thru 8 queues each queue mapped to different TC and all 8 TCs configured as strict priority. The link partner is blocking transmission speed to 8,6 Gbit/s. 

Expected behaviour is that highest priority TCs would be able to transmit all traffic, while lowest priority TCs would be the ones dropping the traffic.

However this is not the case. Even higher priority TCs are loosing traffic.

regards,

Matevz

[Matevz]

The link partner is blocking transmission speed to 8,6 Gbit/s using XON/XOFF PAUSE frames.

[CarlosAM_INTEL]

Hello, @Matevz:

Thanks for your clarification.

Based on the information provided in previous communications, could you please let us know the procedure that you are following to obtain the documentation accessible with an RDC privileged account?

We are waiting for your clarification.

Best regards,

@CarlosAM_INTEL.

[Matevz]

Hi,

we login to RDC and download documents.

best regards,

Matevz

[CarlosAM_INTEL]

Hello, @Matevz:

Thanks for your clarification.

We sent an email to the address related to your account of this tool with some suggestions that may help you to solve this situation. 

Best regards

@CarlosAM_INTEL.