Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
5287 Discussions

X710-DA4 not respecting Base Processor

RTill
Beginner
3,707 Views

Have a dual socket HP Z840 with NIC attached to CPU1 runing windows server 2016.

Have a X710-DA4 with 22.6 driver installed.

Utilizing 2 ports on NIC, trying to set 1 port to Numa 1, core 0 and Numa 1 core 3.

Using Closest static profile.

Only the first NIC port for Numa 1 core 0 is being "respected.

The second NIC port for Numa 1, core 3 is landing on core 2...

I have tried changing around the second port to be on other cores as well to no avail...

Any idea if there is a bug to numa node base processor core binding?

Each NIC set to max process or 1 and queue 1...

Any suggestions would be appreciated...

0 Kudos
16 Replies
idata
Employee
2,299 Views

Hi Tilla,

 

 

Thank you for posting in Wired Communities. Can you provide the tool use to check on the status of the RSS output or where did you see the NIC port that core 3 is landed on Core 2? If possible can you share the steps on how you configure?

 

 

Thanks,

 

Sharon

 

 

 

 

 

0 Kudos
RTill
Beginner
2,299 Views

Thank you so much for responding, I appreciate any help that you can offer...

This is a bit long, but I will give you as much info on the physical setup and app as I can.

Physical Hardware Setup:

So I have a 2 physical processor HP Z840.

I have an X710-DA4 in PCIe slot # 3, which is "attached" to CPU1.

I have a device that streams heavy UDP data over 1-N of the ports on the X710-DA4.

I want each port of the 710-DA4 to "land" on a different core of CPU1.

Application:

I have an application that reads the streams that I have affinitize to CPU0 (this is to observe the receive load placed on CPU1 by the socket guaranteeing that the load is not from the app).

I then initiate a data stream from the device to my application and observe CPU load on task manager.

NIC Setup, Test # 1:

Preferred NUMA Node: 1

Max Num RSS Processors: 1

Max Num RSS Queues: 1

RSS: enabled

RSS Base Processor Number: 0

RSS Load Balancing profile: ClosestProcessorStatic

NIC Setup, Test # 2:

Preferred NUMA Node: 1

Max Num RSS Processors: 1

Max Num RSS Queues: 1

RSS: enabled

RSS Base Processor Number: 1

RSS Load Balancing profile: ClosestProcessorStatic

Expectations:

In the graphs above, CPU0 is the top 4 graphs, CPU1 is the bottom 4 graphs.

By changing the base processor number, I expected the socket receive CPU load to shift from processor 0 to processor 1 on CPU# 1.

Nothing else is running on the machine.

As you can see form the graphs, the recv socket load did not move to CPU1 from CPU0 despite the change in configuration...

Thanks for your time...

0 Kudos
RTill
Beginner
2,299 Views

I have tried many different permutations of configs as well, perhaps there is something about the config that I am not understanding or interpreting properly...

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Thank you for the detail information. I will check on this then update you.

 

 

Regards,

 

Sharon

 

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Can you provide the output of Get-NetAdapterRss for test# 1 and test# 2 ? Thank you.

 

 

Regards,

 

Sharon

 

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Please feel free to provide the information requested for me to better investigate.

 

 

Thanks,

 

Sharon

 

0 Kudos
RTill
Beginner
2,299 Views

Sorry for late response, I was on vaca last week...

The box has gone to the field, so I do not have direct access to it any more

I will see if I can find another box to recreate.

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Thank you for the update and hope you had a great vacation. Feel free to update me when you find another system for the setup.

 

 

Thanks,

 

Sharon

 

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Please feel free to update me in case you already have the setup.

 

 

Thanks,

 

Sharon

 

0 Kudos
RTill
Beginner
2,299 Views

Test # 1:

Name : 10GB-1-10.1

InterfaceDescription : Intel(R) Ethernet Converged Network Adapter X710

Enabled : True

NumberOfReceiveQueues : 1

Profile : ClosestStatic

BaseProcessor: [Group:Number] : :0

MaxProcessor: [Group:Number] : :

MaxProcessors : 1

RssProcessorArray: [Group:Number/NUMA Distance] :

IndirectionTable: [Group:Number] :

Test # 2

Name : 10GB-1-10.1

InterfaceDescription : Intel(R) Ethernet Converged Network Adapter X710

Enabled : True

NumberOfReceiveQueues : 1

Profile : ClosestStatic

BaseProcessor: [Group:Number] : :1

MaxProcessor: [Group:Number] : :

MaxProcessors : 1

RssProcessorArray: [Group:Number/NUMA Distance] :

IndirectionTable: [Group:Number] :

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Thank you for the information. I will further check on this.

 

 

Regards,

 

Sharon

 

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Further checking, please set Max Num RSS Processors = 2 and "RSS Base Processor Number" to 1.

 

 

Please feel free to update me.

 

 

Thanks,

 

Sharon

 

0 Kudos
RTill
Beginner
2,299 Views

I could try that, but in essence isnt that saying "use either core 1 or core 2" (base processor=1 and max processors = 2)?

I desire to be able to pin CPU load for a specific NIC port to a single specific processor in order to efficiently load balance my high network traffic.

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

Thank you for the reply. If you set "Max Num RSS Processors" to 1, it only allows RSS to use CPU 0. Setting "Max Num RSS Processors" to 2 and "RSS Base Processor Number" to 1 would force all to CPU 1.

Max processors = 1 can use only CPU 0

Max processors = 2 can use CPU 0 and 1

Max processors = 3 can use CPU 0,1,2

Max processors = 4 can use CPU 0,1,2,3

 

 

Please feel free to update me.

 

 

Thanks,

 

Sharon

 

0 Kudos
RTill
Beginner
2,299 Views

Ahhh, OK, thanks for clearing up my confusion

0 Kudos
idata
Employee
2,299 Views

Hi Tilla,

 

 

You are welcome and please do let me know if further assistance needed.

 

 

Thanks,

 

Sharon

 

0 Kudos
Reply