topic Allocating Memory Bandwidth in Intel® Xeon® Processor and Server Products

Allocating Memory Bandwidth

amoghavs — Thu, 04 Mar 2021 20:55:57 GMT

I am getting familiar with memory bandwidth allocation and I wanted to make sure I am doing the right thing.

Commands:
sudo pqos -R
sudo pqos -e "mba@0:0=50;mba@0:1=50"
sudo pqos -a "llc:0=0-3;llc:1=4-9"

Output: sudo pqos -s

NOTE: Mixed use of MSR and kernel interfaces to manage
CAT or CMT & MBM may lead to unexpected behavior.
L3CA/MBA COS definitions for Socket 0:
L3CA COS0 => MASK 0x7ff
L3CA COS1 => MASK 0x7ff
L3CA COS2 => MASK 0x7ff
L3CA COS3 => MASK 0x7ff
L3CA COS4 => MASK 0x7ff
L3CA COS5 => MASK 0x7ff
L3CA COS6 => MASK 0x7ff
L3CA COS7 => MASK 0x7ff
L3CA COS8 => MASK 0x7ff
L3CA COS9 => MASK 0x7ff
...
L3CA COS15 => MASK 0x7ff

MBA COS0 => 50% available
MBA COS1 => 50% available
MBA COS2 => 100% available
MBA COS3 => 100% available
MBA COS4 => 100% available
MBA COS5 => 100% available
MBA COS6 => 100% available
MBA COS7 => 100% available
...
Core information for socket 0:
Core 0, L2ID 0, L3ID 0 => COS0, RMID0
Core 1, L2ID 1, L3ID 0 => COS0, RMID0
Core 2, L2ID 2, L3ID 0 => COS0, RMID0
Core 3, L2ID 3, L3ID 0 => COS0, RMID0
Core 4, L2ID 4, L3ID 0 => COS1, RMID0
Core 5, L2ID 8, L3ID 0 => COS1, RMID0
Core 6, L2ID 9, L3ID 0 => COS1, RMID0
Core 7, L2ID 10, L3ID 0 => COS1, RMID0
Core 8, L2ID 11, L3ID 0 => COS1, RMID0
Core 9, L2ID 12, L3ID 0 => COS1, RMID0

Confusion:

AFAIK RMIDs are used to associate the specified configuration (as per this link: https://01.org/cache-monitoring-technology) but the RMID values for both the CLOS is same in my case. Is this alright, if not, how can I change that?

Re:Allocating Memory Bandwidth

SergioS_Intel — Fri, 05 Mar 2021 02:47:55 GMT

Hello amoghavs,

Thank you for contacting Intel Customer Support.

In order to better assist you, can you please provide us the model of the Intel(R) product that you need assistance with?

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

Re: Re:Allocating Memory Bandwidth

amoghavs — Sat, 06 Mar 2021 17:07:38 GMT

I found the clarification about this in the intel-cmt-cat wiki: https://github.com/intel/intel-cmt-cat/wiki/MBM-MBA-how-to-guide , thank you.

Re:Allocating Memory Bandwidth

SergioS_Intel — Tue, 09 Mar 2021 02:32:53 GMT

Hello amoghavs,

We are glad to hear you were able to resolve your problem. We are going to proceed and close this thread.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

Re: Re:Allocating Memory Bandwidth

amoghavs — Wed, 10 Mar 2021 23:23:35 GMT

This is an incomplete answer.

I had a couple of questions while I posted the question,

1. Should allocation impact RMID or are the commands used "right"?

Answer: NO. RMID is related to monitoring, creating/altering COS should not impact RMID.
More on RMIDs: https://software.intel.com/content/www/us/en/develop/blogs/intel-s-cache-monitoring-technology-software-visible-interfaces.html

2. When does RMID actually change.
Answer: RMIDs change when we are monitoring
Command:

cmd-1: sudo pqos -m "all:[0-3],[4-9]"

cmd-2: sudo pqos -s

While cmd-1 is running if we check on RMID status through cmd-2 in a different shell, this is the output we get:

Core information for socket 0:
Core 0, L2ID 0, L3ID 0 => COS0, RMID1
Core 1, L2ID 1, L3ID 0 => COS0, RMID1
Core 2, L2ID 2, L3ID 0 => COS0, RMID1
Core 3, L2ID 3, L3ID 0 => COS0, RMID1
Core 4, L2ID 4, L3ID 0 => COS1, RMID2
Core 5, L2ID 8, L3ID 0 => COS1, RMID2
Core 6, L2ID 9, L3ID 0 => COS1, RMID2
Core 7, L2ID 10, L3ID 0 => COS1, RMID2
Core 8, L2ID 11, L3ID 0 => COS1, RMID2
Core 9, L2ID 12, L3ID 0 => COS1, RMID2

The output clearly shows that RMIDs are used based on groups of cores.