Measuring row-buffer hit ratio?

Michael_J_1 — Fri, 29 Aug 2014 22:47:24 GMT

Hello,

I am a current user of the Intel Performance Counter tool, and I have found it to be very helpful for gathering information on DRAM power consumption and other things like the L2 and L3 cache hit ratio. I was wondering if this tool could be used to measure things like the row-buffer hit ratio or the number of bank conflicts. If not, do you know of any other tools or counters I could use to gather this information? If it makes any difference, my platform uses an Intel E5-26xx series processor with Linux as its operating system.

Thanks in advance.

Mike

The Xeon E5-2600 Family

McCalpinJohn — Tue, 02 Sep 2014 21:50:00 GMT

The Xeon E5-2600 Family Uncore Performance Monitoring Guide (document 327043-001, March 2012) describes the available performance counters in the memory controller portion of the uncore in section 2.5.

Performance counter events are available to count page hits, page misses, and page conflicts -- BUT, you need to pay careful attention to the terminology. What Intel calls a "page miss" is usually referred to as a "page conflict". The event counts precharge commands that occur when the page is open, but has the wrong row in it. The other metrics can be obtained by arithmetic, as discussed below.

There are four programmable performance counters in each of the four memory controllers on each chip. Fortunately, events are available that allow you to get the most important data in a single run. My standard set is:

ACT_COUNT, "DRAM Activate Count", Event 0x01, Umask 0x00
CAS_COUNT.RD, "All DRAM Reads", Event 0x04, Umask 0x03
CAS_COUNT.WR, "All DRAM Writes", Event 0x04, Umask 0x0C
PRE_COUNT.PAGE_MISS, "DRAM Precharge Events due to Page Miss (i.e., Conflict)", Event 0x02, Umask 0x01

The derived events are:

PCT_REQUESTS_PAGE_EMPTY = (ACT_COUNT - PRE_COUNT.PAGE_MISS) / (CAS_COUNT.RD + CAS_COUNT.WR)
PCT_REQUESTS_PAGE_MISS (i.e., conflict) = PCT_COUNT.PAGE_MISS / (CASE_COUNT.RD + CAS_COUNT.WR)
PCT_REQUESTS_PAGE_HIT = 1 - PCT_REQUESTS_PAGE_EMPTY - PCT_REQUESTS_PAGE_MISS

These events are not perfect -- they don't count precharges that come from the "PRECHARGE ALL" command, for example -- but they seem to be quite reasonable in the tests I have run. ("PRECHARGE ALL" should only happen when refreshing the memory or in some power state changes and these are infrequent -- but there is a separate event for it if you want to see how often it happens.)

Of course these events also give the total memory traffic:

MEM_BW_READS = 64 * CAS_COUNT.RD
MEM_BW_WRITES = 64 * CAS_COUNT.WR

This agrees very closely with expectations in the experiments that I have done, but it is not going to be perfect. There are lots of special cases (uncached loads, partially completed streaming stores, IO references to memory, etc) that introduce a small amount of variability into the results.

In newer versions of Linux, the "perf" subsystem might support accessing these events relatively directly. I have not used the Intel Performance Counter Monitor tool recently, so I don't know if it has these events built-in.

Dr. B:

Zhu_G_ — Wed, 19 Nov 2014 04:35:49 GMT

Dr. B:

Hi, I read the guide and I think the umask is wrong.

CAS_COUNT.WR, "All DRAM Writes", Event 0x04, Umask 0xC0

I think the umask of WR_CAS is 0x0C. Am I wrong or I missed something here?

Thanks for catching that

McCalpinJohn — Fri, 21 Nov 2014 22:14:31 GMT

Thanks for catching that error -- I have corrected the post.

topic Dr. B: in Software Tuning, Performance Optimization & Platform Monitoring

Measuring row-buffer hit ratio?

The Xeon E5-2600 Family

Dr. B:

Thanks for catching that