UNCORE EVENT information on KNL

[DeokJae_Oh_]

Hello.

In order to measure the DDR4 and performance of MCDRAM memory of KNL system, I'm looking for uncore events associated with the memory.

I was looking for a reference document, there is only CAS_COUNT event.

(I know there are events, ACT_COUNT, PRE_COUNT, CAS_COUNT, DRAM_REFRESH, DRAM_PRE_ALL, MAJOR_MODES, PREEMPTION ... on Xeon system)

Please tell me it is possible to obtain a variety of event information, such as PRE_COUNT and ACT_COUNT on KNL.

[McCalpinJohn]

Both the core and uncore performance monitoring capabilities of the Xeon Phi x200 series are described in the 2-volume set of documents available at https://software.intel.com/en-us/articles/intel-xeon-phi-x200-family-processor-performance-monitoring-reference-manual

I agree that there are a surprisingly small number of events documented for the counters.  Perhaps more will be documented in the future, but right now the CAS.READS and CAS.WRITES are the only events available for DDR4.  Both these events appear to be accurate.  (UCLK and DCLK events are also available and appear to be correct, but they are not particularly interesting.)

The MCDRAM counters have to be interpreted carefully, but from my experiments RPQ.INSERTS and WPQ.INSERTS give the correct number of cache-line reads and writes to MCDRAM when running in Flat mode.

When running MCDRAM in cached mode, the formulas that Intel provides in Section 3.1 of Volume 2 of the Intel Xeon Phi Performance Monitoring Reference Manual seem reasonable -- especially for reads.  Essentially the MCDRAM is read for every "memory" reference, but the EDC.MISS_* events don't return data.   For writes I am seeing some discrepancies that I don't understand yet -- that analysis is still in progress.

[DeokJae_Oh_]

Thank you for your response.

Saying that support for the future mean that the additional uncore events are supported on the current KNL system??

(Additional uncore events = ACT_COUNT, PRE_COUNT, CAS_COUNT, DRAM_REFRESH, DRAM_PRE_ALL, MAJOR_MODES, PREEMPTION)

[McCalpinJohn]

Intel has clearly chosen to document a very small number of events, but went to the trouble of generating a 604-page "Volume 1" manual describing the performance counter infrastructure.   This suggests that there was, at some point in time, an intent to build a comprehensive performance monitoring infrastructure -- which would have to include more events than the tiny set documented now.

Lots of things might have happened....  Some possibilities include:

• The counters might exist, but be inadequately tested at launch, with a plan to roll out more documentation over time.
• The counters might exist, but be broken.
  • If broken, the counters might still be accessible (but without documentation).
    • This is the case with many Intel processors. 
      • Example: the initial documentation for the Ivy Bridge processors excluded the 0x10 and 0x11 floating-point events (which were quite buggy, but still useful for some purposes), but these were present in the hardware and were later documented.
      • Example: the 0xC7 event is not documented on the Haswell processor, but it can be programmed and it does count something.  My current guess is that it counts the events that were defined under the 0xC7 event on Nehalem/Westmere, but I have only done a few superficial tests.
  • If broken, the counters might have been disabled.
    • Example: The 0x10 and 0x11 Floating Point events don't count anything on Haswell.

I don't think that the more detailed DRAM counters are likely to be useful on Xeon Phi x200 -- there are too many cores fighting for access to the DRAM controllers for it to be practical to modify code to improve DRAM behavior.  I am happy that the CAS counts appear to be correct.