topic The Sandy Bridge core has 8 in Software Tuning, Performance Optimization & Platform Monitoring

[PCM] Adding extra events "MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM" and "MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM"

wang__chi-lung — Wed, 09 Oct 2013 21:22:07 GMT

I am modifying PCM to monitor other events that are not included in current release.

More specifically, I am interested in "MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM" and "MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM".

The "Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3 (3A, 3B & 3C): System Programming Guide" comments "Disable BL bypass and direct2core (see MSR 0x3C9)".

I assume that I have to disable BL bypass and direct2core first. Then, I can obtain the values of these events.

However, I did not find anyplace that gives instructions to disable BL bypass and direct2core.

I would like to know:

1. Do I need to disable BL bypass and direct2core for getting the values of "MEM_LOAD_UOPS_LLC_MISS_RETIRED.LOCAL_DRAM" and "MEM_LOAD_UOPS_LLC_MISS_RETIRED.REMOTE_DRAM"?

2. If the answer is yes to 1, how do I do that?

I appreciate any information and guidance to answer these questions.

Thank you very much.

Ron C. Chiang

Have you looked at

Bernard — Thu, 10 Oct 2013 05:49:06 GMT

Have you looked at description of MSR 0x3C9 does it contain some info on how to disable BL bypass and direct2core?

Thank you for your reply,

wang__chi-lung — Thu, 10 Oct 2013 15:10:36 GMT

Thank you for your reply,

I have checked the description of MSR 0x3C9 in the manual, and searched "BL bypass" and "direct2core" in both the manual and internet.

Unfortunately, I did not find any clear information.

Is there any additional document for MSR?

>>>Is there any additional

Bernard — Thu, 10 Oct 2013 16:04:02 GMT

>>>Is there any additional document for MSR?>>>

I am afraid that beside the SDM documentation there is no freely available more in-depth documentation.

Thank you for your reply,

wang__chi-lung — Thu, 10 Oct 2013 17:13:48 GMT

Thank you for your reply,

Does that imply the answer is in some documents that are not freely available?

Also, do I need BIOS support to disable BL bypass and direct2core? If not, can BL bypass and direct2cire be disabled by setting MSR?

Thank you very much!

I suppose that more advanced

Bernard — Fri, 11 Oct 2013 05:44:34 GMT

I suppose that more advanced technical documentation will be available for Bios vendors based upon some kind of NDA agreement.

Unfortunately I do not have an answer to your last question.

Please see this article

Roman_D_Intel — Fri, 11 Oct 2013 06:25:05 GMT

Please see this article describing the workaround: http://software.intel.com/en-us/articles/performance-monitoring-on-intel-xeon-processor-e5-family

Intel PCM implements it in PCM::enableJKTWorkaround function.

Roman

Hi Ron

Bernard — Fri, 11 Oct 2013 07:45:36 GMT

Hi Ron

where in the manual is description of MSR 0x3C9?

Sorry for the late reply and

wang__chi-lung — Thu, 17 Oct 2013 03:24:00 GMT

Sorry for the late reply and Thanks for the response from iliyapolak and Roman.

iliyapolak,

I can only find Table 18-39 and 35-18 (in the manual Vol.3) mentioned MSR 0x3c9. But, I did not find any explanation there.

Roman, Thank you for the hint.

My machine is Xeon E5-2620. PCM detects it as "Intel(r) microarchitecture codename Sandy Bridge-EP/Jaketown"

After seeing your reply and trace the code again, I found that workaround is already enabled. So, I started to debug my modification.

The original pcm.x monitors 4 events on this machine. My first attempt was extending "coreEventDesc" in cpucounters.cpp. For example, I add the following code for one extra event.

"coreEventDesc[4].event_number = MEM_LOAD_UOPS_LLC_MISS_RETIRED_LOCAL_DRAM_EVTNR;

coreEventDesc[4].umask_value = MEM_LOAD_UOPS_LLC_MISS_RETIRED_LOCAL_DRAM_UMASK;"

I also changed corresponding numbers, e.g, core_gen_counter_num_used = 5; //It was 4 before the modification.

However, I never get a valid number from this implementation.

Then, I tried another way today. I change one of the 4 originally mornitored events to the one I want, instead of adding extra one to "coreEventDesc".

It works. Now I can collect other events.

I am happy to have this problem solved. But, I have one more question. pcm.x shows that "Number of core PMU generic (programmable) counters: 8". So, I thought that I have 8 counters to use and started my first attempt. However, it seems there are only 4 available?

Ron

The Sandy Bridge core has 8

McCalpinJohn — Thu, 17 Oct 2013 16:01:20 GMT

The Sandy Bridge core has 8 programmable performance counters, but these are split into two groups of four when HyperThreading is enabled.

Thank you, John,

wang__chi-lung — Thu, 17 Oct 2013 16:11:53 GMT

Thank you, John,

I did disable HyperThreading on this machine.

Hi,

Roman_D_Intel — Tue, 22 Oct 2013 11:28:00 GMT

Hi,

when Hyperthreading is disabled there should be 8 programmable counters in hardware on your system. 4 hardware programmable counters are available if Hyperthreading is enabled. Intel PCM currently supports 4 programmable counters. We might address this in a future version of Intel PCM.

Roman