Turning off the cache coherence directory system wide in Intel Xeon Gold 6242 processor

[vimalm]

Intel Architecture Optimization Reference Manual suggests in section 12.3.1 that it's possible to turn off the cache coherence directory system-wide and use snoopy coherence protocol instead. 

I need to do this to verify that some of the extra writes to memory is because of coherence directory writes or not.

Could you please help me with how to turn off the coherence directory in Intel Xeon Gold 6242 processor?

Thanks.

[McCalpinJohn · Accepted solution]

This will be controlled by the BIOS.  On my Dell R650 system the BIOS option is called "DirectoryMode" which is set to "enable" by default.  Presumably setting it to "disable" and rebooting would turn off the memory directories, but I have not tested this option.

[vimalm]

Thanks for your reply, Mr. John.

Actually, the current system I am working on does not have that option but there was this option called "I/O Directory Cache". I'm attaching the explanation for the same taken from the system provider.

Now, it seems to me that this is about changing the configuration of cache for memory directory entries and not about turning off the entire memory directory itself. I have tested three of those options and in all of those, it seems to me that there are writes happening to the memory directory. The options which I have tested whether the memory directory updates are happening or not are - "Disable", "Auto", "Enable for Remote InvLtoM Hybrid Push".

To count the number of memory directory updates I'm using performance monitor unit event "unc_m2m_directory_update_any" using ocperf - A wrapper for perf. 

I'm trying to contact my system provider.

But is it correct to say that this setting controls the cache for cache coherence directory entries and doesn't control the use of the memory directory?

[McCalpinJohn]

I think your interpretation is correct -- those options are related to caching of IO operations and not related to the memory directory that tracks whether a cache line might be dirty in another socket.

In earlier processors, the alternatives to "memory directory" were usually referred to as "home snoop" and "early snoop".  It is possible that your BIOS might list something related to those names....

I can't tell from your notes whether the issue is to understand the memory writes due to directory updates or to eliminate them.

It should be possible to determine how many of the memory updates are due to accesses from the other socket that cause directory updates, but I have not tested all the relevant performance counter events myself.  The documentation in the uncore performance monitoring guide is minimal, but careful reading of the SMI3 opcode tables should provide enough clues to get started on directed microbenchmarks to test specific hypotheses.  

[vimalm]

Thank you very much for the suggestions.