Will Intel Optane DC sharing memory bus cause increased DRAM contention?

[Vijayendral]

Hello,

Basic question: I install Intel® Optane™ persistent memory (PMem)  in App Direct mode and my application is issuing loads/stores to both DRAM and PMEM, simultaneously. 

Because PMEM resides on the same memory bus/channel as DRAM, will my application's DRAM traffic not see increased contention. The hitherto DRAM only traffic now has to compete (share memory bus, cache etc) with PMEM traffic and therefore might actually see degradation? Is this the case? If yes, how much is the degradation expected?  And how do I measure it, for my specific application? Given typical DRAM capacity w.r.t to PMEM capacity 32GB vs 512 GB, (with all other things being equal) will we not have 8X increased traffic? What are the tools available to mitigate it?

If not, why not? Is there documentation / white paper that describes these effects?

[Emeth_O_Intel]

Hello Vijayendral, 

Thank you for contacting Intel DCPMM Community. 

I will be more than happy to assist you with this concern. 

Can you provide us more details about the identification process or tool you used in order to conclude that your application has been facing issues in order to load and store on the DRAM and PMEM? Also, please let us what type of application are you configuring to use the DCPMM Features?

Answering your question about "Because PMEM resides on the same memory bus/channel as DRAM, will my application's DRAM traffic not see increased conflict."

The host memory controller communicates with the Intel® Optane™ DC persistent

memory using a unique transactional protocol (DDR-T) that is mapped to the standard DDR4 channel signals. Commands and addresses are packetized, along with reading and write data, and transferred across a standard DDR4 channel in multiple UI frames sent over contiguous bus cycles. Separate specifications detail the DDR4 electrical details, the DDR4 connector and pinning, and the DDRT protocol.

Meanwhile, you provide me with the details asked above, I will investigate more details about the information you mentioned in order to provide you the most accurate details. 

As soon as possible, I will be getting back to you in order to proceed with the next step. 

Regards, 

Emeth O. 

Intel Server Specialist. 

[Vijayendral]

1. I have still not measured any degradation, I am just studying and hypothesising 'IF' there could be degradation. My application is a database software.
2. Lets start from the beginning: 
   1. Cache (L1d/L1i/L2/L3) -- because of the additional cache lines occupied by the loads/stores destined to PMEM. 
   2. I found the attached .jpeg in the internet which says, there are additional peripherals for DDR-T protocol. (And your previous answer also suggests the same.) So there is a new memory controller protocol (DDR-T) which will specifically handle all the traffic to DC PMem. Thanks!

I was hoping to monitor the cache usage using Intel RDT's CMT (See here) but segregating the ld/st whose destination is PMEM with the ld/st whose destination is DRAM 'within' my application is a challenge for me. Let me know if you have any further ideas. And how to estimate the degradation (if any) because increased cache line occupancy.

Thanks

[Emeth_O_Intel]

Hello Vijayendral,

Thank you for your prompt reply.

I would like to ask you some additional details just to make sure we are on the same page.

• Are you trying to write the same data to both DIMM and PMem? If so, can you provide me with more details about why you would like to achieve this configuration?
• On the other hand, have you tried using the Intel® Memory Latency Checker (Intel® MLC)?

The Intel® MLC tool is used to measure memory latencies and bandwidth and show how they change when increasing the load on the system. Intel MLC also provides several options for a more fine-grained investigation where bandwidth and latencies from a specific set of cores to caches or memory can be measured.

Please check the Intel MLC and other benchmarking tools and analysis tools in the following websites:

• https://software.intel.com/content/www/us/en/develop/articles/configure-manage-and-profile-intel-optane-dc-persistent-memory-modules.html
• https://www.intel.com/content/www/us/en/support/articles/000032880/memory-and-storage/data-center-persistent-memory.html

Please let me check the answer to the question above and also what do you consider about the Intel MLC tool.

Have a wonderful day.

Regards,

Emeth O.

Intel Server Specialist.

[Emeth_O_Intel]

Hello Vijayendral,

I was reviewing your thread and I would like to know if the information provided help you to have a better understanding of this matter and also if the Intel MLC and other benchmarking and analysis tools suggested help to monitor the performance of the PMEM.

If you have more questions, please do not hesitate and let me know and I will be more than happy to help you.

Regards,

Emeth O.

Intel Server Specialist.

[Emeth_O_Intel]

Hello Vijayendral,

I was reviewing your thread and I have not seen any activity recently.

Please, if you have more questions; do not hesitate and let me know and I will be more than happy to help you.

Regards,

Emeth O.

Intel Server Specialist.

[Vijayendral]

Ok thanks for the pointers.

We will evaluate Application's (which uses mmap() of a file onto PMEM FS (DAX)) performance using Linux perf-stat tool and also the Intel MLC tool that you have suggested.

Specifically inside perf-stat we plan to measure the following Hardware cache events:

LLC-load-misses

LLC-store-misses

I will let you know, how the experiments go, meanwhile, please let me know, if there are any more suggestions from your side.

Thanks a lot!