Calling CLWB Cause Memory Reading

[liangyang]

I run the snippets as shown below in Xeon Gold 5218, and I will see that the memory reading throughput is very high. Why is it?

 #include "memory.h"
 #include "xmmintrin.h"

 #define CACHE_LINE_SIZE 64
 #define CACHE_LINE_NUM 128

 static inline void
 clwb(const void *addr) {
     asm volatile(".byte 0x66; xsaveopt %0" : "+m" (*(volatile char *)(addr)));
 }

 int main() {
     void* dram_addr = malloc(CACHE_LINE_NUM * CACHE_LINE_SIZE);
     while (1) {
     ¦   int i;
     ¦   for (i = 0; i < CACHE_LINE_NUM; i++) {
     ¦   ¦   clwb(dram_addr + i * CACHE_LINE_SIZE);
     ¦   }
     ¦   _mm_sfence();
     }
     free(dram_addr);
     return 1;
 }

pcm-memory output:

|-- NODE 0 Mem Read (MB/s) :  7407.89 --||-- NODE 1 Mem Read (MB/s) :    24.63 --|
|-- NODE 0 Mem Write(MB/s) :    30.63 --||-- NODE 1 Mem Write(MB/s) :     9.34 --|
|-- NODE 0 PMM Read (MB/s):      0.00 --||-- NODE 1 PMM Read (MB/s):      0.00 --|
|-- NODE 0 PMM Write(MB/s):      0.00 --||-- NODE 1 PMM Write(MB/s):      0.00 --|
|-- NODE 0.0 NM read hit rate :  1.00 --||-- NODE 1.0 NM read hit rate :  0.95 --|
|-- NODE 0.1 NM read hit rate :  1.00 --||-- NODE 1.1 NM read hit rate :  0.95 --|
|-- NODE 0 Memory (MB/s):     7438.52 --||-- NODE 1 Memory (MB/s):       33.97 --|
|---------------------------------------||---------------------------------------|
|---------------------------------------||---------------------------------------|
|--            System DRAM Read Throughput(MB/s):       7432.52                --|
|--           System DRAM Write Throughput(MB/s):         39.96                --|
|--             System PMM Read Throughput(MB/s):          0.00                --|
|--            System PMM Write Throughput(MB/s):          0.00                --|
|--                 System Read Throughput(MB/s):       7432.52                --|
|--                System Write Throughput(MB/s):         39.96                --|
|--               System Memory Throughput(MB/s):       7472.48                --|
|---------------------------------------||---------------------------------------|

[SergioS_Intel]

Hello liangyang,

Can you please let us know when you started noticing this issue? Was there any hardware changes made recently?

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

[liangyang]

I am doing an experiment that copies data on PMem. In contrast with DRAM, it is necessary for PMem to flush cache line (CLWB) to avoid losing data when power failure. I know flushing dirty cache line cause memory(or PMem) reading, but I wonder why flushing clean or invalid cache line cause memory(or PMem) reading. Absolutely, the reading brings performance penalty. how can I avoid it?

[SergioS_Intel]

Hello liangyang,

Please allow us to check on your question and we will get back to you as soon as possible.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

[SergioS_Intel]

Hello liangyang,

Can you please provide us the part number of the memory that you are using and the model of the server board?

We will be looking forward to your response.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

[SergioS_Intel]

Hello liangyang,

We are following your case and would like to know if you were able to gather the model of the server board and the part number of the memory.

We will be looking forward to your response.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

[SergioS_Intel]

Hello liangyang,

We are following your case and would like to know if you need more assistance or if we can close this thread.

We will be looking forward to your response.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios

[SergioS_Intel]

Hello liangyang,

We are following up on a case that is still open with the issue "CLWB Cause Memory Reading", we know that this is important for you to get it resolved and it is also equally important for us to get you the right solution.

Since we have not seen an update for several days, the case will automatically close after 2 business days. You can still re-open the case by logging into your web portal or replying to this email.

Please note that if the case is re-opened more than 15 days after closure, a new case will be created with a reference to the original case.

Best regards,

Sergio S.

Intel Customer Support Technician

For firmware updates and troubleshooting tips, visit :https://intel.com/support/serverbios