topic Write Combining Buffer in Software Tuning, Performance Optimization & Platform Monitoring

Write Combining Buffer

Srinath_Sridharan — Tue, 15 May 2012 23:44:00 GMT

Hi,

I am trying to leverage the write combining buffer in X86 processors to perform some memory and io optimizations.

My question is,

is it possible for me probe into the write combining buffer and know exactly how many bytes are getting evicted out ? Are there any hacks/performance counters that can give me this information ?

Thanks

Srinath

Write Combining Buffer

Patrick_F_Intel1 — Wed, 16 May 2012 02:16:29 GMT

Hello Srinath,
I don't think there is any way do what you want to do.
On older processors there was a partial wc buffer write event (I think) but this event doesn't exist on current processors. And the event didn't tell how many bytes were written, just the number of times a partial write occurred.
Sorry,
Pat

Write Combining Buffer

Srinath_Sridharan — Wed, 16 May 2012 04:14:01 GMT

Thanks Patrick. Thanks for the quick response.

Is there a way to retrive data addresses (virtual or physical) accessed by loads and stores (I am specifically looking for non-temporal stores) ? I was initially using PIN to do that and the performance was terrible. Is there a way to get that info from the hardware directly, say at the time of instruction retirement or something ?

Thanks

Srinath

Write Combining Buffer

Srinath_Sridharan — Wed, 16 May 2012 17:11:46 GMT

Hey

I actually have one more question. Is there any information that I can get from write combining buffers on modern processors ?

Srinath

Write Combining Buffer

TimP — Wed, 16 May 2012 18:16:31 GMT

Only by looking it up under the current equivalent name "fill buffer," and not a great deal.

Write Combining Buffer

levicki — Fri, 18 May 2012 00:00:24 GMT

Let me see if I understand your questions correctly -- are you trying to use non-temporal stores but you are not seeing the improvement?

Write Combining Buffer

Srinath_Sridharan — Fri, 18 May 2012 18:10:36 GMT

My orignial question is as follows:

Is there any meta information (possibly through hardware counters) that I can retrieve from Write Combing Buffers (WCB) (I mmap-ed by address space to WC mode to bypass cache) ?

My question was not about performance. I am trying to save some cache space by bypassing writes that don't need any locality, directly into a storage device. Similar to bypassing framebuffers into the graphics device.

One of the problems I am facing is that, I need to provide some correctness guarantees. I need to verify if the cache-lines flushed from the WCBs have all reached the destination. But to do that I need some information from the processor side such as

1) physical address of cache lines flushed from WCB

2) Number of bytes

3) Atlleast number of partial/full lines flushed

Any combination of 1), 2) or 3) is fine.

Is there a way for me retrieve such information somewhere from the processor ?

Srinath

Write Combining Buffer

levicki — Sat, 19 May 2012 12:30:30 GMT

As you probably know, non-temporal load/store instructions (movntps, movntdq, etc) are a way to bypass the cache on read and write.

Since non-temporal stores are weakly ordered before using the data you need to issue mfence/lfence/sfence (depending on what you are doing with the data, most likely sfence in your case).

Those fencing instructions are the only guarantee that the data has reached the destination before you use it.

As far as I know, the metrics that you are looking for are not available as CPU counters -- perhaps they can be observed through ITP debugging but I am not sure about that, and such hardware is extremely expensive anyway.