Regarding machine clears

[Jia_Rao]

Hi all,

What are the reasons to issue machine clear signals. Do these signals relate to pipeline clears?

On Intel westmere, there are events breaking down the clear signals: MACHINE_CLEARS.CYCLES, MACHINE_CLEAR.MEM_ORDER, MACHINE_CLEARS.SMC. I understand that memory order violations and self-modifying code will flush the pipeline. 

What else? It seems that the trend in event MACHINE_CLEARS.CYCLES does not agree with the trend in MEM_ORDER or SMC.

Thanks.

[David_A_Intel1]

Hi Jia:

Excellent question!  The documentation says that MACHINE_CLEARS.CYCLES is "Cycles that the machine clear (nuke) signal is asserted."  While the other two counters count "events", that is, the number of times a MACHINE_CLEAR was issued for that reason.  It looks like in following architectures, we stopped using the .CYCLES event, for example, in Sandy Bridge processors (described here, page 52).

[Jia_Rao]

Thanks a lot!

Yes, I used some micro-benchmark with multiple threads contending for a spinlock. Thus, almost all the pipeline flushes are caused by memory ordering violations. I profiled the per-thread (on different cores, separate PMUs) MACHINE_CLEARS.CYCLES and MACHINE_CLEAR.MEM_ORDER events. Intuitively, threads with large number of MEM_ORDER events should have large clear CYCLES. However, it seems that there is no such correlations. 

Can each pipeline flush/clear incur different latency penalties (i.e., cycles)? So that some flushes are more expensive than others?

[David_A_Intel1]

I would definitely think so, although I have no knowledge of what kind of impact we are talking about.  Have you checked the Software Developer's Manual for Westmere?  I would have to look it up, as well.  The questions would be 1) can you change your code to remove some or all of the machine clears and 2) what increase in performance would you realize?  If you can't remove the machine clears, then the point is moot.  If you can, try it and measure the app to see what kind of performance increase you get.

[Bernard]

>>>I understand that memory order violations and self-modifying code will flush the pipeline.>>>

Also branch misprediction.

[Peter_W_Intel]

Another possibility is false sharing on same memory space from two threads, without lock protection. It will be very expensive for cache invalidation. 

[Alexander_Alexeev]

Looks like this event will be available on Broadwell again.

C3H 01H MACHINE_CLEARS.CYCLES Counts cycles while a machine clears. stalled forward progress of a logical processor or a processor core.

[Bernard]

>>>Can each pipeline flush/clear incur different latency penalties (i.e., cycles)? So that some flushes are more expensive than others?>>>

I think that yes and it depends on the code  which was executed prior to the pipeline clear/flush. Moreover the exact pipeline stage at which code execution was stopped because of flush will have impact on the time in cycles needed to re-execute the same code.