- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hi all,

I'm trying to measure the resource stalls on a SandyBridge machine. As the SDM introduces, I can break the stalls down to different causes as shown below.

RESOURCE_STALLS:ANY - 0x5301a2

RESOURCE_STALLS:LB - 0x5302a2

RESOURCE_STALLS:RS - 0x5304a2

RESOURCE_STALLS:SB - 0x5308a2

RESOURCE_STALLS:ROB - 0x5310a2

RESOURCE_STALLS:FCSW- 0x5320a2

RESOURCE_STALLS:MXCSR - 0x5340a2

To my understanding, the number of ANY is supposed to be equal to the sum of the other 6 numbers. But it seems to be wrong based on my experiments.

RESOURCE_STALLS:ANY: 397349463242

RESOURCE_STALLS:LB: 5379423

RESOURCE_STALLS:RS: 304513727418

RESOURCE_STALLS:SB: 2248871709

RESOURCE_STALLS:ROB: 18753462189

RESOURCE_STALLS:FCSW: 0

RESOURCE_STALLS:MXCSR: 0

Is that right, or there are more other causes of resource stalls that could not be counted? What are those causes?

Thank you very much!

Link Copied

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hello Yunqi,

From http://software.intel.com/sites/products/documentation/doclib/stdxe/2013/amplifierxe/win/ug_docs/ref..., it doesn't look like the .ANY event counts the other events.

Pat

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hi Patrick,

I assume so, but the numbers are telling something different from that assumption. :-(

Do you know anyone or anyhow I could verify this with Intel?

Thank you very much!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

When you say "the numbers are telling something different from that assumption"... can you be more explicit? What is the assumption you are making and how are the numbers proving/disproving the assumption?

Pat

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hi Patrick,

Sorry for the confusion. I have the results of one experiment in the first post, where you could do the calculation and find that the .ANY is not equal to the sum of the rest stalls.

.ANY = 397349463242

REST = 5379423(.LB) + 304513727418(.RS) + 2248871709(.SB) + 18753462189(.ROB) + 0(.FCSW) + 0(.MXCSR) = 325521440739

Thank you very much for your help!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Right.

I would not expect the .ANY to be equal the rest.

What makes you think that .ANY would be equal to the rest?

Pat

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hmm, I thought .ANY was the sum of the resource stalls no matter what the cause is. Problem solve. Thank you so much! :-)

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

The name is not the most clear but they already used .OTHER for another sub-event.

Usually, when an event includes all the sub-events then the umask is the bitwise OR of the sub-events. In this case if you wanted to make a .ALL event, the umask would be 0xff.

Pat

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hi Yunqi,

jumping late to interesting discussion:) Below is detailed explanation of the event RESOURCE_STALLS.ANY meaning.

The number of instructions in the pipeline waiting for execution or retirement reached the limit the processor can handle.

The number of load or store instructions in the pipeline waiting for retirement reached the limit the processor can handle.

There is an instruction in the pipe that can be executed only when all previous stores complete and their data is committed in the caches or memory. Fore example, SFENCE and MFENCE instructions require this behavior.

The pipeline recovers from a mispredicted branched that was executed.

The floating-point unit ( FPU ) control word is written.

As you can this event is not a sum of the events measured by you in your post.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Cool! Thank you very much iliyapolak! :-)

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

You are always welcome :)

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Is there any way to convert a RESOURCE_STALL.SB count back into how much performance impact I am getting? I am doing a large FFT operation with FFTW and MKL and I have well vectorized floating-point, but lots of poorly vectorized load/store operations since complex numbers are interleaved. I suspect that is exhausting the store buffer, but how do I quantify this effect?

Brian

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

The cost in cycles can be as large as the count from the RESOURCE_STALLS.SB event, or as small as zero, depending on the extent to which the store buffer full stalls overlap with other stalls and/or other work.

If this is not bad enough, it is even more confusing in parallel workloads because of the additional 0%-100% range of possible overlap of stalls across threads. So in the parallel case, one increment of the RESOURCE_STALLS.SB event can have a net execution time cost of anywhere between zero cycles and 1 cycle times the number of cores used in the job.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Yunqi Z. wrote:Hi Patrick,

Sorry for the confusion. I have the results of one experiment in the first post, where you could do the calculation and find that the .ANY is not equal to the sum of the rest stalls.

.ANY = 397349463242

REST = 5379423(.LB) + 304513727418(.RS) + 2248871709(.SB) + 18753462189(.ROB) + 0(.FCSW) + 0(.MXCSR) = 325521440739

Thank you very much for your help!

In this case, the umask does not operate the same as other events: each umask bit does not represent independent, mutually exclusive events. All the bits *except* 0x01 (the .ANY event) operate like that, but the ANY bit includes those events *plus *some other stall causes which are not covered by the remaining bits. So I believe that umask=0x01 will always be greater than or equal to umask=0xFE.

The likely cause is that the full list of stall events are split across at least two performance counter events: RESOURCE_STALLS and RESOURCE_STALLS2. If a stall cause falls into the STALLS2 categories, it won't show up in any of the specific stall causes for RESOURCE_STALLS, but it *will *show up in the ANY bit, so you can could all stalls regardless of cause with only event (rather than trying to add STALLS and STALLS2.

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page