Monitoring PCI performance from software?

[David_Cherkus]

Hi,

I am working on a driver performance issue and I could really use something like:

http://pcitop.berlios.de/

It's description says:

It pokes and prods counters in the root bridge chipsets found in a system, parses the data, and produces a display showing utilization as a percentage for each root bridge. Note that the tool does not show bandwidth, just the ratio of use to the maximum use possible.

• Measures PCI, PCI-X, and PCIe bus utilization.
• Sampling interval can be set by the user.
• Filters can be used to only display specific buses, based on slots, PCI domains, and PCI bus numbers.
• Outputs useful PCI bus information.

And indeed it operates just like a version of the well known *ix 'top' utility, except it shows PCI stats instead of CPU stats.

Screen shot:

But the rub is:

1) Pcitop is currently only supported on Linux on HP Itanium (IA-64)

platforms running a kernel version of 2.6.16 or later. We hope to add

more platforms later as vendors release the needed root bridge interface

specifications.

So, does anyone know if Intel has a copy of this utility that works for current systems (ie nahalem, westmere and sandy bridge based xeon systems)?

Or is there any other similar tool or API to get this kind of info?

Thanks,

Dave

[Hussam_Mousa__Intel_]

Hi,

The closest tool I can think of is the Performance Counter Monitorwhich will only collect events at the uncore level.

You can review Table 19.8 in:
Intel 64 and IA-32 Architectures Software Developer's Manual Volume 3B: System Programming Guide, Part 2

For available uncore events that might help your PCI level analysis.
I am persoallyunaware of other tools that will replicate the pcitop functionality on x86.

-Hussam

[Roman_D_Intel]

Hi,

Intel Xeon Processor E5-2600 Series (previously codenamed "Sandy Bridge-EP") has a monitoring device to monitor ring and IIO traffic from/to PCIe. It is documented here (see section 2.8 - R2PCIe Performance Monitoring).

--
Roman

[SergeyKostrov]

Quoting David Cherkus

...Or is there any other similar tool or API to get this kind of info?...

WMI should provide that information and please take a look atCIM_Controller or CIM_PCIController classes.

Best regards,
Sergey

[alextui]

why don't you use Intel PCM, is it not enough for monitoring?

[David_Cherkus]

I wasn't aware of "performance counter monitor" and its capabilities till now.

I did start a thread called "Which Intel tool for the job?", but got no responses.
See http://software.intel.com/en-us/forums/showthread.php?t=104760&o=a&s=lr

In any case, I have built pcm for linux and it's almost working.

software.intel.com/en-us/articles/intel-performance-counter-monitor says:

Dear readers and users,

for questions and comments about Intel PCM and its use-cases, we recommend the Software Tuning, Performance Optimization & Platform Monitoring forum: http://software.intel.com/en-us/forums/platform-monitoring/

If you post a question/comment on this forum regarding Intel PCM, please mention "Intel PCM" in the title of your forum topic to catch our attention. Thanks, Intel PCM team

... so I will do that right now.

Thanks,
Dave

PS: Thanks to Hussam and Roman for the references to the Intel docs.
PPS: Sergey, Google is hinting that WMI is for Windows, no? If so, I can't use it at this time.

[Roman_D_Intel]

Dave,

just to avoid missunderstanding: latest Intel PCM 2.0 does notdisplay/support PCIecounters (although the monitoring capability exists in hardware as I mentioned earlier in this thread).

--
Roman

[David_Cherkus]

Re: no PCIe counter support in PCM 2.0: Thanks for the clarification.

[Patrick_F_Intel1]

Hello David,
Have you considered using OS counters to monitor PCI traffic?
On windows this would be the MS Perfmon counters like 'physical disk read/write' counters?
On linux this would be sysstat iostat counters.
MS Perfmon and sysstat also have counters for network traffic.
Pat

[AYADA9]

Hi Roman,

I am working on Dual Intel Xeon Socket E5-2600 based board and need to find the PCIe utilization. You have mentioned in this thread about the Uncore Guide and I have only skimmed through section 2.8 - R2PCIe Performance Monitoring. But I had 2 questions:

A. Even with the latest PCM Version 2.4, there doesnt seem to be any support for PCIe counters. Is there any Beta version or another software to get this utilization?

B. Is there any document I can refer to on how to interpret these results/counters in terms of whether the PCIe is a bottleneck or not during my  test?

Thanks

Alam

[Bernard]

David,

You can also use Xperf to monitor disk I/O activity.Use this command to start measurement xperf.exe -on latency

[Bernard]

Hi ALAM ,

if you mean device 8 and 9  so it is dependent on BIOS support.

[Roman_D_Intel]

Dave,

we have just added a PCIe montoring capability into Intel PCM 2.5: http://software.intel.com/en-us/forums/topic/393529

Roman

[Prince_Joseph]

can use Xperf for testing

[Bernard]

>>>can use Xperf for testing>>>

Bios locking is system wide so Xperf can be also affected.

[Anil_A_]

Is there a software release that supports PCM on Haswell.

I get an unsupported error when running on the same:

./pcm.x -nc

 Intel(r) Performance Counter Monitor V2.6 (2013-11-04 13:43:31 +0100 ID=db05e43)

 Copyright (c) 2009-2013 Intel Corporation

Error: unsupported processor. Only Intel(R) processors are supported (Atom(R) and microarchitecture codename Nehalem, Westmere, Sandy Bridge and Ivy Bridge). CPU model number: 63 Brand: "Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz"
Access to Intel(r) Performance Counter Monitor has denied (no MSR or PCI CFG space access).