Questions regarding measuring PCIe performance using PCM

[Yiwen_Z_]

Hi All,

I've been trying to measure PCIe performance on the two types of machines on Linux I work with but neither of them work out well. I summarize my questions below and any help is welcomed. I always ran PCM as root.

1. Xeon E5-2650v2, Ivy Bridge, Linux Kernel 3.13

Keep getting "

ERROR: QPI LL monitoring device (0:7f:8:2) is missing. The QPI statistics will be incomplete or missing.

ERROR: QPI LL monitoring device (0:7f:9:2) is missing. The QPI statistics will be incomplete or missing.

" and the PCM cannot be successfully launched. Can PCM run on this type of machines, or impossible due to the missing device reported?

2. Xeon E5-2450, Sandy Bridge, Linux Kernel 3.13

PCM can run on this types of machines. The tool automatically select the older version of parameters where MMIO measurement is missing. I manually set the tool to always choose the newer version, but "WiL" returns 0 all the time when I ran a small RDMA flow. How can I use the PCM tool to measure MMIO traffic in this type of machines?

Thank you in advance for your great help!

Sincerely,

Yiwen Zhang

[Patrick_L_Intel]

Hi Yiwen,

Can you please post the entire log for ./pcm-pcie.x for E5-2650v2 machine? For PCIe performance monitoring, we are using CBo counter, so QPI error should not affect the ability of monitoring PCIe.

For Sandy Bridge version of Xeon (E5-2600), I checked the SDM [1] and unfortunately WiL and PRd are not available, so we can't monitor MMIO events.

[1]: https://www.intel.com/content/dam/www/public/us/en/documents/design-guides/xeon-e5-2600-uncore-guide.pdf

[Yiwen_Z_]

Thank you for you reply Patrick!

The entire log of running ./pcm-pcie.x for E5-2650v2 are posted below:

Number of physical cores: 16
Number of logical cores: 32
Number of online logical cores: 32
Threads (logical cores) per physical core: 2
Num sockets: 2
Physical cores per socket: 8
Core PMU (perfmon) version: 3
Number of core PMU generic (programmable) counters: 4
Width of generic (programmable) counters: 48 bits
Number of core PMU fixed counters: 3
Width of fixed counters: 48 bits
Nominal core frequency: 2600000000 Hz
Package thermal spec power: 95 Watt; Package minimum power: 60 Watt; Package maximum power: 150 Watt;
ERROR: QPI LL monitoring device (0:7f:8:2) is missing. The QPI statistics will be incomplete or missing.
ERROR: QPI LL monitoring device (0:7f:9:2) is missing. The QPI statistics will be incomplete or missing.
Socket 0: 1 memory controllers detected with total number of 4 channels. 0 QPI ports detected.
ERROR: QPI LL monitoring device (0:ff:8:2) is missing. The QPI statistics will be incomplete or missing.
ERROR: QPI LL monitoring device (0:ff:9:2) is missing. The QPI statistics will be incomplete or missing.
Socket 1: 1 memory controllers detected with total number of 4 channels. 0 QPI ports detected.
Trying to use Linux perf events...
Linux Perf: Error on programming generic event #0 error: Invalid argument
Access to Processor Counter Monitor has denied (Unknown error).
Cleaning up

I also notice in this type of machines PCM also recognize its CPU model as old and pick the older version of measurement parameters. Does that imply we can't measure MMIO on E5-2650v2 either?

Best Regards,

Yiwen Zhang

[Patrick_L_Intel]

Hi Yiwen,

In the Makefile, you can comment out #CXXFLAGS += -DPCM_USE_PERF and recompile pcm. This will use the native driver in PCM to program performance counters.

The event code for monitoring MMIO Read/Write (PRd, WiL) is introduced in E5-v3 generation of processor only.

Patrick

[Yiwen_Z_]

The Makefile change did solve the error. Thank you so much for your help Patrick.

And the information regarding on MMIO is helpful as well.

Best Regards,

Yiwen Zhang