topic Quote:Pavel Gerasimov (Intel) in Analyzers

[Solved][Yocto linux] [Media sdk] gpu profiling

TurtleCrazy — Tue, 15 May 2018 13:04:36 GMT

Hi there.

The intel media sdk is installed on the target and works well. I' m mainly using the `sample_encode` tool. But the intel driver isn't recognized by vtune:

I follow the steps from https://software.intel.com/en-us/vtune-amplifier-help-2019-beta-intel-media-sdk-program-analysis#MFX and the requested paths (INTEL_LIBITTNOTIFY64=/opt/intel/vtune_amplifier_xe/lib64/runtime/libittnotify_collector.so) are exported.

/opt/intel/system_studio_2018/vtune_amplifier/bin64/amplxe-cl -v -target-system=ssh:root@X.Y.0.5 -collect gpu-hotspots datatest/test.sh
amplxe: Using target: ssh:root@X.Y.0.5
amplxe: Error: Intel Graphics kernel module is not detected.
amplxe: Error: Cannot collect GPU hardware metrics. Make sure the Intel OpenCL SDK or Intel Media SDK is installed.

The Media SDK and HD driver were compiled from the opensource github repo:

How to fulfill the vtune requirements about HD gpu ?

libva output:

 vainfo
error: can't connect to X server!
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /usr/lib/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.1.pre1)
vainfo: Driver version: Intel iHD driver - 2.0.0
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointFEI
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointFEI
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointFEI
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileVP8Version0_3          : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointFEI
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD

Hi,

PAVEL_G_Intel — Tue, 15 May 2018 14:02:00 GMT

Hi,

Based on the error message. We may conclude that you have no libmd.so =)

The library is distributed as a part of Intel OpenCl package.

To solve the problem you have 2 options:
1) Install OpenCl
2) Build and install opensource libmd.so. You are able to get sources form: https://github.com/intel/metrics-discovery
After the compilation add the library path to LD_LIBRARY_PATH environment variable.

Quote:Pavel Gerasimov (Intel)

TurtleCrazy — Tue, 15 May 2018 15:07:01 GMT

Pavel Gerasimov (Intel) wrote:

Hi,

Based on the error message. We may conclude that you have no libmd.so =)

I do not see any mention of this library is the error message !?

The library is distributed as a part of Intel OpenCl package.

To solve the problem you have 2 options:
1) Install OpenCl
2) Build and install opensource libmd.so. You are able to get sources form: https://github.com/intel/metrics-discovery
After the compilation add the library path to LD_LIBRARY_PATH environment variable.

I quickly wrote a yocto recipe that installs the "libmd.so" library, compiled from the github repo, into the "/lib", "/lib64" and "/usr/lib" folders. (to make sure)

I still get the same error.

NB, the `install` target is missing from the CMake file of the metrics project.

It may be a problem of

PAVEL_G_Intel — Tue, 15 May 2018 15:21:48 GMT

It may be a problem of different libstdc++ used on system and in VTune Amplifier.

Let's run context-value-list with and without environment variables. Please send both of the result here.

/opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss --context-value-list
LD_PRELOAD=<path_to_libstdc++.so> LD_LIBRARY_PATH=<path_to_parent_dir_of_libmd.so> /opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss --context-value-list

First run:

TurtleCrazy — Tue, 15 May 2018 15:47:06 GMT

First run:

/opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss --context-value-list
targetOS: Linux
OS: Linux
OSBuildNumber: 33
OSBitness: 64
RootPrivileges: true
isPtraceScopeLimited: false
isTSXAvailable: false
isHTEnabled: false
fpgaOnBoard: None
pciClassParts:
isSGXAvailable: false
LinuxRelease: 4.14.33-intel-pk-standard
IsNUMANodeWithoutCPUsPresent: false
Hypervisor: None
PerfmonVersion: 4
isPtraceAvailable: true
areGpuHardwareMetricsAvailable: UnsupportedInterfaceVersion
i915Status: MissingDriver
isFtraceAvailable: yes
isMdfEtwAvailable: false
isCSwitchAvailable: yes
isGpuBusynessAvailable: notAccessible
isGpuWaitAvailable: yes
isFunctionTracingAvailable: yes
isIowaitTracingAvailable: yes
isVSyncAvailable: yes
isPAXDriverLoaded: true
isHyperVEnabled: false
isDeviceOrCredentialGuardEnabled: false
isSEPDriverAvailable: true
platformType: 106
CPU_NAME: Intel(R) Processor code named Broxton
PMU: goldmont
referenceFrequency: 1600000000
isVTSSPPDriverAvailable: true
isNMIWatchDogTimerRunning: false
LinuxPerfCredentials: Unlimited
LinuxPerfCapabilities: breakpoint:raw;cpu:raw,format,events;cstate_core:raw,format,events;cstate_pkg:raw,format,events;intel_bts:raw;intel_pt:raw,format;msr:raw,format,events;power:raw,format,events;software:raw;tracepoint:raw
LinuxPerfStackCapabilities: fp,dwarf
isTPSSAvailable: true
isPytraceAvailable: true
isGENDebugInfoAvailable: false
isGTPinCollectionAvailable: Yes
isSTTAvailable: no
isCOHNPKCtrlLibAvailable: false

Second run:

LD_PRELOAD=/usr/lib/libstdc++.so LD_LIBRARY_PATH=/usr/lib /opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss --context-value-list
targetOS: Linux
OS: Linux
OSBuildNumber: 33
OSBitness: 64
RootPrivileges: true
isPtraceScopeLimited: false
isTSXAvailable: false
isHTEnabled: false
fpgaOnBoard: None
pciClassParts:
isSGXAvailable: false
LinuxRelease: 4.14.33-intel-pk-standard
IsNUMANodeWithoutCPUsPresent: false
Hypervisor: None
PerfmonVersion: 4
isPtraceAvailable: true
areGpuHardwareMetricsAvailable: true
i915Status: MissingDriver
isPAVPEnabled: false
gpuPlatformIndex: 10
isFtraceAvailable: yes
isMdfEtwAvailable: false
isCSwitchAvailable: yes
isGpuBusynessAvailable: notAccessible
isGpuWaitAvailable: yes
isFunctionTracingAvailable: yes
isIowaitTracingAvailable: yes
isVSyncAvailable: yes
isPAXDriverLoaded: true
isHyperVEnabled: false
isDeviceOrCredentialGuardEnabled: false
isSEPDriverAvailable: true
platformType: 106
CPU_NAME: Intel(R) Processor code named Broxton
PMU: goldmont
referenceFrequency: 1600000000
isVTSSPPDriverAvailable: true
isNMIWatchDogTimerRunning: false
LinuxPerfCredentials: Unlimited
LinuxPerfCapabilities: breakpoint:raw;cpu:raw,format,events;cstate_core:raw,format,events;cstate_pkg:raw,format,events;intel_bts:raw;intel_pt:raw,format;msr:raw,format,events;power:raw,format,events;software:raw;tracepoint:raw
LinuxPerfStackCapabilities: fp,dwarf
isTPSSAvailable: true
isPytraceAvailable: true
isGENDebugInfoAvailable: false
isGTPinCollectionAvailable: Yes
isSTTAvailable: no
isCOHNPKCtrlLibAvailable: false

Nice! We almost done.

PAVEL_G_Intel — Tue, 15 May 2018 16:22:07 GMT

Nice! We almost done.

But we have one more problem. Your system kernel is built without CONFIG_DRM_I915_LOW_LEVEL_TRACEPOINTS, CONFIG_EXPERT. Enable this configurations and get context-value-list one more time.

Quote:Pavel Gerasimov (Intel)

TurtleCrazy — Wed, 16 May 2018 07:42:39 GMT

Pavel Gerasimov (Intel) wrote:

Nice! We almost done.

OK, ( the LD_LIBRARY_PATH is useless as far I put the library into a default place)

< areGpuHardwareMetricsAvailable: UnsupportedInterfaceVersion
---
> areGpuHardwareMetricsAvailable: true
18a19,20
> isPAVPEnabled: false
> gpuPlatformIndex: 10

Its a bit annoying then, how to make it run automatically ? I mean, as the vtune drivers on the target are loaded up on a request from the host ? I must admit that I do not understand why the standard c++ library has to be preloaded ... I can link `stdlibc++` from `../lib64` to the system's, hoping that any others tools will not complain about this.

But we have one more problem. Your system kernel is built without CONFIG_DRM_I915_LOW_LEVEL_TRACEPOINTS, CONFIG_EXPERT. Enable this configurations and get context-value-list one more time.

I have started a new image generation for my distro with these configuration knobs. The i915 driver is build as a separate module. May I move it in-tree ?

Hi,

TurtleCrazy — Wed, 16 May 2018 11:25:15 GMT

Hi,

It still failed to detect the driver with the new build of the kernel.

The only difference is :

 isGpuBusynessAvailable: yes

Hi,

PAVEL_G_Intel — Wed, 16 May 2018 12:13:45 GMT

Hi,

Why we need to use library preload?
VTune Amplifier uses its own libstdc++.so and the version of this library is GLIBCXX_3.4.20. But libmd.so depends on another libstdc++.so version. Therefore VTune Amplifier cannot load it.

The changes of context values says, that the GPU analysis is possible right now. But you have a problem with the preload during remote collection.
I have never did it before... Try to replace amplxe-runss on target with bash script which will start amplxe-runss with correct version of libstdc++ preloaded.

Something like this:

#!/bin/bash
LD_PRELOAD=/usr/lib/libstdc++.so /opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss.orig

Quote:Pavel Gerasimov (Intel)

TurtleCrazy — Wed, 16 May 2018 13:16:04 GMT

Pavel Gerasimov (Intel) wrote:

Hi,

Why we need to use library preload?
VTune Amplifier uses its own libstdc++.so and the version of this library is GLIBCXX_3.4.20. But libmd.so depends on another libstdc++.so version. Therefore VTune Amplifier cannot load it.

Wow, as far as there is only short differences between the two (6.20 -> 6.24).

The changes of context values says, that the GPU analysis is possible right now. But you have a problem with the preload during remote collection.
I have never did it before... Try to replace amplxe-runss on target with bash script which will start amplxe-runss with correct version of libstdc++ preloaded.

Fortunately `libstdc++.so.6` is not a regular file but a link; So I re-link libstdc++.so.6 to the system library and it seems to work. Actually, I was able to run a "gpu-hotspot" profiling suite. I have now to make sure that I get relevant results. Thanks.

I m currently running an in tree i915 module; I will check this on a off-tree module later on. Here is the differences between the two:

+pciClassParts:
+isGpuBusynessAvailable: i915TaintUnknown

What sounds weird is also this:

i915Status: MissingDriver

Reading this I first thought that I would not be able to profile the GPU.

As for context values there

PAVEL_G_Intel — Wed, 16 May 2018 13:38:13 GMT

As for context values there are not for end-user. The confusing values are used for better error reporting and not influence the collection.

Quote:Pavel Gerasimov (Intel)

TurtleCrazy — Wed, 16 May 2018 15:24:36 GMT

Pavel Gerasimov (Intel) wrote:

As for context values there are not for end-user. The confusing values are used for better error reporting and not influence the collection.

OK, it sounds that this part works well now. I am facing a new issue with cpugpu-concurrency analysis but it is somehow a different problem.

So I started a new thread; thanks again.

Quote:Pavel Gerasimov (Intel)

TurtleCrazy — Thu, 17 May 2018 12:32:32 GMT

Pavel Gerasimov (Intel) wrote:

#!/bin/bash
LD_PRELOAD=/usr/lib/libstdc++.so /opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss.orig

Just in case, append `$@` to populate arguments.

#!/bin/sh
LD_PRELOAD=/usr/lib/libstdc++.so /opt/intel/vtune_amplifier_2018.2.0.551022/bin64/amplxe-runss.orig ${@}

I have never did it before

I did. It seems to work also. my other solution follows:

cd /opt/intel/vtune_amplifier_2018.2.0.551022/lib64
unlink libstdc++.so.6
ln -s /usr/lib/libstdc++.so.6.24  libstdc++.so.6