Performance degradation after update from 2017R3 to 2018R1

[VASILY_V_]

Hi, 

We've encounter performance degradation after update from 2017R3 to 2018R1 Intel Media SDK community edition for Linux on Intel(R) Xeon(R) CPU E3-1585L v5 @ 3.00GHz

After update the overall performance decreases up to 25% and now our server can handle less streams then before update (we use only AVC as codec)

What we can see as difference, that on 2017R3 metrics_monitor shows us VIDEO2 module and load on it:

RENDER usage: 90.00,	VIDEO usage: 36.00,	VIDEO_E usage: 68.00	VIDEO2 usage: 45.00	GT Freq: 1100.00
RENDER usage: 79.00,	VIDEO usage: 21.00,	VIDEO_E usage: 53.00	VIDEO2 usage: 45.00	GT Freq: 1100.00
RENDER usage: 82.00,	VIDEO usage: 32.00,	VIDEO_E usage: 58.00	VIDEO2 usage: 43.00	GT Freq: 1100.00
RENDER usage: 92.00,	VIDEO usage: 26.00,	VIDEO_E usage: 72.00	VIDEO2 usage: 48.00	GT Freq: 1100.00
RENDER usage: 83.00,	VIDEO usage: 21.00,	VIDEO_E usage: 63.00	VIDEO2 usage: 49.00	GT Freq: 1100.00
RENDER usage: 79.00,	VIDEO usage: 24.00,	VIDEO_E usage: 61.00	VIDEO2 usage: 38.00	GT Freq: 1100.00

but after update, on 2018R1 we can't see VIDEO2 and RENDER module always 100% load:

RENDER usage: 100.00,	VIDEO usage: 49.00,	VIDEO_E usage: 73.00	GT Freq: 1100.00
RENDER usage: 100.00,	VIDEO usage: 41.00,	VIDEO_E usage: 70.00	GT Freq: 1100.00
RENDER usage: 100.00,	VIDEO usage: 47.00,	VIDEO_E usage: 73.00	GT Freq: 1100.00
RENDER usage: 100.00,	VIDEO usage: 47.00,	VIDEO_E usage: 71.00	GT Freq: 1100.00
RENDER usage: 100.00,	VIDEO usage: 45.00,	VIDEO_E usage: 75.00	GT Freq: 1100.00
RENDER usage: 99.00,	VIDEO usage: 42.00,	VIDEO_E usage: 74.00	GT Freq: 1050.00
RENDER usage: 100.00,	VIDEO usage: 38.00,	VIDEO_E usage: 71.00	GT Freq: 1050.00

Also you may notice that GT Freq has small fluctuations that we also don't observe on 2017R3

Dmesg initialization log seems the same for both SDK versions:

[Mon Dec 24 12:49:26 2018] [drm] Initialized drm 1.1.0 20060810
[Mon Dec 24 12:49:26 2018] [drm] Found 128MB of eLLC
[Mon Dec 24 12:49:26 2018] [drm] Memory usable by graphics device = 4096M
[Mon Dec 24 12:49:26 2018] [drm] Replacing VGA console driver
[Mon Dec 24 12:49:26 2018] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[Mon Dec 24 12:49:26 2018] [drm] Driver supports precise vblank timestamp query.
[Mon Dec 24 12:49:27 2018] [drm] failed to retrieve link info, disabling eDP
[Mon Dec 24 12:49:27 2018] [drm] RC6 disabled, disabling runtime PM support
[Mon Dec 24 12:49:27 2018] [drm] Initialized i915 1.6.0 20151010 for 0000:00:02.0 on minor 0
[Mon Dec 24 12:49:27 2018] [drm] Cannot find any crtc or sizes - going 1024x768
[Mon Dec 24 12:49:27 2018] i915 0000:00:02.0: fb1: inteldrmfb frame buffer device
[Mon Dec 24 12:49:28 2018] [drm] RC6 off
[Mon Dec 24 12:49:28 2018] [drm] The Ring/GT multiplier is 2

What can we do? How to debug this problem? Why that can be? 

[Dmitry_E_Intel]

Hi Vasily,

Please say which OS and Linux kernel you use. Please provide  output from "uname -a", "modinfo i915", "cat /proc/cmdline".

Have you tried MSS 2018 R2?  If not, please do.

Can you please try to reproduce the issue on open source media stack: https://github.com/Intel-Media-SDK/MediaSDK/releases/tag/intel-mediasdk-18.3.0 with one of the latest 4.19 kernel?

Regards,

Dmitry

[VASILY_V_]

I've attached listings for system with and without problem. cat /proc/cmdline it both cases just 'auto'

No, I haven't tried MSS 2018 R2, It's not such simple to migrate my system for this SDK. Also I haven't tried opensource version of SDK. 

But it's definitely should be OK on 2018R1, don't it?

Or it's known problem with performance regression?

[Dmitry_E_Intel]

There is no such regression observed on our side.

I guess somehow the 2nd VDBOX has been disabled within kernel update. 

>>we use only AVC as codec

What you your particular usage case ? Single or multiple AVC encode? Or transcode to AVC?  If multiple, what is the number of channels?

Please provide us a reproducer via sample_encode or sample_multi_transcode which would be similar to the pipeline used in your application. Besides, I'd like to see performance results using perf command, like:

perf stat -a -e i915/bcs0-busy/,i915/rcs0-busy/,i915/vcs0-busy/,i915/vcs1-busy/,i915/vecs0-busy/ ./sample_multi_transcode <command line options>

for both 2017 R3 and 2018 R1.

Please also send output from "ls -l /sys/kernel/debug/dri/ && cat /sys/kernel/debug/dri/*/i915_ringstats"