I understand what you mean

Cetrini__Fabio · ‎04-10-2019

Hi, I'm trying to integrate MSDK hw decoding feature inside my existing solution.

Video stream is provided and parsed using GStreamer libraries, the bitstream is then decoded alternatively using sw decoder (LIBAV gstreamer implementation) / nvdec / MSDK, and finally rendered through dx11/dxgi/dx9/wpf surface.

MSDK implementation is really basic, picking just necessary from sample_decode: session init (MFX_IMPL_HARDWARE/MFX_IMPL_VIA_D3D11/MFX_GPUCOPY_DEFAULT/MFX_IOPATTERN_OUT_VIDEO_MEMORY), decode header, prepare frame allocator, decode async, bind out surface to d3d11. It's all working fine, no leak detected on system/gpu memory after some minutes.

Problem appears during stop/start transient: on each stop decoder is flushed and closed, session is closed too, all pointers freed, directx11 layer is destroyed, then all items are created brand new at the next start.

On each stop/start cycle, task manager detect a +20Megs leak on system memory, while GPU dedicated memory grows of 100Megs each 2/3 cycle. When the application is closed, the directx11 debug layer print out:

D3D11 WARNING: Process is terminating. Using simple reporting. Please call ReportLiveObjects() at runtime for standard reporting. [ STATE_CREATION WARNING #0: UNKNOWN]

D3D11 WARNING: Live Producer at 0x0000013F35EB8850, Refcount: 2. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35EBB770, Refcount: 1. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35EF7020, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2CF55570, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35D5C200, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C2CEE0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C306F0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35AF10C0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F03260, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F041C0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35EACBC0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35D36090, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F06BB0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C2DD80, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F04F90, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35EAD530, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C8FDC0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live                         Object :     16 [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Producer at 0x0000013F36103270, Refcount: 2. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F36160650, Refcount: 1. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F6F170, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2CF55A90, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35F64D90, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C2EE90, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C2EC20, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360DC680, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CD18D0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CB00E0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CB0EB0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35D36590, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CB2260, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35C2FFA0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35D7E530, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CB2BA0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360F3A30, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live                         Object :     16 [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Producer at 0x0000013F360CAFF0, Refcount: 2. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F36128000, Refcount: 1. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35DAD340, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2CF53E60, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2F6703B0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360E4380, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360E5490, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35CC9AA0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360CCED0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F36101DD0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35DBA7C0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35D35E10, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2F8ABEE0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360E6330, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F2F8993B0, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F35DBB130, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live Object at 0x0000013F360CDE30, Refcount: 0. [ STATE_CREATION WARNING #0: UNKNOWN]
D3D11 WARNING: Live                         Object :     16 [ STATE_CREATION WARNING #0: UNKNOWN]
DXGI WARNING: Live Producer at 0x0000013F2F8B6B08, Refcount: 4. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Object at 0x0000013F35C562A0, Refcount: 3. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live                         Object :      1 [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Producer at 0x0000013F35B7A338, Refcount: 4. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Object at 0x0000013F35C56770, Refcount: 3. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live                         Object :      1 [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Producer at 0x0000013F2F9CD678, Refcount: 4. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live Object at 0x0000013F35C57110, Refcount: 3. [ STATE_CREATION WARNING #0: ]
DXGI WARNING: Live                         Object :      1 [ STATE_CREATION WARNING #0: ]

It seems that d3d11_device/d3d11_textures remains referred inside decoder heap, but I can't figure out how to free them, besides MFXVideoDECODE_Close/MFXClose.

Input stream is 2592x1520@20fps h264 from network samsung camera, no problem detected using libav/nvdec implementation.

System analyzer log:

Intel(R) Media Server Studio 2017 - System Analyzer (64-bit)

The following versions of Media SDK API are supported by platform/driver
[opportunistic detection of MSDK API > 1.20]:

Version Target Supported Dec Enc
1.0 HW Yes  X X
1.0 SW Yes  X X
1.1 HW Yes  X X
1.1 SW Yes  X X
1.2 HW Yes  X X
1.2 SW Yes  X X
1.3 HW Yes  X X
1.3 SW Yes  X X
1.4 HW Yes  X X
1.4 SW Yes  X X
1.5 HW Yes  X X
1.5 SW Yes  X X
1.6 HW Yes  X X
1.6 SW Yes  X X
1.7 HW Yes  X X
1.7 SW Yes  X X
1.8 HW Yes  X X
1.8 SW Yes  X X
1.9 HW Yes  X X
1.9 SW Yes  X X
1.10 HW Yes  X X
1.10 SW Yes  X X
1.11 HW Yes  X X
1.11 SW Yes  X X
1.12 HW Yes  X X
1.12 SW Yes  X X
1.13 HW Yes  X X
1.13 SW Yes  X X
1.14 HW Yes  X X
1.14 SW Yes  X X
1.15 HW Yes  X X
1.15 SW Yes  X X
1.16 HW Yes  X X
1.16 SW Yes  X X
1.17 HW Yes  X X
1.17 SW Yes  X X
1.18 HW Yes  X X
1.18 SW Yes  X X
1.19 HW Yes  X X
1.19 SW Yes  X X
1.20 HW Yes  X X
1.20 SW Yes  X X
1.21 HW Yes  X X
1.21 SW Yes  X X
1.22 HW Yes  X X
1.22 SW Yes  X X
1.23 HW Yes  X X
1.23 SW Yes  X X
1.24 HW Yes  X X
1.24 SW Yes  X X
1.25 HW Yes  X X
1.25 SW Yes  X X
1.26 HW Yes  X X
1.26 SW Yes  X X
1.27 HW Yes  X X
1.27 SW Yes  X X

Graphics Devices:
Name                                         Version             State
NVIDIA GeForce GTX 1050                      25.21.14.1967       Offline
Intel(R) UHD Graphics 630                    24.20.100.6286      Running / Full Power

System info:
CPU: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
OS: Microsoft Windows 10 Pro
Arch: 64 bit

Installed Media SDK packages (be patient...processing takes some time):
Intel(R) Media SDK 2018 R2 - HEVC GPU accelerated Encoder
Intel(R) Media SDK 2018 R2 - Media Samples
Intel(R) Media Server Studio 2017 - Video Quality Caliper
Intel(R) Media SDK 2018 R2 - Software Development Kit
Intel(R) Media SDK 2018 R2 - Documentation for HEVC
Intel(R) Media SDK 2018 R2 - HEVC SW Encoder
Intel(R) Media SDK 2018 R2 - HEVC SW Decoder

Installed Media SDK DirectShow filters:

Installed Intel Media Foundation Transforms:
Intel(R) Hardware M-JPEG Decoder MFT : {00C69F81-0524-48C0-A353-4DD9D54F9A6E}

Analysis complete... [press ENTER]

Am I missing something?

Mark_L_Intel1 · ‎04-10-2019

Hi Cetrini,

Could you check the following post to see if it helps?

https://software.intel.com/en-us/forums/intel-media-sdk/topic/801208

Mark

Cetrini__Fabio · ‎04-11-2019

Hi Mark,

yes I've looked at that post, but the solution is not so clear...they start talking about sdk sample leak of d3d pointers, but there is nothing about production code fixing.

However, I'm not using d3d11_device.cpp.

My solution is made of C# frontend (wpf/d3d9/dxgi/d3d11) and a C++ plugin for merely MSDK apis.

The mem leak appears even if I completely bypass the texture rendering C# side: any ideas?

Thank you,

Fabio

Dmitry_E_Intel · ‎02-18-2020

Hi Fabio,

I saw your code posted at https://software.intel.com/en-us/forums/media/topic/801208#comment-1952965 .

Well, I didn't find something obvious which could cause such huge memory leak. Perhaps, one thing doesn't look correctly to me: at the finalizer section you call DecodeFrameAsync with NULL bitstream to get cached surfaces but don't call SyncOperation. But this shouldn't cause memory leak.

Can I ask you to share with us a reproducer for this issue made on top the code you posted? I think it can be relatively simple: "core" part is the code you posted plus main () function should feed the core part with data, get output surfaces and Init/Destroy the core part let's say each 100 frames.The only condition to it is the leak should be still reproducible. I understand that it will require some efforts but from my side I can promise it won't be in vain. We'll definitely investigate the issue locally. Otherwise most likely we'll keep changing blind guesses which may result in nothing.

Regards,

Dmitry

Cetrini__Fabio · ‎02-20-2020

Hi Dmitry, working on the reproducer I've found the catch: setting the multithread mode on the Directx11 context and not releasing the COM reference.

In SharpDX I've just added the using statement:

using (var mt = this.device_11.ImmediateContext.QueryInterface<Multithread>() as Multithread)
	mt.SetMultithreadProtected(true);

...and it solved all :-) No issue in my C++ wrapper.

In your C++ "sample_decode" I've not found the explicit release of that interface, maybe it would leek too?

Thank you for your patience, my regards.

Dmitry_E_Intel · ‎02-20-2020

Hi Fabio,

Good to know!:) Thank you.

It looks like sample code is safe because of CComQIPtr:

https://github.com/Intel-Media-SDK/MediaSDK/blob/ec44711b13b1051e2bcfea3bd7455bea732cb93c/samples/sample_common/src/d3d11_device.cpp#L135

https://github.com/Intel-Media-SDK/MediaSDK/blob/5dd2ea4f232626570c347cdf45193f019cb5f460/tutorials/common/common_directx11.cpp#L117

Cetrini__Fabio · ‎02-20-2020

Clear, my C++ is not so good :-D

One other question...is there some API to query GPU status? I'd like to show on my application the decoding load, a sort of diagnostic I mean.

Thank you again,

Fabio

Dmitry_E_Intel · ‎02-20-2020

I understand what you mean and in the past tried to discover the solution. I didn't find it. For Linux there are many ways programmably to get GPU utilization, but there are no public ways on Windows. Sorry about it ...

For manual profiling/debugging you may want to use GPUView. A good introduction article is here: https://graphics.stanford.edu/~mdfisher/GPUView.html

Another way for manual GPU profiling is VTune.

Regards,

Dmitry

System AND GPU memory leak decoding/stopping/starting - 2018 R2.1 (windows)