JPEG Resize Pipeline fails when next JPEG has different sizes

[Riedl__Christian]

I know that the Media SDK is not intended for use with still JPEGs. But based on the "mediasdkjpeg-man.pdf" and the available samples I was able to create a working transcoder :

sts = _mfxDECODE->DecodeFrameAsync(&mfxIBS, &_decSurface, &outSurface, &syncDecode);
sts = _mfxVPP->RunFrameVPPAsync(outSurface, &_encSurface[0], NULL, &syncVpp);
sts = _mfxENCODE->EncodeFrameAsync(&encodeCtrl, &_encSurface[0], &mfxOBS, &syncEncode);
sts = _mfxSession.SyncOperation(syncEncode, MSDK_WAIT_INTERVAL);

As mfxVPP->Init takes quite long (~800 ms) I use Reset (...) for all the pipeline steps when processing the next image. As long as the image and resize dimensions stay equal everything works fast and pretty. If the dimensions are different I get an UNDEFINED BEHAVIOR error back from SyncOperation and no surface is delivered. The problem is caused by the VPP step. I tested DECODE and ENCODE, they are able to work with Reset and changed image dimensions. 

For my application this is absolutely a show-stopper. Is there a workarround ?

Here my session init sequnce. I get back version 1.27 :

initPar.Version.Major = 1;
initPar.Version.Minor = 0;
initPar.GPUCopy = 0;
initPar.Implementation = MFX_IMPL_SOFTWARE;

mfxStatus sts = _mfxSession.InitEx(initPar);
sts = _mfxSession.QueryVersion(&_mfxVersion);
sts = _mfxSession.QueryIMPL(&impl);                            
sts = _mfxSession.Init(impl, &_mfxVersion);
 

[Mark_L_Intel1]

Hi Christian,

How did you call the reset? 

From the program reference of the following URL at page 48, you will get an error if you use different video parameters.

https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.pdf ;

It looks like you have to call init after reset because the init call will allocate memory, if you have a larger output size, init call is required because of more memory is required.

Did you call init after reset?

Mark

[Riedl__Christian]

• I did not call mfxVPP->Init because it takes  800 ms ...
• But even when I call it, it does not help ... 

[Mark_L_Intel1]

If my understanding is correct, in the failure case, you are changing the size of the original file. So there should be 2 cases, 

1. You failed on any size changing.

2. You only failed when you first run without changing size and then reset it and then change the size.

If it is #1, you can try sample_multi_transcode to see if it run succeed.

Mark

[Riedl__Christian]

Mark,

thanks for the answer. What I want to do is to use a DECODE -> RESIZE -> ENCODE JPEG-pipeline inside a web application, to implement something like a slideshow. My problem is that mfxVPP->Init (~800 ms) is very slow compared to mfxVPP->Reset (~50 ms). I understand now that  Reset does not reallocate buffers. 

So I implemented now simply :

sts = _mfxVPP->Reset(&_vppParam);
if (sts != MFX_ERR_NONE) 
    sts = _mfxVPP->Init(&_vppParam);
 

It is working now but not really satisfying, every switch landscape <-> portrait produces a mfxVPP->Init => 800 ms. 

See my results (milliseconds), all files have different dimensions, the destination height is always the same : 

File C:\temp\landscape_x.jpg : read  92 initvpp 827 decode 405 resize  26 encode   6 save   4
File C:\temp\landscape_y.jpg : read  70 resvpp   48 decode 258 resize  23 encode   4 save   4
File C:\temp\landscape_x.jpg : read  86 resvpp   47 decode 399 resize  24 encode   6 save   5
File C:\temp\landscape_y.jpg : read  67 resvpp  52 decode 266 resize  20 encode   4 save   4
File C:\temp\portrait_z.jpg  : read  69 initvpp 812 decode 319 resize  16 encode   5 save   4
File C:\temp\landscape_x.jpg : read  83 initvpp 874 decode 393 resize  25 encode   6 save   4
File C:\temp\landscape_y.jpg : read  71 resvpp  47 decode 249 resize  22 encode   4 save   4
File C:\temp\landscape_x.jpg : read  89 resvpp  63 decode 411 resize  24 encode   6 save   4
File C:\temp\landscape_y.jpg : read  65 resvpp  55 decode 270 resize  21 encode   5 save   4
File C:\temp\portrait_z.jpg  : read  72 initvpp 797 decode 324 resize  17 encode   5 save   4

Perhaps it is a good Idea to have 2 different VPPs one for portrait and one for landscape images. I am also thinking about implementaiton of my own resize step,  I have good experience with Intel IPP bitmap resizing, it is also quite fast.

Why is this mfxVPP->Init so extremly slow ?

By the way, I have no knowledge of the JPEG algorithms. But do you have an explanation why encode is so much faster than decode ?    

Thank you and greetings from Austria, Vienna

[Mark_L_Intel1]

Thanks for the use case description, I understand it now.

I am trying to check mfxVPP->Init call to analyze where it gets slower, I expected to see library loading operations but I can't, here is the code:

https://github.com/Intel-Media-SDK/MediaSDK/blob/006f96adec57ee8a5f7a120e769a5feab5fb7f2c/_studio/mfx_lib/shared/src/libmfxsw_vpp.cpp#L145

The other place that might slow down is the browser itself, is that any framework limitation here?

By the way, we have open-sourced all our source code, you could just use gdb to trace init call to find the bottleneck. Here are reference you can use as the build method:

https://github.com/Intel-Media-SDK/MediaSDK/wiki/Build-Media-SDK-on-Ubuntu

https://software.intel.com/en-us/articles/build-and-debug-open-source-media-stack

Let me know if you have other questions.

About JPEG encoder question, are you talking about general or MSDK context? I don't know much about JPEG algorithm, but if you use HW encoder of MSDK while software decoder, that is possible.

Mark

[Riedl__Christian]

Mark,

after moving my tests to another platform with Intel Graphics I am able to use hardware support for JPEG decoding. The result is really encouraging (JPEG decoding 5 times faster) and the problem with the extremly slow mfxVPP->Init disappeared.

Originally I planned to run my app on a Windows Server 2016 with latest Xeon E-2146G (Intel UHD Graphics P630).

But I see that MediaSDK does not officially support it. Ok I can use a 8th Gen Core I7 instead, but I see that officially only Windows Server 2012 (R2) is supported. That's out of the question. I was looking in the forum for Windows Server 2016 compatibility, but did not find any information.

Probably I should open a new thread for this question, but perhaps you can answer it. 

Thanks, Christian

Many thanks, Christian

[Mark_L_Intel1]

Hi Christian,

Let me clarify your question, do you want to use Media SDK windows release for 8th generation Core processor on Windows Server 2016?

You can refer to the following release notes of Media SDK 2018 R1, which should be able to support this configuration.
https://software.intel.com/sites/default/files/mediasdk_release_notes.pdf

Let me know if this answers your questions.

Mark