How can I decrease the performance load for video frame scaling by IPP?

[Detlev_Petersen]

Dear all,

at the moment my video decoder delivers frames in UYVY. Then a color conversion from UYVY to YUY2 is done. After this the video frame is scaled by an IPP call from 352x288 to 800x600. At last the color conversion from YUY2 back to UYVY is done to have the frame ready for Windows' Blt() call.

Now I try to improve my code. At first I think about the possibility to decode each video frame in YUY2 format. I found out that my DirectDraw interface can be set up for YUY2 instead of UYVY. Therefore it seems to be an option to save two color conversions. Without the conversion UYVY->YUY2 and YUY2->UYVY less performance load can be expected. Try to find a configuration so that there is no need anymore to use the color conversion calls ippiCbYCr422ToYCbCr422_8u_C2R() and ippiYCbCr422ToCbYCr422_8u_C2R().

This aim leads to following questions.

I saw that the pitch is the width*2 with UYVY. I guess I can calculate the pitch in the same way for YUY2. Is this correct?

Does YUY2 have any drawbacks in comparison with UYVY? Maybe the video decoder consumes much more performance load for decoding frames in YUY2 instead of UYVY.

Then I have the following selection of the video decoder in my sources. I saw that a filter method IPPI_INTER_LINEAR is defined:

     umcRes = CodecPipeline::SelectVideoDecoder(
      pItem->m_videostreaminfo,
      &videoInputData,
      IPPI_INTER_LINEAR,
      lDeinterlacingMode,
      1,
      UMC::FLAG_VDEC_REORDER,
      *pItem->m_pColorConverter,
      pItem->m_pVideoDecoder);

If I use a later scaling by IPP with help of the call ippiResizeYUV422_8u_C2R() is it possible to select another filter for the frame decoding itself without having less decoding quality on the screens at the end? I can imagine that there are other configurations for the video decoder to my software faster.

Best regards,

Detlev 

[Detlev_Petersen]

Hi,

is it necessary to use an interpolation like IPPI_INTER_CUBIC while decoding a video?

In my case I scale the video to a bigger resolution than the source resolution. There I use a resize call with an interpolation method. At the end I call an interpolation method twice. Is this really necessary?

I can save performance load if I do it with the last resize call once. The result of the decoder will be of less quality then but does this play any role?

Are there any experiences how I can go on or do I have to do some tests on my own?

It seems to be that the interpolation is IPPI_INTER_NN at least and that there is no possibility to select the video decoder without using any interpolation.

If I suppose the decoding of video files I do not see any reasons for bad video quality in comparison with a video live stream via satellite. There it should be sufficient to do interpolation once.

Are there any other reasons to do interpolation twice while decoding and resizing a video?

Best regards,

Detlev

[Sergey_K_Intel]

Hi Detlev,

I asked our "resize" guy.

Regards,
Sergey 

[Valentin_K_Intel]

Hi Detlev,

I will try to answer some of your questions.

Detlev Petersen wrote:

I saw that the pitch is the width*2 with UYVY. I guess I can calculate the pitch in the same way for YUY2. Is this correct?

Yes, it is correct.

Detlev Petersen wrote:

If I use a later scaling by IPP with help of the call ippiResizeYUV422_8u_C2R() is it possible to select another filter for the frame decoding itself without having less decoding quality on the screens at the end? I can imagine that there are other configurations for the video decoder to my software faster.

When you use the function ippiResizeYUV422_8u_C2R, you can apply the following interpolation methods for image enlarging: IPPI_INTER_NN, IPPI_INTER_LINEAR, IPPI_INTER_CUBIC. So IPPI_INTER_NN method has better perfomance than IPPI_INTER_LINEAR, but worse quality, contrariwise IPPI_INTER_CUBIC method has worse performance than IPPI_INTER_LINEAR, but better quality.

Best regards,
Valentin

[Detlev_Petersen]

Hi Valentin,

I try to ask in another way. Is it useful to use IPP_INTER_LINEAR for the SelectVideoDecoder() and the ippiResizeYUV422_8u_C2R() calls? If yes, I call the same interpolation twice. The first time I interpolate the video source. The second time I interpolate the video frame to be enlarged. From my point of view I do the same interpolation twice and I ask myself if there is any better quality to do it in this way. For example I can use IPP_INTER_NN for the SelectVideoDecoder() call and at least IPP_INTER_LINEAR for the resize call. I cannot see any quality difference between SelectVideoDecoder() with IPP_INTER_NN and IPP_INTER_LINEAR in this case, cause the resize call does the IPP_INTER_LINEAR interpolation. Are there any theoretical arguments to call SelectVideoDecoder() with IPP_INTER_LINEAR if each frame is enlarged by a resize call using IPP_INTER_LINEAR, too?

I think the answer for SelectVideoDecoder() is easy if I do not want to resize the decoded frame, but in the resize case I can save performance load to use IPP_INTER_LINEAR for the resize call only. But what is about the video quality then?

Best regards,
Detlev

[Pavel_V_Intel]

Hi Detlev,

Interpolation is used only for postprocessing resize, it has nothing to do with decoding itself. The only reason it is in Decoder parameters because Postrocessor is a part of decoder output function. You specified it as "*pItem->m_pColorConverter," parameter in your call. It is not only color conversion it is also the resize and deinterlace function

Postprocessing only happens then your output VideoData info is different from internal VideoData info (which is usually the same as VideoDecoder::GetInfo or Splitter::GetInfo) during VideoDecoder::GetFrame call. If you specify custom VideoData parameters then frame must be resized in order to fit output buffer. But if output VideoData parameters is the same as decoded frame then no processing will be done, only copying.

If you want to output decoded frame to YUY2 800x600 format you can just init output VideoData as this:

videoData.Init(800, 600, YUY2);

Does YUY2 have any drawbacks in comparison with UYVY? Maybe the video decoder consumes much more performance load for decoding frames in YUY2 instead of UYVY.

Decoder doesn't know about output format, it decode frames only in YUV420, 422 or 444 and then PostProcessor rescales them and converts to match destination buffer format. And there is no big difference in conversion between YUV422->YUY2 or YUV422->UYVY.