Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

De-Interlacing support.

kishore_C_
Beginner
1,999 Views

Hi,

We are using intel media SDK 2014 version 1.9.

We are facing some problems with the deinterlacing using sdk .

We are using field interleaved input(1080i60 HD, 60 fields) with input frame rate (VPPParams.vpp.In.FrameRateExtN ) as 30 ,i.e 60 fields,

also input height and input width same as the 1920 and 1080.

            a) Along with the above we are mentioning output frame rate,output width and output height as 30,1920 and 1080 receptively.This c                                   configuration is giving de-interlaced output.

           b) Instead of 30 if you ,mention output frame rate as 60 , the application is returning on error of -11.

Along with config of a) if we try to use external VPP of mfxExtVPPDeinterlacing with following config before init call.

   " mfxExtVPPDeinterlacing  deinterlacingCfg;
    memset(&deinterlacingCfg,0,sizeof(mfxExtVPPDeinterlacing));
    deinterlacingCfg.Header.BufferId = MFX_EXTBUFF_VPP_DEINTERLACING;
    deinterlacingCfg.Header.BufferSz = sizeof(mfxExtVPPDeinterlacing);
    deinterlacingCfg.Mode             =    MFX_DEINTERLACING_BOB; 

    mfxExtBuffer* ExtBuffer[1];
    ExtBuffer[0] = (mfxExtBuffer*)&deinterlacingCfg;
    VPPParams.NumExtParam = 0;
    VPPParams.ExtParam = (mfxExtBuffer**)&ExtBuffer[0];  "

Even this configuration , the application is returning with  error of -15.

Please let us know if any additional tools needed to be included or if  there are any mistakes in the configuration we used.

Thanks in advance.

 

 

 

0 Kudos
21 Replies
Surbhi_M_Intel
Employee
1,842 Views

Hi Kishore,

Thank you for the question. Like you said that it is working for 30fps but not for 60fps. My initial gut is that it will require frame rate conversion to do that, you can find FRC in the list of the technical article- https://software.intel.com/en-us/forums/topic/530561 
One thing I was wondering is which sample/tutorial are you using and also the system details of your machine. You can find that by running system analyzer. Again details can be found here - https://software.intel.com/en-us/forums/topic/531083

Thanks,
-Surbhi

0 Kudos
Surbhi_M_Intel
Employee
1,842 Views

Hi Kishore, 

I take back that we need to use frame rate conversion as converting from 30frames per second(60 fields) to 60frames is a supported conversion under deinterlacing, sorry about that. It is mentioned in table for the supported conversion on the Media SDK manual ( can be downloaded from the documents of the Media Solution Portal)
I tested this scenario with sample_encode, downloaded from solution portal under samples. The parameters i'm changing is - 
When initialize VPP params in pipeline_encode.cpp, Intialize the extended buffer and intialize the output vpp params, so that o/p is progressive and no. of frames are doubled

//MFX_EXTBUFF_VPP_DEINTERLACING 
m_Vppdeinterlace.Header.BufferId = MFX_EXTBUFF_VPP_DEINTERLACING;
m_Vppdeinterlace.Header.BufferSz = sizeof(mfxExtVPPDeinterlacing);
m_Vppdeinterlace.Mode = MFX_DEINTERLACING_BOB;
m_mfxVppParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
m_mfxVppParams.vpp.Out.FrameRateExtN = 60;
m_mfxVppParams.vpp.Out.FrameRateExtD = 1;

To run the o/p at 60fps, we need to change the destination frame rate to be 60fps, which need to initialize when initializing the encode params in pipeline_encode.cpp

m_mfxEncParams.mfx.FrameInfo.FrameRateExtN = 60;
m_mfxEncParams.mfx.FrameInfo.FrameRateExtD = 1;

Can you please use the sample and try this scenario and see if you can successful de-interlace and convert it to 60fps. If not, then can you please send us your input so that we can see it our end, if we are able to successfully do it. Also let us the know your system capabilities as I mentioned in the last post. 

Thanks,
-Surbhi

 

 

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Surbhi,

Thank you very much for the information provided. We are trying the experiments in the media sdk example simple_4_vpp_resize_procamp.

For an input of 60 fields per sec, we could get 30 fps progressive content as per configuaration below:

VPPParams.vpp.In.PicStruct = MFX_PICSTRUCT_FIELD_TFF;

VPPParams.vpp.In.FrameRateExtN = 30;

VPPParams.vpp.In.FrameRateExtD = 1;

VPPParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;

VPPParams.vpp.Out.FrameRateExtN = 30;

VPPParams.vpp.Out.FrameRateExtD = 1;

Issue 1:

For an input of 60 fields per sec, to get 60 fps progressive content we tried setting VPPParams.vpp.Out.FrameRateExtN to 60. We get MFX_ERR_MORE_SURFACE error from mfxVPP.RunFrameVPPAsync call.

Issue 2:

For an input of 60 fields per sec, to get 30 fps progressive content with required configuaration of algorithm as below, we get MFX_ERR_INVALID_VIDEO_PARAM error from mfxVPP.Init call.

mfxExtVPPDeinterlacing deinterlaceConfig;

memset(&deinterlaceConfig,0,sizeof(mfxExtVPPDeinterlacing));

deinterlaceConfig.Header.BufferId = MFX_EXTBUFF_VPP_DEINTERLACING;

deinterlaceConfig.Header.BufferSz = sizeof(mfxExtVPPDeinterlacing);

deinterlaceConfig.Mode = MFX_DEINTERLACING_BOB;

mfxExtBuffer* ExtBuffer[1];

ExtBuffer[0] = (mfxExtBuffer*)&deinterlaceConfig;

VPPParams.NumExtParam = 1;

VPPParams.ExtParam = (mfxExtBuffer**)&ExtBuffer[0];

We are doing tests on Intel core i5-2500 CPU @ 3.1 GHz (ARM of 4 GB) with Intel media SDK API version 1.9.  Do you see any issue with configuaration/CPU version?

Can you help us in finding out the changes required to the test application to solve above issues?

Regards,

Ranjit

 

 

 

0 Kudos
Surbhi_M_Intel
Employee
1,842 Views

Hi Ranjit,

Last week had been quite busy which took me long to get back to you. So here is the solution for Issue 1:
To handle the error surface please check the code mentioned in this article - https://software.intel.com/en-us/articles/frame-rate-conversion-frc-in-intel-media-sdk#GeneralStructure 
For issue 2: I will get back to you on that. Please make sure to use the i/p pic struct to be interlaced at frameN to be 30. I don't see a problem with your config but can you let me know the graphics driver as well. you can run system analyzer to find that out. details can be found here - https://software.intel.com/en-us/articles/details-necessary-for-filing-forum-questions#Analyzer

Thanks,
-Surbhi

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Surbhi,

Thanks for the reference for issue 1. We tried doing the changes as per the article. However we see that the order of the frames in the output yuv is not in display order. We see back and forth motion in the same. Please note that the alternate timestamps (corresponding to output data) are -1. Do you have any clue about the reason for the back and forth motion mentioned?

Please find the information about graphics driver below (issue2)

Intel Media SDK System Analyzer (32 bit)


The following versions of Media SDK API are supported by platform/driver:

        Version Target  Supported       Dec     Enc
        1.0     HW      No
        1.0     SW      Yes             X       X
        1.1     HW      No
        1.1     SW      Yes             X       X
        1.3     HW      No
        1.3     SW      Yes             X       X
        1.4     HW      No
        1.4     SW      Yes             X       X
        1.5     HW      No
        1.5     SW      Yes             X       X
        1.6     HW      No
        1.6     SW      Yes             X       X
        1.7     HW      No
        1.7     SW      Yes             X       X
        1.8     HW      No
        1.8     SW      Yes             X       X

Graphics Devices:
        Name                                         Version             State
        Intel(R) HD Graphics Family                  8.15.10.2696        Active

System info:
        CPU:    Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
        OS:     Microsoft Windows 7 Professional
        Arch:   32-bit

Installed Media SDK packages (be patient...processing takes some time):
        Intel« Media SDK Video Encoding Sample 5.0.461.91752
        Intel« Media SDK 2014 R2 for Clients (x86)

Installed Media SDK DirectShow filters:

Installed Intel Media Foundation Transforms:
  Intel« Hardware VC-1 Decoder MFT : {059A5BAE-5D7A-4C5E-8F7A-BFD57D1D6AAA}
  Intel« Hardware H.264 Decoder MFT : {45E5CE07-5AC7-4509-94E9-62DB27CF8F96}
  Intel« Hardware MPEG-2 Decoder MFT : {CD5BA7FF-9071-40E9-A462-8DC5152B1776}
  Intel« Quick Sync Video H.264 Encoder MFT : {4BE8D3C0-0515-4A37-AD55-E4BAE19AF471}
  Intel« Hardware Preprocessing MFT : {EE69B504-1CBF-4EA6-8137-BB10F806B014}

 


Tips:
 - HW target does not work: If you expect it should, then make sure to install
   latest Intel gfx driver, and that Intel gfx is selected as primary driver
 - If Intel driver is associated with secondary adapter, make sure to
   initialize DirectX device used with Media SDK with corresponding adapter

 

Regards,

Ranjit

 

 

0 Kudos
Surbhi_M_Intel
Employee
1,842 Views

Hi Ranjit,

Thanks for sending the system analyzer data. You are using a Sandy bridge(2nd Generation) which is not in the supported list in the Media SDK R2. Release notes: https://software.intel.com/sites/default/files/mediasdk_release_notes_0.pdf. Due to which you might not be able to fully utilize the capabilities of Media SDK.  I would suggest you to update your drivers from downloadcenter.intel.com. You current graphics driver seems to be quite old, lot of fixes have been made since then.

Can you please explain this better -  "However we see that the order of the frames in the output yuv is not in display order. We see back and forth motion in the same.Please note that the alternate timestamps (corresponding to output data) are -1"
That article is for Frame rate conversion where time stamps are required to ensure that the algorithm of preserving or distributing timestamp is working. I think i wasn't very clear that in that article you can check how to handle MFX_ERR_MORE_SURFACE error. 
I have never tried using time stamps for de-interlacer, can you please send us the print of i/p and o/p time stamps with the code changes. I will reproduce the issue and will be able to help you better. 

Thanks,
-Surbhi

 

 

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi

The frame reordering issue I reported was on i7-4770 machine where we could use hardware for deinterlacing.

Suppose the order of frames is 0,1,2,3,4,5,6,7,8,9,10

In case of reordering the order of frames is 0,2,1,4,3,6,5,8,7,10,9

Following is the log input and output timestamps:

In Timestamp 7200
Out Timestamp 7200


In Timestamp 10800
Out Timestamp -1


In Timestamp 10800
Out Timestamp 10800


In Timestamp 14400
Out Timestamp -1


In Timestamp 14400
Out Timestamp 14400


In Timestamp 18000
Out Timestamp -1


In Timestamp 18000
Out Timestamp 18000


In Timestamp 21600
Out Timestamp -1


In Timestamp 21600
Out Timestamp 21600


In Timestamp 25200
Out Timestamp -1


In Timestamp 25200
Out Timestamp 25200


In Timestamp 28800
Out Timestamp -1


In Timestamp 28800
Out Timestamp 28800


In Timestamp 32400
Out Timestamp -1


In Timestamp 32400
Out Timestamp 32400


In Timestamp 36000
Out Timestamp -1


In Timestamp 36000
Out Timestamp 36000


In Timestamp 39600
Out Timestamp -1


In Timestamp 39600
Out Timestamp 39600

Please let us know if I need to give any other information.

Regards,

Ranjit

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Surbhi,

I hope you have received the files that I have sent you. Please let me know how we can solve this problem?

Regards,

Ranjit

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Surbhi,

Thanks for the support provided so far. We are able to root cause the issue and the issue seems to be with source (it has bottom field captured first). We tried testing the application with other contents and we did not see any issue.

Regrds,

Ranjit

0 Kudos
Ranjit_T_1
Beginner
1,842 Views
Hi,
 
We have some queries on configuring of deinterlacing algorithm. We are not able to successfully configure the deinterlacing algorithm (bob/advanced). We tried making a query() call after filling the VPPparams (with deinterlacing mode as advanced),
1. In software mode, in the ext params, mode is being set to 0 which does not map to bob or advanced. However we see that deintelacing happens fine. But we don't know the algorithm used.
2. In hardware mode, the BufferId of extparams which is set to MFX_EXTBUFF_VPP_DEINTERLACING (before query call) is being set to 0, after query call. So init call is returning error. 
What could the reason be for the same?
 
We have another query, what would the main difference be in the pre-processing algorithms from MEDIA SDK API version 1.0 to 1.9? Is it just API has changed or algorithm has also changed?
 
Regards,
Ranjit
0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi experts,

Do you have any comments on above queries?

Regards,

Ranjit

0 Kudos
Surbhi_M_Intel
Employee
1,842 Views

Hi Ranjit, 

I am looking into it and finding more concrete answers for it. I will get back to you within a day's time. 

-Surbhi

0 Kudos
Surbhi_M_Intel
Employee
1,842 Views

Hi Ranjit,

After few experiments on Windows and Linux system, here is the answer- 

1. In software mode, in the ext params, mode is being set to 0 which does not map to bob or advanced. However we see that deintelacing happens fine. But we don't know the algorithm used.
Unfortunately, there is one algorithm being set up by the graphics driver only on windows OS. There is no switch between bob or advance or by default. If you check the o/p of all three it would be all same. There is no easy way to decide what algorithm is being used. From an architecture point of view IVB system should support BOB and HSW system just have support for Advanced Deinterlacing. Having said that system drivers play an important role here and choose the algorithm.
This switch is possible in Linux system and you can see a difference in quality of bob and advanced. So I have compared the quality of the o/p of Linux to windows to figure out. I see the quality closer to BOB instead of Advanced. Also to point out the external buffer for deinterlacer is ignored as far as I have tested since not even setting external buffers results in same quality. 
So I would say you are most probably getting a BOB quality. 
 

2. In hardware mode, the BufferId of extparams which is set to MFX_EXTBUFF_VPP_DEINTERLACING (before query call) is being set to 0, after query call. So init call is returning error. 
Are you using vmem example or set -hw flag for simple_4_vpp_denoise example? I am assuming you are setting -hw flag. I have checked on my system and it sets the the value to be 32. I am attaching the simple_vpp.cpp. Please use that and let us know if you see any issue. 

Thanks,
-Surbhi

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Surbhi,

       We have been doing the testing on windows OS so far.

       We understand that in windows OS you are able to configure extparams (as bob or advanced), but still the quality is on par across all cases. Whereas we are not able to configure extparams at all (ie, init call returns error "MFX_ERR_INVALID_VIDEO_PARAM"). We tried doing testing the same with the source file you have shared and found that still init call returns error "MFX_ERR_INVALID_VIDEO_PARAM"

​      We shall try doing above experiments in Linux OS and get back to you about queries.

Regards,

Ranjit

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Experts,

We have one query. Is it that software mode of video processing(VPP)/encoding/decoding is not supported in Linux OS?

We tried running denoising module in hardware mode (on Cent OS), there was no problem. When we tried running the same in software mode, the binary returns an error. Before debugging, we just wanted to check if video processing is supported in software mode on cent OS?

In addition, can anybody point out the path for the latest installation of intel media sdk for Ubuntu and Cent OS?

Our objective is to verify the quality we can achieve through software and hardware configuration?

Regards,

Ranjit

0 Kudos
Nina_K_Intel
Employee
1,842 Views

Hi Ranjit,

There is no software library in Linux version. There is only a limited software fallback in hw library for functions that may be unsupported by a platform, e.g. JPEG encode is software in current version.

For latest Media Server Studio check out the evaluation version download at this website: https://software.intel.com/en-us/intel-media-server-studio. The 2015 R3 version has been released recently.

Regards,

Nina

 

 

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Nina,

     Thank you very much for the quick reply. Our plan was to use software mode to do H.264/mpeg2 encoding/decoding and some of the pre-processing algorithms like deinterlacing . Now we see that this is not possible in Linux OS . Please confirm.

     However we understand that above can be achieved the in hardware mode. 

     Also are there any older versions of Media SDK where software mode is supported in Linux OS?

 Regards,

Ranjit

0 Kudos
Nina_K_Intel
Employee
1,842 Views

Hi Ranjit,

I confirm sw processing is not possible for these functions (h264/mpeg2 encoding/decoding and vpp) in Linux OS, there was as well no software mode in older versions of Media SDK for Linux OS. SW mode is available only on Windows OS. 

We are considering a possibility to add some of the functions in sw mode for Linux OS in the future, stay tuned for our product updates.

Regards,

Nina

0 Kudos
Ranjit_T_1
Beginner
1,842 Views

Hi Nina,

       Thanks for the confirmation.

Regards,

Ranjit

0 Kudos
Ranjit_T_1
Beginner
1,636 Views

Hi Nina,

        Is it possible to get early access to some version of Intel Media SDK which supports processing (encode + decode + VPP) in software mode along with hardware mode? Also can you let us know when you are planning for the software mode support in Intel Media SDK

      We feel that having some components in software mode and some in hardware mode will improve the overall trans coding system's throughput fps. Hence we are querying for above.

Regards,

Ranjit

 

0 Kudos
Reply