Interlaced video detection

[ODavi1]

Hi,

Is there any mean to detect if a video is interlaced or not, in IMSDK ?

I can't find it.

Thanks.

[Petter_L_Intel]

Hi Oliver,

I'm assuming the case you are asking for is Media SDK decode. Is that correct?

When decoding a stream the first operation is to provide the Media SDK decoder with the stream header via the DecodeHeader call. Upon successful call to DecodeHeader it populates the mfxVideoParam structure. Nested in this you can find: mfx.FrameInfo.PicStruct. PicStruct will tell you if the stream is progressive or interlaced etc. You can refer to the "sample_decode" sample for details on how this works.

However, that said, some streams have a mix of progressive and interlaced frames, and for some of those cases DecodeHeader will be not able to tell what type of stream is and thus set PicStruct to MFX_PICSTRUCT_UNKNOWN. For that case the user must determine the type of frame after each completed DecodeFrameAsync/SyncOperation. (conveyed via PicStruct in the surface, the "Info" structure).

Regards,
Petter 

[innes]

Hi Petter,

Using Media SDK decode, I would like to know if a video is encoded as interlaced or progressive. 

My video stream has following settings (mediainfo output) :

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Format settings, CABAC : Yes
Format settings, ReFrames : 3 frames
Bit rate : 8 000 Kbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 5:4
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Bits/(Pixel*Frame) : 0.772
Writing library : x264 core 118 r2092 6eac7c3
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=tff / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=1 / b_adapt=0 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=0 / keyint=100 / keyint_min=10 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=8000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

As a video stream which may have a mix of interlaced and progressive frames (MBAFF), I expect the video deocder to set PicStruct to  MFX_PICTSTRUCT_UNKNOWN. However, I always have MFX_PICTSTRUCT_PROGRESSIVE. Is this a normal behavior ?

I use the last (2013) Media SDK, software implementation.

Thank you

[Petter_L_Intel]

Hi,

Right, depending on how the stream has been encoded, Media SDK DecodeHeader may conclude that the stream is progressive or interlaced.

I you are implementing a decoder to support streams that has a mix of interlaced and progressive frames you will have to determine the type after each DecodeFrameAsync/SyncOperation call as described in my previous post.

Regards,
Petter

[Nina_K_Intel]

Hi Oliver,

Could you please share your stream by uploading it to ftp.intel.com\incoming? Once uploaded, please tell the exact name of the file and note that it should be less than 100 mb. It'll be stored for 24 hours after upload.

There is one known MSDK issue with decoding MBAFF streams, probably you bumped into that but I'd like to verify first. Meanwhile could you please try setting mfxVideoParam.mfx.ExtendedPicStruct = 1 for Decode.Init. This will enable the decoder to output extended picture structure values for additional display attributes. See the PicStruct field description in mediasdk_man.pdf for details. 

Thanks,

Nina

[SergeyKostrov]

Hi everybody, >>...There is one known MSDK issue with decoding MBAFF streams, probably you bumped into that... I'd like to verify that it is described in Release Notes of IMSDK and could you provide more details, please? Best regards, Sergey

[innes]

Hi Nina, Petter and Sergey

I uploaded my video stream on intel's ftp as requested. Its name is "interlaced_issue.264".

I just tried to run decode_sample.exe setting mfxVideoParam.mfx.ExtendedPicStruct to 1. Now, PicStruct for output frames is set to 3 (MFX_PICSTRUCT_PROGRESSIVE | MFX_PICSTRUCT_FIELD_TFF). I refer to media_sdk_man.pdf : "The output frame is progressive, display as two fields, top field first". 

Thanks,

[Nina_K_Intel]

Hi bzkl,

Thank you for sharing the stream, it confirmed my assumption.

So, current MSDK implementations including 2013 SW library have the following issue: for a stream with mb_adaptive_frame_field_flag = 1 and SEI.pic_struct = {top field, bottom_field} = 3 decoder will set a mixed value PicStruct = PROGRESSIVE | TFF for a frame. This is correct for many scenarios, e.g. for further display as it intructs to display in fields. But it is incorrect for some scenarios where MSDK should be treating the content as interlaced but will treat it as progressive according to the mediasdk_man.pdf. For instance, if such frame goes to VPP.resize after decode, VPP will treat the frame as essentially progressive and will mix fields in resizing algorithm. If the fields are from noticeably distant moments of time this mixing will lead to undesired visual effects. And fields from distant moments can be coded as frame (SEI.pic_struct = 3) within MBAFF model.

The correct behavour of decoder here is to check mbaff flag and set PicStruct = TFF for a frame with SEI.pic_struct = 3 if mbaff = 1 and TFF | PROGRESSIVE if mbaff = 0. A fix for SW library will be available in the next MSDK release and for HW library with driver updates for respective platforms. In the meantime a feasible workaround on application level would be to check for mbaff flag = 1 in stream header and remove the additional | PROGRESSIVE flag from the decoded frame PicStruct field. 

I took TFF for example only. The same applies to BFF.

To Sergey - this is a recently discovered problem so it wasn't mentioned in the latest release notes.

Regards,

Nina

[SergeyKostrov]

>>>>...I'd like to verify that it is described in Release Notes of IMSDK... >> >>...this is a recently discovered problem so it wasn't mentioned in the latest release notes. Thank you for the answer.

[innes]

Hi Nina,

Nina Kurina (Intel) wrote:

So, current MSDK implementations including 2013 SW library have the following issue

Does that mean that hw implementations are safe for interlacement detection using PicStruct values ?

Thanks,

[Nina_K_Intel]

I'm sorry for being unclear, what I meant was that current generation of HW implementations have the issue as well as the SW implementation from MSDK 2013.

Regards,

Nina

[innes]

Ok. 

Thank you for your answer.