Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

H.246 HW encoding interlaced video

andrasjpeg
Beginner
1,288 Views
I am trying to encode 1080i30 video to h264, using HW acceleration.

I am using theMFX_EXTBUFF_CODING_OPTION, and setting FramePicture toMFX_CODINGOPTION_ON, andPicStruct toMFX_PICSTRUCT_FIELD_TFF, but I get an error during encoder initialization.

However, if I just use MFX_PICSTRUCT_PROGRESSIVE, then everything works, but then the image will get encoded as progressive, even though it's interlaced. This results in reduced compression quality, and also playback devices won't properly deinterlace the video.

Is this a known limitation? Or am I missing something?


Would it be possible to manually separate the interlaced frames to fields, and compress them as progressive, and set some flags, so that playback devices can comb the fields together, and display the full deinterlaced field?

Thanks,

Andras
0 Kudos
14 Replies
Anthony_P_Intel
Employee
1,288 Views
Hi Andras,

There is a constraint for encode that PicStruct must be set during SDK initialization and during operation.See the MediaSDKReferenceManual (mediasdk-man.pdf) Appendix A.

If this is not the cause of your issue, can you please provide some information about what error you are seeing? You can use the "tools\mediasdk_tracer\tracer.exe" tool to capture a log if needed.

-Tony
0 Kudos
andrasjpeg
Beginner
1,287 Views
Hi Tony,

I've captured a log. Since it's rather large, I uploaded it to dropbox. You can download it from here:http://dl.dropbox.com/u/33604139/analyzer.log

And here's a short excerpt from the bottom of the log:
SDK DLL File Version=3.12.2.8
SDK DLL Product Version=3.0.361.39233
SDK DLL File Size=12577720
MFXInit.impl=5
MFXInit.ver=1.1
MFXInit.status=MFX_ERR_NONE
MFXQueryIMPL.impl=5
MFXQueryVersion.ver=1.3
encode.Query.in.AsyncDepth=0
encode.Query.in.mfx.FrameInfo.FourCC=NV12
encode.Query.in.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.in.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.in.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.in.mfx.FrameInfo.AspectRatio=1:1
encode.Query.in.mfx.FrameInfo.PicStruct=FIELD_TFF
encode.Query.in.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.in.mfx.CodecId=AVC
encode.Query.in.mfx.CodecProfile=100
encode.Query.in.mfx.CodecLevel=42
encode.Query.in.mfx.NumThread=0
encode.Query.in.mfx.TargetUsage=4
encode.Query.in.mfx.GopPicSize=0
encode.Query.in.mfx.GopRefDist=0
encode.Query.in.mfx.GopOptFlag=0
encode.Query.in.mfx.IdrInterval=0
encode.Query.in.mfx.RateControlMethod=CBR
encode.Query.in.mfx.InitialDelayInKB=0
encode.Query.in.mfx.BufferSizeInKB=0
encode.Query.in.mfx.TargetKbps=8000
encode.Query.in.mfx.MaxKbps=0
encode.Query.in.mfx.NumSlice=0
encode.Query.in.mfx.NumRefFrame=0
encode.Query.in.mfx.EncodedOrder=0
encode.Query.in.Protected=0
encode.Query.in.IOPattern=IN_SYSTEM_MEMORY
encode.Query.in.NumExtParam=1
encode.Query.in.ExtParam=mfxExtCodingOption,
encode.Query.in.ExtParam.mfxExtCodingOption.RateDistortionOpt=0
encode.Query.in.ExtParam.mfxExtCodingOption.MECostType=0
encode.Query.in.ExtParam.mfxExtCodingOption.MESearchType=0
encode.Query.in.ExtParam.mfxExtCodingOption.MVSearchWindow=(0,0)
encode.Query.in.ExtParam.mfxExtCodingOption.EndOfSequence=0
encode.Query.in.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.in.ExtParam.mfxExtCodingOption.CAVLC=0
encode.Query.in.ExtParam.mfxExtCodingOption.NalHrdConformance=0
encode.Query.in.ExtParam.mfxExtCodingOption.SingleSeiNalUnit=0
encode.Query.in.ExtParam.mfxExtCodingOption.VuiVclHrdParameters=0
encode.Query.in.ExtParam.mfxExtCodingOption.RefPicListReordering=0
encode.Query.in.ExtParam.mfxExtCodingOption.ResetRefList=0
encode.Query.in.ExtParam.mfxExtCodingOption.RefPicMarkRep=0
encode.Query.in.ExtParam.mfxExtCodingOption.FieldOutput=0
encode.Query.in.ExtParam.mfxExtCodingOption.IntraPredBlockSize=0
encode.Query.in.ExtParam.mfxExtCodingOption.InterPredBlockSize=0
encode.Query.in.ExtParam.mfxExtCodingOption.MVPrecision=0
encode.Query.in.ExtParam.mfxExtCodingOption.MaxDecFrameBuffering=0
encode.Query.in.ExtParam.mfxExtCodingOption.AUDelimiter=0
encode.Query.in.ExtParam.mfxExtCodingOption.EndOfStream=0
encode.Query.in.ExtParam.mfxExtCodingOption.PicTimingSEI=0
encode.Query.in.ExtParam.mfxExtCodingOption.VuiNalHrdParameters=0
encode.Query.out.AsyncDepth=0
encode.Query.out.mfx.FrameInfo.FourCC=NV12
encode.Query.out.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.out.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.out.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.out.mfx.FrameInfo.AspectRatio=1:1
encode.Query.out.mfx.FrameInfo.PicStruct=PROGRESSIVE
encode.Query.out.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.out.mfx.CodecId=AVC
encode.Query.out.mfx.CodecProfile=100
encode.Query.out.mfx.CodecLevel=42
encode.Query.out.mfx.NumThread=0
encode.Query.out.mfx.TargetUsage=4
encode.Query.out.mfx.GopPicSize=0
encode.Query.out.mfx.GopRefDist=0
encode.Query.out.mfx.GopOptFlag=0
encode.Query.out.mfx.IdrInterval=0
encode.Query.out.mfx.RateControlMethod=CBR
encode.Query.out.mfx.InitialDelayInKB=0
encode.Query.out.mfx.BufferSizeInKB=0
encode.Query.out.mfx.TargetKbps=8000
encode.Query.out.mfx.MaxKbps=0
encode.Query.out.mfx.NumSlice=0
encode.Query.out.mfx.NumRefFrame=0
encode.Query.out.mfx.EncodedOrder=0
encode.Query.out.Protected=0
encode.Query.out.IOPattern=IN_SYSTEM_MEMORY
encode.Query.out.NumExtParam=1
encode.Query.out.ExtParam=mfxExtCodingOption,
encode.Query.out.ExtParam.mfxExtCodingOption.RateDistortionOpt=0
encode.Query.out.ExtParam.mfxExtCodingOption.MECostType=0
encode.Query.out.ExtParam.mfxExtCodingOption.MESearchType=0
encode.Query.out.ExtParam.mfxExtCodingOption.MVSearchWindow=(0,0)
encode.Query.out.ExtParam.mfxExtCodingOption.EndOfSequence=0
encode.Query.out.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.out.ExtParam.mfxExtCodingOption.CAVLC=0
encode.Query.out.ExtParam.mfxExtCodingOption.NalHrdConformance=0
encode.Query.out.ExtParam.mfxExtCodingOption.SingleSeiNalUnit=0
encode.Query.out.ExtParam.mfxExtCodingOption.VuiVclHrdParameters=0
encode.Query.out.ExtParam.mfxExtCodingOption.RefPicListReordering=0
encode.Query.out.ExtParam.mfxExtCodingOption.ResetRefList=0
encode.Query.out.ExtParam.mfxExtCodingOption.RefPicMarkRep=0
encode.Query.out.ExtParam.mfxExtCodingOption.FieldOutput=0
encode.Query.out.ExtParam.mfxExtCodingOption.IntraPredBlockSize=0
encode.Query.out.ExtParam.mfxExtCodingOption.InterPredBlockSize=0
encode.Query.out.ExtParam.mfxExtCodingOption.MVPrecision=0
encode.Query.out.ExtParam.mfxExtCodingOption.MaxDecFrameBuffering=0
encode.Query.out.ExtParam.mfxExtCodingOption.AUDelimiter=0
encode.Query.out.ExtParam.mfxExtCodingOption.EndOfStream=0
encode.Query.out.ExtParam.mfxExtCodingOption.PicTimingSEI=0
encode.Query.out.ExtParam.mfxExtCodingOption.VuiNalHrdParameters=0
encode.Query.status=MFX_WRN_INCOMPATIBLE_VIDEO_PARAM
0 Kudos
andrasjpeg
Beginner
1,288 Views
(I've submitted this reply once already, but didn't seem to appear, so I'm trying again. Sorry, if it ends up duplicated..)

Tony,

I've created a log with the tracer. Let me know if you'd like to see the entire log, it's quite big. Here's an excerpt from the bottom of the log:

SDK DLL File Version=3.12.2.8
SDK DLL Product Version=3.0.361.39233
SDK DLL File Size=12577720
MFXInit.impl=5
MFXInit.ver=1.1
MFXInit.status=MFX_ERR_NONE
MFXQueryIMPL.impl=5
MFXQueryVersion.ver=1.3
encode.Query.in.AsyncDepth=0
encode.Query.in.mfx.FrameInfo.FourCC=NV12
encode.Query.in.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.in.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.in.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.in.mfx.FrameInfo.AspectRatio=1:1
encode.Query.in.mfx.FrameInfo.PicStruct=FIELD_TFF
encode.Query.in.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.in.mfx.CodecId=AVC
encode.Query.in.mfx.CodecProfile=100
encode.Query.in.mfx.CodecLevel=42
encode.Query.in.mfx.NumThread=0
encode.Query.in.mfx.TargetUsage=4
encode.Query.in.mfx.GopPicSize=0
encode.Query.in.mfx.GopRefDist=0
encode.Query.in.mfx.GopOptFlag=0
encode.Query.in.mfx.IdrInterval=0
encode.Query.in.mfx.RateControlMethod=CBR
encode.Query.in.mfx.InitialDelayInKB=0
encode.Query.in.mfx.BufferSizeInKB=0
encode.Query.in.mfx.TargetKbps=8000
encode.Query.in.mfx.MaxKbps=0
encode.Query.in.mfx.NumSlice=0
encode.Query.in.mfx.NumRefFrame=0
encode.Query.in.mfx.EncodedOrder=0
encode.Query.in.Protected=0
encode.Query.in.IOPattern=IN_SYSTEM_MEMORY
encode.Query.in.NumExtParam=1
encode.Query.in.ExtParam=mfxExtCodingOption,
encode.Query.in.ExtParam.mfxExtCodingOption.RateDistortionOpt=0
encode.Query.in.ExtParam.mfxExtCodingOption.MECostType=0
encode.Query.in.ExtParam.mfxExtCodingOption.MESearchType=0
encode.Query.in.ExtParam.mfxExtCodingOption.MVSearchWindow=(0,0)
encode.Query.in.ExtParam.mfxExtCodingOption.EndOfSequence=0
encode.Query.in.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.in.ExtParam.mfxExtCodingOption.CAVLC=0
encode.Query.in.ExtParam.mfxExtCodingOption.NalHrdConformance=0
encode.Query.in.ExtParam.mfxExtCodingOption.SingleSeiNalUnit=0
encode.Query.in.ExtParam.mfxExtCodingOption.VuiVclHrdParameters=0
encode.Query.in.ExtParam.mfxExtCodingOption.RefPicListReordering=0
encode.Query.in.ExtParam.mfxExtCodingOption.ResetRefList=0
encode.Query.in.ExtParam.mfxExtCodingOption.RefPicMarkRep=0
encode.Query.in.ExtParam.mfxExtCodingOption.FieldOutput=0
encode.Query.in.ExtParam.mfxExtCodingOption.IntraPredBlockSize=0
encode.Query.in.ExtParam.mfxExtCodingOption.InterPredBlockSize=0
encode.Query.in.ExtParam.mfxExtCodingOption.MVPrecision=0
encode.Query.in.ExtParam.mfxExtCodingOption.MaxDecFrameBuffering=0
encode.Query.in.ExtParam.mfxExtCodingOption.AUDelimiter=0
encode.Query.in.ExtParam.mfxExtCodingOption.EndOfStream=0
encode.Query.in.ExtParam.mfxExtCodingOption.PicTimingSEI=0
encode.Query.in.ExtParam.mfxExtCodingOption.VuiNalHrdParameters=0
encode.Query.out.AsyncDepth=0
encode.Query.out.mfx.FrameInfo.FourCC=NV12
encode.Query.out.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.out.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.out.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.out.mfx.FrameInfo.AspectRatio=1:1
encode.Query.out.mfx.FrameInfo.PicStruct=PROGRESSIVE
encode.Query.out.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.out.mfx.CodecId=AVC
encode.Query.out.mfx.CodecProfile=100
encode.Query.out.mfx.CodecLevel=42
encode.Query.out.mfx.NumThread=0
encode.Query.out.mfx.TargetUsage=4
encode.Query.out.mfx.GopPicSize=0
encode.Query.out.mfx.GopRefDist=0
encode.Query.out.mfx.GopOptFlag=0
encode.Query.out.mfx.IdrInterval=0
encode.Query.out.mfx.RateControlMethod=CBR
encode.Query.out.mfx.InitialDelayInKB=0
encode.Query.out.mfx.BufferSizeInKB=0
encode.Query.out.mfx.TargetKbps=8000
encode.Query.out.mfx.MaxKbps=0
encode.Query.out.mfx.NumSlice=0
encode.Query.out.mfx.NumRefFrame=0
encode.Query.out.mfx.EncodedOrder=0
encode.Query.out.Protected=0
encode.Query.out.IOPattern=IN_SYSTEM_MEMORY
encode.Query.out.NumExtParam=1
encode.Query.out.ExtParam=mfxExtCodingOption,
encode.Query.out.ExtParam.mfxExtCodingOption.RateDistortionOpt=0
encode.Query.out.ExtParam.mfxExtCodingOption.MECostType=0
encode.Query.out.ExtParam.mfxExtCodingOption.MESearchType=0
encode.Query.out.ExtParam.mfxExtCodingOption.MVSearchWindow=(0,0)
encode.Query.out.ExtParam.mfxExtCodingOption.EndOfSequence=0
encode.Query.out.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.out.ExtParam.mfxExtCodingOption.CAVLC=0
encode.Query.out.ExtParam.mfxExtCodingOption.NalHrdConformance=0
encode.Query.out.ExtParam.mfxExtCodingOption.SingleSeiNalUnit=0
encode.Query.out.ExtParam.mfxExtCodingOption.VuiVclHrdParameters=0
encode.Query.out.ExtParam.mfxExtCodingOption.RefPicListReordering=0
encode.Query.out.ExtParam.mfxExtCodingOption.ResetRefList=0
encode.Query.out.ExtParam.mfxExtCodingOption.RefPicMarkRep=0
encode.Query.out.ExtParam.mfxExtCodingOption.FieldOutput=0
encode.Query.out.ExtParam.mfxExtCodingOption.IntraPredBlockSize=0
encode.Query.out.ExtParam.mfxExtCodingOption.InterPredBlockSize=0
encode.Query.out.ExtParam.mfxExtCodingOption.MVPrecision=0
encode.Query.out.ExtParam.mfxExtCodingOption.MaxDecFrameBuffering=0
encode.Query.out.ExtParam.mfxExtCodingOption.AUDelimiter=0
encode.Query.out.ExtParam.mfxExtCodingOption.EndOfStream=0
encode.Query.out.ExtParam.mfxExtCodingOption.PicTimingSEI=0
encode.Query.out.ExtParam.mfxExtCodingOption.VuiNalHrdParameters=0
encode.Query.status=MFX_WRN_INCOMPATIBLE_VIDEO_PARAM


What I notice is that MFXVideoENCODE_Query changed the PicStruct to PROGRESSIVE, then it returned the warning.. Any ideas?
0 Kudos
andrasjpeg
Beginner
1,288 Views
[Trying to re-post a shorter version, because the long one didn't seem to get through]
Tony, here's an excerpt from the trace log (I've removed the startup portion, and the lines that are set to zero, if you need to see the whole log, let me know how I can send it):

SDK DLL File Version=3.12.2.8
SDK DLL Product Version=3.0.361.39233
SDK DLL File Size=12577720
MFXInit.impl=5
MFXInit.ver=1.1
MFXInit.status=MFX_ERR_NONE
MFXQueryIMPL.impl=5
MFXQueryVersion.ver=1.3
encode.Query.in.mfx.FrameInfo.FourCC=NV12
encode.Query.in.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.in.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.in.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.in.mfx.FrameInfo.AspectRatio=1:1
encode.Query.in.mfx.FrameInfo.PicStruct=FIELD_TFF
encode.Query.in.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.in.mfx.CodecId=AVC
encode.Query.in.mfx.CodecProfile=100
encode.Query.in.mfx.CodecLevel=42
encode.Query.in.mfx.TargetUsage=4
encode.Query.in.mfx.RateControlMethod=CBR
encode.Query.in.mfx.TargetKbps=8000
encode.Query.in.IOPattern=IN_SYSTEM_MEMORY
encode.Query.in.NumExtParam=1
encode.Query.in.ExtParam=mfxExtCodingOption,
encode.Query.in.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.out.mfx.FrameInfo.FourCC=NV12
encode.Query.out.mfx.FrameInfo.Resolution=(1920,1088)
encode.Query.out.mfx.FrameInfo.Crop=(0,0,1920,1080)
encode.Query.out.mfx.FrameInfo.FrameRate=30000/1001
encode.Query.out.mfx.FrameInfo.AspectRatio=1:1
encode.Query.out.mfx.FrameInfo.PicStruct=PROGRESSIVE
encode.Query.out.mfx.FrameInfo.ChromaFormat=4:2:0
encode.Query.out.mfx.CodecId=AVC
encode.Query.out.mfx.CodecProfile=100
encode.Query.out.mfx.CodecLevel=42
encode.Query.out.mfx.TargetUsage=4
encode.Query.out.mfx.RateControlMethod=CBR
encode.Query.out.mfx.TargetKbps=8000
encode.Query.out.IOPattern=IN_SYSTEM_MEMORY
encode.Query.out.NumExtParam=1
encode.Query.out.ExtParam=mfxExtCodingOption,
encode.Query.out.ExtParam.mfxExtCodingOption.FramePicture=16
encode.Query.status=MFX_WRN_INCOMPATIBLE_VIDEO_PARAM

0 Kudos
andrasjpeg
Beginner
1,288 Views
Tony, I have tried replying to your post with a trace log 3 times today, but none of them got through. Perhaps it was too long, I don't know. I'll try to send a link this time:http://dl.dropbox.com/u/33604139/analyzer.log
0 Kudos
andrasjpeg
Beginner
1,288 Views
Is there anything I can do to move this thread forward? Thanks,

-Andras
0 Kudos
Anthony_P_Intel
Employee
1,288 Views
Hi Andras,

Sorry for the delay. I got your log and I'm taking a look at it now.
I should have an answer today.

-Tony
0 Kudos
Anthony_P_Intel
Employee
1,288 Views
Thanks you very much for your question.

I have not been able to identify the cause of the modification (and I've asked for help).
We hope to have an answer soon.

-Tony
0 Kudos
andrasjpeg
Beginner
1,288 Views
Ok, thanks for looking into this!
0 Kudos
Anthony_P_Intel
Employee
1,288 Views
Hi,

Sorry this the slow update.

I know realize that the core issue here is simply that the input & output of the 'encode' module must be the same. If the input to the encoder is interlaced, the only supported output is interlace, and if the input is progressive, the output is progressive. The MediaSDK supports conversion with the VPP component (not the encoder component). If you desire to create progressive content from interlaced content, then the input of VPP should be interlaced and the output of VPP should be progressive (and then the progressive output of VPP is fed into the encoder component).

-Tony
0 Kudos
andrasjpeg
Beginner
1,288 Views
Well, my problem is this: The actual source data is formatted the same way, whether it's interlaced or progressive. It's just that alternating lines represent different moment in time for interlaced video. So there needs to be a way to tell the encoder that the frame it's getting is interlaced.


Now, here's the issue: Normally, when I have an interlaced video file, the media player will recognize that it's interlaced and deinterlace it during display. However, even though I tell the encoder that my frames are interlaced, the media players treat my encoded video as progressive, not attempting to deinterlace it. So the end result is that the video has "mice teeth" when watching on a player..


Any thoughts on that?


0 Kudos
Petter_L_Intel
Employee
1,288 Views
Hi Andras,
sorry for the delayed reply.

We revisited this issue and found a potential issue with the SDK for this use case. We are investigating solutions.

In the meantime please set CodecLevel to 0 to let Media SDK select the suitable codec level. By doing this the output PicStruct will not me changed, as should be the expected behavior. Hopefully this is ok for your needs?

Regards,
Petter
0 Kudos
Petter_L_Intel
Employee
1,288 Views
Hi again,

Correction. This was investigated further and deemed not to be a bug.

The H.264 standard (chapter A.3.3.2) states that particular codec levels should obey "frame_mbs_only_flag=1" restriction (table A-4). This means that for these levels both interlaced and MBAFF encoding modes are prohibited.

So, you have the option of setting CodecLevel to 0 as suggested earlier or manually selecting supported level for interlaced mode.

Regards,
Petter
0 Kudos
andrasjpeg
Beginner
1,288 Views
Hey, sorry I got sidetracked on other projects, and stopped following this thread for a while..

Thanks a bunch for figuring this out, it all makes sense now!I'll give it a try as soon as I can, and report back!

Cheers,

Andras
0 Kudos
Reply