DecodeFrameAsync returns MFX_ERR_MORE_DATA until second I-frame

Bob_Kirnum · ‎01-26-2016

I have much of the Media SDK functionality working within our real-time streaming media framework (enhanced media server). I have work around some previous issues (i.e. can't get VPP to do NV12 to YV12 color model conversion). However, I have one issue I have been struggling with. For some reason the DecodeFrameAsync always returns MFX_ERR_MORE_DATA from the first I-frame until the next one. All preceding initialization steps work fine without error. I am logging the bitstream presented to the DecodeFrameAsync API to file (attached) and there are no errors reported in our video analyzer.

Debug output with key structure dumps before and after decoder API calls..

0(MFX_ERR_NONE) = MFXQueryVersion(0x7fee24000aa0, 0x7feeae22aedc) 1.16

sInputParams: videoType AVC mode 0 memType 0 bUseHWLib true bIsMVC false bLowLat false bCalLat false nMaxFPS 30 nWallCell 0 nWallW 0 nWallH 0 nWallMonitor 0 bWallNoTitle false nWallTimeout 0 numViews 0 nRotation 0 nAsyncDepth 1 gpuCopy 0 nThreadsNum 0 SchedulingType 0 Priority 0 width 704 height 576 fourcc YV12 nFrames 0

mfxBitstream: EncryptedData (nil) ExtParam (nil) NumExtParam 0 DecodeTimeStamp 5313600 TimeStamp 0 Data 0x7fedc7758018 DataOffset 0 DataLength 28212 MaxLength 28212 PicStruct 0 FrameType 0 DataFlag 1

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 34( IN_SYSTEM OUT_SYSTEM) ExtParam (nil) NumExtParam 0

mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 0 CodecLevel 0 NumThread 0

mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC Width 0 Height 0 Crop (0, 0, 0, 0) ChromaFormat 0(MONO) PicStruct 0
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0

Call to decode header with above bitstream (first I-frame containing SPS / PPS) and video parameters passes. Offset remains at 0, offset to SPS / PPS. Video parameters updated and correlate to bitstream.

0(MFX_ERR_NONE) = m_pmfxDEC->DecodeHeader(0x7fedf914a1d8, 0x7fedf914a238)

mfxBitstream: EncryptedData (nil) ExtParam (nil) NumExtParam 0 DecodeTimeStamp 5313600 TimeStamp 0 Data 0x7fedc7758018 DataOffset 0 DataLength 28212 MaxLength 28212 PicStruct 0 FrameType 0 DataFlag 1
mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 34( IN_SYSTEM OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 704, 576) ChromaFormat 1(YUV420) PicStruct 1
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0

pretending that stream is 30fps one
pretending that aspect ratio is 1:1

CreateAllocator: mfxIMPL: MFX_IMPL_HARDWARE:MFX_IMPL_VIA_VAAPI
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0

Call to Query passes and video parameters are unchanged but for the Crop values.

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 704, 576) ChromaFormat 1(YUV420) PicStruct 1
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0

0(MFX_ERR_NONE) = m_pmfxDEC->Query(0x7fedf914a238, 0x7fedf914a238)

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 0, 0) ChromaFormat 1(YUV420) PicStruct 1

Call to QueryIOSurf passes.

mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0
mfxFrameAllocRequest: Info: BitDepthLuma 0 BitDepthChroma 0 Shift 0 FourCC Width 0 Height 0 CropX 0 CropY 0 CropW 0 CropH 0 FrameRateExtN 0 FrameRateExtD 0 AspectRatioW 0 AspectRatioH 0 PicStruct 0 ChromaFormat 0(MONO)
mfxFrameAllocRequest: AllocId 0 Type 0() NumFrameMin 0 NumFrameSuggested 0

0(MFX_ERR_NONE) = m_pmfxDEC->QueryIOSurf(0x7fedf914a238, 0x7fee602b6360)

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 0, 0) ChromaFormat 1(YUV420) PicStruct 1

mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0
mfxFrameAllocRequest: Info: BitDepthLuma 0 BitDepthChroma 0 Shift 0 FourCC NV12 Width 704 Height 576 CropX 0 CropY 0 CropW 0 CropH 0 FrameRateExtN 30 FrameRateExtD 1 AspectRatioW 1 AspectRatioH 1 PicStruct 1 ChromaFormat 1(YUV420)
mfxFrameAllocRequest: AllocId 0 Type 0x242( SYSTEM FROM_DECODE EXTERNAL) NumFrameMin 1 NumFrameSuggested 1
mfxIMPL: MFX_IMPL_HARDWARE:MFX_IMPL_VIA_VAAPI

Call to Alloc passes.

0(MFX_ERR_NONE) = m_pMFXAllocator->Alloc(0x7fee240264c0, 0x7fee602b6360, 0x7fedf914a148)

Call to Init passes with same video parameters above.

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 0, 0) ChromaFormat 1(YUV420) PicStruct 1
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0

0(MFX_ERR_NONE) = m_pmfxDEC->Init(0x7fedf914a238)

mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 0
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 0, 0) ChromaFormat 1(YUV420) PicStruct 1
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0
0(MFX_ERR_NONE) = m_pmfxDEC->GetVideoParam(0x7fedf914a238)

The same frame used above is passed to DecodeFrameAsync which returns 'more data'. The entire frame is consumed as reflected in the offset. This continues for the subsequent P-frames until the next I-frame arrives.

mfxBitstream: EncryptedData (nil) ExtParam (nil) NumExtParam 0 DecodeTimeStamp 5313600 TimeStamp 0 Data 0x7fedc7758018 DataOffset 0 DataLength 28212 MaxLength 28212 PicStruct 0 FrameType 0 DataFlag 1
mfxVideoParam: AsyncDepth 1 Protected 0 IOPattern 32( OUT_SYSTEM) ExtParam (nil) NumExtParam 0
mfxVideoParam: mfx: LowPower 0 BRCParamMultiplier 0 CodecId AVC CodecProfile 66 CodecLevel 31 NumThread 1
mfxVideoParam: mfx: FrameInfo BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 Crop (0, 0, 0, 0) ChromaFormat 1(YUV420) PicStruct 1
mfxVideoParam: mfx: DecodedOrder 0 ExtendedPicStruct 0 TimeStampCalc 0 SliceGroupsPresent 0 MaxDecFrameBuffering 0
mfxBitstream: EncryptedData (nil) ExtParam (nil) NumExtParam 0 DecodeTimeStamp 5313600 TimeStamp 0 Data 0x7fedc7758018 DataOffset 0 DataLength 28212 MaxLength 28212 PicStruct 0 FrameType 0 DataFlag 1

-10(MFX_ERR_MORE_DATA) = m_pmfxDEC->DecodeFrameAsync(0x7fedf914a1d8, 0x7fee240265e0, 0x7fee602b6790, 0x7fee24026408)

mfxFrameSurface1: Info: BitDepthLuma 0 BitDepthChroma 0 Shift 0 FrameId: [TemporalId 0 PriorityId 0 DependencyId 0 ViewId 0] FourCC NV12 Width 704 Height 576 CropX 0 CropY 0 CropW 0 CropH 0 FrameRateExtN 30 FrameRateExtD 1 AspectRatioW 1 AspectRatioH 1 PicStruct 1 ChromaFormat 1(YUV420)
mfxFrameSurface1: Data: ExtParam (nil) NumExtParam 0 PitchHigh 0 TimeStamp 0 FrameOrder 0 Locked 0 Pitch 704 Y 0x7fedc42670a0 (405504) CbCr 0x7fedc42ca0a0 (202752) A (nil) Corrupted 0 DataFlag 0
mfxBitstream: EncryptedData (nil) ExtParam (nil) NumExtParam 0 DecodeTimeStamp 5313600 TimeStamp 0 Data 0x7fedc7758018 DataOffset 28212 DataLength 0 MaxLength 28212 PicStruct 0 FrameType 0 DataFlag 1

Environment

CentOS 7, MediaServerStudioProfessionalEvaluation2015R6, SDK2015Production16.4.2.1

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 60
model name      : Intel(R) Xeon(R) CPU E3-1285 v3 @ 3.60GHz
stepping        : 3
microcode       : 0x1c
cpu MHz         : 3437.296
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 1
cpu cores       : 4
apicid          : 2
initial apicid : 2
fpu             : yes
fpu_exception   : yes

Bjoern_B_Intel · ‎01-28-2016

Hi Bob,

I will look at this but please give me some time to reproduce.

Best,

Bjoern