Main10 monochrome H.265 Encoding with Kaby Lake: MFX_ERR_UNDEFINED_BEHAVIOR (-16)

[Fahrni__Christoph]

The monochrome 10bit images from openCV should be compressed in a H.265 file with Kaby Lake HW accelerator (Ubuntu 19.10).

Because the supported formats are not b/w I changed to an RGB (CV_16UC3) image which is converted to YUV using cvtBGRtoTwoPlaneYUV.

	int fourcc = cv::VideoWriter::fourcc('H', '2', '6', '5');
	...
	params.mfx.CodecId = codecId;
	params.mfx.TargetUsage = MFX_TARGETUSAGE_BALANCED;
	params.mfx.TargetKbps = (mfxU16)cvRound(frameSize.area() * fps / 500); // TODO: set in options
	params.mfx.RateControlMethod = MFX_RATECONTROL_VBR;
	params.mfx.FrameInfo.FrameRateExtN = cvRound(fps * 1000);
	params.mfx.FrameInfo.FrameRateExtD = 1000;
	params.mfx.FrameInfo.BitDepthLuma = 10; // added by customer
	params.mfx.FrameInfo.BitDepthChroma = 10; // added by customer
	params.mfx.FrameInfo.FourCC = MFX_FOURCC_P010; // modified by customer
	params.mfx.FrameInfo.Shift = 1; // added by customer
	params.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
	params.mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
	params.mfx.FrameInfo.CropX = 0;
	params.mfx.FrameInfo.CropY = 0;
	params.mfx.FrameInfo.CropW = (mfxU16)frameSize.width;
	params.mfx.FrameInfo.CropH = (mfxU16)frameSize.height;
	params.mfx.FrameInfo.Width = (mfxU16)alignSize(frameSize.width, 32);
	params.mfx.FrameInfo.Height = (mfxU16)alignSize(frameSize.height, 32);
	params.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY;

The changes from the openCV videoio code are highlighted. This results in the output:

FrameInfo: 
| FourCC: P010 (0x30313050)
| Size: 288x288
| ROI: (0;0) 288x288
| BitDepth(L/C): 10 / 10
| Shift: 1
| TemporalID: 0
| FrameRate: 10000/1000
| AspectRatio: 0x0
| PicStruct: 1
| ChromaFormat: 1

By calling write_one I get the output:

MFX GetVideoParam: MFX_ERR_NONE (0)
requested 414 kB
BS Open /home/appVideo/data/o.mp4 -> 1
BS Allocate 847872 bytes (0.808594 Mb)
Calling with surface: 0x7fffec042450
ERR_MORE_DATA
Calling with surface: 0x7fffec042508
ERR_MORE_DATA
Calling with surface: 0x7fffec0425c0
ERR_MORE_DATA
Calling with surface: 0x7fffec042678
ERR_MORE_DATA
Calling with surface: 0x7fffec042730
MFX: Sync error: MFX_ERR_UNDEFINED_BEHAVIOR (-16)
Calling with surface: 0x7fffec0427e8
MFX: Sync error: MFX_ERR_UNDEFINED_BEHAVIOR (-16)
Calling with surface: 0x7fffec0428a0
MFX: Sync error: MFX_ERR_UNDEFINED_BEHAVIOR (-16)
Calling with surface: 0x7fffec042958
MFX: Sync error: MFX_ERR_UNDEFINED_BEHAVIOR (-16)
Calling with surface: 0x7fffec042a10

The ERR_MORE_DATA is normal according to the documentation. Because I did not find any documentation about the message MFX_ERR_UNDEFINED_BEHAVIOR I would like to ask the experts for support. Please let me know if I should take other formats to handle 10bit mono frames, etc. - Thank you, Christoph

[Dmitry_E_Intel · Accepted solution]

Since you're on Linux you can debug it manually using opensource MediaSDK. But most likely the error is due to wrong surface initialization:

        surface.Data.Y = dataPtr;
        surface.Data.UV = dataPtr + width * height;
        surface.Data.PitchLow = width & 0xFFFF;
        surface.Data.PitchHigh = (width >> 16) & 0xFFFF;

You want 10 bit surface. 10 bits are stored in two bytes, not in 1,25 bytes. So pointers to Y/U/V has to be set accordingly: https://github.com/Intel-Media-SDK/MediaSDK/blob/master/samples/sample_common/src/sysmem_allocator.cpp#L176

This is also wrong: "oneSize(width * height * bpp / 8)".