Fails to encode with ROI

[刚_隆_]

I'm using Intel MSS 2015 R6 on Windows 7 64bit, Core i5 4590 processor with very recent driver. And latest Intel Media Samples 6.0.0.68.

When I tried to modify sample_encode to enable ROI encoding test, two problems arose:

1. I added mfxExtEncoderROI with only 1 ROI to mfxEncodeCtrl before calling EncodeFrameAsync() per frame, the returned status seemed OK but after all frames were encoded, I decoded the bitstream and compared the uncompressed YUV to the one processed without ROI parameters, the results turned out to be exactly the same. That's to say, ROI encoding setting took no effect at all!

2. Alternatively, I also tried to add mfxExtEncoderROI to mfxVideoParam at the initialization phase, but such parameter would cause MFXVideoENCODE_Init to return MFX_ERR_INVALID_VIDEO_PARAM directly.

Note: I was using SW encoder for test.

Basically, I changed sample_encode at the minimum need, for example: still H.264 encoding and CBR mode. Kind of being similar to those mentioned in https://software.intel.com/en-us/forums/topic/537260 except for mfxExtCodingOption.

Is there any way to help me work out these problems, especially issue#1 ?

Thanks!

[Alexey_F_Intel]

try it in const QP mode

[刚_隆_]

Then I tried CQP and even VBR mode, with "priority" = 3 for a ROI: top, left, right, bottom = (0, 0, 128, 128). It still made no difference to the results without adding mfxExtEncoderROI per frame.

This puzzled me a lot :(

[david123456]

For CQP mode ,try Priority =-50 or  50 for ROI, you will see great differ.

Priority uses value (-51~+51) to modify real QP.

[刚_隆_]

I did learn ROI-relevant description in SDK manual carefully, and I did know the CQP range. But I tried big delta QP on ROI like david123456 said, it just made no difference at all.
Actually I have been using exact binary-to-binary comparison in my every test which exclude potential mistakes by visual judgment. There was just not a single bit of difference.

So I think there must be something wrong else.

[Surbhi_M_Intel]

Hi there, 

I tried your use case at our end setting ROI structure and CQP bit rate control method, I am able to replicate your issue. We are working on it, this could be a possible sample bug. Please give us a day or two time to debug and find the root cause of this issue. 

Thanks,
Surbhi

[刚_隆_]

Hi Surbhi,

Thanks for the feedback. Here I just add a few: actually I had also tested H.264 hw encoding, HEVC sw encoding, and HEVC gacc encoding, all resulted in the same issue.

So in all: H.264 sw/hw & HEVC sw/GACC, none of the four modes works with ROI in per-frame mfxEncodeCtrl. And I'd like to mention our first priority is HEVC w/ GAcc.

Thanks

[Surbhi_M_Intel · Accepted solution]

Hi there, 

I am successfully able to use the ROI structure with CQP through sample_encode. I have attached the code working at our end. 
cmd line : sample_encode.exe h264 -i ducks_1080p.yuv -o out_roi_async_1.h264 -cqp -qpi 17 -qpb 17 -qpp 17 -async 1 -hw  -f 30 -w 1920 -h 1080
In case, it doesn't work please check your driver version. Also, can you please let us know how are you verifying the ROI change? ROI structure is not supported for HEVC, present only for H264 at the moment.
Let us know if you still see any issue. 

Thanks,
Surbhi

[刚_隆_]

Hi Surbhi,

The way I verify ROI change is two-fold: (1) in binary comparison with non-ROI bitstream to see if there is any bit of difference, and (2) decoding both bitstreams to compare YUVs visually. And I well know the first criteria is more strict.

I was able to reproduce successful H.264 hw encoding like you said, in which ROI is meant to take effect since initialization phase. And alternatively, it's OK to add ROI to mfxEncodeCtrl before calling EncodeFrameAsync() per frame, given it's H.264 HW encoding (p.s. SW encoding didn't work, though it's not my concern). 

My concern is, when I tried rate control mode like VBR instead of CQP, it didn't work: e.g. if I set VBR and add ROI to mfxEncodeCtrl per frame, EncodeFrameAsync() will report MFX_WRN_INCOMPATIBLE_VIDEO_PARAM, and the bitstream becomes exactly the same as non-ROI counterpart.

So it seems to me that currently ROI is really supported only in H.264 HW mode under CQP rate control. Please let me know if something wrong in my conclusion.

From customer point of view, I would appreciate very much if Intel MSS can soon support ROI to the following cases: 1. HEVC encoding, and 2. VBR encoding. We are looking forward to the progress of ROI in HEVC+VBR.

Thanks!

[Alexey_F_Intel]

your conclusion is correct. If you are MSS Pro customer - please submit IPS bug report/feature request.

Thanks, Alexey

[Hang_L_Intel]

Hi,

I encountered one problem while using the mfxExtEncoderROI in the Media SDK, the problem is after the ROI information have been set, the Query() function failed with MFX_ERR_UNSUPPORTED, I wonder is this feature supported in my current development environment.

The Media SDK version I am using is: Intel Media SDK 2017 R1, the API version is 1.23.

I am using a NUC (https://www.amazon.com/Intel-NUC6i7KYK-Mini-i7-6770HQ-Windows/dp/B01FLDS5SO) with a i7-6770HQ CPU with Iris Pro Graphics 580. My OS is Windows 10 Enterprise 64Bit.

Some parts of the code are listed as follows:

m_mfxROI.NumROI = 1;

m_mfxROI.ROI[0].Left = 480;

m_mfxROI.ROI[0].Right = 1440;

m_mfxROI.ROI[0].Top = 270;

m_mfxROI.ROI[0].Bottom = 810;

m_mfxROI.ROI[0].Priority = -50; // CQP mode

m_EncExtParams.push_back((mfxExtBuffer*)&m_mfxROI);

...

the failed function is:

 sts = GetFirstEncoder()->Query(&m_mfxEncParams, &m_mfxEncParams);
 MSDK_CHECK_STATUS(sts, "Query (for encoder) failed");