RGB4 to H264 Encode Support

[Ramashankar]

Hi,

I noticed in Intel Media SDK 2018 R1 release and its sample code that it can now directly encode from RGB4 to H264 without using VPP.

 m_mfxEncParams.mfx.FrameInfo.FourCC       = pInParams->EncodeFourCC; //=> this I set as MFX_FOURCC_RGB4

I tried this in my PC and it actually worked fine (my PC hw is 6th gen and exposing API version 1.20). But if I try this same code on any old hw PC (3rd gen CPU, exposing API version 1.11) then encoder initialization get fails with error MFX_ERR_INVALID_VIDEO_PARAM.

So I want to know from which API version this RGB4 to h264 direct encode will work? Or if there is any way I can get it working on old HW too?

Note: In earlier SDK version I used to convert RGB4 to NV12 through VPP first and then feeding encoder with NV12 frame, but there is known issue of text color blur in RGB4 to NV12 VPP conversion ( https://software.intel.com/en-us/forums/intel-media-sdk/topic/657754), so I am much interested to use this RGB4 to H264 direct encoding feature).

[Mark_L_Intel1]

Hi Ramashankar,

After checking the code history, I believe this was changed in API 1.20 at the release of Media Server Studio 2017R2.

https://software.intel.com/en-us/articles/intel-media-sdk-release-notes

Let me know if you still have the accuracy of the timeline.

Mark

[Ramashankar]

Hi Mark,

Thanks for your reply. It seems that in sample code it may have changed from API version 1.20, but in driver dll (libmfxhw64.dll) it may have supported from earlier version itself. I could test this latest encoder on some pc with old intel hw & drivers, and results are as following:

1. Windows 8.1 (64 bit) + Intel i3-6098P CPU + intel HD Graphics 510 GPU + driver 21.20.16.4565 => exposing API 1.20 => RGB4 to H264 direct encode is successful.

2. Windows 8.1 (64 bit) + Intel i3-4150 CPU + intel HD Graphics 4400 GPU + driver 10.18.14.4578 => exposing API 1.17 => RGB4 to H264 direct encode is successful.

3. Windows 8.1 (64 bit) + Intel i3-4130 CPU + intel HD Graphics 4400 GPU + driver 10.18.14.4294 => exposing API 1.16 => RGB4 to H264 direct encode is successful.

4. Windows 10 (64 bit) 1803 build + Intel i5-3337u CPU + intel HD Graphics 4000 GPU + driver 10.18.10.4358 => exposing API 1.11 => RGB4 to H264 fails with MFX_ERR_INVALID_VIDEO_PARAM

Unfortunately I couldn't find any HW which expose API between 1.11 to 1.16 so not sure of behaviour there. But based on above results it seems 1.16 can be considered the threshold level. Now I am bit curious to know more on followings:

1. Is it safe to rely on this API version for using new RGB4->H264 direct encode feature? I mean can there be any environment where API version is exposed higher (say higher than 1.16) but encoder still fails to support this feature?

2. How the API version is actually exposed/decided by driver? On what other factors or combination it depends on? 

[Mark_L_Intel1]

Thank so much for the detailed testing,

Or we can say, we should support above 4th generation and above API 1.16.

In general, API version should depend on driver version and driver is limited by the hardware.

Anyway, for the question, "Which API version support RGB4 to H264 direct encoding?" Since there is no document about it, do you want to submit a request?

Mark

[Ramashankar]

Hi Mark,

Anyway, for the question, "Which API version support RGB4 to H264 direct encoding?" Since there is no document about it, do you want to submit a request?

Yes, it would be better if I get this confirmation as I need to exploit this feature in our product (against the issue mentioned in note section in first post).

Further, can you provide your input on my 1st qn: Is it safe to rely on this API version for using new RGB4->H264 direct encode feature? I mean can there be any environment where API version is exposed higher (say higher than 1.16) but encoder still fails to support this feature?

[Mark_L_Intel1]

Hi Ramashankar,

I have created a ticket and I will keep you updated.

Mark

[Artem_S_Intel]

RGB input supported only starting from Skylake(6th generation Core), previous platforms doesn't support.

To manage in Application you can use MFXVideoENCODE_Query interface, or MFXVideoCORE_QueryPlatform to detected platform before creating pipeline, detect platform and run VPP+Encode or Encode only.

[Artem_S_Intel]

Ramashankar wrote:

Hi Mark,

Anyway, for the question, "Which API version support RGB4 to H264 direct encoding?" Since there is no document about it, do you want to submit a request?

Yes, it would be better if I get this confirmation as I need to exploit this feature in our product (against the issue mentioned in note section in first post).

Further, can you provide your input on my 1st qn: Is it safe to rely on this API version for using new RGB4->H264 direct encode feature? I mean can there be any environment where API version is exposed higher (say higher than 1.16) but encoder still fails to support this feature?

It is not right to rely on API version for feature support, one or another feature can be supported on some platforms or not, API version can guarantee that for particular API not defined in this version is not supported, but not vice versa.

In case of RGB it is even not new API, but feature support.

[Ramashankar]

Hi Artem,

RGB input supported only starting from Skylake(6th generation Core), previous platforms doesn't support.

As I am able to use RGB input successfully on Haswell(4th gen core) also (refer point 2 and 3 in third post above), so I am unable to understand your above conclusion. Could you please clarify it more?

Do you mean to say that on Skylake onwards platform this is definitely supported, but on previous platform it may be or may not be supported?

[yu__leon]

hi all,

does  software encoder  also support  RGB4 to H264 encode?   Currently I am converting RGB to NV12  before passing the data to encoder. 

[Artem_S_Intel]

Ramashankar wrote:

Do you mean to say that on Skylake onwards platform this is definitely supported, but on previous platform it may be or may not be supported?

Yes, on earlier platforms it can be supported or not, SKL has support always

[Artem_S_Intel]

yu, leon wrote:

hi all,

does  software encoder  also support  RGB4 to H264 encode?   Currently I am converting RGB to NV12  before passing the data to encoder. 

Yes, it is supported.

[yu__leon]

the sample_encode readme file specifically  says

-rgb4 Signals that the input stream is in RGB4, works for JPEG encode only

that means  this is just a limitation of sample_encode program.  

is RGB3 supported?  my uncompressed frame is actually RGB 3-color, 8 bits per color.  I hope I don't need to convert it to RGBA for the encoder if possible.   thank you.

[Artem_S_Intel]

Yes, looks like sample missing right message for it.

RGB3 is not supported, only RGBA format.

[Ramashankar]

Hi Artem,

Yes, on earlier platforms it can be supported or not, SKL has support always

Understood. Thanks for the confirmation.

[yu__leon]

Artem S. (Intel) wrote:

Yes, looks like sample missing right message for it.

RGB3 is not supported, only RGBA format.

thank you

[Ramashankar]

Hi Yu,

-rgb4 Signals that the input stream is in RGB4, works for JPEG encode only

I think it is just issue of "Readme not updated". If you even check the code of sample encode exe, RGB4 is supported there for h264 encoding also.

[Mark_L_Intel1]

Hi Ramashankar,

Does Artem's answer satisfy your question? If yes, I can close this case.

Mark

[Ramashankar]

Hi Mark,

Yes you can close this case now. Thanks for your support.

Just another request: if you can help in finding the solution of my original issue (https://software.intel.com/en-us/forums/intel-media-sdk/topic/657754) then it would be a great help as I am still having this issue in one of my product (for all platform below Skylake). 

Thanks.

[iroeville]

Interesting to see here that encoding direct from RGB4 (without having to use preceding VPP to convert RGB4 to NV12) is supported from Skylake platform and later.

Is there increased processing speed in encoding directly from RGB4 ?

I currently use a preceding VPP stage for color conversion - so VPP input set to RGB4, VPP output set to NV12 and ENCODE input  set to NV12.

But I sometimes also use the VPP stage for other things like deinterlacing, so to encode from RGB4 I guess the VPP input and VPP output and ENCODE input would all be set to RGB4. Are there any VPP operations that would stop working because this would be forcing all of the VPP operations to work with RGB4 (instead of NV12) ?

[sg]

Good to see that.  Is the other direction also available (e.g. nv12/yuvj420p  to RGB) ?