Color Conversion RGB4 to NV12

[iroeville]

Our application does H.264 encoding of RGB4 (RGB32) frames. It does this by having a VPP stage before encoding that converts RGB4 to the NV12 that is required as the input to the encoding stage. The VPP stage also does deinterlacing if the input video is interlaced.

It has always worked fine, but now we have a report from one particular client that it fails on both of their computers which are:

1. micro PC/NUC manufactured by Intel "Intel Skull Canyon".

Graphics Devices:
        Name                                         Version             State
        Intel(R) Iris(R) Pro Graphics 580            23.20.16.4944       Active

System info:
        CPU:    Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz
        OS:     Microsoft Windows 10 Pro
        Arch:   64-bit
 

2. Intel NUC

Graphics Devices:
        Name                                         Version             State
        Intel(R) Iris(R) Graphics 540                23.20.16.4982       08
        Microsoft Basic Display Adapter              10.0.16299.15       Active

System info:
        CPU:    Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
        CPU:    Intel(R) Core(TM) i5-6260U CPU @ 1.80GHz
        OS:     Microsoft Windows 10 Pro
        Arch:   64-bit

If it is fed with NV12 instead of RGB4 so that it does not have to do color conversion then encoding succeeds on both computers.

The deinterlacing works OK, so it seems to be the color conversion that is causing it to fail.

Are there some platforms that are not capable of doing RGB4 to NV12 conversion ?

[Tamer_Assad]

Hi Iroeville,

Could you provide the error code?

What are the failed frame sizes?

What type of memory do you use within the VPP?

Best regards,

Tamer

[iroeville]

Error code is -12 which happens at the beginning of encoding.

Frame size 1440x1080

Memory Type:
m_mfxParamsVPP IOPattern is 17
m_mfxParams IOPattern is 2

[iroeville]

These seem to be Intel manufactured computers.

So please can someone from Intel actually try RGB4 to NV12 on these processors to confirm if there is a problem with the hardware drivers on these processors/platforms ?

[iroeville]

I have upgraded to the latest media SDK 2018 R1 and the same problem still exists on the client's Intel-manufactured computers (details above).

When RGB4 to NV12 color conversion VPP happens before the H.264 or MPEG-2 encoder, SyncOperation() returns either MFX_ERR_ABORTED or MFX_ERR_UNDEFINED_BEHAVIOR.

When the encoder is fed NV12 (so RGB4 to NV12 VPP is not triggered) then the encoding works fine.

The existing code has always worked OK on countless client machines, so there can't be anything wrong with the setup of the parameters.

The above 2 client machines are the only instances of this problem. I suspect there is a problem with the hardware drivers on these machines. The client tells me that that the Intel utility says that they already have the most recent version of the graphics drivers installed.

Can someone at Intel PLEASE look into this. I really think this is a problem with the Intel drivers.

[Dmitry_E_Intel]

 Hi Iroeville,

RGB->NV12 CSC has to work on there systems.

1) If would be great if we can reproduce the issue locally. Is it possible to share a reproducer? it might be created from sample_encode with minor changes to do color conversion or perhaps you can share simplified version of your application.

 2) Can you share msdk traces? 

3) Does the error occur randomly or on some particular frame (e.g. the 1st frame)? If it's random, how often does it happen?

Regards,

Dmitry

[Mark_L_Intel1]

Hi Iroeville,

I have a SkyLake machine and I tried our sample_vpp on Win10 with MSDK 2018R2, CSC works ok. Could you try this on your side?

Here are the details.

1. Extract ES file to RGB4 and confirm result:
   # ffmpeg -i big_buck_bunny_1080p_100.264 -vf format=rgba -f rawvideo big_buck_bunny_1080p_100.rgb4
   # ffplay -i big_buck_bunny_1080p_100.rgb4 -f rawvideo -pix_fmt rgba -s 1920x1080
    
2. Run sample_vpp and confirm the result:
   # sample_vpp.exe -sw 1920 -sh 1080 -scc rgb4 -dw 854 -dh 480 -dcc nv12 -iopattern d3d_to_d3d -i big_buck_bunny_1080p_100.rgb4 -o out.yuv
   # ffplay -i out.yuv -f rawvideo -pix_fmt nv12 -s 854x480

​Hope this helps.

Mark

[iroeville]

Thanks for your comments.

The client has run tracer.exe and provided this trace attached. It seems to fail on the first SyncOperation. I hope it helps.