I have a problem to use QueryIOSurf, please help

[MEI-YUN_C_]

Dear Sir:

I use MSDK 3.0 beta5 to encode and pass the encoded data to sink through wifi display, but the data encoded have some problem.

When I set vppParam.vpp.In.ForCC = MFX_MAKEFOURCC('R','G','B','4') and encParams.mfx.FrameInfo.FourCC = MFX_FOURCC_NV12;

the encoded data have some prolem:

when open a pdf file in source NB to encode and set page size to 50% to let word in pdf file seems very small,

the small word in the encoded data will have strange red/yellow color, witch originally is pure black small word.

To solve the problem, I set vppParam.vpp.In.ForCC = MFX_MAKEFOURCC('R','G','B','3') instead of ('R','G','B','4')

the stange color in small word is resolved !!!

But in my 4 notebooks(core i5), 3 notebooks with this solution work perfect,

but 1 notebook have problem when I set sts = m_pmfxVPP->QueryIOSurf(&m_mfxVppParams, VppRequest) in AllocFrames,

it will return MFX_ERR_INVALID_VIDEO_PARAM and then fail to encode.

Could you suggest what parameter I need to set when I use RGB3??

Thank you very much !!

[Petter_L_Intel]

Hi,

Could you plase share some more information regarding your encoder configuraiton so that we can get a better understanding of your workload.

Color converison using VPP has been supported via Media SDK for quite awhile now. But as the HW and graphics driver has evolved it is possible that the color conversion algorithms have also improved, thus the differnt behavior you are observing.

Please also share the exact machine config and graphics driver version for which you see undesirable artifacts vs. for which config you do not see it.

Please observe the following:
- Media SDK 3.0 beta 5 is not an official verison of Media SDK. I suggest you revisit your tests using either Media SDK 2012 R3 or the very recent dia SDK 2013 release.

- Color format "RGB3" usage via VPP is deprected and should NOT be used. Actually, I'm very surprised you got it to work at all.

Regards,
Petter

[MEI-YUN_C_]

Dear Sir
 
 Thank you for your reply.
 
 Since you mention that RGB3 is not be used, I list the parameter for VPP and encoder for RGB4.
 
 Could you help me point out the error for distortion of small words:
 
mfxStatus RTKWidiVideoEncoder::Init()
{
    mfxStatus sts;
    mfxVideoParam encParams;
    mfxVideoParam vppParams;
    mfxExtCodingOption encExtCO;

    sts = CEncodingPipeline::Init();
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    MSDK_ZERO_MEMORY(encParams);
    MSDK_ZERO_MEMORY(vppParams);
    init_ext_buffer (encExtCO);

    encParams.mfx.CodecId            = MFX_CODEC_AVC;
    encParams.mfx.GopPicSize         = 60;
    encParams.mfx.GopRefDist         = 1; // IP-only for low-delay applications
    encParams.mfx.RateControlMethod  = MFX_RATECONTROL_VBR;
    encParams.mfx.TargetKbps         = 800; // in Kbps
    encParams.mfx.MaxKbps            = 2 * encParams.mfx.TargetKbps; // VBR: common practice is to cap at doubled average
    encParams.mfx.InitialDelayInKB   = 500 * encParams.mfx.TargetKbps / 8000; // CBR/VBR: 0.5s initial CPB delay
//  encParams.mfx.Accuracy           = 500; // AVBR: 50% rate fluctuation after convergence
//  encParams.mfx.Convergence        = 1; // AVBR: 100 frames to converge
    encParams.mfx.BufferSizeInKB     = 1000; // should be more than enough
    encParams.mfx.NumRefFrame        = 1;
  encParams.mfx.CodecProfile = MFX_PROFILE_AVC_BASELINE;

    ConvertFrameRate(60.,
   &encParams.mfx.FrameInfo.FrameRateExtN,
   &encParams.mfx.FrameInfo.FrameRateExtD);
    encParams.mfx.FrameInfo.FourCC       = MFX_FOURCC_NV12;
    encParams.mfx.FrameInfo.Width        = 352;
    encParams.mfx.FrameInfo.Height       = 288;
    encParams.mfx.FrameInfo.CropX        = 0;
    encParams.mfx.FrameInfo.CropY        = 0;
    encParams.mfx.FrameInfo.CropW        = encParams.mfx.FrameInfo.Width;
    encParams.mfx.FrameInfo.CropH        = encParams.mfx.FrameInfo.Height;
    encParams.mfx.FrameInfo.AspectRatioW = 1;
    encParams.mfx.FrameInfo.AspectRatioH = 1; // PC display always uses square samples
    encParams.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    encParams.mfx.FrameInfo.PicStruct    = MFX_PICSTRUCT_PROGRESSIVE;

    switch (GetImplementation())
    {
        case MFX_IMPL_HARDWARE:

            m_mfxEncParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_QUALITY;

            encExtCO.RateDistortionOpt = MFX_CODINGOPTION_ON;

            break;

        case MFX_IMPL_SOFTWARE:

            m_mfxEncParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;

            encExtCO.RateDistortionOpt = MFX_CODINGOPTION_OFF;

            break;

        default:

            m_mfxEncParams.mfx.TargetUsage = MFX_TARGETUSAGE_BALANCED;
            break;
    }

 // Try to encode fast currently now, please change it later.
 m_mfxEncParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;
 encExtCO.RateDistortionOpt = MFX_CODINGOPTION_OFF;
 
    encExtCO.MaxDecFrameBuffering = 1; // allow only one frame in the DPB to reduce decode latency

    // VPP output: same as Encoder input
    memcpy(&vppParams.vpp.Out, &encParams.mfx.FrameInfo, sizeof(mfxFrameInfo));

    // VPP input: same as VPP output except format
    memcpy(&vppParams.vpp.In, &vppParams.vpp.Out, sizeof(mfxFrameInfo));
    vppParams.vpp.In.FourCC = MFX_FOURCC_RGB4;

    sts = InitMfxEncParams(&encParams, &encExtCO);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    sts = InitMfxVppParams(&vppParams);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

  sts = SetAsyncDepth(1);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    return sts;
}

[Petter_L_Intel]

Hi,

Thanks for sharing some more details.

Unfortunately without having access to the content input to encoder I cannot draw any conclusions regarding the encoding artifacts you encountered.
Could you please share an example of RGB4 raw data which showcases the issue? Please also share the encoded stream showcasing the issue.

Also, using RGB4 CC, from your first post it seems the the artifacts is only observed on some of your machines. Is that correct? If so, please provide details about the machine config, driver version and if HW or SW encode was used for the cases you have tested.

Regards,
Petter 

[MEI-YUN_C_]

Dear Sir:

Thank you for your reply.

I attach 2 files in question.7z

1.driver.raw -> please use XnView to open, and for reading option, please select "interleave" and "BGR" to open the file.
                     -> Mirror driver put data in (pVirMonitor->pVideoMemory).
                         But I think it's not RGB32, because I only get 24 bit outof 32 bit in (pVirMonitor->pVideoMemory),
                         and can use BGR mode in XnView to open it correctly.
          // I got one frame and file out to driver.raw
          if(frame_counter == 500)
    {    
        if (encWidth == width)
        {
            for(i=0; i < width * height; i++)
                OUTFILE_->write((const char *)pVirMonitor->pVideoMemory + i*4, 3);
        }
        else
        {
            int j=0;
            for (i = 0 ; i < height ; i++)
            {
                for(j=0; j<width ; j++)
                    OUTFILE_->write((const char *)pVirMonitor->pVideoMemory + i * width * 4 + j * 4, 3);
            }
        }
    }                      
2.demo_video_memory2.mpg -> For data encoded, I file out the data after mux audio and video.
                                                       You can use VLC to open it.
                                                       And you will see that  small words in driver.raw is correct, but has distortion in  demo_video_memory2.mpg.

And all my machines in RGB4 mode will have this distortion issue.

If I set RGB3 instead of RGB4 for convertion to NV12, 5 machines won't have this distortion, but the other 2 machines will QueryIOSurf fail.
                                                                      

[Petter_L_Intel]

Hi,

so you're seeing the exact same thing using RGB4 for all your machines, right?

I tested similar setup as yours by grabbing screen desktop frames as RGB32 then using Media SDK VPP to color convert to NV12 surface then encode to H.264 using similar bitrate as your setup. Desktop screen resolution (1680x1050) was encoded at native resolution (VPP was not used to scale the frame).

Using recent driver 15.28.2857 I do some slight artifacts (much less than for the encoded content you provided) when encoding screen content with very small font and text sections with text such as "llll". The effect is slightly more visible when encoding using lower quality settings such as MFX_TARGETUSAGE_BEST_SPEED and not using B-frame (GopRefDist =1).

I also explored scaling the frame to half the resolution using VPP but still do not see any additional effects. But as expected this results in quite ugly text.

In essence I do not think there is much that can be done to improve the behavior. Please note that the HW encoder and color conversion routines exposed via Media SDK was created and optimized for typical video content. Encoder can certainly be used to encode frames containing text but the results will not be optimal. I suggest trying various encoder configurations exploring if there is a setup that produces results closer to what you need.

Again, I do not encourage using RGB3 since it's a deprecated feature.

Regards,
Petter

[MEI-YUN_C_]

Dear Sir:

Thank you for your reply.

Yes, I saw the exact same thing using RGB4 for all my machines.

As for you mentioned "I suggest trying various encoder configurations", could you suggest some variables for me to try??

[Petter_L_Intel]

Please try some of the following:

- GOPRefDist > 1.   Implies encoding of B-frames, thus improved quality.
- TargetUsage = MFX_TARGETUSAGE_BEST_QUALITY
- profile = MFX_PROFILE_AVC_HIGH
- NumSlice = 1 may have a slight impact 
- You can also let Media SDK set BufferSizeInKB instead of setting it yourself. This will ensure enough buffer is allocated. A too small buffer may affect quality
- You can also try CQP rate control mode to explicty control frame quality.

Regards,
Petter 

[MEI-YUN_C_]

Dear Sir:

Thank you for your reply.

But the parameters you suggest don't work quiet well.

I change rate control mode from VBR to CQP even will cause serious mosaics.

Could you give me another suggest for RGB32 convert to NV12??

Or, now I use RGB24 convert to NV12, most of my notebooks can work very well (although you don't encourage using RGB3) but some will QueryIOSurf fail.

Could you suggest method to overcome QueryIOSurf fail so I can use RGB24 instead??

[Petter_L_Intel]

Hi,

I see no reason for CQP causing quality issues. Make sure you set appropriate QP values.

The VPP RGB32->NV12 conversion method cannot be modified. Alternatively you can explore third party color conversion routines or Intel Integrated Performance Primitives (IPP) library which has a wide range of color conversion routines including RGB32 and RGB24 to NV12. However, such approach naturally does not use the GPU.

VPP RGB24->NV12 color conversion method is deprectated. We do not recommend using it and do not plan to implement workarounds to support it.

Regards,
Petter