Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Software Reference 10 bit encoding.

RobinsonUK
New Contributor I
2,198 Views

With an AV1 encoder, I pass in a format in my encode_parameters.mfx.FrameInfo.FourCC as MFX_FOURCC_I010 (the software reference doesn't seem to support P010LE, which is what NVIDIA uses for 10 bit)

 

The function MFXVideoENCODE_Query succeeds.  It does not modify my parameters, i.e. the format is still MFX_FOURCC_I010.  MFXVideoENCODE_Init succeeds with these parameters.

 

When I fetch a frame to encode with using MFXMemory_GetSurfaceForEncode, the surface.surface->Info.FourCC format comes back as VUYY ( 0x56555959) which does not correspond to any known fourcc code.

 

Can anyone explain this discrepency?

 

The impl I'm using is: oneAPI VPL CPU Implementation v2.7 (Build: 131079)

0 Kudos
15 Replies
RobinsonUK
New Contributor I
2,190 Views

Even stranger, the returned surface has:

 

    BitDepthLuma 8

    BitDepthChroma 8

 

What can I conclude from this? 

 

Also tried AV1 with 10 bit MFX_FOURCC_I010 in hello_encode.  I don't grab any frames from disk (that part is commented out).  I just take the returned surface and feed it to the encoder.  It also gives 0x56555959 as the fourcc code for the surface, with an 8 bit luma and chroma.  Everything succeeds and I get output.

 

Can someone check this with the boffins?  I think it's a bug.  At the very least what is supported by the software reference isn't made clear (my code that enumerates formats suggests IYUV and I010 are supported, i.e. MFX_FOURCC_I420 and MFX_FOURCC_I010).

0 Kudos
ThasneemV_Intel
Moderator
2,159 Views

Hi


Thanks for posting in Intel Communities.


To assist you better, could you please share the following details.

1) Sample reproducer code.

2) The exact steps to reproduce the issue.

3) System infrastructure details.


Regards,

Thasneem Vazim


0 Kudos
RobinsonUK
New Contributor I
2,150 Views

This is my somewhat modified hello_encode.cpp:

 

 

//==============================================================================
// Copyright Intel Corporation
//
// SPDX-License-Identifier: MIT
//==============================================================================

///
/// A minimal oneAPI Video Processing Library (oneVPL) encode application,
/// using 2.x API with internal memory management
///
/// @file
#define _CRT_SECURE_NO_WARNINGS
#include "util.h"

#define TARGETKBPS                 20000
#define FRAMERATE                  30
#define OUTPUT_FILE                "out.av1"
#define BITSTREAM_BUFFER_SIZE      10000000
#define MAJOR_API_VERSION_REQUIRED 2
#define MINOR_API_VERSION_REQUIRED 2

void Usage(void) {
    printf("\n");
    printf("   Usage  :  hello-encode\n");
    printf("     -hw        use hardware implementation\n");
    printf("     -sw        use software implementation\n");
    printf("     -i input file name (raw frames)\n");
    printf("     -w input width\n");
    printf("     -h input height\n\n");
    printf("   Example:  hello-encode -i in.i420 -w 320 -h 240\n");
    printf("   To view:  ffplay %s\n\n", OUTPUT_FILE);
    printf(" * Encode raw frames to HEVC/H265 elementary stream in %s\n\n", OUTPUT_FILE);
    printf("   CPU native color format is I420/yuv420p.  GPU native color format is "
           "NV12\n");
    return;
}

int main(int argc, char *argv[]) {
    // Variables used for legacy and 2.x
    bool isDraining                = false;
    bool isStillGoing              = true;
    bool isFailed                  = false;
    FILE *sink                     = NULL;
    FILE *source                   = NULL;
    mfxBitstream bitstream         = {};
    mfxFrameSurface1 *encSurfaceIn = NULL;
    mfxSession session             = NULL;
    mfxSyncPoint syncp             = {};
    mfxU32 framenum                = 0;
    mfxU32 framenumc = 0;
    mfxStatus sts                  = MFX_ERR_NONE;
    mfxStatus sts_r                = MFX_ERR_NONE;
    Params cliParams               = {};
    mfxVideoParam encodeParams     = {};

    // variables used only in 2.x version
    mfxConfig cfg[3];
    mfxVariant cfgVal[3];
    mfxLoader loader = NULL;

    // Parse command line args to cliParams
    if (ParseArgsAndValidate(argc, argv, &cliParams, PARAMS_ENCODE) == false) {
        Usage();
        return 1; // return 1 as error code
    }

    source = fopen(cliParams.infileName, "rb");
    VERIFY(source, "Could not open input file");

    sink = fopen(OUTPUT_FILE, "wb");
    VERIFY(sink, "Could not create output file");

    // Initialize VPL session
    loader = MFXLoad();
    VERIFY(NULL != loader, "MFXLoad failed -- is implementation in path?");

    // Implementation used must be the type requested from command line
    cfg[0] = MFXCreateConfig(loader);
    VERIFY(NULL != cfg[0], "MFXCreateConfig failed")

    sts =
        MFXSetConfigFilterProperty(cfg[0], (mfxU8 *)"mfxImplDescription.Impl", cliParams.implValue);
    VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for Impl");

    // Implementation must provide an HEVC encoder
    cfg[1] = MFXCreateConfig(loader);
    VERIFY(NULL != cfg[1], "MFXCreateConfig failed")
    cfgVal[1].Type     = MFX_VARIANT_TYPE_U32;
    cfgVal[1].Data.U32 = MFX_CODEC_AV1;
    sts                = MFXSetConfigFilterProperty(
        cfg[1],
        (mfxU8 *)"mfxImplDescription.mfxEncoderDescription.encoder.CodecID",
        cfgVal[1]);
    VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for encoder CodecID");

    // Implementation used must provide API version 2.2 or newer
    cfg[2] = MFXCreateConfig(loader);
    VERIFY(NULL != cfg[2], "MFXCreateConfig failed")
    cfgVal[2].Type     = MFX_VARIANT_TYPE_U32;
    cfgVal[2].Data.U32 = VPLVERSION(MAJOR_API_VERSION_REQUIRED, MINOR_API_VERSION_REQUIRED);
    sts                = MFXSetConfigFilterProperty(cfg[2],
                                     (mfxU8 *)"mfxImplDescription.ApiVersion.Version",
                                     cfgVal[2]);
    VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for API version");

    sts = MFXCreateSession(loader, 0, &session);
    VERIFY(MFX_ERR_NONE == sts,
           "Cannot create session -- no implementations meet selection criteria");

    // Print info about implementation loaded
    ShowImplementationInfo(loader, 0);

    // Initialize encode parameters
    encodeParams.AsyncDepth = 4;
    encodeParams.mfx.EncodedOrder = 0;
    encodeParams.mfx.CodecId = MFX_CODEC_AV1;
    encodeParams.mfx.CodecProfile = MFX_PROFILE_AV1_MAIN;
    encodeParams.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    encodeParams.mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
    encodeParams.mfx.FrameInfo.FourCC = MFX_FOURCC_I010;
    encodeParams.mfx.FrameInfo.CropX = 0;
    encodeParams.mfx.FrameInfo.CropY = 0;
    encodeParams.mfx.FrameInfo.CropW = cliParams.srcWidth;
    encodeParams.mfx.FrameInfo.CropH = cliParams.srcHeight;
    encodeParams.mfx.FrameInfo.Width = cliParams.srcWidth;
    encodeParams.mfx.FrameInfo.Height = cliParams.srcHeight;
    encodeParams.mfx.GopRefDist = 4;
    encodeParams.mfx.GopPicSize = static_cast<mfxU16>(60);
    encodeParams.mfx.GopOptFlag = MFX_GOP_CLOSED;
    encodeParams.mfx.NumRefFrame = 3;
    encodeParams.mfx.NumSlice = 0;
    encodeParams.mfx.IdrInterval = 0;
    encodeParams.mfx.TargetKbps = TARGETKBPS;
    encodeParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;
    encodeParams.mfx.RateControlMethod = MFX_RATECONTROL_CBR;
    encodeParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY;
    encodeParams.mfx.FrameInfo.FrameRateExtN = 60;
    encodeParams.mfx.FrameInfo.FrameRateExtD = 1;


    // Initialize encoder
    sts = MFXVideoENCODE_Init(session, &encodeParams);
    VERIFY(MFX_ERR_NONE == sts, "Encode init failed");

    // Prepare output bitstream
    bitstream.MaxLength = BITSTREAM_BUFFER_SIZE;
    bitstream.Data      = (mfxU8 *)calloc(bitstream.MaxLength, sizeof(mfxU8));

    printf("Encoding %s -> %s\n", cliParams.infileName, OUTPUT_FILE);

    printf("Input colorspace: ");
    switch (encodeParams.mfx.FrameInfo.FourCC) {
        case MFX_FOURCC_I420: // CPU input
            printf("I420 (aka yuv420p)\n");
            break;
        case MFX_FOURCC_NV12: // GPU input
            printf("NV12\n");
            break;
        default:
            printf("Unsupported color format\n");
            isFailed = true;
            //goto end;
            break;
    }

    while (isStillGoing == true) {
        // Load a new frame if not draining
        if (isDraining == false) {
            sts = MFXMemory_GetSurfaceForEncode(session, &encSurfaceIn);
            VERIFY(MFX_ERR_NONE == sts, "Could not get encode surface");

          //  sts = ReadRawFrame_InternalMem(encSurfaceIn, source);
          //  if (sts != MFX_ERR_NONE)
           //     isDraining = true;
        }

        sts = MFXVideoENCODE_EncodeFrameAsync(session,
                                              NULL,
                                              (isDraining == true) ? NULL : encSurfaceIn,
                                              &bitstream,
                                              &syncp);
        framenumc++;
        if (!isDraining) {
            sts_r = encSurfaceIn->FrameInterface->Release(encSurfaceIn);
            VERIFY(MFX_ERR_NONE == sts_r, "mfxFrameSurfaceInterface->Release failed");
        }
        switch (sts) {
            case MFX_ERR_NONE:
                // MFX_ERR_NONE and syncp indicate output is available
                if (syncp) {
                    // Encode output is not available on CPU until sync operation
                    // completes
                    sts = MFXVideoCORE_SyncOperation(session, syncp, WAIT_100_MILLISECONDS);
                    VERIFY(MFX_ERR_NONE == sts, "MFXVideoCORE_SyncOperation error");

                    WriteEncodedStream(bitstream, sink);
                    framenum++;
                    printf("frame: %d, %d\n", framenum, framenumc);
                }
                break;
            case MFX_ERR_NOT_ENOUGH_BUFFER:
                // This example deliberatly uses a large output buffer with immediate
                // write to disk for simplicity. Handle when frame size exceeds
                // available buffer here
                break;
            case MFX_ERR_MORE_DATA:
                // The function requires more data to generate any output
                if (isDraining == true)
                    isStillGoing = false;
                break;
            case MFX_ERR_DEVICE_LOST:
                // For non-CPU implementations,
                // Cleanup if device is lost
                break;
            case MFX_WRN_DEVICE_BUSY:
                // For non-CPU implementations,
                // Wait a few milliseconds then try again
                break;
            default:
                printf("unknown status %d\n", sts);
                isStillGoing = false;
                break;
        }
    }

end:
    printf("Encoded %d frames\n", framenum);

    // Clean up resources - It is recommended to close components first, before
    // releasing allocated surfaces, since some surfaces may still be locked by
    // internal resources.
    if (source)
        fclose(source);

    if (sink)
        fclose(sink);

    MFXVideoENCODE_Close(session);
    MFXClose(session);

    if (bitstream.Data)
        free(bitstream.Data);

    if (loader)
        MFXUnload(loader);

    if (isFailed) {
        return -1;
    }
    else {
        return 0;
    }
}

 

As you can see, I've modified it to filter for AV1 encoders:

 

 

cfg[1] = MFXCreateConfig(loader);
VERIFY(NULL != cfg[1], "MFXCreateConfig failed")
cfgVal[1].Type     = MFX_VARIANT_TYPE_U32;
cfgVal[1].Data.U32 = MFX_CODEC_AV1;
sts                = MFXSetConfigFilterProperty(
    cfg[1],
    (mfxU8 *)"mfxImplDescription.mfxEncoderDescription.encoder.CodecID",
    cfgVal[1]);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for encoder CodecID");

 

I am using -sw on the commandline, to get the software impl, with -width 1280 and -height 720.  The software impl reports:

 


Implementation details:
ApiVersion: 2.7
Implementation type: SW
AccelerationMode via: NA
Path: C:\Program Files (x86)\Common Files\Intel\Shared Libraries\intel64\bin\libvplswref64.dll

I have modified the code to prevent it bombing out in default, as below:

 

 

switch (encodeParams.mfx.FrameInfo.FourCC) {
    case MFX_FOURCC_I420: // CPU input
        printf("I420 (aka yuv420p)\n");
        break;
    case MFX_FOURCC_NV12: // GPU input
        printf("NV12\n");
        break;
    default:
        printf("Unsupported color format\n");
        isFailed = true;
        //goto end;
        break;
}

 

 

Although it claims MFX_FOURCC_I010 is an unsupported colour format in the sample, this format is present when you enumerate implementations and look at the description.  MFXVideoENCODE_Init succeeds.  However,  MFXMemory_GetSurfaceForEncode returns a surface with an unknown format.

 

I have modified the code to not read a frame from the disk, just to get the surface for encoding and send it into the encoder:

 

 

if (isDraining == false) {
sts = MFXMemory_GetSurfaceForEncode(session, &encSurfaceIn);
VERIFY(MFX_ERR_NONE == sts, "Could not get encode surface");

// sts = ReadRawFrame_InternalMem(encSurfaceIn, source);
// if (sts != MFX_ERR_NONE)
// isDraining = true;
}

 

 

If you put MFXVideoENCODE_Query before MFXVideoENCODE_Init, MFXVideoENCODE_Query succeeds.

 

My setup is an AMD 3900X with 128GB RAM, Windows 11.  I have an NVIDIA 3060 and an Intel Arc 380 plugged into it right now.  However the above was performed with an NVIDIA 4080 plugged in and no Arc, i.e. purely using the software reference impl.  I have up to date Windows and all drivers. 

0 Kudos
ThasneemV_Intel
Moderator
2,027 Views

Hi,


Thank you for sharing the code. To debug the issue further, could you please provide the input video file and the exact commands used to reproduce the issue.


Regards,

Thasneem vazim


0 Kudos
ThasneemV_Intel
Moderator
1,954 Views

Hi,


We have not heard back from you. Could you please give us an update?


Regards,

Thasneem Vazim


0 Kudos
RobinsonUK
New Contributor I
1,942 Views

The input command would have been:

 

-sw -i "1588.raw" -w 1280 -h 720

 

I have a folder of single P010LE files extracted from an HD movie.  I use these frames to test my code.  I passed a single one to hello_encode.  I have attached 1588.raw to this comment.  It has also been verified on rawpixels, as shown in the image below.

 

raw_pixels.jpg

Please note that I modified the code to not bother reading or writing frames, as you can see from my previous comments.  I am simply fetching a surface:

MFXMemory_GetSurfaceForEncode

and encoding whatever it gives me:

MFXVideoENCODE_EncodeFrameAsync

,  without copying any data to it.  So in a way with the code above it doesn't matter what the input is and nor should it.  The issue is the image format of the returned frame.

 

If anything I've written here is unclear or you need more from me, please don't hesitate to ask.

 

Thank you.

 

 

 

0 Kudos
ThasneemV_Intel
Moderator
1,881 Views

Hi,


Thank you for sharing the requested details. We tried compiling your code, but it seems like your code needs some dependent header files. Could you please share the necessary files so that we could triage your issue further?


Regards,

Thasneem Vazim


0 Kudos
RobinsonUK
New Contributor I
1,847 Views

Thanks Thasneem.  I have attached util.hpp.  It's the util.hpp from Intel's hello_encode project.

0 Kudos
ThasneemV_Intel
Moderator
1,753 Views

Hi,


Thank you for sharing the details. We are checking on this internally. We will get back to you soon with an update.


Regards,

Thasneem Vazim



0 Kudos
ThasneemV_Intel
Moderator
1,657 Views

Hi,

 

When we tried compiling your modified hello encode sample, we are getting compilation error as shown below:

ThasneemV_Intel_0-1701183818483.jpeg

Please find the resources MFXSetConfigFilterProperty and mfxImplDescription for more details.

Could you assist us in providing the specific code you compiled so that we could triage your issue further?

 

Regards,

Thasneem Vazim

 

0 Kudos
RobinsonUK
New Contributor I
1,641 Views

Ah,  I think I gave the wrong util.hpp file.  I've attached both source files as compiled by me a moment ago without error, in Visual Studio 2022.  I have libvplswref64.dll in the output directory.  I am linking with vpl.lib.  The The issue remains, i.e. specify 10 bit AV1, get back a weird FOURCC code in the surface returned by MFXMemory_GetSurfaceForEncode.

 

I hope your people are able to compile this.

 

 

 

 

0 Kudos
ThasneemV_Intel
Moderator
1,575 Views

Hi,


We are checking on this internally. We will get back to you soon with an update.


Regards,

Thasneem Vazim


0 Kudos
ThasneemV_Intel
Moderator
1,355 Views

Hi,

 

We regret to inform you that the application is using Implementation of VPL CPU Runtime(SW) which is no longer active. The core project and GPU runtime continues to be actively maintained. For CPU video processing the recommended solution is to use software codecs that are part of media frameworks such as FFMpeg and GStreamer. Intel® provides plugins for frameworks to support this usage.

 

Reference: https://github.com/oneapi-src/oneVPL-cpu

More information can be found in the link below: https://www.intel.com/content/www/us/en/developer/tools/vpl/overview.html#gs.1vrs9s

 

Kindly confirm if this information and references resolves your query and do let us know if we can stop monitoring this thread.

 

Regards,

Thasneem Vazim

 

 

0 Kudos
RobinsonUK
New Contributor I
1,300 Views

 

OK, we ditched the software encoding.  It was too slow anyway.  Thanks.  You can close this out.

0 Kudos
Pamela_H_Intel
Moderator
919 Views
0 Kudos
Reply