- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
With an AV1 encoder, I pass in a format in my encode_parameters.mfx.FrameInfo.FourCC as MFX_FOURCC_I010 (the software reference doesn't seem to support P010LE, which is what NVIDIA uses for 10 bit)
The function MFXVideoENCODE_Query succeeds. It does not modify my parameters, i.e. the format is still MFX_FOURCC_I010. MFXVideoENCODE_Init succeeds with these parameters.
When I fetch a frame to encode with using MFXMemory_GetSurfaceForEncode, the surface.surface->Info.FourCC format comes back as VUYY ( 0x56555959) which does not correspond to any known fourcc code.
Can anyone explain this discrepency?
The impl I'm using is: oneAPI VPL CPU Implementation v2.7 (Build: 131079)
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Even stranger, the returned surface has:
BitDepthLuma 8
BitDepthChroma 8
What can I conclude from this?
Also tried AV1 with 10 bit MFX_FOURCC_I010 in hello_encode. I don't grab any frames from disk (that part is commented out). I just take the returned surface and feed it to the encoder. It also gives 0x56555959 as the fourcc code for the surface, with an 8 bit luma and chroma. Everything succeeds and I get output.
Can someone check this with the boffins? I think it's a bug. At the very least what is supported by the software reference isn't made clear (my code that enumerates formats suggests IYUV and I010 are supported, i.e. MFX_FOURCC_I420 and MFX_FOURCC_I010).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
Thanks for posting in Intel Communities.
To assist you better, could you please share the following details.
1) Sample reproducer code.
2) The exact steps to reproduce the issue.
3) System infrastructure details.
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is my somewhat modified hello_encode.cpp:
//==============================================================================
// Copyright Intel Corporation
//
// SPDX-License-Identifier: MIT
//==============================================================================
///
/// A minimal oneAPI Video Processing Library (oneVPL) encode application,
/// using 2.x API with internal memory management
///
/// @file
#define _CRT_SECURE_NO_WARNINGS
#include "util.h"
#define TARGETKBPS 20000
#define FRAMERATE 30
#define OUTPUT_FILE "out.av1"
#define BITSTREAM_BUFFER_SIZE 10000000
#define MAJOR_API_VERSION_REQUIRED 2
#define MINOR_API_VERSION_REQUIRED 2
void Usage(void) {
printf("\n");
printf(" Usage : hello-encode\n");
printf(" -hw use hardware implementation\n");
printf(" -sw use software implementation\n");
printf(" -i input file name (raw frames)\n");
printf(" -w input width\n");
printf(" -h input height\n\n");
printf(" Example: hello-encode -i in.i420 -w 320 -h 240\n");
printf(" To view: ffplay %s\n\n", OUTPUT_FILE);
printf(" * Encode raw frames to HEVC/H265 elementary stream in %s\n\n", OUTPUT_FILE);
printf(" CPU native color format is I420/yuv420p. GPU native color format is "
"NV12\n");
return;
}
int main(int argc, char *argv[]) {
// Variables used for legacy and 2.x
bool isDraining = false;
bool isStillGoing = true;
bool isFailed = false;
FILE *sink = NULL;
FILE *source = NULL;
mfxBitstream bitstream = {};
mfxFrameSurface1 *encSurfaceIn = NULL;
mfxSession session = NULL;
mfxSyncPoint syncp = {};
mfxU32 framenum = 0;
mfxU32 framenumc = 0;
mfxStatus sts = MFX_ERR_NONE;
mfxStatus sts_r = MFX_ERR_NONE;
Params cliParams = {};
mfxVideoParam encodeParams = {};
// variables used only in 2.x version
mfxConfig cfg[3];
mfxVariant cfgVal[3];
mfxLoader loader = NULL;
// Parse command line args to cliParams
if (ParseArgsAndValidate(argc, argv, &cliParams, PARAMS_ENCODE) == false) {
Usage();
return 1; // return 1 as error code
}
source = fopen(cliParams.infileName, "rb");
VERIFY(source, "Could not open input file");
sink = fopen(OUTPUT_FILE, "wb");
VERIFY(sink, "Could not create output file");
// Initialize VPL session
loader = MFXLoad();
VERIFY(NULL != loader, "MFXLoad failed -- is implementation in path?");
// Implementation used must be the type requested from command line
cfg[0] = MFXCreateConfig(loader);
VERIFY(NULL != cfg[0], "MFXCreateConfig failed")
sts =
MFXSetConfigFilterProperty(cfg[0], (mfxU8 *)"mfxImplDescription.Impl", cliParams.implValue);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for Impl");
// Implementation must provide an HEVC encoder
cfg[1] = MFXCreateConfig(loader);
VERIFY(NULL != cfg[1], "MFXCreateConfig failed")
cfgVal[1].Type = MFX_VARIANT_TYPE_U32;
cfgVal[1].Data.U32 = MFX_CODEC_AV1;
sts = MFXSetConfigFilterProperty(
cfg[1],
(mfxU8 *)"mfxImplDescription.mfxEncoderDescription.encoder.CodecID",
cfgVal[1]);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for encoder CodecID");
// Implementation used must provide API version 2.2 or newer
cfg[2] = MFXCreateConfig(loader);
VERIFY(NULL != cfg[2], "MFXCreateConfig failed")
cfgVal[2].Type = MFX_VARIANT_TYPE_U32;
cfgVal[2].Data.U32 = VPLVERSION(MAJOR_API_VERSION_REQUIRED, MINOR_API_VERSION_REQUIRED);
sts = MFXSetConfigFilterProperty(cfg[2],
(mfxU8 *)"mfxImplDescription.ApiVersion.Version",
cfgVal[2]);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for API version");
sts = MFXCreateSession(loader, 0, &session);
VERIFY(MFX_ERR_NONE == sts,
"Cannot create session -- no implementations meet selection criteria");
// Print info about implementation loaded
ShowImplementationInfo(loader, 0);
// Initialize encode parameters
encodeParams.AsyncDepth = 4;
encodeParams.mfx.EncodedOrder = 0;
encodeParams.mfx.CodecId = MFX_CODEC_AV1;
encodeParams.mfx.CodecProfile = MFX_PROFILE_AV1_MAIN;
encodeParams.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
encodeParams.mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
encodeParams.mfx.FrameInfo.FourCC = MFX_FOURCC_I010;
encodeParams.mfx.FrameInfo.CropX = 0;
encodeParams.mfx.FrameInfo.CropY = 0;
encodeParams.mfx.FrameInfo.CropW = cliParams.srcWidth;
encodeParams.mfx.FrameInfo.CropH = cliParams.srcHeight;
encodeParams.mfx.FrameInfo.Width = cliParams.srcWidth;
encodeParams.mfx.FrameInfo.Height = cliParams.srcHeight;
encodeParams.mfx.GopRefDist = 4;
encodeParams.mfx.GopPicSize = static_cast<mfxU16>(60);
encodeParams.mfx.GopOptFlag = MFX_GOP_CLOSED;
encodeParams.mfx.NumRefFrame = 3;
encodeParams.mfx.NumSlice = 0;
encodeParams.mfx.IdrInterval = 0;
encodeParams.mfx.TargetKbps = TARGETKBPS;
encodeParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;
encodeParams.mfx.RateControlMethod = MFX_RATECONTROL_CBR;
encodeParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY;
encodeParams.mfx.FrameInfo.FrameRateExtN = 60;
encodeParams.mfx.FrameInfo.FrameRateExtD = 1;
// Initialize encoder
sts = MFXVideoENCODE_Init(session, &encodeParams);
VERIFY(MFX_ERR_NONE == sts, "Encode init failed");
// Prepare output bitstream
bitstream.MaxLength = BITSTREAM_BUFFER_SIZE;
bitstream.Data = (mfxU8 *)calloc(bitstream.MaxLength, sizeof(mfxU8));
printf("Encoding %s -> %s\n", cliParams.infileName, OUTPUT_FILE);
printf("Input colorspace: ");
switch (encodeParams.mfx.FrameInfo.FourCC) {
case MFX_FOURCC_I420: // CPU input
printf("I420 (aka yuv420p)\n");
break;
case MFX_FOURCC_NV12: // GPU input
printf("NV12\n");
break;
default:
printf("Unsupported color format\n");
isFailed = true;
//goto end;
break;
}
while (isStillGoing == true) {
// Load a new frame if not draining
if (isDraining == false) {
sts = MFXMemory_GetSurfaceForEncode(session, &encSurfaceIn);
VERIFY(MFX_ERR_NONE == sts, "Could not get encode surface");
// sts = ReadRawFrame_InternalMem(encSurfaceIn, source);
// if (sts != MFX_ERR_NONE)
// isDraining = true;
}
sts = MFXVideoENCODE_EncodeFrameAsync(session,
NULL,
(isDraining == true) ? NULL : encSurfaceIn,
&bitstream,
&syncp);
framenumc++;
if (!isDraining) {
sts_r = encSurfaceIn->FrameInterface->Release(encSurfaceIn);
VERIFY(MFX_ERR_NONE == sts_r, "mfxFrameSurfaceInterface->Release failed");
}
switch (sts) {
case MFX_ERR_NONE:
// MFX_ERR_NONE and syncp indicate output is available
if (syncp) {
// Encode output is not available on CPU until sync operation
// completes
sts = MFXVideoCORE_SyncOperation(session, syncp, WAIT_100_MILLISECONDS);
VERIFY(MFX_ERR_NONE == sts, "MFXVideoCORE_SyncOperation error");
WriteEncodedStream(bitstream, sink);
framenum++;
printf("frame: %d, %d\n", framenum, framenumc);
}
break;
case MFX_ERR_NOT_ENOUGH_BUFFER:
// This example deliberatly uses a large output buffer with immediate
// write to disk for simplicity. Handle when frame size exceeds
// available buffer here
break;
case MFX_ERR_MORE_DATA:
// The function requires more data to generate any output
if (isDraining == true)
isStillGoing = false;
break;
case MFX_ERR_DEVICE_LOST:
// For non-CPU implementations,
// Cleanup if device is lost
break;
case MFX_WRN_DEVICE_BUSY:
// For non-CPU implementations,
// Wait a few milliseconds then try again
break;
default:
printf("unknown status %d\n", sts);
isStillGoing = false;
break;
}
}
end:
printf("Encoded %d frames\n", framenum);
// Clean up resources - It is recommended to close components first, before
// releasing allocated surfaces, since some surfaces may still be locked by
// internal resources.
if (source)
fclose(source);
if (sink)
fclose(sink);
MFXVideoENCODE_Close(session);
MFXClose(session);
if (bitstream.Data)
free(bitstream.Data);
if (loader)
MFXUnload(loader);
if (isFailed) {
return -1;
}
else {
return 0;
}
}
As you can see, I've modified it to filter for AV1 encoders:
cfg[1] = MFXCreateConfig(loader);
VERIFY(NULL != cfg[1], "MFXCreateConfig failed")
cfgVal[1].Type = MFX_VARIANT_TYPE_U32;
cfgVal[1].Data.U32 = MFX_CODEC_AV1;
sts = MFXSetConfigFilterProperty(
cfg[1],
(mfxU8 *)"mfxImplDescription.mfxEncoderDescription.encoder.CodecID",
cfgVal[1]);
VERIFY(MFX_ERR_NONE == sts, "MFXSetConfigFilterProperty failed for encoder CodecID");
I am using -sw on the commandline, to get the software impl, with -width 1280 and -height 720. The software impl reports:
Implementation details:
ApiVersion: 2.7
Implementation type: SW
AccelerationMode via: NA
Path: C:\Program Files (x86)\Common Files\Intel\Shared Libraries\intel64\bin\libvplswref64.dll
I have modified the code to prevent it bombing out in default, as below:
switch (encodeParams.mfx.FrameInfo.FourCC) {
case MFX_FOURCC_I420: // CPU input
printf("I420 (aka yuv420p)\n");
break;
case MFX_FOURCC_NV12: // GPU input
printf("NV12\n");
break;
default:
printf("Unsupported color format\n");
isFailed = true;
//goto end;
break;
}
Although it claims MFX_FOURCC_I010 is an unsupported colour format in the sample, this format is present when you enumerate implementations and look at the description. MFXVideoENCODE_Init succeeds. However, MFXMemory_GetSurfaceForEncode returns a surface with an unknown format.
I have modified the code to not read a frame from the disk, just to get the surface for encoding and send it into the encoder:
if (isDraining == false) {
sts = MFXMemory_GetSurfaceForEncode(session, &encSurfaceIn);
VERIFY(MFX_ERR_NONE == sts, "Could not get encode surface");
// sts = ReadRawFrame_InternalMem(encSurfaceIn, source);
// if (sts != MFX_ERR_NONE)
// isDraining = true;
}
If you put MFXVideoENCODE_Query before MFXVideoENCODE_Init, MFXVideoENCODE_Query succeeds.
My setup is an AMD 3900X with 128GB RAM, Windows 11. I have an NVIDIA 3060 and an Intel Arc 380 plugged into it right now. However the above was performed with an NVIDIA 4080 plugged in and no Arc, i.e. purely using the software reference impl. I have up to date Windows and all drivers.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for sharing the code. To debug the issue further, could you please provide the input video file and the exact commands used to reproduce the issue.
Regards,
Thasneem vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We have not heard back from you. Could you please give us an update?
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The input command would have been:
-sw -i "1588.raw" -w 1280 -h 720
I have a folder of single P010LE files extracted from an HD movie. I use these frames to test my code. I passed a single one to hello_encode. I have attached 1588.raw to this comment. It has also been verified on rawpixels, as shown in the image below.
Please note that I modified the code to not bother reading or writing frames, as you can see from my previous comments. I am simply fetching a surface:
MFXMemory_GetSurfaceForEncode
and encoding whatever it gives me:
MFXVideoENCODE_EncodeFrameAsync
, without copying any data to it. So in a way with the code above it doesn't matter what the input is and nor should it. The issue is the image format of the returned frame.
If anything I've written here is unclear or you need more from me, please don't hesitate to ask.
Thank you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for sharing the requested details. We tried compiling your code, but it seems like your code needs some dependent header files. Could you please share the necessary files so that we could triage your issue further?
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Thasneem. I have attached util.hpp. It's the util.hpp from Intel's hello_encode project.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for sharing the details. We are checking on this internally. We will get back to you soon with an update.
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
When we tried compiling your modified hello encode sample, we are getting compilation error as shown below:
Please find the resources MFXSetConfigFilterProperty and mfxImplDescription for more details.
Could you assist us in providing the specific code you compiled so that we could triage your issue further?
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ah, I think I gave the wrong util.hpp file. I've attached both source files as compiled by me a moment ago without error, in Visual Studio 2022. I have libvplswref64.dll in the output directory. I am linking with vpl.lib. The The issue remains, i.e. specify 10 bit AV1, get back a weird FOURCC code in the surface returned by MFXMemory_GetSurfaceForEncode.
I hope your people are able to compile this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We are checking on this internally. We will get back to you soon with an update.
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We regret to inform you that the application is using Implementation of VPL CPU Runtime(SW) which is no longer active. The core project and GPU runtime continues to be actively maintained. For CPU video processing the recommended solution is to use software codecs that are part of media frameworks such as FFMpeg and GStreamer. Intel® provides plugins for frameworks to support this usage.
Reference: https://github.com/oneapi-src/oneVPL-cpu
More information can be found in the link below: https://www.intel.com/content/www/us/en/developer/tools/vpl/overview.html#gs.1vrs9s
Kindly confirm if this information and references resolves your query and do let us know if we can stop monitoring this thread.
Regards,
Thasneem Vazim
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OK, we ditched the software encoding. It was too slow anyway. Thanks. You can close this out.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page