Sample Encode

[Arjun_M_]

Hello,

I am trying to use sample_encode to encode raw YCbCr frames captured from a webcam using EmguCV in a C# project. I understand that sample_encode expects a YUV/NV12 file as input in order to encode. Is there any way to modify the sample_encode in order to work with individual frames rather than an entire file, as I would like to be able to compress a live feed. 

I found a blog post that suggests that the Run() method be modified so that LoadNextFrame is not called, and manually enter new frames. But I did not understand how to load frames. Currently I have simply put the captured frames into a queue. 

[Sravanthi_K_Intel]

Hello Arjun,

sample_encode and our tutorials (which are far simpler code examples https://software.intel.com/en-us/intel-media-server-studio-support/training#tutorial) encode frame-by-frame. Since you are encoding, on each LoadNextFrame() or equivalent LoadRawFrame() in the tutorial, the function reads WxH number of bytes of the frame and converts to NV12 format. You should be able to replace this function with your own function.

[Arjun_M_]

Thank you Sravanthi for your response. I did in fact come across the simpler examples. 

With regards to replacing LoadRawFrame(), I noticed that LoadRawFrame() takes in memory surfaces and the input file as parameters. If I was to replace this with my own function, is it correct to assume that I would have to handle these surfaces? Are there any resources available on how to do this, as I am not familiar with them. 

[Sravanthi_K_Intel]

Hi Arjun,

The LoadRawFrame() function is populating some of the basic parameters when reading the file - dimensions, crop values and of course the pointer to the frame that is read. So, you will have to do the same. I don't see any other "handling" of surfaces in specific. One thing to note though is that the surfaces require input format to be NV12. So, if you are capturing in a different format, write a function to convert it to NV12 and feed to the surface data pointer.

So, here is the flow:

Populate pSurface->Info variables (width, height, crop values - as in LoadRawFrame()

Capture format -> NV12 format conversion, and feed to pSurface->Data planes.

[Arjun_M_]

I had a follow-up with regards to the simple_encode code. I have incorporated the code into my own function, but I am facing a problem with specifically the allocation of surface headers for the encoder.

mfxFrameSurface1** pEncSurfaces = new mfxFrameSurface1 *[nEncSurfNum];

I get an SEHException for this. The stack trace is inconsistent, sometimes it points to a function call and other times its just null.

In addition, in the output log, I also get an "Invalid handle." message. I am not quite sure what this means, but it points to:

MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

As a result, my application is crashing. Could you please assist me with this? Thank you.

[Sravanthi_K_Intel]

Hello Arjun, Can you give some more details on what modification you did to sample_encode? From our previous conversation, the only change you needed to make to the pipeline you had was to modify the LoadRawFrame() or equiv function to read data from your capture buffer. I am not sure why you also modified the surface allocation part as above. Sample_Encode itslef has surface allocation that allocates memory for the number of surfaces (as suggested by QueryIOSurf function). That surface allocation should suffice for you and you dont have to add a new one to the pipeline.

Can you please give us more details of the changes you did to the code, and what you are trying to achieve?

[Arjun_M_]

Hi Sravanthi,

I did not modify the surface allocation, this is the surface allocation that is already done by simple_encode. 

The only part I really changed was: 

	for (int i = 0; i < nEncSurfNum; i++) {
		
		pEncSurfaces = new mfxFrameSurface1;
		memset(pEncSurfaces, 0, sizeof(mfxFrameSurface1));
		memcpy(&(pEncSurfaces->Info), &(mfxEncParams.mfx.FrameInfo), sizeof(mfxFrameInfo));
		//pEncSurfaces->Data.Y = &surfaceBuffers[surfaceSize * i]; // point directly to pointer result
		pEncSurfaces->Data.Y = result;
		pEncSurfaces->Data.U = &surfaceBuffers[surfaceSize * i] + width * height;
		pEncSurfaces->Data.V = pEncSurfaces->Data.U + 1;
		pEncSurfaces->Data.Pitch = width;
		/*if (!bEnableInput) {
			ClearYUVSurfaceSysMem(pEncSurfaces, width, height);
		}*/
	}

The commented out part is what was there originally. I am currently just passing a pointer to a gray image in my C# application to this code and assigning it to the Y data as I am only concerned with the intensity right now. 

[Arjun_M_]

Could you also assist me with another error I'm getting (I think its actually related to the "invalid handle" part)

I am defining my MFXVideoENCODE object as a pointer in my constructor as I would like to preserve this object throughout execution, since I have moved the actual encoding functionality into a separate function. I am doing: 

MFXVideoENCODE* mfxENC = new MFXVideoENCODE(session);

Later, when I am performing any functionality such as querying the number of surfaces required by the encoder, or validating video encode parameters (shown below):

// Validate video encode parameters (optional)
	// - In this example the validation result is written to same structure
	// - MFX_WRN_INCOMPATIBLE_VIDEO_PARAM is returned if some of the video parameters are not supported,
	//   instead the encoder will select suitable parameters closest matching the requested configuration
	sts = mfxENC->Query(&mfxEncParams, &mfxEncParams);
	MSDK_IGNORE_MFX_STS(sts, MFX_WRN_INCOMPATIBLE_VIDEO_PARAM);
	System::Console::WriteLine(sts);
	MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);
	
	// Query number of required surfaces for encoder
	//mfxFrameAllocRequest EncRequest;
	memset(&EncRequest, 0, sizeof(EncRequest));
	sts = mfxENC->QueryIOSurf(&mfxEncParams, &EncRequest);
	System::Console::WriteLine(sts);
	MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

the code errors. I check the sts value, and it is -6, which is MFX_ERR_INVALID_HANDLE. Could you please tell me why this is the case, and if there is any solution? 

Also, the session that is passed to MFXVideoENCODE, does that refer to the MFXVideoSession created earlier? Or does it refer to the member variable m_session created when MFXVideoSession is instantiated?

Thank you.