Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3075 Discussions

Feeding D3D surfaces to Quick Sync encoder MFT does not work

Vincent_L_1
Beginner
1,130 Views

Media SDK Client or Media Server Studio version installed: unknown

Processor Type (required both for Linux & Windows OS): Intel(R) Core(TM) i7-5557U CPU

Driver Version(required only for windows OS): 20.19.15.4531

Operating System (required both for Linux & Windows OS): Windows 10 Pro 10.0.14393

Media SDK System Analyzer(only for windows OS): N/A

Concise Description of the Issue:

I want to encode video using the "Intel® Quick Sync Video H.264 Encoder MFT". I'm using the MFT manually, without using a paired decoder MFT, or any other MediaFoundation components. Feeding normal buffers (IMFSamples with buffers created by MFCreateAlignedMemoryBuffer) works well.

Now I'm investigating whether I can feed it ID3D11Texture2D surfaces as input (DXGI_FORMAT_NV12, 1280x720) in order to improve performance. I tried to pass IMFSample instances created with MFCreateVideoSampleFromSurface or MFCreateDXGISurfaceBuffer to IMFTransform::ProcessInput and made multiple experiments (trying different texture creation flags), but the best result was that all input samples were accepted, but no output samples produced. In case it matters, I never actually tried uploading data to the textures, assuming this would not make a difference from textures filled with garbage pixel data.

Additionally, I tried setting a IMFDXGIDeviceManager instance with IMFTransform::ProcessMessage and MFT_MESSAGE_SET_D3D_MANAGER, but this call returned E_FAIL. (The same works when doing hardware decoding with the Microsoft H264 decoder MFT.) In my understanding, this is a prerequisite for the hardware encoder to work.

Am I doing something wrong? Is this a bug in the Intel MFT? Is it just not supported?

I wish I could post a mftrace log, but the Intel MFT will fail with "Merit validation failed" under mftrace - there are others who seem to have this problem, but I've seen no solution.

 

There's another problem unrelated to the one described above. The Intel encoder MFT does not initialize the output IMFMediaType with MF_MT_MPEG_SEQUENCE_HEADER. This happens only after you send it a few input frames. Then the MFT will signal MF_E_TRANSFORM_STREAM_CHANGE, and add MF_MT_MPEG_SEQUENCE_HEADER.

This is very inconvenient. Usually you need MF_MT_MPEG_SEQUENCE_HEADER at the start of the transcoding process to e.g. write the CodecPrivate element for Matroska. The Microsoft encoder MFT does not show this unexpected behavior.

I'm working this around with a bad hack by encoding a dummy frame with CODECAPI_AVEncVideoForceKeyFrame and then discarding the first output sample (since trying to reset/flush the decoder MFT to forget previous input doesn't work either). I know that the underlying quick sync API can retrieve this information via MFX_EXTBUFF_CODING_OPTION_SPSPPS before sending any input frames, so this seems like an unnecessary complication.

0 Kudos
2 Replies
Tao_H_
Beginner
1,130 Views

I've encountered the same problem, Any progress?

0 Kudos
Tao_H_
Beginner
1,130 Views

I've encountered the same problem,Any progress?

0 Kudos
Reply