Problems with hardware encoding in Metro app...

[richard_s_2]

I am wanting to get QuickSync working for its hardware encoding support but am having trouble making a Metro app that uses QuickSync's hardware encoding.

I tried to make a Metro app that mimics the low-latency video conference example (sample_videoconf).  However, when I tried to create objects such as MFXVideoSession in Metro apps, they seemed to try to link to Desktop/non-Metro libraries that are not available for Metro apps.

I then looked at the sample_simple_transcoder example since it was designed for Metro.  However, it seems it only performs transcoding, not encoding.  What is the suggested approach to have a Metro app do encoding w/ QuickSync hardware support?

System details: I have the release version of Windows 8 installed.  mediasdk_sys_analyzer reports sw+hw support for up to v1.4 and the Intel HD Graphics driver is the latest.  All the out-of-the-box examples seem to work fine.

[Petter_L_Intel]

Hi, It is not possible to use the Intel Media SDK via the Media SDK API in Windows 8 New UI (aka. Metro). Due to the Microsoft enforced sandbox environment developers have to resort to building Media Foundation applications instead. HW acceleration is available indirectly via Intel H.264 encode MFT which is part of the Intel Graphics driver. "sample_simple_transcoder" provides an example on how to implement a transcode pipeline for Windows 8 New UI. At the moment no other Windows 8 New UI samples are available as part of the Media SDK package. For Windows 8 Desktop mode, Intel Media SDK works in the same way as it does for Windows 7. Regards, Petter

[richard_s_2]

Petter - thanks for the quick response. Is there any documentation that shows how Media Foundation is hooking into the Intel H.264 encoder? Also, using Media Foundation and HW Intel H.264 encode indirectly, are there any technical hurdles to implementing something similar to "sample_videoconf" for a low-latency scenario? Thanks. Yes, Win 8 desktop mode is running the other examples fine.

[richard_s_2]

From my understanding of browsing Media Foundation documentation, they don't have APIs for just encoding...?

[Petter_L_Intel]

Hi. we do not provide any specific documentation in this area except the sample code I referred to in previous post. When it comes to Media Foundation the behavior and limitations is primarily dictated my Microsoft. The sample code uses the H.264 encoder MFT in a transcode centered context. At this point we do not have a sample that showcases video conferencing using MFTs, but it is something we are exploring. I suggest you look at the provided sample, it should give the necessary details with regards to how to configure the MFT. Aside from that I suggest you also look at the wide variety of Win8 samples from Microsoft. Regards, Petter

[richard_s_2]

After testing the transcoding sample code, I can see that it is indeed engaging the Intel hardware encoder since the CPU utilization drops a lot when hardware encoding option is enabled from the GUI. However, I am not understanding how the Intel H.264 encoder MFT (set as the Sink Writer) is able to read the data from the Source Reader (which is the file chosen from the file picker); is the encoder responsible for decoding whatever format the Source Reader is in, and then reencoding that as H.264? Or is there an intermediate raw format that the Intel H.264 encoder MFT is consuming? If I want to hook up a new type of Source Reader, what format should it emit for H.264 encoder to be compatible? Second, I don't understand how Media Foundation understands that it should be using the H.264 encoder MFT (even though from the CPU profiling, it does appear it is being used). The only flag I see is MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS but this doesn't say anything about Intel H.264 Encoder...? Thanks again!

[Petter_L_Intel]

Hi, The Microsoft Media Foundation framework is responsible for delivering frames/samples between connected components. This is out of context of Media SDK. Underneath the Media foundation surface the encode transform does receive raw NV12 frame surfaces from the up stream component. The Source Reader selects appropriate demuxer and decoder based on the file type/content. This is completely up Microsoft using the default MFT components they provide. For the case when the input stream is an H.264 stream, a generic HW accelerated (via DXVA) decoder from Microsoft will be used. Intel does not deliver this. Intel does provide a HW accelerated H.264 encode transform component as part of the Intel Graphics driver. This component has higher merit than the default Microsoft SW H.264 encode transform, thus the Intel transform will be selected by default, unless HW acceleration is not selected. With regards to creating your own source reader, I suggest checking out samples from Microsoft or forum discussions. Regards, Petter