How to use Media Foundation plugins provided by IntelMediaSDK sample code?

[hbo4577]

In the folder is Intel Media SDK sample code about Media Foundation plugins (Intel MFT).

C:\\Program Files\\Intel\\Media SDK\\2.0.12.24071\\samples\\sample_plugins\\mfoundation\\

I implemented a program for do H.264 encoding based on Microsoft Media Foundation (using sinkwriter method, not media session).

and I call regsvr32.exe in the command line to forcelly register mfx_mft_h264ve_32.dll in the system (windows7 + Intel SandyBridge x64).

and then I run the program. The CPU utilization is really very low. It is about 15 ~ 25%.

It seems that the HW encoding function is really enabled.

However, when I use Intel Media Checker, the report said that the Intel Media SDK SW and HW encoding functions are both not called. (SW and HW both ZERO times)

In conclusion, the program does not really run Intel MFT.

I asked Mircosoft engineers. They told me I could register Intel MFT locally before Media Foundation sinkwriter will select Intel MFT to execuate.

I register MFT locally just like this:

InputTypes.guidMajorType = MFMediaType_Video;

InputTypes.guidSubtype = MFVideoFormat_NV12 ;

OutputTypes.guidMajorType = MFMediaType_Video;

OutputTypes.guidSubtype = MFVideoFormat_H264;

hr_SW = MFTRegisterLocalByCLSID( CLSID_MF_H264EncFilter, MFT_CATEGORY_VIDEO_EFFECT, L"Intel\\xae Media SDK H.264 Encoder MFT", MFT_ENUM_FLAG_ALL, 1, &InputTypes, 1, &OutputTypes);

However, when I try to enumerate MFTs, this Intel MFT always can not appear locally.

hr_SW = MFTEnumEx( MFT_CATEGORY_VIDEO_ENCODER, MFT_ENUM_FLAG_LOCALMFT , &InputTypes, &OutputTypes, &ppMFTActivate, &cMFTActivate );

CLSID m_clsid;

hr_SW = ppMFTActivate[0]->GetGUID( MFT_TRANSFORM_CLSID_Attribute, &m_clsid );//get CLSID from this enumerated MFT

m_clsid is 6CA50344-051A-4DES-9779-A43305165E35, but CLSID_MF_H264EncFilter shoud not be this number.

I look up the registry, 6CA50344-051A-4DES-9779-A43305165E35 should mean Micorsoft H.264 Encoder MFT.

and if I call like this : hr_SW = MFTEnumEx( MFT_CATEGORY_VIDEO_ENCODER, MFT_ENUM_FLAG_ASYNCMFT | MFT_ENUM_FLAG_LOCALMFT, &InputTypes, &OutputTypes, &ppMFTActivate, &cMFTActivate );

then CLSID_MF_H264EncFilter would be enumerated.

Do you know why this case happen? It seems that CLSID_MF_H264EncFilter is already registered in the system, but can not be registered locally. In this case, I do not know how to make sinkwriter choose CLSID_MF_H264EncFilter to do encoding, not default Microsoft H.264 Encoder MFT.

[Nikolay_A_Intel]

hello,

in case of registering:

hr_SW = MFTRegisterLocalByCLSID( CLSID_MF_H264EncFilter, MFT_CATEGORY_VIDEO_EFFECT, L"Intel\xae Media SDK H.264 Encoder MFT", MFT_ENUM_FLAG_ALL, 1, &InputTypes, 1, &OutputTypes);

you useMFT_CATEGORY_VIDEO_EFFECT,

and after enum MFT_CATEGORY_VIDEO_ENCODER:

hr_SW = MFTEnumEx( MFT_CATEGORY_VIDEO_ENCODER, MFT_ENUM_FLAG_ASYNCMFT | MFT_ENUM_FLAG_LOCALMFT, &InputTypes, &OutputTypes, &ppMFTActivate, &cMFTActivate );

could you please try to use MFT_CATEGORY_VIDEO_ENCODER during registering?

[hbo4577]

Thanks.

I tried your suggestion, but the result is the same.

When enumeration locally (by MFT_ENUM_FLAG_LOCALMFT) , I only get Microsoft H.264 Encoder MFT, not Intel MFT.

However, when enumeration by MFT_ENUM_FLAG_ASYNCMFT, I can get Intel MFT not locally.

When enumeration by MFT_ENUM_FLAG_LOCALMFT |

MFT_ENUM_FLAG_ASYNCMFT, I only can get Intel MFT.

Could you give me more direction?

[Nikolay_A_Intel]

In msdn documentation for registering and enumerating of MFT plug-ins you can find out that asynchronous MFT's are not enumerated by default and you need explicitly set MFT_ENUM_FLAG_ASYNCMFT flag to get them enumerated. But if in MFTEnum you set only MFT_ENUM_FLAG_ASYNCMFT, then all other, non async plug-ins are not enumirated. So, the behavior is correct from documentation point of view.

To get both encoders enumerated, you need to set both flags.

[hbo4577]

Do you mean Intel MFT (CLSID_MF_H264EncFilter)isasync plug-ins??

In my program, I just callMFTRegisterLocalByCLSID( CLSID_MF_H264EncFilter, MFT_CATEGORY_VIDEO_ENCODER, L"Intel\xae Media SDK H.264 Encoder MFT", MFT_ENUM_FLAG_ALL, 1, &InputTypes, 1, &OutputTypes);

According to the MSDN,CLSID_MF_H264EncFilter the Intel MFT should be registered locally.

Hence,CLSID_MF_H264EncFilter should be async and local, right?

I thought the operator '|' means 'OR', so whenI only setMFT_ENUM_FLAG_LOCALMFT, the Intel MFT should be enumerated.

How about Microsoft H.264 MFT?

when I setMFT_ENUM_FLAG_LOCALMFT |MFT_ENUM_FLAG_ASYNCMFT, why is it not enumerated?

and why can Intel Media Checker "not" show any call number after registering locally Intel Encoder MFT if this MFT is really registered locally?

[Nikolay_A_Intel]

Intel MFT's are async.

So, the best way to get Intel MFT enumerated is to useMFT_ENUM_FLAG_ASYNCMFT flag in MFTEnumEx function.

We did not experimented with different combinations of MTFEnumEx flags, since this is Media Fountation functionality owned by Microsoft.

Also registering MFT withMFT_ENUM_FLAG_ALL flag seems to be a little bit risky, since it registers MFT with all possible flags except one.

[hbo4577]

Thank you.

but the real problem is thatIntel Media Checker "can not" show any calling-times when I execuate my program if this MFT is really registered?

When I use debug-mode, I can not trace my code in dll file (MFT-plugins), which means this code should not be execuated.

Do you know how to make sinkwriter (Media Foundation) execuate Intel MFT registered locally or aync?

[Nikolay_A_Intel]

could you take a look at the resulted file? not how it is played, but on bytes inside of file. do you see any "Microsoft" words inside of file? If yes, it means that inbox (Microsoft) encoder was used.

[hbo4577]

Attached please see the report, which is just Intel Media Checker created.

My issue is really easy.
I call MFTRegisterLocalByCLSID to register Intel MFT.
Sinkwriter of Microsoft Media Foundation should choose this local MFT as encoder.
Then, I can use Intel MFT to do HW encoding on SandyBridge. --> my goal


However, Intel Media Checker report nothing about Intel Media SDK call.
It is really not normal.

[Nikolay_A_Intel]

are you sure that Intel MFT encoder was used in processing?(if not you can use the BKM I wrote earlier or use some other way)

withoutconfidence about MFT's used in chain we have too broad ranges of possible roots for the issue.

[hbo4577]

I am sure Intel MFT is registered locally, because when I callhr_SW = MFTEnumEx( MFT_CATEGORY_VIDEO_ENCODER, MFT_ENUM_FLAG_LOCALMFT, NULL, NULL, &ppMFTActivate, &cMFTActivate );

I could enumerate Intel MFT.

Microsoft engineers told me if I register a MFT locally sucessfully, sinkwriter in Media Foundation would choose it as encoder. I assume they are right, so I think Intel MFT should be used in processing.

I don't know BKM you mentioned. What is BKM ? Can it help me know whether Intel MFT is running? Please tell me more detail about BKM.

another issue: Today I tried another program implemented by Intel IPP (for MPEG2 encoding), the CPU utilization is only 10~15% when running on SandyBridge with 20~30 fps. This is so amazing.

As I know, Intel IPP is definitely implemented for software version, not hardware acceleration. Why the CPU utilization of the one made by Intel IPP is lower than the one of another program made by Intel MFT althought Intel MFT is for H.264 naturally more complex than MPEG2?

[Nikolay_A_Intel]

BKM is Best Known Method, in our case it is to look at the resulted file (not play it, but look inside of it on the bytes) and check for "Microsoft" string in it. Do you have any "Microsoft" word in resulted file?

[hbo4577]

I tried to use Microsoft tool MFTRACE.exe, and I found something interesting:

The log, made by MFTRACE.exe, in the beginning, would show

CoCreateInstance @ New MFT @00ABAB48, MF_TRANSFORM_ASYNC=1;MFT_SUPPORT_DYNAMIC_FORMAT_CHANGE=1

CoCreateInstance @ Created {08B2F572-51BF-4E93-8B15-33864546DC9A} Intel Media SDK H.264 Encoder MFT (C:\Program Files\Intel\Media SDK\2.0.12.24071\samples\_build\Win32\Debug\mfx_mft_h264ve_32.dll) @00ABAB48 - traced interfaces: IMFTransform @00ABAB48,

and I tried every stream media input types for H.264 encoder listed in MSDN (http://207.46.16.248/en-us/library/dd797816(VS.85).aspx):

• MFVideoFormat_I420
• MFVideoFormat_IYUV
• MFVideoFormat_NV12
• MFVideoFormat_YUY2
• MFVideoFormat_YV12

All failed butMFVideoFormat_NV12.

The log always show

0028 796,638 03:53:18.67152 CMFTransformDetours::SetInputType @00ABAB48 Failed MT: MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_SUBTYPE=the media type I set such as the above exceptMFVideoFormat_NV12;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_SIZE=4398046511872 (1024,768);MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297

(1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1

Only when I use MFVideoFormat_NV12, the log would showSetInputType @00ABAB48

Succeeded MT: MF_MT_MAJOR_TYPE=MEDIATYPE_Video;MF_MT_SUBTYPE=MFVideoFormat_NV12;MF_MT_INTERLACE_MODE=2;MF_MT_FRAME_SIZE=4398046511872 (1024,768);MF_MT_FRAME_RATE=128849018881 (30,1);MF_MT_PIXEL_ASPECT_RATIO=4294967297

(1,1);MF_MT_ALL_SAMPLES_INDEPENDENT=1

Microsoft MFT would not be enumerated now!

Right now, I think I am really using Intel MFT.

However, there is nothing written in the output file when the sinkwirter of Media Foundation choose Intel MFT as H.264 encoder now.

I don't know why and is trying to solve this issue.

If you know anything helpful, please propose it to discuss. Thanks.

[Nikolay_A_Intel]

Thanks for the information, it looks like Intel MFT is used in chain, so this is strange that no output is generated.

Could you, please, share mftrace log?

[sjbaran]

FYI, we are also discussing this on the msdn forum at:

http://social.msdn.microsoft.com/Forums/en/mediafoundationdevelopment/thread/137b2208-8601-4633-88f9-30cc79cb7794

[sjbaran · Accepted solution]

Here are the problems I've run into trying to use the Intel Media SDK H264 Encoder using Media Foundation:

1) The encoder requires a NV12 input: "The Intel Media SDK sample filters require the input data to be in NV12 format. NV12 is the native format of the GPU, and thus the filters need this format to pass the data to the hardware for acceleration. Developers may find the need to insert a color conversion filter upstream of the sample decoder filter in order to connect the graph successfully."

2) There is no readily available transform to do this color converstion. The color converter transform from Microsoft does not support NV12 (http://msdn.microsoft.com/en-us/library/ff819078(v=VS.85).aspx) and "The Intel Media SDK does not provide a sample color conversion filter at this time. This is left to the developer to implement"

3) I was not even able to test the Intel encoder by reading in a NV12 formatted file because MFCreateSourceReaderFromURL is returning 0xC00D36C4 (Unsupported Bytestream) on any NV12 files I create.

4) so in order to use the Intel Media SDK H264 Encoder with Media Foundation, I will have to write a transform to output to NV12.

The Intel VPP can do the color conversion I would need in the transform (see sample_vpp from the Intel Media SDK), but it may make more sense to just use the API provided in the Intel Media SDK to do the entire video pipeline. The samples have the necessarily pieces (sample_encode can use NV12(or YUV420) to encode with H264 and sampe_vpp can convert from YV12, YUY2, RGB3, and RGB4 to NV12) - and the sample_multi_transcode does the whole pipeline.