Decoding video from a live stream using Intel Media SDK and Directshow

[wadrwadr]

Some weeks ago I builded a directshow graph based application to decode live video stream.
The video stream comes from a multicast IP, compressed with h264 and encapsulated in MPEG-TS packets under the UDP protocol.
Filters in graph were connected as I explain next:
Elecard Network Source Plus -> Elecard Push Demultiplexer -> Elecard H264 Video Decoder

Graph also works using the next h264 decoders:
- Microsoft DTV-DVD Video Decoder
- CoreAVC Video Decoder

Now my company decided to use Intel Media SDK Directshow filters to improve decoding performance on a multiprocessor system.
Problem is that I can't find a single way to build a similar graph as the mentioned above using the Intel Media SDK h264 decoder.

I tried to replace Elecard H264 Video Decoder with Intel Media SDK h264 decoder but filter connection with push multiplexer throws an error... ( pin negotiation fails ).
Fact is I couldn't connect Intel h264 decoder to any source network/push demuxer I know.

Someone could successfully decode live video streams with similar format as I described above using Intel h264 or mpeg-2 decoders with Directshow?
Do you know some network source/push demuxer filters compatibles with intel media sdk decoders?

Thanks in advance.
Walter.

[gthomaz]

I don't know wich format is available in your old filter. But Intel Media SDK supports NV12 color format on input pins.

The conversion RGB to NV12 or YUV/I420/etc to NV12 is not available. You need to build your own filter...

There's a lot of tips here in this forum.. please search for "NV12" here....

Thanks!!

[wadrwadr]

Hello Guilhermer thanks for reply.

Correct me if I'm wronkg but I'm triying to decode here, color format is the output not the input of a decoder.
My problem is when I try to connect the intel h264 decorder input pin to push demuxer video output pin, video is still encoded in this stage.
Some well know input formats of a common h264 decoder filter are vc1, h264, x264, VSSH, Maiconcept h.264, ArcSoft H.264.
I did some research and found that push demux filter exposes the next media types in the video output pin:

MEDIATYPE_Video 73646976-0000-0010-8000-00AA00389B71
MEDIASUBTYPE_H264 8D2D71CB-243F-45E3-B2D8-5FD7967EC09B

MEDIATYPE_MPEG2_PES e06d8020-db46-11cf-b4d1-00805f6cbbea
MEDIASUBTYPE_H264 8D2D71CB-243F-45E3-B2D8-5FD7967EC09B

MEDIATYPE_Video 73646976-0000-0010-8000-00AA00389B71
MEDIASUBTYPE_h264 34363268-0000-0010-8000-00AA00389B71

None of these types seems to negotiate a connection with intel h264 decoder input pin but works ok with the decoders i mentioned early.

Regards
Walter.

[gthomaz]

Hello Walter!

Pardon, I'm wrong. I read fast and didn't realize about "DECODER".

I'm not work with live stream, but I did a decoder(player) here for a simple test.
I connected the filters like this :

H264 Source (file async) -> Intel MP4 Splitter -> (video) Intel h264 Decoder -> NV12toRGB(my filter) ->V.Rend
-> (audio) AAC Decoder -> Audio Renderer

It helps you?
Pardon for my fist reply

[wadrwadr]

Hi Guilherme.

Yes I tried with MP4 splitter but no luck.
I found another thing:
I wrote a small piece of code to get the media types expossed by intel decoder output and input pins and for my surprise result was 0 no media types exposed at all.
Very strange... do I have to setup something from code before connect ?

Thanks.
Walter.

[gthomaz]

Hi again Walter,

Really strange...

I did any special configuration... only the declaration of H264Decoder using the respective GUID ("CCCE52FD-02CB-482C-AC81-1E55EF1D61EE").

I've a suggestion to you:
Could you install an application like GraphEdit and check the media types available for H264 decoder?

For me, H264 Decoder has:
Input Pin -> Major Type: MEDIATYPE_Video
Subtype: fourcc (34363248) H264

Output Pin -> Major Type: MEDIATYPE_Video
Subtype: fourcc (3231564e) NV12
(to check this, please go to Graph-> Insert Filters...-> DirectShow-> Intel*)

[wadrwadr]

Hello Guilherme.

I found the problem.
For some reason exists two h264 media subtypes GUIDs.

The one supported by Intel h264 decoder input pin and defined in uuids.h at windows sdk is:

// H.264 compressed video stream
// 34363248-0000-0010-8000-00AA00389B71 'H264' == MEDIASUBTYPE_H264
OUR_GUID_ENTRY(MEDIASUBTYPE_H264,
0x34363248, 0x0000, 0x0010, 0x80, 0x00, 0x00, 0xaa, 0x00, 0x38, 0x9b, 0x71)


But exists another h264 subtype which is not supported by the filter, I found it debugging the filter and searching at the web:

//{8D2D71CB-243F-45E3-B2D8-5FD7967EC09B}
static GUID MEDIASUBTYPE_h264= {0x8D2D71CB, 0x243F, 0x45E3, {0xB2, 0xD8, 0x5F, 0xD7, 0x96, 0x7E, 0xC0, 0x9B}};

To fix the problem I had to modify the method CheckInputType in filter source code to support the new subtype.
Here's the modification:

HRESULT CH264DecVideoFilter::CheckInputType(const CMediaType *mtIn)
{
MSDK_CHECK_POINTER(mtIn, E_POINTER);

if (MEDIASUBTYPE_H264 != *mtIn->Subtype() && MEDIASUBTYPE_h264 != *mtIn->Subtype())
{
return VFW_E_INVALIDMEDIATYPE;
}

return CDecVideoFilter::CheckInputType(mtIn);
};

I test it with graphedit and works like a charm.
Regards.
Walter.

[gthomaz]

Ok Walter!
I'm happy that you could solve your problem!!!
I have been working in C#, so I use my decoder in a different way. I didn't have this same problem..
But, anyway, graphedit help us always! lol

Good job!

[Devora_B_Intel]

Hi ,

I built a similar graph using graph-edit tool.
Elecard Network Source Plus -> Elecard Push Demultiplexer -> MPC video decoder ->

I tried to implement the same graph programmatically using C++ and DirectShow - no success.

1. I don't know how to set the properties of "Elecard Network Source Plus" (in graphedit I just right click on the filter).

2. I don't know how to create an output pin for the elecard Push demultiplexer - how do I know what are the right configuration for the AM_MEDIA_TYPE struct.

Do you know how to do it programmatically?

Thanks in advance,
Devora

[Anthony_P_Intel]

Hi Devora,

I believe your questions are specific to Elecard's components and their documentation would likely answer the specifics of their APIs and component properties.

-Tony

[flobadobatwork]

Quoting gthomaz

I don't know wich format is available in your old filter. But Intel Media SDK supports NV12 color format on input pins.

The conversion RGB to NV12 or YUV/I420/etc to NV12 is not available. You need to build your own filter...

There's a lot of tips here in this forum.. please search for "NV12" here....

Thanks!!

Guilherme, I need to write an NV12 to RGB24 converter filter. I've looked for code samples / algorithms all over this forum but can't find any. Do you have a code snippet or algorithm to do this?

thanks.