MVC Decoding of m2ts files

[Markus_Pingel]

I have a question about decoding MVC with the SDK 3 beta:

How is this supposed to work if the encoded stream is separated like in the m2ts files of AVCHD and Blu-Ray?

How does decodeheader work? Do you have to call it on the base and the dependent stream?

What does DecodeFrameAsync expect? Do I need to merge the two streams in one? Do I interleave frames or can I interleave NALUs?

Thanks,

Markus

[Anthony_P_Intel]

Hi Markus,

The Media SDK 3.0 Beta 3 release is focused on single MVC streams that contain all the views of each frame. If each view is in a separate transport stream, you would need to merge these into a single MVC elementary stream. I believe you would want to interleave the frames.

As I am new to MVC myself, I will ask some colleges to make sure I am correct.

Also, we hope to have more sample code in future releases of this SDK that should help.

-Tony

[IDZ_A_Intel]

I am not sure how blueray and AVCHD separate stereo views into different streams. If they are separate maybe they are totally independent and you can just use h.264 AVC decoder?

If it is still an MVC stream with base and dependent views, you need a marged stream to be passed to decoder both to DecodeHeader and DecodeFrameAsync.Decoder outputs frames for each view interleaving them. That is 1st frame for base view, 1st frame for dependent view, 2nd frame for base view, 2nd frame for dependent view, etc.

[Markus_Pingel]

The streams are separated into MVC base viewand MVC dependent view. Either as 2 elementary streams muxed in a ts or as two separate files. One stream with profile high, the other with stereo high.

So it seems I have to merge the streams before passing it to the decoder. Should they be merged frame by frame or on any NALU border?

[IDZ_A_Intel]

You need to merge bitstreams into a whole MVC access unit. It should contain all slice and picture headers, then all slices for the base view, then all slices for the dependent views.

[Markus_Pingel]

Ok, Thanks. I will try this as soon as I have the splitter ready.

But what about encoding? The spec says "The generated bitstream contains the complete picture (multiple views.)". Do I have to parse the encoded bitstream to separate the views?

[IDZ_A_Intel]

Yes MVC encoder outputs bitstream in chunks of whole MVC access units. They contain base view and all dependent views (one for stereo). If you need to separate base and dependent views currently the only way is to parse bitstream.

[Markus_Pingel]

OK, if there are no plans to provide the info on where the access unit could be split, there is no other way than to write my own parser.
I won't look a givenSDK horse in the mouth too deep :).