Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

MVC Decoding of m2ts files

Markus_Pingel
New Contributor I
793 Views

I have a question about decoding MVC with the SDK 3 beta:

How is this supposed to work if the encoded stream is separated like in the m2ts files of AVCHD and Blu-Ray?

How does decodeheader work? Do you have to call it on the base and the dependent stream?

What does DecodeFrameAsync expect? Do I need to merge the two streams in one? Do I interleave frames or can I interleave NALUs?

Thanks,

Markus

0 Kudos
7 Replies
Anthony_P_Intel
Employee
793 Views
Hi Markus,

The Media SDK 3.0 Beta 3 release is focused on single MVC streams that contain all the views of each frame. If each view is in a separate transport stream, you would need to merge these into a single MVC elementary stream. I believe you would want to interleave the frames.

As I am new to MVC myself, I will ask some colleges to make sure I am correct.

Also, we hope to have more sample code in future releases of this SDK that should help.

-Tony
0 Kudos
IDZ_A_Intel
Employee
793 Views
I am not sure how blueray and AVCHD separate stereo views into different streams. If they are separate maybe they are totally independent and you can just use h.264 AVC decoder?

If it is still an MVC stream with base and dependent views, you need a marged stream to be passed to decoder both to DecodeHeader and DecodeFrameAsync.Decoder outputs frames for each view interleaving them. That is 1st frame for base view, 1st frame for dependent view, 2nd frame for base view, 2nd frame for dependent view, etc.
0 Kudos
Markus_Pingel
New Contributor I
793 Views

The streams are separated into MVC base viewand MVC dependent view. Either as 2 elementary streams muxed in a ts or as two separate files. One stream with profile high, the other with stereo high.

So it seems I have to merge the streams before passing it to the decoder. Should they be merged frame by frame or on any NALU border?

0 Kudos
IDZ_A_Intel
Employee
793 Views
You need to merge bitstreams into a whole MVC access unit. It should contain all slice and picture headers, then all slices for the base view, then all slices for the dependent views.
0 Kudos
Markus_Pingel
New Contributor I
793 Views
Ok, Thanks. I will try this as soon as I have the splitter ready.

But what about encoding? The spec says "The generated bitstream contains the complete picture (multiple views.)". Do I have to parse the encoded bitstream to separate the views?
0 Kudos
IDZ_A_Intel
Employee
793 Views
Yes MVC encoder outputs bitstream in chunks of whole MVC access units. They contain base view and all dependent views (one for stereo). If you need to separate base and dependent views currently the only way is to parse bitstream.
0 Kudos
Markus_Pingel
New Contributor I
793 Views
OK, if there are no plans to provide the info on where the access unit could be split, there is no other way than to write my own parser.
I won't look a givenSDK horse in the mouth too deep :).
0 Kudos
Reply