Pass immutable parameter through IMSDK decoder

[Anita_C_]

Is there an IMSDK decoder parameter that we can set for the input bitstream buffer and get the same along with the output surface? We are looking for a parameter which is tied to one buffer and decoder should not modify that parameter. Decoder should just pass the parameter from input to output. 

Please let us know if similar thing can be done for encoder and preprocessor also ?

Note : We are feeding the input to decoder frame by frame by setting MFX_BITSTREAM_COMPLETE​_FRAME.

[Jeffrey_M_Intel1]

Hi Anita,

You have two options:

1. SEI messages
2. Timestamps

Timestamps are straightforward, and can be passed through decode, VPP, and encode unchanged if there is no framerate conversion or deinterlace.  Aside from these cases, how Media SDK treats timestamps matches your requirement description.  However, you're probably already using them for their intended purpose.

SEI messages are a more general answer.  A starting point is here: https://software.intel.com/en-us/blogs/2014/08/18/how-to-add-closed-caption-messages-in-avc-and-mpeg2-streams. It will be up to your application to help navigate through VPP however.

Regards, Jeff

[Anita_C_]

Hi Jeffrey,

Thanks for the quick response. As you correctly pointed out, we are already using timestamps for their intended purpose. 

SEI messages(as explained here) can be used when we want to add generic data to encoded stream and retrieve it from the decoded frames. But our usecase is different, We have some generic data and we want to pass it along with the decoder input bitstream. We want to retrieve this generic data at the decoder output. Similarly, we want to pass the generic data through encoder and VPP also. Can you please help us with such a parameter ?

Regards, Anita

[Jeffrey_M_Intel1]

As far as I know there are no additional parameters beyond the two already mentioned built into Media SDK.  It's a good feature request, but as you know prioritization is complex. 

There may be another option: The Video Metadata Framework (https://software.intel.com/en-us/vmf-sdk/)

I have not tried it yet, and am certainly no expert, but it may be at least as useful as SEI messages for your scenario.  In both cases your application would have to manage through VPP stages.  

Perhaps you could consider some sort of map for this metadata, with timestamps as keys?  This would let you use the existing Media SDK mechanisms to track data through the pipeline, but would allow you to continue using timestamps as timestamps too.  Another advantage would be the ability to store more data, since timestamps are fixed at 64 bits. 

[Anita_C_]

Hi Jeffrey,

Thanks for the suggestions.

Based on your suggestion we are using timestamps and expecting that they will be passed through decoder unchanged. We are setting pointer to C structure as value for the Timestamp and DecodeTimestamp field (for input bitstream) and expecting the same at the output. But below are the issues that we are facing :

1. We expect the value of DataFlag for the output surface to be MFX_FRAMEDATA_ORIGINAL_TIMESTAMP(As per IMSDK manual, this value Indicates the time stamp of this frame is not calculated and is a pass-through of the original time stamp) but IMSDK decoder is not setting DataFlag as MFX_FRAMEDATA_ORIGINAL_TIMESTAMP for some frames(observed for IMSDK MPEG2 decoder). Can you please help us understand this behavior of IMSDK MPEG2 decoder ?

Please note that we have not observed this issue with IMSDK H264 decoder, we also wish to understand this differenence in the behaviour of IMSDK MPEG2 decoder and H264 decoder.

2. When the DataFlag is not set to MFX_FRAMEDATA_ORIGINAL_TIMESTAMP and we try to dereference the pointer, we found that the corresponding memory is not usable since the same value is not passed. 

Thanks and Regards, Anita

[Jeffrey_M_Intel1]

Hi Anita,

You can see the simple_6 lowlatency tutorials as examples of using timestamps to map to other data (in this case wallclock times).  The map approach in the tutorials is safer and will allow more error controls by your application.  If a timestamp is changed or isn't the expected result your application may still be able to intelligently find the right thing, and it could be done without the risks from incorrect pointers.

As far as I know different timestamp behavior for MPEG2 and H264 decoders is not expected.  I can try to reproduce here, but any additional details you can provide in terms of code an input streams will be very helpful.

Regards, Jeff