Passing an opaque pointer through the Media SDK encoder

[Rosen__Daniel]

Is there a way to pass an arbitrary pointer or data structure through the Media SDK encoder so that it can be retrieved along with the encoded compressed frame?

[Sravanthi_K_Intel]

I think you are referring to embedding SEI messages in the encoding stream. If so, yes, you can. Before you call the frame encode function (MFXVideoENCODE_EncodeFrameAsync), you have to populate the mfxEncodeCtrl structure with the payload data and pass this structure as the first parameter of the encode function. And when you decode the frame, use GetPayload() function to retrieve the payload. You can find more information on SEI and payloads in the manuals, and an example in the developers guide for the same (Section 4.13 p48).

If I am misunderstanding your question, please let me know.

[Rosen__Daniel]

Thanks for the reply.  No, we don't want to create an SEI NAL with this data.  Rather, we keep some metadata from the original uncompressed frames which we would like to tag the compressed frames with in our application, but this data never appears in the encoded stream.  For example, the x.264 api has an arbitrary pointer which you can send along with the uncompressed frame data and then retrieve, via the api, with the compressed frame data.  

For example, we maintain an external database of the time codes, non-standard captioning and and chapter information all from the original source file and want to associate this information with the returned encoded frame.  One could think of it as unregistered SEI data, that isn't written to the output stream.  We have a couple workarounds including creating SEI data and then stripping it later, but it would be so much easier if this data could be passed to the encoder and retrieved either from a a variable in mfxBitstream or via an API.

[Sravanthi_K_Intel]

Currently, Media SDK does not have direct support for what you are looking for. But like you pointed out, SEI messages are a good workaround. Other than that, one other option is to use the timestamps of the frames to hash into the metadata data, assuming the timestamps do not get modified during the media pipeline (encode to decode).

[Rosen__Daniel]

Thanks, I was afraid there wouldn't be such support, but was just checking.