Simple_endcode_d3d - frame to stream instead of file

[Matthew_C_7]

I am looking at the simple_encode_d3d.cpp example and trying to write something similiar that will instead take an individual frame at a time instead of reading from a file and then will write a h264 bytestream out to a socket and had a few questions.

First of all, what is the difference between Stage 1 Encoding loop and Stage 2 which says it retrieves buffered encoded frames? From a code perspective, it appears that they do the exact same thing and I'm not sure why you need to do it twice.  How different would this process be if I'm doing it one frame at a time?

Also, regarding the bitstream, do you know how I could split that up into individual packets (or I-frames & B-frames) that could be recieved by a h264 player on the other end after it reads this from a socket?

Thank you!

Matt

[Petter_L_Intel · Accepted solution]

Hi Matthew,

Regarding difference between stage 1 and 2 (BTW. You can see the same layout in sample_encode sample, part of the SDK package ). When you are encoding a stream you will often use an encoder configuration requiring the encoder to keep references to frames (P to I, or P to P etc...). So, to completely flush out all the frames requested to be encoded there a draining stage (stage 2) which instructs the encoder that no new frames are available and to start draining the frames.  You can spot the difference in the EncodeFrameAsync call (the second parameter) between the stages.

That said, for streaming or video conferencing usages the draining stage is often ignored, thus the buffered frames are ignored (discarded).

To your second question. The encoder outputs individual frames (I, P or B) so no need to split. If the question is about how to efficiently packetize frames, or part of frames into network packets for streaming, then this is outside the context of Media SDK and not something we consult on.

Regards,
Petter

[Matthew_C_7]

How likely is the draining stage to catch a new frame when used for something like video conferencing?  I may have to experiment with the draining stage and see how much of a difference it makes in speed and quality.

Ok that makes sense, so I can just take the encoder output and send it out the socket and the other end should see it as an incoming h264 stream.  Will try.

Thank you Petter!!

[Matthew_C_7]

Also, are there any functions somewhere already designed for writing a YUV frame to a surface, the only ones I see are similiar to LoadRawFrame, which loads an entire file.  I've already started dissecting it and making it my own but would be nice to know if something like that is already written.

Thanks again.

[Petter_L_Intel]

Hi Matthew,

The Media SDK samples provide utility functions for both reading and writing raw YUV frames. Please refer to the sample_utils.h/cpp files.

The Media SDK Tutorial sample code also have YUV read/write utilities in common_utils.h/cpp

Regards,
Petter