Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Simple_endcode_d3d - frame to stream instead of file

Matthew_C_7
Beginner
592 Views

I am looking at the simple_encode_d3d.cpp example and trying to write something similiar that will instead take an individual frame at a time instead of reading from a file and then will write a h264 bytestream out to a socket and had a few questions.

First of all, what is the difference between Stage 1 Encoding loop and Stage 2 which says it retrieves buffered encoded frames? From a code perspective, it appears that they do the exact same thing and I'm not sure why you need to do it twice.  How different would this process be if I'm doing it one frame at a time?

Also, regarding the bitstream, do you know how I could split that up into individual packets (or I-frames & B-frames) that could be recieved by a h264 player on the other end after it reads this from a socket?

Thank you!

Matt

0 Kudos
1 Solution
Petter_L_Intel
Employee
592 Views

Hi Matthew,

Regarding difference between stage 1 and 2 (BTW. You can see the same layout in sample_encode sample, part of the SDK package ). When you are encoding a stream you will often use an encoder configuration requiring the encoder to keep references to frames (P to I, or P to P etc...). So, to completely flush out all the frames requested to be encoded there a draining stage (stage 2) which instructs the encoder that no new frames are available and to start draining the frames.  You can spot the difference in the EncodeFrameAsync call (the second parameter) between the stages.

That said, for streaming or video conferencing usages the draining stage is often ignored, thus the buffered frames are ignored (discarded).

To your second question. The encoder outputs individual frames (I, P or B) so no need to split. If the question is about how to efficiently packetize frames, or part of frames into network packets for streaming, then this is outside the context of Media SDK and not something we consult on.

Regards,
Petter

View solution in original post

0 Kudos
4 Replies
Petter_L_Intel
Employee
593 Views

Hi Matthew,

Regarding difference between stage 1 and 2 (BTW. You can see the same layout in sample_encode sample, part of the SDK package ). When you are encoding a stream you will often use an encoder configuration requiring the encoder to keep references to frames (P to I, or P to P etc...). So, to completely flush out all the frames requested to be encoded there a draining stage (stage 2) which instructs the encoder that no new frames are available and to start draining the frames.  You can spot the difference in the EncodeFrameAsync call (the second parameter) between the stages.

That said, for streaming or video conferencing usages the draining stage is often ignored, thus the buffered frames are ignored (discarded).

To your second question. The encoder outputs individual frames (I, P or B) so no need to split. If the question is about how to efficiently packetize frames, or part of frames into network packets for streaming, then this is outside the context of Media SDK and not something we consult on.

Regards,
Petter

0 Kudos
Matthew_C_7
Beginner
592 Views

How likely is the draining stage to catch a new frame when used for something like video conferencing?  I may have to experiment with the draining stage and see how much of a difference it makes in speed and quality.

Ok that makes sense, so I can just take the encoder output and send it out the socket and the other end should see it as an incoming h264 stream.  Will try.

 

Thank you Petter!!

0 Kudos
Matthew_C_7
Beginner
592 Views

Also, are there any functions somewhere already designed for writing a YUV frame to a surface, the only ones I see are similiar to LoadRawFrame, which loads an entire file.  I've already started dissecting it and making it my own but would be nice to know if something like that is already written.

Thanks again.

0 Kudos
Petter_L_Intel
Employee
592 Views

Hi Matthew,

The Media SDK samples provide utility functions for both reading and writing raw YUV frames. Please refer to the sample_utils.h/cpp files.

The Media SDK Tutorial sample code also have YUV read/write utilities in common_utils.h/cpp

Regards,
Petter

0 Kudos
Reply