We do not provide such sample as part of the SDK but implementing direct buffer interface instead of reading data from file for your specific case should not be difficult at all. mfxBitstream carries the data that is fed to the encoder. mfxBitstream represents the buffer input to encoder and it does not matter where the data originates (file, other buffer etc...). Just keep in mind to insert appropriate timestamps in mfxBitstream for each frame. The sample_decode sample provides a good base for this purpose.
Other developers on this forum may also be able to share some thier best practices on this subject.
I am having some difficulty decoding H264frame by frame can you tell me what SDK functions and in what order I must call them and delete them to decode one frame at a time.
Currently I can decode the first frame ok which is the keyframe and contains the header information. My camera sends each key frame every 10th frame. I did manage to decode the second frame as well but I got memory leaks so the decoding was short lived.
The order I am currently calling the sdk functions to decode are as follows and taken from the sample_decode.exe
1: I createone session
2: I create a MFX bitstream to hold the frame InitMfxBitstream(&m_mfxBS,1024 * 1024)
3: I iniit the decoder MFXVideoDECODE(session)
4: I DecodeHeader MFXVideoDECODE_DecodeHeader(..)
m_mfxBS.Data = pBuffer //H265 buffer from camera
m_mfxBS.DataLength = pSize //length of buffer
m_mfxBS.TimeStamp = pTimeline //timestamp from camera
5: I specify memory type which I set to SYSTEM MEMORY
6: I create a memoryallocator
7: I init the decoder MFXVideoDECODER_Init(..)
8: I find a working surface GetFreeSurfaceIndex(...)
9: I decodeframeAsync(&m_mfxBS,...); this always returns MFX_ERR_DATA then I recall GetFreeSurfaceIndex
10: Recall GetFreeSurfaceIndex
11: I recall decodeframeAsync(NULL,...) without m_mfxBS return is now MFX_ERR_MORE_SURFACE
12: I then call GetFreeSurfaceIndex(..) again nIndex = 1 now
13: I then call DecodeFrameAsync(NULL,..) return now MFX_ERR_NONE
14: I then call SyscOperation(..)
15: I now have the firstdecoded frame
Now for the second frame
16:I now just repeat from 8: ~ 14: except I pass the next frame buffer to m_mfxBSbefore I call DecodeFrameAsync(m_mfxBS,..)
m_mfxBS.Data = pBuffer //H264 buffer from camera
m_mfxBS.DataLength = pSize //size of video buffer
m_mfxBS.TimeStamp = pTimeline //Time stamp from camera
17: DecodeFrameAsync(m_mfxBS,...) returns MFX_ERR_MORE_DATA the same as the first frame did
18: I then call GetFreeSurfaceIndex again where nIndex = 0;
19: DecodeFrameAsync(NULL,...) return MFX_ERR_MORE_DATA so I added MFX_ERR_MORE_DATA to the while loop and call GetFreeSurface(..) again where mfxResponse.NumFrameActual = 9
problem is nIndex dosnt implement to 1 and the code just continously loops due to MFX_ERR_MORE_DATA being returned by DecodeFrameAsync(NULL,..)
Hop you can help me out here as im at a loose as what to do and Im new to the media sdk and still trying to understand how to use it.
Looking at your pseudocode thereare some things that stand out.
9 - 11: Calling DecodeFrameAsync with mfxBS=NULL will effectively start "draining" the decoder the retrieve the buffered frames. This is normally what is done after there are no more video stream data input to decoder.If you look at sample_decode you will see how this works. The first loop processes the input stream until it's empty (or ends). The second loop handles decoder buffer "draining as explained above.
Since (9)DecodeFrameAsync(mfxBS) returns MFX_ERR_MORE_DATA it means that the mfxBS does not have enough stream data to be able to process at least one frame. As can be seen in sample_decode, the mfxBS is then appended with more data from stream file. So my guess is that you do not have complete stream data to be able to start decoding of the fist frame in mfxBS.
In your case it seems you're using the SDK for a streaming or video conferencing usage scenario. If so please refer to this article:http://software.intel.com/en-us/articles/video-conferencing-features-of-intel-media-software-development-kit/
I have now sucessfully decoding each frame but I have two issues
1: I create an empty buffer for each new frame that is passed by my frame grabber using
InitBitstream(mfxBitstream* pBitstream, mfxU32 nSize) but when I try to delete it after the frame has been decoded I get an error from WipeMfxBitstream(&m_mfxBS) when it trys to delete MSDK_SAFE_DELETE_ARRAY(pBitstream->Data);
Any Ideas as I have a big memory leak from this.
2: my decoded frames are very pixelated meaning each decoded frame looks like a very compressed jpeg image im decoding via hardware and using the i7-3770K third generation CPU
what would cause the decoding to be this way
1) I suspect your memory area has been corrupted somehow. Please compare with behavior of sample_decode
2) I have no clue to why this would happen. If you write the stream that is received from your camera to file then decode the file using sample_decode. Do you encounter the same results?
The memory issue is solved but I seem to have a problem with hardware decoding and software decoding.
When I decode using software option I retreive from (mfxFrameSurface1* pmfxOutSurface) a NV12 color space which I then convert to RGB which works fine but when I decode using Hardware option pmfxOutSurface nolonger seems to be the same NV12 color space can you point me into the right direction as to what is going on here, as my frame is now all distorted but fine if I use software decode option.
Also when I use the option -r from the sample_decode.exe I only get the first frame in the rendered window but the console is telling me it is decoding each frame untill all frames are complete I added a small extra function to just see if each frame was being processed this function just copys the NV12 color space to RGB and saves a bmp image each frame is being processed and saved but the rendered surface remains with the first frame only.
Surfaces are referenced in different ways depending on if you use D3D or system memory surfaces. In the case of D3D, the surface handle (effectively a IDirect3DSurface pointer) is stored in Data.MemId. In the case of using system memory the surface is referenced via the memory plane pointers (Data.Y, Data.U, Data.V).
I do not know the reason that you only see the first frame getting rendered. Have you verified that the input stream is not broken?