dshow H264 sample filters does not work with high resolutions

[oar]

Hi,

I use the following direct show graph (in graphstudio):

SourceFilter(Logitech QuickCam Pro 9000)->ColorSpaceConverter->Intel Media SDK H.264 Encoder->Intel Media SDK H.264 Decoder->Video Renderer

When I set a low resolution for the camera, like 320x240, 640x480 or 800x600, the graph is running fine and the video is displayed by the Video Renderer. But, when I use resolutions like 1280x720, 1600x1200 or 1920x1080 the graph runs for two seconds and then stops - the Video Renderer show only a black image.

I debugged the source code and found out that in base_encoder.cpp in the method DeliverNextFrame(...) the following statement:

[cpp]sts = pEncoder->m_mfxVideoSession.SyncOperation(*it->psyncP, 0);[/cpp]

returns with -1 (MFX_ERR_UNKNOWN), afterwards the graph is stopped. This happens only when using high resolutions. The same statement returns always 1 (MFX_WRN_IN_EXECUTION) when using low resolutions.

Media SDKs used for testing:
2.0 gold
3.0 beta 3
3.0 beta 4

Thanks,
oar

[Anthony_P_Intel]

Hi oar,

Thank you for your question. Would it be possible to include a log from the mediasdk tracer tool?

Is this running on a Intel system with encode acceleration, or is this seen even if software/CPU encoding is used?

Thanks
-Tony

[Eric_S_Intel]

Thats a lot of data coming from the camera at those resolutions. Usually, Cameras will output MJPEG for Hi-def. Does your camera support this? Can you change the graph to:

Camera->MJPEG Decompressor -> Intel Media SDK Encoder -> Intel Media SDK Decoder -> Render

The MJPEG Decompressor usually output RGB32, and we added RGB32->NV12 in the encoder filter using VPP for this purpose. Thus, you will be able to eliminate the color conversion filter from the graph.

-Eric

[oar]

Hi,

I tested the graph on two systems, with one using software encoding and the other using hardware (GPU). On both system I can observe the same problem/behaviour as described in my initial post.

I did some further tests, also considering the advice from Eric, and generated a log file with the tracer tool for each case.

Cases not using mjpeg:

1. The normal case: SourceFilter(Logitech HD Pro Webcam C910)->ColorSpaceConverter->Intel Media SDK H.264 Encoder->Intel Media SDK H.264 Decoder->Video Renderer. Using 1920x1080 as resolution with 15 fps. The colors space converter outputs NV12 (input for intel h264 encoder). The graphs stops as described in my initial post. The corresponding log file: nv12_1920x1080_analyzer.log
2. Like the first case, only this time forcing the color space converter to convert to RGB32. The graph does not (always) stop. But only in the first few seconds (maybe 2 or 3) the video renderer receives new frames, then no more frames are received. The graph runs longer than in case 1, but stops from time to time. The corresponding log file: rgb32_1920x1080_analyzer.log

As the logitech cameras do not support mjpeg output, I had to use a Microsoft LifeCam HD-5001 for the next cases, which only supports a resolution of 1280x720 as the highest possible resolution.
Cases using mjpeg:

1. The graph: SourceFilter(Microsoft LifeCam HD-5001)->MJPEG Decompressor->Intel Media SDK H.264 Encoder->Intel Media SDK H.264 Decoder->Video Renderer. Using 1280x720 as resolution with 30 fps. The graph keeps running but only at the first few seconds frames are received by the video renderer. The corresponding log file: mjpeg_1280x720_30fps_analyzer.log. As a sidenote: If I do not use the intel h264 encoder and decoder in the graph so that the graph looks like that: SourceFilter(Microsoft LifeCam HD-5001)->MJPEG Decompressor->ColorSpaceConverter->Video Renderer, the graph keeps running at 30 fps.
2. Like the first (mjpeg) case, but with 10 fps instead of 30. The graph keeps running and frames are received continuously (the "quality" property page shows ~10fps). The corresponding log file: mjpeg_1280x720_10fps_analyzer.log

Thanks
oar