Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

question on simple_decode_vmem

MyMother
Beginner
484 Views

hello Intel friends,

OS: Ubuntu 12.04

       MediaSamples_Linux_6.0.16043175.175

       MediaServerStudioEssentials2015R6

Platform:  i5-4570S

      I use two bit-stream to be decoded correctly Just like the following and got failure as using combined bit-stream. Basically, the parameters are validated in each bit-stream and it should be OK. Why it fail to decode combined bit-stream??

[_build] $ pwd
/home/dspuser/Downloads/mediasdk-tutorials-0.0.3/_build
[_build] $ sudo ./simple_decode_vmem -auto test_stream.264     ==> validate first bit-stream
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0

Execution time: 0.07 s (1421.59 fps)
[_build] $ sudo ./simple_decode_vmem -auto 1080p_4M.h264     ==> validate second bit-stream
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0

Execution time: 1.04 s (611.63 fps)

     And I combined these two bit-streams and validate with ffmpeg. But I got failure as using simple_decode_vmem

[_build] $ cp test_stream.264 combined.264              ==> create combined bit-stream  
[_build] $ cat 1080p_4M.h264 >> combined.264        ==> create combined bit-stream  
[_build] $ ffmpeg -i combined.264  -vcodec rawvideo -pix_fmt yuv420p combined.yuv       ==> validate with ffmpeg

ffmpeg version 2.1.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug  5 2014 05:02:01 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-gpl --enable-nonfree --enable-pthreads --enable-x11grab --enable-version3 --enable-libx264 --enable-encoder=libx264 --enable-libvpx --disable-zlib --disable-debug
  libavutil      52. 48.101 / 52. 48.101
  libavcodec     55. 39.101 / 55. 39.101
  libavformat    55. 19.104 / 55. 19.104
  libavdevice    55.  5.100 / 55.  5.100
  libavfilter     3. 90.100 /  3. 90.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, h264, from 'combined.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 176x96 [SAR 1:1 DAR 11:6], 30 fps, 30 tbr, 1200k tbn, 60 tbc
Output #0, rawvideo, to 'combined.yuv':
  Metadata:
    encoder         : Lavf55.19.104
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 176x96 [SAR 1:1 DAR 11:6], q=2-31, 200 kb/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> rawvideo)
Press to stop, [?] for help
[h264 @ 0x21c2580] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x21c2580] Reinit context to 1920x1088, pix_fmt: yuv420p
[h264 @ 0x21c2e40] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x21b2100] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x21b29c0] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x2195440] Delayed frames seen. Reenabling low delay requires a codec flush.
Input stream #0:0 frame changed from size:176x96 fmt:yuv420p to size:1920x1080 fmt:yuv420p
[h264 @ 0x21b2100] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x21b29c0] Delayed frames seen. Reenabling low delay requires a codec flush.
    Last message repeated 1 times
frame=  736 fps=419 q=0.0 Lsize=   18216kB time=00:00:24.53 bitrate=6082.6kbits/s
video:18216kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.000000%
[_build] $ sudo ./simple_decode_vmem -auto combined.264    ==> got failure
[sudo] password for dspuser:
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0

 Incompatible video parameters. src/simple_decode_vmem.cpp 200
[_build] $

0 Kudos
7 Replies
Surbhi_M_Intel
Employee
484 Views

Hi Medwin, 

The reason it would have failed with simple decode tutorial is that it read the header information of the first video and didn't reset the parameters for the second video. I see your first video to be of lower resolution than second, which means it would have requested for lesser surfaces to handle the high resolution stream. You have to call reset to handle this scenario. Please check sample_decode part of the samples package to find details on it.  

Thanks,
Surbhi

0 Kudos
MyMother
Beginner
484 Views

hi Super Surbhi ,

   Thanks for your response. And I still have several questions as below

Q1.  I see your first video to be of lower resolution than second, which means it would have requested for lesser surfaces to handle the high resolution stream. 

>> But I try to change the combining order as the following, and it looks like not problem about resolution. ~ Any idea??

[_build] $ cp 1080p_4M.h264 combined2.264           ==> create combined bit-stream

[_build] $ cat test_stream.264 >> combined2.264     ==> create combined bit-stream
[_build] $ ffmpeg -i combined2.264  -vcodec rawvideo -pix_fmt yuv420p combined2.yuv    ==> validate with ffmpeg

ffmpeg version 2.1.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Aug  5 2014 05:02:01 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  configuration: --enable-gpl --enable-nonfree --enable-pthreads --enable-x11grab --enable-version3 --enable-libx264 --enable-encoder=libx264 --enable-libvpx --disable-zlib --disable-debug
  libavutil      52. 48.101 / 52. 48.101
  libavcodec     55. 39.101 / 55. 39.101
  libavformat    55. 19.104 / 55. 19.104
  libavdevice    55.  5.100 / 55.  5.100
  libavfilter     3. 90.100 /  3. 90.100
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, h264, from 'combined2.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 1920x1080, 30 fps, 30 tbr, 1200k tbn, 60 tbc
Output #0, rawvideo, to 'combined2.yuv':
  Metadata:
    encoder         : Lavf55.19.104
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080, q=2-31, 200 kb/s, 90k tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> rawvideo)
Press to stop, [?] for help
Reinit context to 176x96, pix_fmt: yuv420ptrate=746496.0kbits/s
Input stream #0:0 frame changed from size:1920x1080 fmt:yuv420p to size:176x96 fmt:yuv420p
frame=  736 fps= 57 q=0.0 Lsize= 2235600kB time=00:00:24.53 bitrate=746496.0kbits/s
video:2235600kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.000000%
[_build] $ sudo ./simple_decode_vmem -auto combined2.264  ==> still got failure, ??
[sudo] password for dspuser:
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0

 Incompatible video parameters. src/simple_decode_vmem.cpp 200
[_build] $

Q2. Please check sample_decode part of the samples package to find details on it.  

>> I try something below and got ERROR message, I can't validate the total RAW data with my handy player ~  Do you have any idea to validate the RAW data?? 

 

[release] $ ./sample_decode_x11 h264 -i combined.264 -o combined.yuv
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Decoding Sample Version 0.0.000.0000


Input video     AVC
Output format   YUV420
Resolution      176x96
Crop X,Y,W,H    0,0,0,0
Frame rate      30.00
Memory type             system
MediaSDK impl           hw
MediaSDK version        1.16

Decoding started
Frame number:  101, fps: 199.639, fread_fps: 0.000, fwrite_fps: 4015.106
ERROR: Incompatible video parameters detected. Recovering...
Frame number:  736, fps: 34.485, fread_fps: 0.000, fwrite_fps: 35.631
Decoding finished
[release] $ ./sample_decode_x11 h264 -i combined2.264 -o combined2.yuv
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Decoding Sample Version 0.0.000.0000


Input video     AVC
Output format   YUV420
Resolution      1920x1088
Crop X,Y,W,H    0,0,0,0
Frame rate      30.00
Memory type             system
MediaSDK impl           hw
MediaSDK version        1.16

Decoding started
Frame number:  635, fps: 29.634, fread_fps: 0.000, fwrite_fps: 30.216
ERROR: Incompatible video parameters detected. Recovering...
Frame number:  736, fps: 34.290, fread_fps: 0.000, fwrite_fps: 34.982
Decoding finished
[release] $

0 Kudos
Surbhi_M_Intel
Employee
484 Views

Hi Medwin, 

As I explained before, decoder needs to be reset to decode streams with different properties, which is not handled in tutorial yet and sample_decode can be used. To check the o/p you can use rendering option by adding "-r" to your cmd line or use yuvplayer which is available for free online. 

Thanks,
Surbhi

0 Kudos
MyMother
Beginner
484 Views

hi Surbhi-giant

   Thanks for your reply.

   Q1. As your speaking, I can use sample_decode to decode and jump out the program. But I got frozen frame and hang after adding "-r" in both combined streams... And the stream can be normally displayed by ffplay. Does that means a bug or issue ??

[release] $ ./sample_decode_x11 h264 -i combined.264 -r    ==> I got frozen frame and hang
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Decoding Sample Version 0.0.000.0000


Input video     AVC
Output format   YUV420
Resolution      176x96
Crop X,Y,W,H    0,0,0,0
Frame rate      30.00
Memory type             d3d
MediaSDK impl           hw
MediaSDK version        1.16

Decoding started
Frame number:  101, fps: 1187.816, fread_fps: 0.000, fwrite_fps: 1308.527
ERROR: Incompatible video parameters detected. Recovering...
Frame number:  101, fps: 994.506, fread_fps: 0.000, fwrite_fps: 1308.5277

 

[release] $ ./sample_decode_x11 h264 -i combined2.264 -r    ==> still got frozen frame and hang
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Decoding Sample Version 0.0.000.0000

Input video     AVC
Output format   YUV420
Resolution      1920x1088
Crop X,Y,W,H    0,0,0,0
Frame rate      30.00
Memory type             d3d
MediaSDK impl           hw
MediaSDK version        1.16

Decoding started
Frame number:  635, fps: 23.663, fread_fps: 0.000, fwrite_fps: 23.676
ERROR: Incompatible video parameters detected. Recovering...
Frame number:  635, fps: 23.660, fread_fps: 0.000, fwrite_fps: 23.676

[release] $ ffplay combined.264    ==> validate with ffplay and it's normally displayed
ffplay version 2.1.1 Copyright (c) 2003-2013 the FFmpeg developers
...
Input #0, h264, from 'combined.264':KB vq=    0KB sq=    0B f=0/0
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 176x96 [SAR 1:1 DAR 11:6], 30 fps, 30 tbr, 1200k tbn, 60 tbc
[h264 @ 0x7f39d000ad80] Delayed frames seen. Reenabling low delay requires a codec flush.
[h264 @ 0x7f39d000ad80] Reinit context to 1920x1088, pix_fmt: yuv420p

 

0 Kudos
Surbhi_M_Intel
Employee
484 Views

In general, Media Server Studio Linux is focused on headless use cases as you would expect from server prospective. X11 samples illustrate how to display via X11 path, but using this isn't recommended as X11 path in the samples isn't well optimized and may contain bugs.
The reason to have x11 samples is to show one of the ways to implement rendering, it isn't a priority of the Media Server Studio. U
you might want to consider using ffplay or vlc for displaying purposes or using OpenGL, there should be ways to render using that but that will be out of scope of Media Server Studio. 

Thanks,
Surbhi

0 Kudos
MyMother
Beginner
484 Views

hi spuer-Surbhi,

    You are really so nice. Thanks. And I still have several question as below.

Q1.  Is the following function thread-safe ?? I didn't find any description about that in https://software.intel.com/sites/default/files/mediasdk-man.pdf

        // Decode a frame asychronously (returns immediately)
        sts = mfxDEC.DecodeFrameAsync(NULL, pmfxSurfaces[nIndex], &pmfxOutSurface, &syncp);

Q2. Any suggestion for thread number?? depend on platforms like core numbers or else ??

 

0 Kudos
Surbhi_M_Intel
Employee
484 Views

Hi there, 

Yes, MFXDecodeFrameAsync is a thread safe function and how threading is done is completely internal to the SDK and user doesn't control on it. Manual provides information about API structure and function and doesn't go in coding details. Feel free to ask this forum for such questions. 
Can you briefly explain what you mean by thread number, since it's internal to SDK and the underlying platform. From an application prospective you won't be able to control that. 

If you have any new questions on Media SDK which is diff than current topic being discussed please start a new thread so all the questions can be tracked separately 

Thanks,
Surbhi

0 Kudos
Reply