Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Ishmael_Sameen
Beginner
2,061 Views

GStreamer Media SDK plugins now available in 01.org! (now with complete Windows 10 support)

Jump to solution

Hi All,

We have developed a set of GStreamer plugins that uses Intel Media SDK on Linux, and we'd like to share out our work here in the forums for those who might find this useful for their projects. This project was done for a major customer targeting the Apollo Lake platform, but we made sure that these GStreamer Media SDK plugins work on Haswell, Broadwell and Skylake platforms as well. The plugins can perform  hardware-accelerated decode, encode, VPP, transcode, rendering with zero-copy, etc. and works for all video formats currently supported by Intel Media Server Studio 2016 / 2017 Community, Essential or Professional Editions.

The validated stable versions can be cloned from Intel's official 01.org repository below:

https://github.com/01org/gstreamer-media-SDK

Windows support is completed, with full D3D11 hardware acceleration including zero-copy feature and deep color support when rendering with mfxsink. Please feel free to test it out from the below repos and report any issues you may find.

https://github.com/ishmael1985/gstreamer-media-SDK
https://github.com/ph0b/gstreamer-media-SDK

Any contributions to these repos are most welcome :)

Since I am no longer associated with Intel, please contact me via my personal email below strictly for GST-MSDK plugin issues / matters. As for other MSDK matters, it's best to contact Intel MSDK team since I do not represent Intel in official capacity.

ishmael1985@gmail.com

Regards,
Ishmael Sameen

0 Kudos
2 Solutions
benjamin_d_3
New Contributor I
1,783 Views

Hi Ishmael,

I just want to confirm that all issues that I have reported have been solved, the memory has been the same for 4 hours, we also have implemented the plugin to work with our internal application you can see the time and memory usage on the screenshot in the attachment. 

Thanks again for the help

Best Regards
Ben

 

  

View solution in original post

benjamin_d_3
New Contributor I
1,837 Views

Hi Ishmael & Mantas,

First I want to thank Ishmael for the hard work and making this great plugin, we already started using it in production and dropped VAAPI due to quality and bug issues. I don't know how you did this but it's amazing for what I can see you are the only one developing this and solved every bug we found, thanks to Mantas as well for helping us find bugs.

Back to my last test results: I did run a pipeline with udpsrc and audio is good, i did make the source mess up by making the bandwidth lower then what source was in my switch to see if it goes out of sync then after it went back in sync again, I wasnt able to reproduce Mantas issue, maybe last fix solved it. 

I tried running other sources as well like rtmp hls tcp they all worked fine.

As far as memory that seems to be good too i have a process running for 21 hours at only 110MB. 

View solution in original post

89 Replies
david123456
Beginner
1,351 Views

Hi Samseen,

Do you mean GStreamer MSDK plugins can run in Apollo lake under LINUX?

Best Regards

David

Ishmael_Sameen
Beginner
1,351 Views

Hi David,

Yes it is confirmed to work under Yocto APL-I platforms with Media SDK Embedded edition. In fact, this whole project was targeted mainly for that platform, but it will work on Haswell (current development platform running Fedora 23 with kernel 3.14.5) and CherryHill (Fedora 23 with default 4.2.3 kernel).

Pls call me Ishmael :)

Regards,

Ishmael

david123456
Beginner
1,351 Views

Hi Ishmael,

Thanks for you quick answer. Where can I download media SDK Embedded edition?

BTW,  Is your apollo lake hardware NUC6CAYH?

 

Best Regards

David

Ishmael_Sameen
Beginner
1,351 Views

Hi David,

As of this time, Media SDK Embedded Edition is not publicly available for download. I would strongly expect though that the Media SDK stack for Apollo Lake would be made publicly available this year Q3 or Q4. Till then, MSS 2016 should suffice, but you will have to bear with Haswell, Broadwell-based systems with Intel HD 4th generation minimum or Iris Pro graphics for now.

As for the target Apollo Lake platform, it was tested on an unreleased Customer Reference Board, but theoretically speaking your enquired APL NUC platform should share the same GPU class, so it should work with that platform equally as well.

Regards,

Ishmael

david123456
Beginner
1,351 Views

Hi Ishmael,

Could you show me the performance of video encode or transcode to H.264 bsaed  APL under GStreamer? We are very interesting in APL low power plateform under linux OS.

Best Regards,

David

 

高__超
Beginner
1,351 Views

Hi Ishmael

I would like to use gst-media-sdk on Haswell platform. The os is Ubuntu 14.04.04 LTS, and the version of gstreamer and its plugins is 1.8.3. I have compiled gstreamer-media-SDK according to README.md. Here is the cmake info:
gc@gc-ubuntu:~/gst-msdk/gstreamer-media-SDK/build$ cmake .. -DWITH_MSS=ON
-- The C compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- MediaSDK distribution found in /opt/intel/mediasdk
-- MediaSDK include directory: /opt/intel/mediasdk/include
-- MediaSDK library directory: /opt/intel/mediasdk/lib/lin_x64/libmfx.a
-- Checking for module 'glib-2.0'
--   Found glib-2.0, version 2.40.2
-- Checking for module 'gobject-2.0'
--   Found gobject-2.0, version 2.40.2
-- Checking for module 'gio-2.0'
--   Found gio-2.0, version 2.40.2
-- Checking for module 'gstreamer-1.0>=1.4'
--   Found gstreamer-1.0, version 1.8.3
-- Checking for module 'gstreamer-video-1.0>=1.4'
--   Found gstreamer-video-1.0, version 1.8.3
-- Checking for module 'gstreamer-allocators-1.0'
--   Found gstreamer-allocators-1.0, version 1.8.3
-- Checking for module 'libva'
--   Found libva, version 0.99.0
-- Checking for module 'libdrm'
--   Found libdrm, version 2.4.54
-- Checking for module 'libdrm_intel'
--   Found libdrm_intel, version 2.4.54
-- Checking for module 'libva-drm'
--   Found libva-drm, version 0.99.0
-- Checking for module 'libudev'
--   Found libudev, version 204
-- Checking for module 'wayland-client'
--   Found wayland-client, version 1.4.0
-- Checking for module 'egl'
--   Found egl, version 10.1.3
-- Checking for module 'wayland-egl'
--   Found wayland-egl, version 10.1.3
-- Checking for module 'xcb-dri3'
--   Found xcb-dri3, version 1.10
-- Checking for module 'xcb-present'
--   Found xcb-present, version 1.10
-- Checking for module 'x11'
--   Found x11, version 1.6.2
-- Checking for module 'xkbcommon'
--   Found xkbcommon, version 0.4.1
-- Checking for module 'xrandr'
--   Found xrandr, version 1.5.0
-- Checking for module 'gstreamer-codecparsers-1.0'
--   Found gstreamer-codecparsers-1.0, version 1.8.3
-- Checking for module 'gstreamer-pbutils-1.0'
--   Found gstreamer-pbutils-1.0, version 1.8.3
Build: release
-- Configuring done
-- Generating done
-- Build files have been written to: /home/gc/gst-msdk/gstreamer-media-SDK/build

Then I made it and installed it successfully. However, when I used the following command in README.example_usage to test it, it failed. The error msg is:
WARNING: erroneous pipeline: no element "mfxdecode"

I cleaned the cache of gst-plugins, and run gst-inspect-1.0 -b, the info is:
(gst-plugin-scanner:2767): GStreamer-WARNING **: Failed to load plugin '/usr/lib/gstreamer-1.0/libgstmfx.so': /usr/lib/gstreamer-1.0/libgstmfx.so: undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE
Blacklisted files:
  libgstmfx.so
Total count: 1 blacklisted file


Then I run ldd -r /usr/lib/gstreamer-1.0/libgstmfx.so, the info is:
    linux-vdso.so.1 =>  (0x00007fff7b351000)
    libgstreamer-1.0.so.0 => /usr/lib/libgstreamer-1.0.so.0 (0x00007fc803605000)
    libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fc803397000)
    libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fc80308f000)
    libgstvideo-1.0.so.0 => /usr/lib/libgstvideo-1.0.so.0 (0x00007fc802e20000)
    libgstbase-1.0.so.0 => /usr/lib/libgstbase-1.0.so.0 (0x00007fc802bc0000)
    libgstallocators-1.0.so.0 => /usr/lib/libgstallocators-1.0.so.0 (0x00007fc8029bc000)
    libva.so.1 => /usr/local/lib/libva.so.1 (0x00007fc80279f000)
    libdrm.so.2 => /usr/local/lib/libdrm.so.2 (0x00007fc802592000)
    libdrm_intel.so.1 => /usr/local/lib/libdrm_intel.so.1 (0x00007fc802371000)
    libva-drm.so.1 => /usr/local/lib/libva-drm.so.1 (0x00007fc80216e000)
    libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007fc801f5c000)
    libwayland-client.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007fc801d4f000)
    libEGL.so.1 => /usr/lib/x86_64-linux-gnu/mesa-egl/libEGL.so.1 (0x00007fc801b2a000)
    libwayland-egl.so.1 => /usr/lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007fc801927000)
    libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007fc80171d000)
    libgstcodecparsers-1.0.so.0 => /usr/lib/libgstcodecparsers-1.0.so.0 (0x00007fc8014da000)
    libgstpbutils-1.0.so.0 => /usr/lib/libgstpbutils-1.0.so.0 (0x00007fc8012a5000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc80108f000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc800cca000)
    libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fc800ac5000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc8007bf000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc8005b7000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc8003b2000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc800194000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fc7fff8c000)
    libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fc7ffd4d000)
    libpciaccess.so.0 => /usr/lib/x86_64-linux-gnu/libpciaccess.so.0 (0x00007fc7ffb44000)
    libcgmanager.so.0 => /lib/x86_64-linux-gnu/libcgmanager.so.0 (0x00007fc7ff928000)
    libnih.so.1 => /lib/x86_64-linux-gnu/libnih.so.1 (0x00007fc7ff710000)
    libnih-dbus.so.1 => /lib/x86_64-linux-gnu/libnih-dbus.so.1 (0x00007fc7ff506000)
    libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fc7ff2c0000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fc803b99000)
    libX11-xcb.so.1 => /usr/lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007fc7ff0be000)
    libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007fc7fed89000)
    libxcb-dri2.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0 (0x00007fc7feb83000)
    libxcb-xfixes.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-xfixes.so.0 (0x00007fc7fe97c000)
    libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fc7fe75d000)
    libwayland-server.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-server.so.0 (0x00007fc7fe54b000)
    libgbm.so.1 => /usr/lib/x86_64-linux-gnu/libgbm.so.1 (0x00007fc7fe343000)
    libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007fc7fe131000)
    libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fc7fdf26000)
    libgstaudio-1.0.so.0 => /usr/lib/libgstaudio-1.0.so.0 (0x00007fc7fdcd3000)
    libgsttag-1.0.so.0 => /usr/lib/libgsttag-1.0.so.0 (0x00007fc7fda9a000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc7fd880000)
    libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007fc7fd67c000)
    libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fc7fd475000)
undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZNKSt9exception4whatEv    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZTVN10__cxxabiv120__si_class_type_infoE    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZTISt9exception    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZNSt8ios_base4InitD1Ev    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: __gxx_personality_v0    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: __cxa_begin_catch    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZdaPv    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _Znwm    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZdlPv    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _Znam    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: xcb_dri3_pixmap_from_buffer    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: __cxa_allocate_exception    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: __cxa_throw    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: __cxa_end_catch    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: xcb_present_pixmap    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZNSt8ios_base4InitC1Ev    (/usr/lib/gstreamer-1.0/libgstmfx.so)
undefined symbol: _ZNSt9exceptionD2Ev    (/usr/lib/gstreamer-1.0/libgstmfx.so)

So how should I solve this problem?

Best Regards
Chao

Ishmael_Sameen
Beginner
1,351 Views

Hi Chao,

Thanks for letting us know about your issue. This seems to be related to incompatible xcb-dri3 libs linked against the gstmfx plugin. If you can uninstall xcb-dri3 related libraries, the plugin should work fine with the EGL render path (assuming you are also using a custom-compiled 3.14.5 kernel with the patches bundled together with MSS 2016 Professional / Community edition).

We'll fix the CMake file to enable an option for enabling / disabling DRI3 render feature since not all systems support DRI3. Meanwhile you can keep track of the latest changes from my fork below, it'll be updated more frequently compared to the main branch.

https://github.com/ishmael1985/gstreamer-media-SDK

Some fixes and new features are coming up very soon in this branch (gst-play-1.0 playbin support and glimagesink compatibility with zero-copy using dmabuf is forthcoming), so do keep track from the branch above. The changes from this branch will be normally merged back to main afterwards once the fixes / features are tested stable.

 

Regards,

Ishmael

 

高__超
Beginner
1,351 Views

Hi Ishmael,

Thank you for your reply. I tried to remove xcb-dri3 libraries, but it depends on libegl1-mesa-dev. When I removed it, cmake failed because egl was not found. So maybe I should wait for the fix of cmake files.

Best Regards,

Chao

Ishmael_Sameen
Beginner
1,351 Views

Hi All,

You can get the latest unstable updates from my fork here:

https://github.com/ishmael1985/gstreamer-media-SDK

The changes will be merged back to the official 01.org once the new changes are tested stable.

Some heavy re-factoring is currently ongoing so as to make the architecture platform-generic, and some other fixes.

Regards,

Ishmael

benjamin_d_3
New Contributor I
1,351 Views

hi Ishmael, 

We are having the same issue where Gstreamer is blacklisting libgstmfx.so and libgstvaapi.so and what I read on this thread you mentioned that it has to do something with xcb-dri3 and we should compile it without it, we were thinking to remove this lib but it depends in so many other libs.

here is the error that we get:

 

gst-inspect-1.0 -b

(gst-plugin-scanner:25782): GStreamer-WARNING **: Failed to load plugin '/usr/local/lib/gstreamer-1.0/libgstmfx.so': /usr/local/lib/gstreamer-1.0/libgstmfx.so: undefined symbol: _ZTVN10__cxxabiv120__si_class_type_infoE
Blacklisted files:
  libgstmfx.so
  libgstvaapi.so

please let me know for any suggestions or workarounds.

Regards
Ben

 

 

 

 

 

Ishmael_Sameen
Beginner
1,351 Views
Hi Benjamin, What is the hardware platform and OS you are using? We understand that for older OS's, the X11 DRI3 libraries are not readily available. So far, we have tested these plugins on Haswell / Skylake on Fedora 23 / 24 / 25 and we haven't faced issues so far. However, we would like to make sure that we can support older OS's as much as we can. The 01.org repo will be updated very soon with the latest and greatest, you can preview that release from my personal fork. We may add some new documentation regarding setting up the plugins (it's actually not much work) for Fedora versions >= 23 onwards. Your feedback would be highly appreciated on our work. Regards, Ishmael
jsatm
Beginner
1,351 Views

Hi Ishmael,

Would be nice with full Windows support as well, since both GStreamer and Media SDK area multi-platform (in the OS-sense) supporting Windows. That would make the implementation even more appealing for certain projects that need the multi-OS support.

Have you considered coordinating efforts with the other MediaSDK-for-GStreamer plugin implementations out there? That would make so much sense consolidating the energy towards a single excellent implementation and avoiding moving in different directions. It would also make it a simpler choice, instead of having to look into and test and eventually choose among the different implementations.

Most importantly, I notice that there is now a (different) set of plugins being integrated into the official GStreamer package (-bad-plugins), where the initial version is a part of the unstable 1.11.1 release from mid-January 2017. See e.g. https://gstreamer.freedesktop.org/releases/gst-plugins-bad/1.11.1.html and https://bugzilla.gnome.org/show_bug.cgi?id=770990 taken from this repo https://github.com/Oblong/gst-msdk. I do not think those plugins are at the level of those you provide.
And then there is yet another implementation (which has even been mentioned in an Intel presentation!): https://github.com/Intel-Media-SDK/gstreamer-plugins. I would really encourage that the various parts of Intel coordinate their efforts here.

Best regards,

 - Jay

PS. Have yet to try your plugins but we have a few upcoming projects that could utilize such Media SDK plugins.

Ishmael_Sameen
Beginner
1,351 Views
Hi Chao, Benjamin, We managed to reproduce your issues with an Ubuntu 16.04 LTS Broadwell setup. The issue is a stdc++ linker problem when building with CMake, hence we fixed the CMake. The fixed, stable version of the plugins can be downloaded from my personal repo link below. https://github.com/ishmael1985/gstreamer-media-SDK Please let me know if the current updated version from the abovementioned link resolves your issue. Regards, Ishmael
Ishmael_Sameen
Beginner
1,351 Views

Hi Jay,

Indeed, it would be really great if full Windows support with D3D video memory usage is enabled. It's part of the TODO list for the plugins. Stay tuned :)

The current GST-MSDK architecture of the IOTG version, which was derived from the GStreamer VAAPI architecture, was heavily modified and redesigned with Windows support in mind. However, due to the customer demand for bug fixes for the IOTG version, we haven't had the time to finally implement it.

Regarding the three separate GST-MSDK implementations, we are fully aware of them. Firstly though, I believe some customers were concerned about the license we used (LGPL), so other groups started implementing the BSD version of the MSDK plugins. The versions they developed though, were nowhere near to having the feature completeness the IOTG version has, and the main reason is because it takes a lot of time and effort to develop entirely new implementations of the plugins that have the more liberal BSD license. We cut a lot on that effort by taking a mature GStreamer plugin architecture (GST-VAAPI) and heavily redesigning certain parts of it to cater for MSDK, and keeping the good parts as well. This decision also allowed us to keep track of GST-VAAPI development with new features and bug fixes, and we can easily sync with their changes. The cost though is to keep the LGPL license since we derived from GST-VAAPI code, but our customers don't seem to be too concerned about it. Also, by separately developing the plugins from the official GStreamer repos, we don't have versioning dependencies, and our customers prefer it that way since they normally decide on a fixed GStreamer version like 1.6 or 1.8 for their proprietary software stack. Of course, we will put our effort in ensuring that the IOTG plugins will work with the latest GStreamer versions, but for now, we will focus on refinement, and of course the big feature, Windows 10 support.

 

Regards,

Ishmael

benjamin_d_3
New Contributor I
1,351 Views

Thank you Ishmael, we were able to compile from master branch and try out these great plugins you are developing, 
but we noticed one issue when trying to transcode live stream source or TS file, the output video is jerking or skipping i should say.

you may download the transcoded output below to see the issue.
mfxh264enc output:  http://albcast.tv/mfxh264enc/out_mfx264.ts

here is the pipeline I used:
gst-launch-1.0 -v filesrc location=src.ts ! tsdemux ! queue ! h264parse ! mfxdecode ! mfxvpp width=1280 height=720 deinterlace-mode=1 ! video/x-raw,framerate=25/1 ! mfxh264enc rate-control=1 bitrate=2000 ! mpegtsmux ! filesink location=out_mfx264.ts

this is the file used on the pipline above as source, you may download it here and test it on your end: http://albcast.tv/testsource/src.ts

 

we also tried transocding live stream and the results are the same when on the video with this pipeline:
 gst-launch-1.0 -v souphttpsrc location='http://xxxx:80/russchannel/winquicksync/mpegts' ! tsdemux ! queue ! h264parse ! mfxdecode ! mfxvpp width=1280 height=720 deinterlace-mode=0 ! video/x-raw,framerate=25/1 ! mfxh264enc rate-control=1 bitrate=1000 ! flvmux ! rtmpsink location='rtmp://xxxxxxx:1935/russchannel/tdsesttadda'

let me know if there is something that im missing to solve the issue

best regards
Ben

Ishmael_Sameen
Beginner
1,351 Views

Hi Benjamin,

I have tried to run the same transcoding pipeline you had (Haswell Fedora 23 setup with MSS 2016), but I was unable to reproduce the issue, the generated transcoded video for me was not jerky like the one you shared. As for the streaming link, I can't access here from the Intel network.

Can you try getting a new copy from my personal repo and try the same pipeline again? The issue you may be seeing could have been an older issue we had that was fixed since then. I know you are using an older version, since deinterlace-mode=0 is not valid anymore when using the latest mfxvpp.

https://github.com/ishmael1985/gstreamer-media-SDK

Also, when using mfxdecode, and then connecting to mfxvpp, you don't have to deinterlace, mfxdecode does an automatic advanced deinterlacing if the original encoded stream is interlaced. It will even do an automatic double frame-rate deinterlacing with 60i videos.

If the issue somehow persists, please do provide as much details as you about the platform setup and the GST logs. We'll try to help when we can with the time we can afford.

Regards,

Ishmael

benjamin_d_3
New Contributor I
1,351 Views

hi Ishmael

It looks like our hardware is quite different.
here is my environment i switched from Ubuntu to Centos thought that was the issue but still no luck:
OS CentOS Release: 7.2.1511
CPU: Xeon(R) CPU E3-1245 v5 @ 3.50GHz SKYLAKE
Intel Media Server Studio 2017 R2 Community
GStreamer version 1.10.4

vainfo:
error: can't connect to X server!
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
vainfo: VA-API version: 0.99 (libva 1.67.0.pre1)
vainfo: Driver version: 16.5.1.59511-ubit
vainfo: Supported profile and entrypoints
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: <unknown entrypoint>
      VAProfileH264ConstrainedBaseline: <unknown entrypoint>
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : <unknown entrypoint>
      VAProfileH264Main               : <unknown entrypoint>
      VAProfileH264High               : VAEntrypointVLD
and more……

my pipeline in verbose:

gst-launch-1.0 -vv rtmpsrc location=rtmp://127.0.0.1:1935/stream/oranews_HD ! flvdemux ! h264parse ! mfxh264dec ! mfxvpp width=1280 height=720 ! videoconvert ! mfxh264enc rate-control=1 bitrate=1200 ! flvmux ! rtmpsink location='rtmp://127.0.0.1:1935/pushrtmp/bench11s_mfx264'
Setting pipeline to PAUSED ...
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Pipeline is PREROLLING ...
Got context from element 'mfxench264-0': gst.mfx.Aggregator=context, gst.mfx.Aggregator=(GstMfxTaskAggregator)NULL;
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, codec_data=(buffer)014d0028ffe1001b274d40289a6280f0088fbc050808080a000003000200000300652801000428fe3880
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)25/1, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstMfxDec_h264:mfxdec_h264-0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)25/1, parsed=(boolean)true, alignment=(string)au, profile=(string)main, level=(string)4
/GstPipeline:pipeline0/GstMfxDec_h264:mfxdec_h264-0.GstPad:src: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstMfxPostproc:mfxpostproc0.GstPad:src: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstMfxEncH264:mfxench264-0.GstPad:sink: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1280, height=(int)720, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstMfxPostproc:mfxpostproc0.GstPad:sink: caps = video/x-raw(memory:MFXSurface), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstMfxEncH264:mfxench264-0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1, codec_data=(buffer)0164001fffe100242764001fac2ec05005bb01100000030010000003032e8400493000124f9bdef81b43862701000428eebcb0, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstFlvMux:flvmux0.GstPad:video: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1, codec_data=(buffer)0164001fffe100242764001fac2ec05005bb01100000030010000003032e8400493000124f9bdef81b43862701000428eebcb0, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstFlvMux:flvmux0: streamable = true
/GstPipeline:pipeline0/GstFlvMux:flvmux0.GstPad:src: caps = video/x-flv, streamheader=(buffer)< 464c5601010000000900000000, 120000f60000000000000002000a6f6e4d657461446174610800000009000c766964656f636f646563696400401c000000000000000577696474680040940000000000000006686569676874004086800000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d6572617465004039000000000000000d766964656f6461746172617465000000000000000000000f6d6574616461746163726561746f720200134753747265616d657220464c56206d75786572000c6372656174696f6e64617465020017576564204d617220382030353a30313a3035203230313700000900000101, 090000380000000000000017000000000164001fffe100242764001fac2ec05005bb01100000030010000003032e8400493000124f9bdef81b43862701000428eebcb000000043 >
/GstPipeline:pipeline0/GstRTMPSink:rtmpsink0.GstPad:sink: caps = video/x-flv, streamheader=(buffer)< 464c5601010000000900000000, 120000f60000000000000002000a6f6e4d657461446174610800000009000c766964656f636f646563696400401c000000000000000577696474680040940000000000000006686569676874004086800000000000000c417370656374526174696f58003ff0000000000000000c417370656374526174696f59003ff000000000000000096672616d6572617465004039000000000000000d766964656f6461746172617465000000000000000000000f6d6574616461746163726561746f720200134753747265616d657220464c56206d75786572000c6372656174696f6e64617465020017576564204d617220382030353a30313a3035203230313700000900000101, 090000380000000000000017000000000164001fffe100242764001fac2ec05005bb01100000030010000003032e8400493000124f9bdef81b43862701000428eebcb000000043 >
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

the video plays in fast mode then freezes you see how it play on VLC i recorded.

http://albcast.tv/mfxh264enc/mfx_output.mp4

 

 

 

Ishmael_Sameen
Beginner
1,351 Views

Hi Benjamin,

This seems to be a timestamping issue. I'll have to set up an rtmp server and try to replicate the issue. The presentation and decoding timestamps are not valid in the final encoded bitstream?

Also, does the stuttering issue you reported still exist?

You could try:

export GST_DEBUG=mfx*:6

gst-launch-1.0 -vv rtmpsrc location=rtmp://127.0.0.1:1935/stream/oranews_HD ! flvdemux ! h264parse ! avdec_h264 ! mfxh264enc rate-control=1 bitrate=1200 ! video/x-h264, profile=baseline ! flvmux ! rtmpsink location='rtmp://127.0.0.1:1935/pushrtmp/bench11s_mfx264'

Note that I used avdec_h264 in the pipeline so that I can isolate only the output from mfxh264enc.

Please do provide the logs you get as well as the output video sample that may get.

Also please update to the latest version present in my personal repo. The latest updates do fix a few encode issues.

Regards,

Ishmael

 

benjamin_d_3
New Contributor I
1,351 Views

hi Ishmael,

I don't think it is a timestamp issue as I tried transcoding same source with avdec_h264 & x264enc and output is fine. 
Also I was able to get better results when using avdec_h264 & mfxh264enc, so the problem persists only when using the mfxdecode or mfxh264dec.

about video stuttering I found out that that if don't set freame rate when using mfxh264dec it stutters but if set ! video/x-raw,framerate=25/1 ! then video plays very fast as same as http://albcast.tv/mfxh264enc/mfx_output.mp4

I updated the gstreamer-media-SDK from your repo with the last commit:
git branch –vv
master e39fea1 [origin/master] Added keyframe synchronization points in encoded bitstream to ensure smooth seeking

Please find the logs in the attachment of this post.

Thanks for your help.
Ben

Ishmael_Sameen
Beginner
326 Views

Hi Benjamin,

I have pushed a commit in my repo that may possibly resolve your encoding issues. Please update from my personal repo and let me know what you get.

Regards,

Ishmael

Reply