Q&A on OpenVINO DL Streamer in 2020.2

[Zoe_C_Intel]

Hey everyone!

Last Thursday, we hosted a live technical deep dive webinar introducing the new OpenVINO DL Streamer. We had such a vibrant and engaged audience. Thanks to everyone who attended live and participated in the Q&A. We received so many great questions, so we're publishing a few of those questions we answered live and answering the questions we didn't get to here!

If you're interested in participating in future technical deep dive webinars, go here. If you missed last Thursday's webinar, you can still watch it on-demand.

Please join us and continue the conversation below!

1. Is DL Streamer open source? Where is the source repository? DL Streamer source code is available on GitHub at https://github.com/opencv/gst-video-analytics. Currently, we do not have plans to rename this repository.
2. What is the level of C++ expertise needed to program with DL Streamer? DL Streamer supports both C++ and Python APIs. So, you can develop application in Python. DL Streamer samples folder has both C++ and python samples. A developer needs to have very basic knowledge of C++ to build C++ application using DL Streamer.
3. Where is the documentation for DL Streamer samples? DL Streamer samples documentation is available here.
4. Is there a tutorial for DL Streamer? Please refer to the links here: https://docs.openvinotoolkit.org/latest/index.html
5. Can you pass live video stream as an input to the pipeline? DL Streamer supports RTSP or web URI, webcamera device and local video files as input source. The samples in DL Streamer/ samples folder will show all the three input sources.
6. Can DL Streamer support more than one sensor (camera)? DL Streamer pipeline (and GStreamer in general) may contain multiple sub-pipelines each processing one source. Refer to benchmark sample to understand how to build multi-source pipeline.
7. Is the output of the face detection the face itself or the coordinates of the face? Is everything else cropped out? The output of gvadetect element is original video frame with metadata objects (GstVideoRegionOfInterestMeta) attached to it. Metadata describes the bounding box, confidence, object class, and other attributes.
8. Does DL Streamer - GVA plugin support and respect timestamps? e.g. to drop frames/processing in order to stay in sync with real-time playback/processing. gvadetect element provides the following experimental property: no-block: (Experimental) Option to help maintain frames per second of incoming stream. Skips inference on an incoming frame if all inference requests are currently processing outstanding frames. See https://github.com/opencv/gst-video-analytics/wiki/gvadetect for full list of properties.
9. Can DL Streamer elements be configured to process only every Nth frame? Yes, you can use inference-interval property available in all inference elements to process only every Nth frame.
10. Does gvatrack support tracking of various object types, for example persons, screws, cars, etc.? gvatrack can be used for tracking, in general. It should work reasonably well on various object types (cars, persons, etc.)
11. Can I add pre- and/or post-processing to the certain gva-plugins in the pipeline? Yes, please refer to how to set C++/Python callback function for custom pre- or post-processing.
12. Should labels in the model-proc JSON file be in the same order as COCO labels, for object detection model trained on COCO dataset Yes, here is s model-proc file example for object detection model trained on COCO dataset.
13. Can DL Streamer be used to analyze texts or links? gvainfernece element can be used for this purpose. It would also require custom post processing of inference results, on C++ or Python, see https://github.com/opencv/gst-video-analytics/wiki/Custom-Processing
14. Can I use several cascaded (serialized) models? A pipeline consisting of detection followed by multiple classification models is supported by gvadetect and gvaclassify elements in DL Streamer.
15. Can I run one inference element of the pipeline on the CPU and another one on the GPU? Yes, different DL Streamer inference elements in the pipeline can run on different processing units such as CPU, GPU, VPU. Each element has a property named 'device' which is used for specifying the target device/ processing unit for that element.
16. Can DL Streamer elements run in parallel with other elements in pipeline? Inference elements are asynchronous elements. Internally they buffer multiple frames for parallel and asynchronous processing.
17. What are the advantages of using the DL Streamer compared to directly using an API call to the OpenVINO Inference Engine? DL Streamer helps to achieve better performance and significant reduction in development time. It provides highly optimized building blocks (GStreamer elements) that you can easily configure and connect with other GStreamer elements (input/output, video decode) to build end-to-end pipeline. Also, there is no need to implement any pipeline management (data flow, parallelization, etc) as GStreamer framework provides it out-of-box.
18. Are the age and emotion algorithms, used in samples,  intel algorithms or something off the shelf? Age and gender classification models are part of Intel Model Zoo: age-gender-recognition-retail-0013.md
19. Can you provide more details on publishing to MQTT/ Kafka? Please refer to gvametapublish samples in gst-video-analytics open-source repository.
20. Will an end to end example of DL Streamer be available? Including the behind code to ensemble everything?Refer to the following reference designs: Smart City: https://github.com/OpenVisualCloud/Smart-City-Sample; Ad Insertion: https://github.com/OpenVisualCloud/Ad-Insertion-Sample
21. Does DL Streamer support Raspberry Pi? The binary version is not validated on Raspberry Pi, but you can try build open-source version of
22. Does DL Streamer use Intel Threading Building Blocks library? Intel Threading Building Blocks is used by OpenVINO Inference Engine for inference on CPU and DL Streamer uses Inference Engine as a backend for executing inference.
23. Can the DL Streamer - GVA plugins process audio and video data? DL Streamer currently supports video data processing. Audio will be supported in the near future.

Did you find this useful? Do you have any other ideas or questions? Please let us know below!

Give OpenVINO DL Streamer a try by downloading the latest version of OpenVINO 2020.2