Image manipulation for DLStreamer pipeline.

[Singh__Prateek]

Hi,

I am using DLStreamer for creating an end-to-end pipeline that uses gvadetect and multiple gvaclassify.  Few questions related to it:

1 - Can I resize every input frame before gvadetect using a callback? If not then what would be the suggested way to achieve this?

2 -Currently ROIs from gvadetect are fed into gvaclassify, is there a way to rotate the cropped image which is fed into gvaclassify?

3 - I'm aware of that JSON method that is used to preprocess, is there an equivalent way using C++ callbacks which allows me to preprocess the video frame before it is sent to a gva element?

Thanks,
Prateek.

[Iffa_Intel]

Greetings,

We are currently investigating this and will get back to you asap.

Sincerely,

Iffa

[Iffa_Intel]

Hi,

Here are the answers to your questions,

Question 1

Input frame resize before gvadetect is achievable using this command: videoscale ! video/x-raw,width=640,height=360

We suggest that you take a look into the "videoscale" element of gstreamer for further reference in this link: resize - How to change the resolution of image in gstbuffer? - Stack Overflow

Question 2

The architecture of DL Streamers was designed to capture gvdetect ROI as input for the gvaclassify model.

It is possible with codes and algorithms tweaking but such request is not officially supported from OpenVINO toolkit perspective.

You also refer to this link for further exploration: dlstreamer_gst/region_of_interest.h at master · openvinotoolkit/dlstreamer_gst (github.com)

Note:

The DL Streamer pipeline

1. gvadetect detects the ROIs in the video and outputs ROIs with the appropriate attributes (person, vehicle, bike) according to its model-proc on every 10th frame, due to inference-interval=10.

2. gvatrack tracks each object detected by gvadetect.

3. gvadetect ROIs are used as inputs for the gvaclassify model.

4. gvaclassify classifies the ROIs and outputs additional attributes according to model-proc, but skips classification for already classified objects for 10 frames, using tracking information from gvatrack to determine whether to classify an object:

• object-class tells gvalcassify which ROIs to classify.

• object-class=vehicle classifies ROIs that have the 'vehicle' attribute.

• reclassify-interval determines how often to reclassify tracked objects. Only valid when used in conjunction with gvatrack.

Question 3

If the goal is to set up a callback on buffer passing through any GVA element and runs video analytics pipeline using GVA API, the sample can be found here under Custom post-processing tutorial - GStreamer Video Analytics (GVA) Plugin: GStreamer Video Analytics API reference (openvinotoolkit.github.io)

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 

Sincerely,

Iffa