OpenVino - Implementing trackers from OpenCV_Contrib

[Tipan__Steven]

Hello,

I'm utilising OpenVino on a Raspberry Pi 3B with the Intel Neural Compute Stick 2 (NCS2) to detect and track objects from a camera feed.

So far I've managed to successfully implement the detection algorithm (TinyYOLO) with the NCS2 and now I want to implement the trackers. However OpenVino doesn't include the extra modules in OpenCV_Contrib for tracking.

I've tried compiling OpenCV4 from source with the extra modules and the inference engine for the NCS2 but I get a generic segmentation fault error when I run my script.

As this dead end took several days I wanted to ask if there's another way of getting these tracking modules in OpenVino. Any help would be appreciated.

Thanks,

[Shubha_R_Intel]

Hi Steven:

The version of OpenVino is pointed to by environment variables in the bin/setupvars.bat. For Ubuntu, of course it would be a *.sh file. You should be able to point these environment variables to a different version of OpenCV which contains the trackers you require.  Is this the method that you tried ? If not, can you tell me exactly the steps which you took  ?

[Tipan__Steven]

Hi, 

I'm using Raspbian Stretch. First I compiled from source Open CV 4.0.0 with the extra modules. Next, I downloaded  The Intel Distribution of OpenVINO toolkit for Raspbian OS and changed  inference_engine_vpu_arm/bin/setupvars.sh file. Since, I had my own version of OpenCV I changed the following part:

if [ -e "$INSTALLDIR/opencv" ]; then
if [ -f "$INSTALLDIR/opencv/setupvars.sh" ]; then
    source "$INSTALLDIR/opencv/setupvars.sh"
else
    export OpenCV_DIR="$INSTALLDIR/opencv/share/OpenCV"
    export LD_LIBRARY_PATH="$INSTALLDIR/opencv/lib:$LD_LIBRARY_PATH"
    export LD_LIBRARY_PATH="$INSTALLDIR/opencv/share/OpenCV/3rdparty/lib:$LD_LIBRARY_PATH"

to 

if [ -e "$INSTALLDIR/opencv" ]; then
if [ -f "$INSTALLDIR/opencv/setupvars.sh" ]; then
    source "$INSTALLDIR/opencv/setupvars.sh"
else
    export LD_LIBRARY_PATH="/opt/opencv-4.0.0/lib:$LD_LIBRARY_PATH"
   

With that I tried to point it  to my OpenCV installation, additionally  I deleted the OpenCV included in the toolkit. Finally, I applied the changes and used the command source to make them effective ( source inference_engine_vpu_arm/bin/setupvars.sh). When trying to import opencv in python I got the following error:

ImportError /home/pi/inference_engine_vpu_arm/python/python3.5/cv2.cypthon-35m-arm-linux-gnueabihf.so:
 undefined symbol: _ZTIN2cv3dnn14dnn4_v201812055LayerE

I also tried to compile OPENCV 4.0.1 with the extra modules and inference engine in order to not use the OpenVino toolkit. But by doing that I get "segmentation fault error" when I run my script.

Thanks,

[Dmitry_K_Intel3]

Hi, Steven! I'd like to ask you to try this guide with X-compilation of OpenCV for Raspbian: https://github.com/opencv/opencv/wiki/Intel%27s-Deep-Learning-Inference-Engine-backend#raspbian-stretch

[Pozzetti__Andrea]

Hi,

i have the same problem of Steven. i want to install opencv_contrib to use trackers.

How can I change the opencv path that openvino create on installation and use my custom opencv version?
Is it possible to avoid that openvino installs opencv?

[Tipan__Steven]

Hi,

First thanks for your suggestion, I was able to make opencv_contrib work with openvino following that link. I faced some problems with some libraries since I needed it to work with Python 3.5 and the link gives information only for Python 2.7. For now, everything is working.

And Andrea, I followed the steps in the previous link (All this is for Raspberry PI): 

1. Cross-compile OpenCV with Opencv_Contrib and Inference Engine (I've got those files in case you need them, I could email you).

2. Install OpenVino for Raspberry pi, following the instructions provided by Intel.

3. Add to Path the libraries needed of OpenCV.

That worked for me, if you need more information just let me know.

[ananya_nrt]

Hi @Tipan__Steven ,

I am facing the same issue as you did with OpenCV contrib for OpenVINO. 

If you could please share our cross-compile files for OpenCV and OpenCV_contrib and inference engine. Also, if you could please assist as to how to add to path the libraries for OpenCV.

Thanks.

[Pozzetti__Andrea]

Hi Steven,
thanks for your help.
If you could send me the compiled file it would be great!! You can send them to dev(AT)fluxedo.com

Can you also explain in details step 3?

Thank you so much.

[Pozzetti__Andrea]

I'm trying to compile opencv and opencv_contrib with python3.5 but unsuccessful.
"OpenVino" is missing when i try to use it in my script.
This is the CMAKE configuration that i used.
Which is the error?

cmake -D CMAKE_BUILD_TYPE=RELEASE \
     -D CMAKE_INSTALL_PREFIX=/opt/opencv-4.0.1 \
     -D OPENCV_EXTRA_MODULES_PATH=~/opencv_all/opencv_contrib-4.0.1/modules \
     -D OPENCV_ENABLE_NONFREE=ON \
     -D PYTHON2_INCLUDE_PATH=/usr/include/python2.7 \
     -D PYTHON2_LIBRARIES=/usr/lib/arm-linux-gnueabihf/libpython2.7.so \
     -D PYTHON2_NUMPY_INCLUDE_DIRS=/usr/lib/python2/dist-packages/numpy/core/include \
     -D PYTHON3_INCLUDE_PATH=/usr/include/python3.5m \
     -D PYTHON3_LIBRARIES=/usr/lib/arm-linux-gnueabihf/libpython3.5m.so \
     -D PYTHON3_NUMPY_INCLUDE_DIRS=/usr/lib/python3/dist-packages/numpy/core/include \
     -D BUILD_OPENCV_PYTHON2=ON \
     -D BUILD_OPENCV_PYTHON3=ON \
     -D ENABLE_NEON=ON \
     -D ENABLE_VFPV3=ON \
     -D WITH_INF_ENGINE=ON \
     -D INF_ENGINE_LIB_DIRS="/root/inference_engine_vpu_arm/deployment_tools/inference_engine/lib/raspbian_9/armv7l" \
     -D INF_ENGINE_INCLUDE_DIRS="/root/inference_engine_vpu_arm/deployment_tools/inference_engine/include" \
     -D CMAKE_FIND_ROOT_PATH="/root/inference_engine_vpu_arm/inference_engine_vpu_arm/" \
     -D WIDTH_GTK=ON \
     -D BUILD_TESTS=OFF \
     -D BUILD_DOCS=OFF \
     -D BUILD_EXAMPLES=OFF ..

[hamze60]

@Pozzetti, Andrea

As I checked, if you run OpenVino's setupvars.sh script before compiling opencv, you do not need to set

-D INF_ENGINE_LIB_DIRS=....
-D INF_ENGINE_INCLUDE_DIRS=....

inference engine is automatically detected then.

[hamze60]

Hello @Tipan, Steven

I appreciate it if you can write the variables you set in step 3.  paths on cross-compile links are not valid, at least for opencv4.0.1. I tried some other variables, but not successful. when I do my own way to connect my installed opencv to openvino, it is detected and I can use trackers, but when I use NCS2, I get "segmentation fault".

[Castro__Renato]

Hi Steven, 

I was tried to configure and follow the link but some packages cannot be installed in my raspbian version. What steps did you follow to succeed? Thanks 

[hamze60]

Dear all,

I recently checked again with OpenCV4.1.0 and OpenVino 2019R1 on an ARM board (not exactly Raspberry pi). Even without cross-compile, It worked easily. The only thing I added after compiling my own OpenCV from source, to find tracker functions was running this:

export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH                #after compile opencv from source, libraries go into /usr/local/lib
export PYTHONPATH=/usr/local/lib/python3.5/dist-packages/:$PYTHONPATH

[karanda__marijiani]

ahangari, hamzeh wrote:

What were the steps exactly , did you install openvino first or opencv first beacuse i tried it a couple of times but i still can not get the extra libraries like tracking

Dear all,

I recently checked again with OpenCV4.1.0 and OpenVino 2019R1 on an ARM board (not exactly Raspberry pi). Even without cross-compile, It worked easily. The only thing I added after compiling my own OpenCV from source, to find tracker functions was running this:

export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH                #after compile opencv from source, libraries go into /usr/local/lib
export PYTHONPATH=/usr/local/lib/python3.5/dist-packages/:$PYTHONPATH

[gupta__himant]

Hello,

Can you please share exact steps of setting paths.

I have done installation of CMAKE, OPENCV with CONTRIB, and DLDT.

Still not able to use the trackers.

Can you share your setupvars file with the changes highlighted? Did you also change /opencv/setupvars.sh?

Also while compiling DLDT - the openCV_DIR is the run time directory or OpenCV? or the directory with Include/opencv4/opencv2?

Please assist  

[mjohannessen]

This is an old thread, but is there a definitive set of instructions to implement OpenCV trackers with OpenVINO and the now-obsolete NCS2 on a RPi 4b - Buster?

My understanding that the last OpenVINO to support the NCS2 was v2020.3.

I attempted to cross-compile using:

https://github.com/opencv/opencv/wiki/Intel-OpenVINO-backend#raspbian-buster

After installing opencv/opencv-contrib 4.5.5 from source:

$ python3

Python 3.7.3 (default, Oct 31 2022, 14:04:00)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.5.5'
>>> tracker = cv2.TrackerCSRT_create()
>>>

However, in a test.py script I have:

...

import cv2

net = cv2.dnn.readNetFromCaffe(_weights, _model)

net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

...

detections = net.forward()

I get the error relating to DNN_TARGET_MYRIAD:

cv2.error: OpenCV(4.5.5) /home/pi/opencv/modules/dnn/src/dnn.cpp:1414: error: (-215:Assertion failed) preferableBackend != DNN_BACKEND_OPENCV || preferableTarget == DNN_TARGET_CPU || preferableTarget == DNN_TARGET_OPENCL || preferableTarget == DNN_TARGET_OPENCL_FP16 in function 'setUpNet'

I then used this to install OpenVINO:

https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_raspbian.html

but using this version of OpenVINO (as the last to support the NCS2):

https://storage.openvinotoolkit.org/repositories/openvino/packages/2020.3/l_openvino_toolkit_runtime_raspbian_p_2020.3.194.tgz

I exported the paths to the new post cross-compiled opencv_install directory:

$ export PYTHONPATH=/home/pi/Desktop/opencv_install/lib/python2.7/dist-packages/:$PYTHONPATH

$ export PYTHONPATH=/home/pi/Desktop/opencv_install/lib/python3.7/site-packages/:$PYTHONPATH

$ export LD_LIBRARY_PATH=/home/pi/Desktop/opencv_install/lib/:$LD_LIBRARY_PATH

I set up the NCS2 with no errors :

$ sudo usermod -a -G users "$(whoami)"

$ sh /opt/intel/openvino_2020.3/install_dependencies/install_NCS_udev_rules.sh

then:

$ source /opt/intel/openvino_2020.3/bin/setupvars.sh

and then checked:

$ python3

Python 3.7.3 (default, Oct 31 2022, 14:04:00)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cv2.__version__
'4.3.0-openvino-2020.3.0'
>>> tracker = cv2.TrackerCSRT_create()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'cv2' has no attribute 'TrackerCSRT_create'
>>>

If I open a new terminal and $ source /opt/intel/openvino_2020.3/bin/setupvars.sh

then run a test.py script:

...

import cv2

net = cv2.dnn.readNetFromCaffe(_weights, _model)

net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)

...

detections = net.forward()

...

I get a segmentation fault.

Thanks for any help! I'd like to put this NCS2 to work.