Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata
Employee
2,589 Views

It may already be released by someone, but I made a cooperative sample of RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3.

 

Distance can also be measured while detecting objects.

 

I'm glad if you can use it as a reference.

 

This time, it is single stick + single thread.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

By the way, sample of multistick + multithread is here.

 

https://github.com/PINTO0309/MobileNet-SSD
0 Kudos
35 Replies
idata
Employee
1,610 Views

However, at the present time it will not work with NCSDK v 2.00 +

0 Kudos
idata
Employee
1,610 Views

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

0 Kudos
idata
Employee
1,610 Views

How many fps were able to maintain.

0 Kudos
idata
Employee
1,610 Views

@chicagobob123

 

 

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

 

Yes.

 

This sample uses programs created for the generation of NCSDK Ver1.

 

It seems that the call part of the API has been updated in Ver2.

 

I just did not test it with Ver.2.

 

I will make it correspond to Ver.2 in the near future.

 

How many fps were able to maintain.

 

By the way, I have not measured FPS for the following programs yet.

 

I will try it today. Please wait for a while.

 

-> MobileNet-SSD-RealSense

 

 

Movie playback and detection are performed asynchronously.

 

Detection is probably about 8FPS for single threads and about 15FPS for 3 threads.

 

FPS changes according to the input resolution of the camera.

 

In order to prioritize speed, the input image is now dropped to 320x240, so it is a coarse image.

 

-> MobileNet-SSD
0 Kudos
idata
Employee
1,610 Views

@chicagobob123

 

MobileNet-SSD-RealSense is about 6.5 FPS.(Detection + Screen drawing)
0 Kudos
idata
Employee
1,610 Views

6.5 at 320x240 very impressive for a PI.

0 Kudos
idata
Employee
1,610 Views

Since the installation of NCSDK v2.05.00.02 is now finished, i will modify the Github program.

0 Kudos
idata
Employee
1,610 Views

That would be great. Thanks.

0 Kudos
idata
Employee
1,610 Views

@chicagobob123

 

It corresponds to NCSDK Ver2.

 

Input image is 640x480.

 

I do not know whether the measurement method is correct, the performance is still 6.5 FPS.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/77cV9fyqJ1w

 

It is an article written by me, but combined the following procedure.

 

Procedure for installing NCSDK v2.05.00.02 to RaspberryPi3(Raspbian Stretch)

 

Procedure for "make examples" NCSDK v2.05.00.02 RaspberryPi3(Raspbian Stretch) / Ubuntu16.04

 

Large screen support (over 1024x768) and multistick/multithread correspondence will be done next time.

0 Kudos
idata
Employee
1,610 Views

The following sample program is compatible with NCSDK v 2.00.05.02 + MobileNetSSD(Caffe) + Asynchronous execution.

 

It was about 12 FPS with one stick.

 

Due to the influence of "Global Interpreter Lock", it does not seem to be able to raise the performance any further unless it corresponds to multiprocess.

 

Sample of Multistick + Multithread.

 

https://github.com/PINTO0309/MobileNet-SSD

 

https://youtu.be/_Cbt0gI8niQ

0 Kudos
idata
Employee
1,610 Views

Hello. And I'm sorry many times.

 

It supports MultiProcessing. (Quad core)

 

It is a rough measurement result, but it is a play speed of about 25 FPS and a prediction rate of about 12 FPS.

 

It is out of frame because it is asynchronous.

 

I hope it will be helpful for everyone.

 

MultiStickSSDwithRealSense.py

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/tAf1u9DKkh4

 

I was referring to @sggriset article.

 

Thank you very much!!

 

https://ncsforum.movidius.com/discussion/950/problems-with-python-multiprocessing-using-sdk-2-0-0-4
0 Kudos
idata
Employee
1,610 Views

@PINTO Thanks for sharing your efforts and contributions to the Intel Movidius NCS community! Can't wait to test this one out.

0 Kudos
idata
Employee
1,610 Views

It is a soliloquy.

 

Supported for USB Camera Mode.

 

MultiStick x4 Boosted.

 

In Japan, we call it "4 bai KAI-O-KEN".

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/GedDpAc0JyQ
0 Kudos
idata
Employee
1,610 Views

Hello everyone.

 

I tried combining Object Detection with MobileNetSSD and Transparent Background.

 

By using SSD, I succeeded transparent the background while responding to dynamically changing distances.

 

Because the load of image processing is high, although only performance of only 5FPS comes out…

 

It may be interesting to look at combination with face detection etc.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/ApyX-mN_dYA
0 Kudos
idata
Employee
1,610 Views

ok , I finish the same work too on a upsquared board(from intel)

0 Kudos
idata
Employee
1,610 Views

@zufeifei

 

If you have a lot of money, that is the correct answer.

 

Performance of UP Board 2 is incomplete, so I would like you to try it on LattePanda Alpha+OpenVINO if you can.

 

I am poor so I can not try it.
0 Kudos
idata
Employee
1,610 Views

emmm I just think it has nothing to do with it's price, you know intel would send many device for free

0 Kudos
idata
Employee
1,610 Views

I thought that many engineers wanted to know the marginal performance.

 

Seriously, I measured the performance for each number of sticks.

 

Video playback rate and detection rate are calculated separately.

 

In 4 core RaspberryPi 3 use of 3 stick or more seems to have no effect.

 

Video device : USB Camera (No RealSense D435)

 

Input resolution : 640x480

 

Output resolution : 640x480

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

1 Stick = 6 FPS

 

https://youtu.be/lNbhutT8hkA

 

2 Sticks = 12 FPS

 

https://youtu.be/zuJOhKWoLwc

 

3 Sticks = 16.5 FPS

 

https://youtu.be/8UDFIJ1Z4v8

 

4 Sticks = 16.5 FPS

 

https://youtu.be/_2xIZ-IZwZc

0 Kudos
idata
Employee
1,610 Views

Do you use a powered hub or plug in the sticks direct? If direct, how do you get them to fit.

0 Kudos
idata
Employee
1,323 Views

@chicagobob123

 

I am sorry I can not meet your expectations.

 

I used a self-powered USB HUB.

 

With RaspberryPi alone, the voltage is not enough from the second stick.
0 Kudos
Reply