Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5530 Discussions

RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata
Community Manager
1,122 Views

It may already be released by someone, but I made a cooperative sample of RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3.

 

Distance can also be measured while detecting objects.

 

I'm glad if you can use it as a reference.

 

This time, it is single stick + single thread.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

By the way, sample of multistick + multithread is here.

 

https://github.com/PINTO0309/MobileNet-SSD
0 Kudos
35 Replies
idata
Community Manager
517 Views

However, at the present time it will not work with NCSDK v 2.00 +

idata
Community Manager
517 Views

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

idata
Community Manager
517 Views

How many fps were able to maintain.

idata
Community Manager
517 Views

@chicagobob123

 

 

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

 

Yes.

 

This sample uses programs created for the generation of NCSDK Ver1.

 

It seems that the call part of the API has been updated in Ver2.

 

I just did not test it with Ver.2.

 

I will make it correspond to Ver.2 in the near future.

 

How many fps were able to maintain.

 

By the way, I have not measured FPS for the following programs yet.

 

I will try it today. Please wait for a while.

 

-> MobileNet-SSD-RealSense

 

 

Movie playback and detection are performed asynchronously.

 

Detection is probably about 8FPS for single threads and about 15FPS for 3 threads.

 

FPS changes according to the input resolution of the camera.

 

In order to prioritize speed, the input image is now dropped to 320x240, so it is a coarse image.

 

-> MobileNet-SSD
idata
Community Manager
517 Views

@chicagobob123

 

MobileNet-SSD-RealSense is about 6.5 FPS.(Detection + Screen drawing)
idata
Community Manager
517 Views

6.5 at 320x240 very impressive for a PI.

idata
Community Manager
517 Views

Since the installation of NCSDK v2.05.00.02 is now finished, i will modify the Github program.

idata
Community Manager
517 Views

That would be great. Thanks.

idata
Community Manager
517 Views

@chicagobob123

 

It corresponds to NCSDK Ver2.

 

Input image is 640x480.

 

I do not know whether the measurement method is correct, the performance is still 6.5 FPS.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/77cV9fyqJ1w

 

It is an article written by me, but combined the following procedure.

 

Procedure for installing NCSDK v2.05.00.02 to RaspberryPi3(Raspbian Stretch)

 

Procedure for "make examples" NCSDK v2.05.00.02 RaspberryPi3(Raspbian Stretch) / Ubuntu16.04

 

Large screen support (over 1024x768) and multistick/multithread correspondence will be done next time.

idata
Community Manager
517 Views

The following sample program is compatible with NCSDK v 2.00.05.02 + MobileNetSSD(Caffe) + Asynchronous execution.

 

It was about 12 FPS with one stick.

 

Due to the influence of "Global Interpreter Lock", it does not seem to be able to raise the performance any further unless it corresponds to multiprocess.

 

Sample of Multistick + Multithread.

 

https://github.com/PINTO0309/MobileNet-SSD

 

https://youtu.be/_Cbt0gI8niQ

idata
Community Manager
517 Views

Hello. And I'm sorry many times.

 

It supports MultiProcessing. (Quad core)

 

It is a rough measurement result, but it is a play speed of about 25 FPS and a prediction rate of about 12 FPS.

 

It is out of frame because it is asynchronous.

 

I hope it will be helpful for everyone.

 

MultiStickSSDwithRealSense.py

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/tAf1u9DKkh4

 

I was referring to @sggriset article.

 

Thank you very much!!

 

https://ncsforum.movidius.com/discussion/950/problems-with-python-multiprocessing-using-sdk-2-0-0-4
idata
Community Manager
517 Views

@PINTO Thanks for sharing your efforts and contributions to the Intel Movidius NCS community! Can't wait to test this one out.

idata
Community Manager
517 Views

It is a soliloquy.

 

Supported for USB Camera Mode.

 

MultiStick x4 Boosted.

 

In Japan, we call it "4 bai KAI-O-KEN".

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/GedDpAc0JyQ
idata
Community Manager
517 Views

Hello everyone.

 

I tried combining Object Detection with MobileNetSSD and Transparent Background.

 

By using SSD, I succeeded transparent the background while responding to dynamically changing distances.

 

Because the load of image processing is high, although only performance of only 5FPS comes out…

 

It may be interesting to look at combination with face detection etc.

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

https://youtu.be/ApyX-mN_dYA
idata
Community Manager
517 Views

ok , I finish the same work too on a upsquared board(from intel)

idata
Community Manager
517 Views

@zufeifei

 

If you have a lot of money, that is the correct answer.

 

Performance of UP Board 2 is incomplete, so I would like you to try it on LattePanda Alpha+OpenVINO if you can.

 

I am poor so I can not try it.
idata
Community Manager
517 Views

emmm I just think it has nothing to do with it's price, you know intel would send many device for free

idata
Community Manager
517 Views

I thought that many engineers wanted to know the marginal performance.

 

Seriously, I measured the performance for each number of sticks.

 

Video playback rate and detection rate are calculated separately.

 

In 4 core RaspberryPi 3 use of 3 stick or more seems to have no effect.

 

Video device : USB Camera (No RealSense D435)

 

Input resolution : 640x480

 

Output resolution : 640x480

 

https://github.com/PINTO0309/MobileNet-SSD-RealSense

 

1 Stick = 6 FPS

 

https://youtu.be/lNbhutT8hkA

 

2 Sticks = 12 FPS

 

https://youtu.be/zuJOhKWoLwc

 

3 Sticks = 16.5 FPS

 

https://youtu.be/8UDFIJ1Z4v8

 

4 Sticks = 16.5 FPS

 

https://youtu.be/_2xIZ-IZwZc

idata
Community Manager
517 Views

Do you use a powered hub or plug in the sticks direct? If direct, how do you get them to fit.

idata
Community Manager
230 Views

@chicagobob123

 

I am sorry I can not meet your expectations.

 

I used a self-powered USB HUB.

 

With RaspberryPi alone, the voltage is not enough from the second stick.
Reply