RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

[idata]

It may already be released by someone, but I made a cooperative sample of RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3.

Distance can also be measured while detecting objects.I'm glad if you can use it as a reference.This time, it is single stick + single thread.https://github.com/PINTO0309/MobileNet-SSD-RealSense

By the way, sample of multistick + multithread is here.

https://github.com/PINTO0309/MobileNet-SSD

[idata]

However, at the present time it will not work with NCSDK v 2.00 +

[idata]

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

[idata]

How many fps were able to maintain.

[idata]

@chicagobob123

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

Yes. This sample uses programs created for the generation of NCSDK Ver1. It seems that the call part of the API has been updated in Ver2. I just did not test it with Ver.2. I will make it correspond to Ver.2 in the near future.

How many fps were able to maintain.

By the way, I have not measured FPS for the following programs yet. I will try it today. Please wait for a while. -> MobileNet-SSD-RealSense

Movie playback and detection are performed asynchronously.

Detection is probably about 8FPS for single threads and about 15FPS for 3 threads.FPS changes according to the input resolution of the camera.In order to prioritize speed, the input image is now dropped to 320x240, so it is a coarse image.-> MobileNet-SSD

[idata]

@chicagobob123

MobileNet-SSD-RealSense is about 6.5 FPS.（Detection + Screen drawing）

[idata]

6.5 at 320x240 very impressive for a PI.

[idata]

Since the installation of NCSDK v2.05.00.02 is now finished, i will modify the Github program.

[idata]

That would be great. Thanks.

[idata]

@chicagobob123

It corresponds to NCSDK Ver2.Input image is 640x480.I do not know whether the measurement method is correct, the performance is still 6.5 FPS.https://github.com/PINTO0309/MobileNet-SSD-RealSensehttps://youtu.be/77cV9fyqJ1w

It is an article written by me, but combined the following procedure.

Procedure for installing NCSDK v2.05.00.02 to RaspberryPi3(Raspbian Stretch)Procedure for "make examples" NCSDK v2.05.00.02 RaspberryPi3(Raspbian Stretch) / Ubuntu16.04

Large screen support (over 1024x768) and multistick/multithread correspondence will be done next time.

[idata]

The following sample program is compatible with NCSDK v 2.00.05.02 + MobileNetSSD(Caffe) + Asynchronous execution.

It was about 12 FPS with one stick.Due to the influence of "Global Interpreter Lock", it does not seem to be able to raise the performance any further unless it corresponds to multiprocess.

Sample of Multistick + Multithread.

https://github.com/PINTO0309/MobileNet-SSD

https://youtu.be/_Cbt0gI8niQ

[idata]

Hello. And I'm sorry many times.

It supports MultiProcessing. (Quad core)It is a rough measurement result, but it is a play speed of about 25 FPS and a prediction rate of about 12 FPS.It is out of frame because it is asynchronous.I hope it will be helpful for everyone.【MultiStickSSDwithRealSense.py】https://github.com/PINTO0309/MobileNet-SSD-RealSensehttps://youtu.be/tAf1u9DKkh4

I was referring to @sggriset article.

Thank you very much!!https://ncsforum.movidius.com/discussion/950/problems-with-python-multiprocessing-using-sdk-2-0-0-4

[idata]

@PINTO Thanks for sharing your efforts and contributions to the Intel Movidius NCS community! Can't wait to test this one out.

[idata]

It is a soliloquy.

Supported for USB Camera Mode.MultiStick x4 Boosted.In Japan, we call it "4 bai KAI-O-KEN".https://github.com/PINTO0309/MobileNet-SSD-RealSensehttps://youtu.be/GedDpAc0JyQ

[idata]

Hello everyone.

I tried combining Object Detection with MobileNetSSD and Transparent Background.By using SSD, I succeeded transparent the background while responding to dynamically changing distances.Because the load of image processing is high, although only performance of only 5FPS comes out…It may be interesting to look at combination with face detection etc.https://github.com/PINTO0309/MobileNet-SSD-RealSensehttps://youtu.be/ApyX-mN_dYA

[idata]

ok , I finish the same work too on a upsquared board(from intel)

[idata]

@zufeifei

If you have a lot of money, that is the correct answer.Performance of UP Board 2 is incomplete, so I would like you to try it on LattePanda Alpha+OpenVINO if you can.I am poor so I can not try it.

[idata]

emmm I just think it has nothing to do with it's price, you know intel would send many device for free

[idata]

I thought that many engineers wanted to know the marginal performance.

Seriously, I measured the performance for each number of sticks.Video playback rate and detection rate are calculated separately.In 4 core RaspberryPi 3 use of 3 stick or more seems to have no effect.Video device　：　USB Camera (No RealSense D435)Input resolution　：　640x480Output resolution　：　640x480https://github.com/PINTO0309/MobileNet-SSD-RealSense1 Stick = 6 FPS

https://youtu.be/lNbhutT8hkA

2 Sticks = 12 FPS

https://youtu.be/zuJOhKWoLwc

3 Sticks = 16.5 FPS

https://youtu.be/8UDFIJ1Z4v8

4 Sticks = 16.5 FPS

https://youtu.be/_2xIZ-IZwZc

[idata]

Do you use a powered hub or plug in the sticks direct? If direct, how do you get them to fit.

[idata]

@chicagobob123

I am sorry I can not meet your expectations.I used a self-powered USB HUB.With RaspberryPi alone, the voltage is not enough from the second stick.