topic Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3 in Intel® Distribution of OpenVINO™ Toolkit

RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Tue, 10 Jul 2018 09:01:23 GMT

It may already be released by someone, but I made a cooperative sample of RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3.

Distance can also be measured while detecting objects.

I'm glad if you can use it as a reference.

This time, it is single stick + single thread.

https://github.com/PINTO0309/MobileNet-SSD-RealSense

By the way, sample of multistick + multithread is here.

https://github.com/PINTO0309/MobileNet-SSD

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Tue, 10 Jul 2018 11:15:09 GMT

However, at the present time it will not work with NCSDK v 2.00 +

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 01:31:40 GMT

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 01:46:42 GMT

How many fps were able to maintain.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 04:13:51 GMT

@chicagobob123

PINTO why will it not work with 2.0 +? Is it because of the initialization ?

Yes.

This sample uses programs created for the generation of NCSDK Ver1.

It seems that the call part of the API has been updated in Ver2.

I just did not test it with Ver.2.

I will make it correspond to Ver.2 in the near future.

How many fps were able to maintain.

By the way, I have not measured FPS for the following programs yet.

I will try it today. Please wait for a while.

-> MobileNet-SSD-RealSense

Movie playback and detection are performed asynchronously.

Detection is probably about 8FPS for single threads and about 15FPS for 3 threads.

FPS changes according to the input resolution of the camera.

In order to prioritize speed, the input image is now dropped to 320x240, so it is a coarse image.

-> MobileNet-SSD

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 17:36:26 GMT

@chicagobob123

MobileNet-SSD-RealSense is about 6.5 FPS.（Detection + Screen drawing）

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 21:13:50 GMT

6.5 at 320x240 very impressive for a PI.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Wed, 11 Jul 2018 21:47:07 GMT

Since the installation of NCSDK v2.05.00.02 is now finished, i will modify the Github program.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Thu, 12 Jul 2018 01:19:15 GMT

That would be great. Thanks.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Sat, 14 Jul 2018 09:43:08 GMT

@chicagobob123

It corresponds to NCSDK Ver2.

Input image is 640x480.

I do not know whether the measurement method is correct, the performance is still 6.5 FPS.

https://github.com/PINTO0309/MobileNet-SSD-RealSense

https://youtu.be/77cV9fyqJ1w

It is an article written by me, but combined the following procedure.

Procedure for installing NCSDK v2.05.00.02 to RaspberryPi3(Raspbian Stretch)

Procedure for "make examples" NCSDK v2.05.00.02 RaspberryPi3(Raspbian Stretch) / Ubuntu16.04

Large screen support (over 1024x768) and multistick/multithread correspondence will be done next time.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Thu, 19 Jul 2018 11:09:59 GMT

The following sample program is compatible with NCSDK v 2.00.05.02 + MobileNetSSD(Caffe) + Asynchronous execution.

It was about 12 FPS with one stick.

Due to the influence of "Global Interpreter Lock", it does not seem to be able to raise the performance any further unless it corresponds to multiprocess.

Sample of Multistick + Multithread.

https://github.com/PINTO0309/MobileNet-SSD

https://youtu.be/_Cbt0gI8niQ

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Sat, 21 Jul 2018 00:27:59 GMT

Hello. And I'm sorry many times.

It supports MultiProcessing. (Quad core)

It is a rough measurement result, but it is a play speed of about 25 FPS and a prediction rate of about 12 FPS.

It is out of frame because it is asynchronous.

I hope it will be helpful for everyone.

【MultiStickSSDwithRealSense.py】

https://github.com/PINTO0309/MobileNet-SSD-RealSense

https://youtu.be/tAf1u9DKkh4

I was referring to @sggriset article.

Thank you very much!!

https://ncsforum.movidius.com/discussion/950/problems-with-python-multiprocessing-using-sdk-2-0-0-4

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Sat, 21 Jul 2018 03:44:11 GMT

@PINTO Thanks for sharing your efforts and contributions to the Intel Movidius NCS community! Can't wait to test this one out.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Mon, 23 Jul 2018 16:34:53 GMT

It is a soliloquy.

Supported for USB Camera Mode.

MultiStick x4 Boosted.

In Japan, we call it "4 bai KAI-O-KEN".

https://github.com/PINTO0309/MobileNet-SSD-RealSense

https://youtu.be/GedDpAc0JyQ

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Sat, 04 Aug 2018 08:25:46 GMT

Hello everyone.

I tried combining Object Detection with MobileNetSSD and Transparent Background.

By using SSD, I succeeded transparent the background while responding to dynamically changing distances.

Because the load of image processing is high, although only performance of only 5FPS comes out…

It may be interesting to look at combination with face detection etc.

https://github.com/PINTO0309/MobileNet-SSD-RealSense

https://youtu.be/ApyX-mN_dYA

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Mon, 13 Aug 2018 13:02:11 GMT

ok , I finish the same work too on a upsquared board(from intel)

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Mon, 13 Aug 2018 14:43:28 GMT

@zufeifei

If you have a lot of money, that is the correct answer.

Performance of UP Board 2 is incomplete, so I would like you to try it on LattePanda Alpha+OpenVINO if you can.

I am poor so I can not try it.

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Mon, 13 Aug 2018 21:23:54 GMT

emmm I just think it has nothing to do with it's price, you know intel would send many device for free

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Sat, 25 Aug 2018 11:14:09 GMT

I thought that many engineers wanted to know the marginal performance.

Seriously, I measured the performance for each number of sticks.

Video playback rate and detection rate are calculated separately.

In 4 core RaspberryPi 3 use of 3 stick or more seems to have no effect.

Video device　：　USB Camera (No RealSense D435)

Input resolution　：　640x480

Output resolution　：　640x480

https://github.com/PINTO0309/MobileNet-SSD-RealSense

1 Stick = 6 FPS

https://youtu.be/lNbhutT8hkA

2 Sticks = 12 FPS

https://youtu.be/zuJOhKWoLwc

3 Sticks = 16.5 FPS

https://youtu.be/8UDFIJ1Z4v8

4 Sticks = 16.5 FPS

https://youtu.be/_2xIZ-IZwZc

Re: RealSense D435 + Neural Compute Stick + MobileNet-SSD + RaspberryPi3

idata — Mon, 27 Aug 2018 05:22:28 GMT

Do you use a powered hub or plug in the sticks direct? If direct, how do you get them to fit.