Tiny YOLO on NCS

[idata]

I've been trying to find a way to run an object detection network on the NCS. After a while trying and converting different models I finally found a one that seems to work; Tiny YOLO. I haven't yet found a way to convert the weights from darknet to caffe to NCS, but with empty weights the performance looks promising. Under 150ms per image!

python3 mvNCProfile.pyc yolo_tiny_deploy.prototxt -s 12

mvNCProfile v02.00, Copyright @ Movidius Ltd 2016

__ WARNING: using empty weights __

USB: Transferring Data…Time to Execute : 140.52 msUSB: Myriad Execution FinishedTime to Execute : 121.39 msUSB: Myriad Execution FinishedUSB: Myriad Connection Closing.USB: Myriad Connection Closed.Network Summary

Detailed Per Layer Profile

Layer Name MFLOPs Bandwidth MB/s time(ms)

0 scale1 173.408 1173.26 8.81

1 pool1 3.211 811.31 7.552 scale2 462.422 964.80 14.303 pool2 1.606 956.24 3.204 scale3 462.422 698.33 9.925 pool3 0.803 1008.54 1.526 scale4 462.422 428.05 8.387 pool4 0.401 1001.39 0.768 scale5 462.422 205.36 11.139 pool5 0.201 980.96 0.3910 scale6 462.422 308.66 10.0811 pool6 0.100 944.83 0.2012 scale7 462.422 809.99 11.6413 scale8 231.211 596.90 8.9814 fc9 0.025 2146.64 16.40 Total inference time 113.27

Does anyone have experience converting the weights? Or knowledge how to draw the bounding boxes based on the YOLO output layer? It appears to output 1470 values.

[idata]

i have convert small yolo from tensorflow to caffe .it seem that all things are right on the caffe ,but on the movidus ,there is some thing wrong about

Myraid error:"status Error"

[idata]

@shixinli What command did you run when that error appeared? mvNCCompile.pyc? Is that all output you get? Usually it says something like which layer failed or so.

I got my tiny YOLO running on the NCS, but I'm still trying to get the output processing right.

[idata]

Please share any installation or scripts you have used to setup TinyYOLO. I would love to get TinyYOLO working on Pi. Great work!

[idata]

@mantu @spinoza1791 I second this. I think a lot of our members could benefit from your discoveries.

[idata]

@mantu here is my result for tiny model imgur.com/a/T7Enu

I just merged yolo_main.py into NCS example. Next step would be to input video from camera.

[idata]

@spinoza1791 @Tome_at_Intel I'm still working on getting the output (box coordinates and classes) right, but I can share some code/instructions when that's done.

@dagofff Awesome! Hopefully I get the classes working soon, too.

To convert Tiny YOLO to NCS, I used caffemodel files found from this repo: https://github.com/xingwangsfu/caffe-yolo

For conversion the needed files are:

1. The prototxt model file: https://raw.githubusercontent.com/xingwangsfu/caffe-yolo/16f30c81e2ad6fc70da4dcaf7ae2bd6efd6ff6ea/prototxt/yolo_tiny_deploy.prototxt ( Thanks @dagoff for pointing out the prototxt found in the repo was incompatible with the weights file and giving the link for the correct one )
2. The weights file https://drive.google.com/file/d/0Bzy9LxvTYIgKNFEzOEdaZ3U0Nms/view?usp=sharing

Then I edited the top parameter of the final layer in yolo_tiny_deploy.prototxt, changing it from "result" to "fc12". After that the files are ready for conversion using the mvNCCompile.pyc script. So pass in the yolo_tiny_deploy.prototxt as the network parameter and yolo_tiny.caffemodel as the weights file. That should get you the NCS ready model. Then I started to modify the stream_infer.py script to work with this new model, but I recommend to do it the way @dagofff did it, so use the yolo_main.py to create the bounding boxes.

[idata]

I put my code to github https://github.com/gudovskiy/yoloNCS

[idata]

@dagofff Awesome! Thanks a lot!

[idata]

@mantu np, modify it if find some bug. I spent few hours to figure out the exact Caffe to NCS image preprocessing. Resize() was tricky.

[idata]

@dagofff Will do. I also spent hours trying to figure out how to get the image right. Good job!

P.S. Just tested the code and it works great!

[idata]

I added yolo_camera.py that processes camera input. Seems that network is producing correct results. Now the question is how to draw bounding boxes in GTK…

[idata]

@dagofff Nice. I tried to find a way to draw rectangles, but nothing worked yet. I tried to follow this example http://pygtk.org/pygtk2tutorial/examples/drawingarea.py and add it to the gtk_output.py file. I also tried using only opencv instead of gtk, so I used cv2.VideoCapture instead of cv2.imread in your yolo_example.py. But then the problem was that I couldn't get threading/multiprocessing right. Experimenting continues…

[idata]

@mantu camera app is working with threads as well without GTK stuff. So, we are done.

[idata]

Well done!

[idata]

Hey @dagofff

Good job on yoloNCS!

I am having a little issue loadueg object_detection.py

https://github.com/gudovskiy/yoloNCS/issues/1

Let me know if you know how to fix it.

Best,

Peter

[idata]

@Nyceane , I replied you about OpenCV installation and added a note at README.md

[idata]

@mantu This may be a good reference point regarding bounding boxes: https://github.com/hojel/caffe-yolo-model/blob/master/yolo_main.py

[idata]

@Tome_at_Intel Thanks, I ended up using the code by @dagofff , which works great for my purpose.

[idata]

@dagofff

Do you have any plan to enable full yolo on NCS?

[idata]

@xhuan28 - "full yolo" model is already supported here. YOLO9000 (or version 2) requires re-engineering a few layers.