Minimalistic NCSDK v2 Example

idata · ‎05-18-2018

Hi Everyone,

I was learning to use the new SDK and am looking forward to using it some of my projects.

I made the following python script as I was learning to learn how to use it. It's well documented with some notes on gotchas and pitfalls.

I am planning to write a blog post on getting started with the SDK soon. Do let me know if you have any feedback or are interested in the blogpost!

The exact code for svhn_data_utils.py is at: https://github.com/gokul-uf/talks/blob/master/Intel%20Adv%20Attacks/svhn_data_utils.py

Cheers :smile:

# Sample code for using the ncsdk2 API
# We assume that you have used the CLI tools (mvNCCheck / mvNCProfile / mvNCCompile)
# to generate the necessary files and is in a folder named `saved_models`

# we use SVHN for our demo, and the exact implementation of SVHN_Processor is not relevant
# All you need to know is it returns normalized images and labels as NumPy arrays

__author__ = "gokul.uf@gmail.com"

import numpy as np
import os

import mvnc.mvncapi as mvnc

from svhn_data_utils import SVHN_Processor


graph_file = "saved_model/graph"

if __name__ == "__main__":


  # 1. open the device
  devices = mvnc.enumerate_devices()
  device = mvnc.Device(devices[0])
  device.open()

  # 2. open the graph file 
  with open(graph_file, "rb") as f:
    graph_blob = f.read()

  # 3. load graph onto device and get I/O queues, input_fifo is WO, output_fifo is RO
  # It is important to set input_fifo_num_elem and output_fifo_num_elem else inference 
  # will block if you add too many elements to the queues

  # Alternatively, always read out the prediction of an input before feeding in the next one

  graph = mvnc.Graph("cnn_classifier")
  input_fifo, output_fifo = graph.allocate_with_fifos(device = device, graph_buffer = graph_blob,
                              input_fifo_num_elem = 10, output_fifo_num_elem = 10) 

  # 4. get images for testing
  svhn_processor = SVHN_Processor(data_dir = "svhn", batch_size = 10)
  train_yielder  = svhn_processor.get_train_batch()
  val_yielder    = svhn_processor.get_val_batch()

  batch_img, batch_label = next(train_yielder)
  print(f"Label: {batch_label}")

  print(f"capacity: {input_fifo.get_option(mvnc.FifoOption.RO_CAPACITY)}")
  # 5. do forward-pass on the device

  # All I/O to and from the stick are handled via FIFO-Queues
  # So, you feed in one or more inputs with the `input_fifo` object
  # And you read the outputs, one-by-one from the `output_fifo` object
  # graph.allocate_with_fifos` and `graph.queue_inference_with_fifo_elem` are helper
  # functions to make your life easier.

  # 5a. feed in multiple images one by one
  for i in range(len(batch_img)):
    print(f"image: {i}")
    graph.queue_inference_with_fifo_elem(input_fifo, output_fifo, batch_img[i], None)

  # 5b. read back the predictions one by one, read until output fifo is empty
  print("reading the results")
  for _ in range(len(batch_img)):
    # user_obj is None because we sent None, can be used for other purpose to tag each input.
    output, user_obj = output_fifo.read_elem() 
    print(f"Preds: {output}")

    # get time for previous inference
    inference_time = graph.get_option(mvnc.GraphOption.RO_TIME_TAKEN)
    # returns an array with layer-wise inference times, take `np.sum` for total 
    print(f"Inference Time: {np.sum(inference_time)}")

  # 6. All Done! Now deallocate, the order is important!
  print("All DONE! Shutting down the device") 
  input_fifo.destroy()
  output_fifo.destroy()
  print("FIFOs destroyed!")

  graph.destroy()
  print("Graph Destroyed!")

  device.close()
  print("Device Closed!")
  device.destroy()
  print("Device Destroyed!")

idata · ‎05-19-2018

Thanks for sharing your example @gokul_uf! If anyone is interested in more examples that use the NCAPI v2, you can find quite a few in the ncsdk2 branch of the ncappzoo repository here: https://github.com/movidius/ncappzoo/tree/ncsdk2

And for anyone that has existing NCAPI v1 code that they want to continue to use with the new release but aren't ready to convert it yet, there is a shim module there that will let them accomplish that very little work: https://github.com/movidius/ncappzoo/tree/ncsdk2/ncapi2_shim

idata · ‎05-23-2018

@gokul_uf Thank you for the share.

I've been trying to a similar script with a video stream and inception_v3 but end up with an invalid_data_length error, did you have any of these?

When I print the size of the input tensor (ndarray) it gives this: (1, 299, 299, 3)

Which is the input size of the model.