Link Copied
HI Hairul,
I do know how to install the drivers, on Windows and Linux. My problem is not how to run sample models, i have just to read the pages the describe the process into the intel website. The installation process is explained.
What i found is complitely missing is an explanation of when someone would like to use his own tensorflow model. I would like to get it clear in mind how to start from a simple .py code -> convert it into IR format and then --> execute it with the stick.
The stick looks to me like something that can run only prebuild models I can't find documentation that help me on starting from .py to use the stick properly. There is something that I'm missing about this device.
When you wrote :
"To perform the inference samples and demos on Intel ® Neural Compute Stick 2 (NCS2),use the -d MYRIAD device flag when executing the samples or demos."
This is fine is easy and straight forward, i already run sumples on the stick, but like a mokey pressing button without having a clear understanding about what i was doing. I want to run my model, my code.
Can you please give me a precise list of action, step by step instruction on how to run this code on the stick:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
class Antirectifier(layers.Layer):
def __init__(self, initializer="he_normal", **kwargs):
super(Antirectifier, self).__init__(**kwargs)
self.initializer = keras.initializers.get(initializer)
def build(self, input_shape):
output_dim = input_shape[-1]
self.kernel = self.add_weight(
shape=(output_dim * 2, output_dim),
initializer=self.initializer,
name="kernel",
trainable=True,
)
def call(self, inputs):
inputs -= tf.reduce_mean(inputs, axis=-1, keepdims=True)
pos = tf.nn.relu(inputs)
neg = tf.nn.relu(-inputs)
concatenated = tf.concat([pos, neg], axis=-1)
mixed = tf.matmul(concatenated, self.kernel)
return mixed
def get_config(self):
# Implement get_config to enable serialization. This is optional.
base_config = super(Antirectifier, self).get_config()
config = {"initializer": keras.initializers.serialize(self.initializer)}
return dict(list(base_config.items()) + list(config.items()))
# Training parameters
batch_size = 128
num_classes = 10
epochs = 3
# The data, split between train and test sets
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1, 784)
x_test = x_test.reshape(-1, 784)
x_train = x_train.astype("float32")
x_test = x_test.astype("float32")
x_train /= 255
x_test /= 255
print(x_train.shape[0], "train samples")
print(x_test.shape[0], "test samples")
# Build the model
model = keras.Sequential(
[
keras.Input(shape=(784,)),
layers.Dense(256),
Antirectifier(),
layers.Dense(256),
Antirectifier(),
layers.Dropout(0.5),
layers.Dense(10),
]
)
# Compile the model
model.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.RMSprop(),
metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
# Train the model
model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=0.15)
# Test the model
model.evaluate(x_test, y_test)
STEP 1 MODEL OPTIMIZER execute this mo-tf.py (that i suppose it does the job of executing the conversion)
- -output dir <OUTPUT " width="778" height="88" border="0">
i convert the file into IR format, so i will have three files.
STEP 2
????
First and foremost, you need to understand & distinguish between a model and inferencing code.
The origin of a model could come from PyTorch, onnx, etc.
This is where you can see they are in a Python script (model.py) which later on will be saved in the specific model format (eg: model.onnx, model.caffe, etc).
Meanwhile, the inferencing code (from OpenVINO perspective) uses the OpenVINO Inference Engine library combined with other required libraries, designed to interpret the model into the desired representation
(eg: object detection, classification,etc).
These are the steps to infer a model with MYRIAD:
1. Download a model(I'm using OpenVINO Model Downloader): python downloader.py --name yolo-v4-tf
2. Convert the model from its native format into IR(xml and bin): python converter.py --name yolo-v4-tf
Note: I'm using a model officially downloaded using OpenVINO Model Downloader, hence, I can use this conversion script.
If you are using a third-party model (downloaded from somewhere else or custom-made) you'll need to use Model Optimizer.
You may refer here: Converting a Tensorflow Model
Ensure your Tensorflow topology is listed in the Supported Topology section, else, it's not supported.
If your model is not frozen, you'll need to do so. All details including baby steps are available in that documentation.
Notice that there are 2 new folders with the name FP32 and FP16. These are the IR files folder. FP32 and FP16 are the names of the IR datatype. Make sure you have 3 files in it (XML, bin, mapping).
3. Infer the IR using OpenVINO Benchmark App (simplest inference sample app ever for testing): python benchmark_app.py -m "C:\Program Files (x86)\Intel\openvino_2021.4.752\deployment_tools\open_model_zoo\tools\downloader\public\yolo-v4-tf\FP16\yolo-v4-tf.xml" -i C:\Users\A221LPEN\Desktop\frenchmotorway.jpg -d MYRIAD
Parse the parameter -d MYRIAD in the command to infer on NCS2.
Please take a note, for MYRIAD, you must use the FP16 format.
If you managed to reach this stage, then you can graduate from the baby steps and proceed into the advanced process.
You need to know when to specify input shape and when to reverse input channels which vary according to your "custom model".
Feed these arguments with the Model Optimizer command and if you fulfill all the requirements, you'll get proper IR files.
All inferencing steps concepts are the same as I had shown above.
Hope this helps!
Sincerely,
Iffa
