Running Inference with a custom vgg19 model with Neural Stick 2 and Raspberry Pi 4B

publio · ‎02-17-2022

Hello, I've been following up on my previous question on how to run a custom vgg19 model on Raspberry Pi 4B. The last post offered me a few paths to follow up on my question, which I have tried all but to no success.

The Hello and Async image classification examples were only validated with the alexnet models and didn't work on my current model that was trained using transfer learning from an initial VGG 16 model.

The five lines inference script did give me results but the output is [1.] for any given image and it doesn't require a label, I am not sure if that result is representative of my model.

This is the code I used to run the five lines inference script:

import cv2 as cv
import os
import glob
from PIL import Image
import numpy as np
# Load the model
net = cv.dnn.readNet('custom_model.xml', 'custom_model.bin')

# Specify target device
net.setPreferableTarget(cv.dnn.DNN_TARGET_MYRIAD)


# Read an image
frame = np.array(Image.open(r"/home/pi/Desktop/images/image_test.png")) 
print(frame.shape)
frame = cv.resize(frame, (128, 160))
frame = np.dstack((frame, frame, frame))
#print(frame.shape)
# Prepare input blob and perform an inference
blob = cv.dnn.blobFromImage(frame, size=(128, 160), ddepth=cv.CV_8U)
net.setInput(blob)
out = net.forward()
     
print(out)

The output for any image is [[1.]]

Additionally, I also tried the guide on how to build an image classifier

#Load network 
from openvino.inference_engine import IENetwork, IECore
#load image library
import cv2
import cv2 as cv
#loading numpy
import numpy as np
import glob
from PIL import Image

#model architecture loaded using xml & bin files
model_xml = "custom_model.xml"
model_bin = "custom_model.bin"

# load the network and the models in the function
def loadNetwork(model_xml, model_bin):
    #load the network
    network = IENetwork(model_xml, model_bin)
    print(network.inputs)
    #inputs placeholder - for a specififc area for inference
    input_placeholder_key = list(network.inputs)[0]
    input_placeholder = network.inputs[input_placeholder_key]
    print(input_placeholder.shape)
    #outputs placeholder - output of that specific area
    output_placeholder_key = list(network.outputs)[0]
    output_placeholder = network.outputs[output_placeholder_key]
    print(output_placeholder.shape)
    return network, input_placeholder_key, output_placeholder_key

#define the image variables such as batch size, RGB value
#Image height and Image width
batch_size = 1
channels = 3
IMG_HEIGHT = 128
IMG_WIDTH = 160


#Call the load network function to see the results
loadNetwork(model_xml, model_bin)
# load the path to source - image
image_path = "test.PNG"

#Define a function to load the source
def load_source(path_to_image): 
    #use opencv to load the image
    image = cv2.imread(path_to_image)
    #view the image dimensions - current
    print(image.shape)
    #change the image dimensions according to the need
    #using opencv
    image = cv2.resize(image,(160,128))
    #print the new dimensions
    print(image.shape)
    return image

#Call the function and declare a new image
image_new = load_source(image_path)

#Adding batches to the image
def add_batches(image):
    image = np.expand_dims(image, 0)
    image = np.transpose(image, [0,3,1,2])
    print(image.shape)
    return image

#Call the function and declare a new image 
image_new = add_batches(image_new)

#Classify the image using a function
def classify(image):
    #specify device which we run the network on
    device = "MYRIAD"
    #network init
    network, input_placeholder_key, output_placeholder_key = loadNetwork(model_xml, model_bin)
    #using inference engine to load the network
    ie = IECore()
    #making an executable of the network
    exec_net = ie.load_network(network= network, device_name = device)
    result = exec_net.infer(inputs = {input_placeholder_key:image})
    result = result[output_placeholder_key]
    return result

 
#Display the results
result = classify(image_new)
result = result[0]
#Finding the top most result
top_result = np.argmax(result)
#Printing the top most result
print(top_result)
print(result[top_result])

The above code outputs the following error:

inf.py:18: DeprecationWarning: Reading network using constructor is deprecated. Please, use IECore.read_network() method instead
  network = IENetwork(model_xml, model_bin)
inf.py:19: DeprecationWarning: 'inputs' property of IENetwork class is deprecated. To access DataPtrs user need to use 'input_data' property of InputInfoPtr objects which can be accessed by 'input_info' property.
  print(network.inputs)
{'x': <openvino.inference_engine.ie_api.DataPtr object at 0xb588aac0>}
[1, 3, 128, 160]
[1, 1]
(1080, 1420, 3)
(128, 160, 3)
(1, 3, 128, 160)
{'x': <openvino.inference_engine.ie_api.DataPtr object at 0xb588aac0>}
[1, 3, 128, 160]
[1, 1]
E: [global] [    613295] [python3] addEvent:360	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    613295] [python3] addEventWithPerf:372	 addEvent(event, timeoutMs) method call failed with an error: 3
E: [global] [    613295] [python3] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime, 0xFFFFFFFF))
E: [ncAPI] [    613295] [python3] ncFifoReadElem:3295	Packet reading is failed.
Traceback (most recent call last):
  File "inf.py", line 85, in <module>
    result = classify(image_new)
  File "inf.py", line 79, in classify
    result = exec_net.infer(inputs = {input_placeholder_key:image})
  File "ie_api.pyx", line 1062, in openvino.inference_engine.ie_api.ExecutableNetwork.infer
  File "ie_api.pyx", line 1441, in openvino.inference_engine.ie_api.InferRequest.infer
  File "ie_api.pyx", line 1463, in openvino.inference_engine.ie_api.InferRequest.infer
RuntimeError: Failed to read output from FIFO: NC_ERROR
E: [global] [    613320] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    613320] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613341] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    613341] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613341] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    613341] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613341] [python3] addEvent:360	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    613341] [python3] addEventWithPerf:372	 addEvent(event, timeoutMs) method call failed with an error: 3
E: [global] [    613341] [python3] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime, 0xFFFFFFFF))
E: [ncAPI] [    613341] [python3] getGraphMonitorResponseValue:1902	XLink error, rc: X_LINK_ERROR
E: [global] [    613341] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    613341] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613342] [python3] addEvent:360	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    613342] [python3] addEventWithPerf:372	 addEvent(event, timeoutMs) method call failed with an error: 3
E: [global] [    613342] [python3] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime, 0xFFFFFFFF))
E: [ncAPI] [    613342] [python3] getGraphMonitorResponseValue:1902	XLink error, rc: X_LINK_ERROR
E: [global] [    613342] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_CLOSE_STREAM_REQ

E: [xLink] [    613342] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613342] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    613342] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [global] [    613342] [python3] addEvent:360	Condition failed: event->header.flags.bitField.ack != 1
E: [global] [    613342] [python3] addEventWithPerf:372	 addEvent(event, timeoutMs) method call failed with an error: 3
E: [global] [    613342] [python3] XLinkReadData:156	Condition failed: (addEventWithPerf(&event, &opTime, 0xFFFFFFFF))
E: [ncAPI] [    613342] [python3] getGraphMonitorResponseValue:1902	XLink error, rc: X_LINK_ERROR
E: [global] [    613352] [Scheduler00Thr] dispatcherEventSend:54	Write failed (header) (err -4) | event XLINK_RESET_REQ

E: [xLink] [    613353] [Scheduler00Thr] sendEvents:1132	Event sending failed
E: [xLink] [    613353] [python3] DispatcherWaitEventComplete:390	Assertion Failed: curr != NULL 

E: [global] [    613353] [python3] XLinkResetRemote:257	Condition failed: DispatcherWaitEventComplete(&link->deviceHandle, XLINK_NO_RW_TIMEOUT)
E: [ncAPI] [    623353] [python3] ncDeviceClose:1852	Device didn't appear after reboot

Lastly, I also tried running the classification_demo.py and I get the following message:

  File "classification_demo.py", line 25, in <module>
    from openvino.model_zoo.model_api.models import Classification, OutputTransform
ModuleNotFoundError: No module named 'openvino.model_zoo'

I am not sure how do I get the dependency openvino.model_zoo...

I wasn't able yet to run my model using my neural compute stick 2, I really would like to get my image classification model working...

This link has my .h5 keras model:

Files - Google Drive

This link has my .pb model converted:

Pasta - Google Drive

This link has the images used for inference:

Google Drive

And the class labels are true and false

I'd like to know what are the steps that I can use to run inference on those images above, I can't seem to get inference working on my model, pls help

Wan_Intel · ‎02-28-2022

Hi Publio,

Thanks for reaching out to us.

We noticed that you have posted a similar question here, and the issue has been resolved.

Therefore, this thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Best regards,

Wan