Problems applying self-trained ResNet50 in NCS2

[alexalex35]

When I use the ResNet50 trained by Matlab to classify images using OpenVino samples (hello_classification.py), There is a problem with the accuracy of the judgment.

My model has a recall value of 89% on MATLAB
But only 40% left in NCS2 deployment, I think it's a big difference but I don't know where is the problem
I also used the same IR file on the CPU but the result is the same as the NCS2

Is it that OpenVINO does not support neural networks trained by Matlab?

[Zulkifli_Intel]

Hello Alex,

Thank you for reaching out to us.

You can train your model on MATLAB, export your model to ONNX, and use OpenVINO's model optimizer to optimize the model.

If it is possible, please share with us your ONNX model and the Model Optimizer command you used to convert the model for replication purposes.  

Sincerely,

Zulkifli 

[alexalex35]

Hello Zulkifli,

Thanks for your reply.

In fact, I trained the model on MATLAB and converted it to ONNX , and then converted it to IR file.

My ONNX Model cannot be used as an attachment because it exceeds 71MB. Could you please leave an E-mail for me to send the file to you?

The following is my process of converting onnx to IR file:
python mo.py --input_model D:\mydata\Desktop\openvino\seg_traindata\resnet_50.onnx --input_shape [1,3,224,224] --mean_values=[104.0,117.0,123.0] --data_type FP16 --reverse_input_channels --output_dir D:\mydata\Desktop\openvino\seg_traindata\

[Zulkifli_Intel]

Hello Alex,

Thank you for sharing the MO command with us.

Here is my email address:

zulkiflix.bin.abdul.halim@intel.com

Sincerely,

Zulkifli

[alexalex35]

Hello Zulkifli,

I have just E-mailed my ONNX file to your mailbox.

I am using the pre-trained ResNet50 model on MATLAB to train the wafer defect discrimination I want to classify, and I am successful on MATLAB.
I also use GoogleNet v1, and its results are better than ResNet50, but there is a gap compared to the results on MATLAB.

Will converting the file to an IR file itself will reduce the accuracy of the model and then judge the image?

[Zulkifli_Intel]

Hello Alex,

Can you tell us which tool did you use to check the accuracy of both models? Are you using Accuracy Checker?

Please share with us the type of dataset you used, is it Pascal VOC, COCO or etc. Also, please share with us the complete dataset with annotation.

Sincerely,

Zulkifli

[alexalex35]

Hello Zu,

I am writing a batch file to automatically repeat my hello_classification and observe and record the classification results each time. There are 10 types of classifications. If the types of photos are sent to you, in some types (TypeA.TypeB.TypeD. TypeE.TypeF.TypeG) don't even get a correct classification result for a single image.

Also, I would like to ask if I can run several networks at the same time on one NCS2 or run several NCS2s at the same time?

[alexalex35]

Hello Zulkifli,

May I know where the process is now?

[Zulkifli_Intel]

Hello Alex,

Yes, it is possible to run several networks on a NCS2 device. For example, the Action Recognition Python Demo is based on two networks.

You can also run multiple NCS 2s at the same time. Please refer to Multiple NCS Devices and Defining and Configuring the Multi-Device Plugin for more info.

Please share with us the dataset with annotation you used to train the model. The reason why we need the annotated dataset is that we want to assist to validate the accuracy of the model using our Accuracy Checker tool.

Sincerely,

Zulkifli

[alexalex35]

Hello Zulkifli,

Thanks for your reply.

This is my dataset.

[Zulkifli_Intel]

Hello Alex,

Thank you for your patience.

We managed to run your model with Hello Classification Python Sample and Image Classification Async Python Sample. I’ve obtained identical results when running with both samples.

Hello Classification Python Sample

Image Classification Async Python Sample

 

However, we wish to inform you that Resnet50 model was not validated with both these samples. Alexnet and googlenet-v1 are the only validated models for Hello Classification Python Sample, while alexnet is the only validated model with Image Classification Async Python Sample. For unvalidated models, we can give it a try and see if it works, it may or may not work, and we cannot guarantee from our side.

You can use Classification C++ Demo instead, since the Resnet50 model is supported for this demo. But, we are unable to run it because of don’t have ground truth (gt) and label files. You can try to run your model with Classification C++ Demo using the following steps:

Build demos

1. Go to demos:

cd C:\Program Files (x86)\Intel\openvino_2021.4.752\deployment_tools\open_model_zoo\demos

2. Run:

build_demos_msvc.bat to build the demos

3. Demos will be built in C:\Users\Documents\Intel\OpenVINO\omz_demos_build\intel64\Release

Preparing labels and gt files:

1. Create your own label file.

2. Create your own gt file.

Please refer to Required Files for more info.

Running demo

1. Go to build directory.

2. Use the following command to run the demo:

classification_demo.exe -m <path_to_classification_model> -i <path_to_folder_with_images> -labels <path_to_file_with_list_of_labels> -gt <path_to_ground_truth_data_file> -u CDM

Sincerely,

Zulkifli

[alexalex35]

Hello Zulkifli,

Thanks for your reply.

I will try Alexnet. Have new questions or results to share with you.
In addition, I would like to ask if using multiple NCS2s at the same time is a network using multiple NCS2s to increase the inference speed.
Or can I run different models on different NCS2 at the same time?
For example, if I have 2 different image databases, one is cropped and the other is the original image to be identified, I can use 2 NCS2 to let him identify at the same time so that I can know whether it has been cropped or not. the difference in identification results?

[Zulkifli_Intel]

Hello Alex,

It is possible to use multiple NCS devices with the OpenVINO Toolkit. The Inference Engine API will distribute inferences to the NCS devices based on the device load. Take a look at Multiple NCS Devices for additional information.

Sincerely,

Zulkifli

[Zulkifli_Intel]

Hello Alex,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Sincerely,

Zulkifli