Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5924 Discussions

What are the supported validation dataset types for DL Workbench YOLOv4 model ?

IDPBY
Beginner
197 Views

I am using OpenVINO 2021.4 version. I would like to conduct accuracy performance test using DL workbench, and generate the accuracy report.  However, when I tried to import the validation dataset with the format of Pascal VOC, ImageNet, it displayed the errors as below (circled in red):

Non-supported.jpg

c84ce982-9e89-443c-83bf-ee027afd47bc.jpg

I found out that only non-annotated format can be used as validation dataset. So, is there any way which I can generate the accuracy report with YOLOv4 model?

 

Thank you.

0 Kudos
1 Solution
Hairul_Intel
Moderator
118 Views

Hi IDPBY

From my end, I've successfully downloaded the yolo-v4-tf model using OpenVINO 2021.4 and converted the model's weights into .savedmodel format with the following commands:

python convert.py "yolo-v4-tf\keras-YOLOv3-model-set\cfg\yolov4.cfg" "yolo-v4-tf\yolov4.weights" "yolo-v4-tf\yolov4.savedmodel" --yolo4_reorder

 

Here are the model parameters and the results for Visualize Output on several images:

model_param.png

dog_output.png

bird_output.png

 

 

Additionally, please share your custom model with us and also provide more details regarding your model, the topology, source repository, etc., for further investigation.

 

 

Regards,

Hairul

 

View solution in original post

5 Replies
Hairul_Intel
Moderator
161 Views

Hi IDPBY,

Thank you for reaching out to us.

 

I've validated from my end using DL Workbench 2021.4 with yolo-v4-tf model by using Common Objects in Context (COCO) dataset. The dataset is 2017 Val images with 2017 Train/Val annotations which is available in Download COCO Dataset section.

 

For Object Detection dataset, use the instances_val2017.json annotation file and organize it in the following file structure:

|-- val

     |-- 0001.jpg

     |-- 0002.jpg

     ...

     |-- n.jpg

|-- annotations

     |-- instances_val.json

 

 

Here is the result for using proper COCO dataset format where DL Workbench will detect the dataset as Object Detection task:

coco.png

 

For more information regarding the dataset types used in DL Workbench, you can refer here.

 

 

Regards,

Hairul

 

IDPBY
Beginner
146 Views

Hi @Hairul_Intel ,

 

Thanks for your prompt reply.

 

I encountered another issue where I could not visualize my custom model output in DL Workbench. I converted the YOLOv4 weight file to saved model using the convert.py in the directory of keras-YOLOv3-model-set/tools/model_converter

sudo python3 convert.py "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/keras-YOLOv3-model-set/cfg/yolov4.cfg" "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/yolov4.weights" "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/yolov4.savedmodel" --yolo4_reorder

IDPBY_1-1655796935888.png

IDPBY_2-1655796958920.png

The saved model is saved as shown above.

Next, I uploaded the saved model folder to the DL workbench and converted into OpenVINO IR format. The conversion setting was as follows:

Conversion setting.jpg

However, when I did the model output visualization, I got the error message as circled in red.

Error1.jpg

 

How to do solve this error?

 

Hairul_Intel
Moderator
119 Views

Hi IDPBY

From my end, I've successfully downloaded the yolo-v4-tf model using OpenVINO 2021.4 and converted the model's weights into .savedmodel format with the following commands:

python convert.py "yolo-v4-tf\keras-YOLOv3-model-set\cfg\yolov4.cfg" "yolo-v4-tf\yolov4.weights" "yolo-v4-tf\yolov4.savedmodel" --yolo4_reorder

 

Here are the model parameters and the results for Visualize Output on several images:

model_param.png

dog_output.png

bird_output.png

 

 

Additionally, please share your custom model with us and also provide more details regarding your model, the topology, source repository, etc., for further investigation.

 

 

Regards,

Hairul

 

IDPBY
Beginner
103 Views

Hi @Hairul_Intel ,

 

Thank you so much.  I have solved the above issue. 

 

Hairul_Intel
Moderator
85 Views

Hi IDPBY,

Glad to know that your issue has been resolved.

 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Hairul

 

Reply