What are the supported validation dataset types for DL Workbench YOLOv4 model ?

[IDPBY]

I am using OpenVINO 2021.4 version. I would like to conduct accuracy performance test using DL workbench, and generate the accuracy report.  However, when I tried to import the validation dataset with the format of Pascal VOC, ImageNet, it displayed the errors as below (circled in red):

I found out that only non-annotated format can be used as validation dataset. So, is there any way which I can generate the accuracy report with YOLOv4 model?

Thank you.

[Hairul_Intel]

Hi IDPBY,

Thank you for reaching out to us.

I've validated from my end using DL Workbench 2021.4 with yolo-v4-tf model by using Common Objects in Context (COCO) dataset. The dataset is 2017 Val images with 2017 Train/Val annotations which is available in Download COCO Dataset section.

For Object Detection dataset, use the instances_val2017.json annotation file and organize it in the following file structure:

|-- val

     |-- 0001.jpg

     |-- 0002.jpg

     ...

     |-- n.jpg

|-- annotations

     |-- instances_val.json

Here is the result for using proper COCO dataset format where DL Workbench will detect the dataset as Object Detection task:

For more information regarding the dataset types used in DL Workbench, you can refer here.

Regards,

Hairul

[IDPBY]

Hi @Hairul_Intel ,

Thanks for your prompt reply.

I encountered another issue where I could not visualize my custom model output in DL Workbench. I converted the YOLOv4 weight file to saved model using the convert.py in the directory of keras-YOLOv3-model-set/tools/model_converter

sudo python3 convert.py "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/keras-YOLOv3-model-set/cfg/yolov4.cfg" "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/yolov4.weights" "/opt/intel/openvino_2021/deployment_tools/tools/model_downloader/public/yolo-v4-tf/yolov4.savedmodel" --yolo4_reorder

The saved model is saved as shown above.

Next, I uploaded the saved model folder to the DL workbench and converted into OpenVINO IR format. The conversion setting was as follows:

However, when I did the model output visualization, I got the error message as circled in red.

How to do solve this error?

[Hairul_Intel · Accepted solution]

Hi IDPBY

From my end, I've successfully downloaded the yolo-v4-tf model using OpenVINO 2021.4 and converted the model's weights into .savedmodel format with the following commands:

python convert.py "yolo-v4-tf\keras-YOLOv3-model-set\cfg\yolov4.cfg" "yolo-v4-tf\yolov4.weights" "yolo-v4-tf\yolov4.savedmodel" --yolo4_reorder

Here are the model parameters and the results for Visualize Output on several images:

Additionally, please share your custom model with us and also provide more details regarding your model, the topology, source repository, etc., for further investigation.

Regards,

Hairul

[IDPBY]

Hi @Hairul_Intel ,

Thank you so much.  I have solved the above issue. 

[Hairul_Intel]

Hi IDPBY,

Glad to know that your issue has been resolved.

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Hairul