Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
6026 Discussions

Object Detection Custom Dataset for INT8 calibration

msolmaz
Employee
519 Views

I have converted a custom trained EfficientDet model to OpenVINO version and the inference works good for full FP32 precision. However, conversion to INT8 precision using default configuration results in very degrading Average Precision (AP) performance and synchronous inference is actually slower. I believe this is because I used Not-Annotated Dataset.

When I try to use this link (https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Dataset_Types.html) to create COCO dataset, the DL Workbench does not import it. I tried changing the folder structure. I made sure the numeric data is in integer format. 

My question is: how does anyone create a custom dataset for OD? I don't think the instructions are clear in this link and they only work for open source datasets. 

 

 

0 Kudos
1 Solution
VladimirG
Employee
405 Views

Hi msolmaz,

It seems like the issue in this case is indeed the lack of required fields. To fix the issue, try adding the "info" and "licenses" entries into the .json file. They don't have to be filled out - you can just add them as empty list/dictionary respectively - however DL Workbench specifically searches for those fields to determine if the dataset is in COCO format.

 

Regards,
Vladimir

View solution in original post

7 Replies
Hairul_Intel
Moderator
487 Views

Hi Msolmaz,

Thank you for reaching out to us.

 

Yes, you are right. If the tradeoff between accuracy and performance is large, you can import an annotated dataset and use AccuracyAware optimization method.

 

For creating custom annotated dataset, you can use any of the following image labelling tools:

 

Ensure that the dataset structure from the annotated images produced are adhered to the supported dataset formats for DL Workbench.

 

Referring to your question on importing COCO dataset into DL Workbench, you'll need to combine both the image folder and annotations folder into one new archive.

 

Object Detection COCO dataset archive is organized as follows:

|-- val

     |-- 0001.jpg

     |-- 0002.jpg

     ...

     |-- n.jpg

|-- annotations

     |-- instances_val.json

 

Object Detection COCO dataset requires the instances_val.json file only inside the annotations folder.

 

 

Regards,

Hairul


msolmaz
Employee
437 Views

Hairul, I believe I adhered to these standards and the annotations folder is exactly like that. I just created a "coco.zip" file with the 2 folders. I will also reply to Vladimir about DL Workbench

VladimirG
Employee
453 Views

Hi Msolmaz,

Thanks for reaching out to us.

I'd need some clarifications on what kind of issue you're running in to - is the dataset you've created not being uploaded at all, or is it not being recognized as suitable for Object Detection?

To clarify - for DL Workbench to recognize the dataset as COCO, it should be sufficient to provide a .json annotation file with all 4 dictionaries (`info`, `licenses`, `images` and `annotations`) present in any form. For specifics tasks however, more information would be necessary to determine the cause. Could you provide an example of your dataset so I can verify any such issues?

Regards,
Vladimir

msolmaz
Employee
436 Views

Vladimir, I just get an error message on the UI as attached. I just zipped annotations folder with instances_val.json in it, and val folder with all the images in it. 

I realized that the json file does not have the "info" and "license". It looks like below:

 

{"images": [{"file_name": "Test_0_0", "height": 646, "width": 632, "id": "Test_0_0"}, {"file_name": "Test_0_1", "height": 646, "width": 632, "id": "Test_0_1"}
"type": "instances", "annotations": [{"area": 152.0, "iscrowd": 0, "bbox": [110.0, 114.0, 19.0, 8.0], "category_id": 1, "ignore": 0, "segmentation": [], "image_id": "Test_0_0", "id": 1}

 

VladimirG
Employee
406 Views

Hi msolmaz,

It seems like the issue in this case is indeed the lack of required fields. To fix the issue, try adding the "info" and "licenses" entries into the .json file. They don't have to be filled out - you can just add them as empty list/dictionary respectively - however DL Workbench specifically searches for those fields to determine if the dataset is in COCO format.

 

Regards,
Vladimir

msolmaz
Employee
381 Views

Thank you!
Adding emtpy info and licenses field solved the problem of importing dataset in DL Workbench

Hairul_Intel
Moderator
363 Views

Hi msolmaz,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Hairul


Reply