Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

a problem in accuracy checker

shirleyliu
New Contributor I
1,932 Views


hello, I encounter a problem when I try to use accuracy checker tools. the error said that my input dataset size(1,3,339,510) doesn't match my model's input size(1,1,339,510). But actually, my input image size is in 8 bits(only one channel) not 24bits(RGB channel). I don't know why this happened? and there is also one problem that even if I changed my lr_suffix and hr_suffix in the confiuration file(yml), the lr_suffix and hr_suffix in CMD_PROMPT is not changed

20210608-142535(WeLinkPC).png

Screenshot.PNG

0 Kudos
1 Solution
Zulkifli_Intel
Moderator
1,774 Views

Hello Liu Xue Chun.

 

We have successfully replicated the case and managed to obtain an accurate PSNR.

You are unable to obtain the correct value due to the .yml file created did not specify the Pre-processing and Post-processing configurations with some missing parameters in the Adapter section.

 

The attached file is the updated and validated version of your .yml file.

 

Regards,

Zulkifli

 

View solution in original post

0 Kudos
15 Replies
shirleyliu
New Contributor I
1,925 Views

I try to add the two lines in my configuration file, I don't know if it's correct to use "gray" flag. my input image is the Y channel which converted from RGB. now I successfully run the accuracy checker. but the PSNR I got is not correct. the PSNR should be around 28-30dB I suppose。

type: opencv_imread
reading_flag: gray

the PSNR got from the accuracy_checker.

  Screenshot.PNG

the dataset I use only contain two y channel images 

Screenshot.PNG

0 Kudos
Zulkifli_Intel
Moderator
1,906 Views

Hello Liu Xue Chun,


Greetings to you.


Please share with us your model, command to use, and the necessary files (such as dataset, configuration files, etc) for us to reproduce the issue.


Regards,

Zulkifli



0 Kudos
shirleyliu
New Contributor I
1,872 Views

hello, I will try to describe my necessary files later(some restrictions for uploading). and BTW when I tried to quantize my model I also meet this problem

Screenshot.PNG

my commad line is 

pot -c SR-fsrcnn.json

my json configuration file is here 

{
/* Model parameters */

"model": {
"model_name": "withoutd_s8_x4_fp32", // Model name
"model": "withoutd_s8_x4_fp32.xml", // Path to model (.xml format)
"weights": "withoutd_s8_x4_fp32.bin" // Path to weights (.bin format)
},

/* Parameters of the engine used for model inference */

"engine": {
"config": "AC-SR-FP32-fsrcnn.yml"
},

/* Optimization hyperparameters */

"compression": {
"target_device": "GPU", // Target device, the specificity of which will be taken
// into account during optimization
"algorithms": [
{
"name": "DefaultQuantization", // Optimization algorithm name
"params": {
"preset": "performance", // Preset [performance, mixed, accuracy] which control the quantization
// mode (symmetric, mixed (weights symmetric and activations asymmetric)
// and fully asymmetric respectively)

"stat_subset_size": 1 // Size of subset to calculate activations statistics that can be used
// for quantization parameters calculation
}
}
]
}
}

my yml configuration file is 

models:
- name: SR-fsrcnn-x4-fp32
launchers:
- framework: dlsdk
model: withoutd_s8_x4_fp32.xml
weights: withoutd_s8_x4_fp32.bin

device: GPU
adapter:
type: super_resolution
reverse_channels: True
datasets:
- name: SR_fsrcnn_x4
reader:
type: opencv_imread
reading_flag: gray
data_source: dataset_x4
annotation_conversion:
converter: super_resolution
data_dir: dataset_x4
lr_suffix: lr_y_
#upsample_suffix: upsample_x4
hr_suffix: hr_y_
#two_streams: True
annotation: SR_fsrcnn_x4.pickle
metrics:
- type: psnr
scale_border: 0
presenter: print_vector
- type: ssim
presenter: print_vector

 

0 Kudos
Zulkifli_Intel
Moderator
1,847 Views

Hello Liu Xue Chun.


Can you try to upload the model, dataset, and all the configuration files on google drive and send the link for us to download?


Regards,

Zulkifli


0 Kudos
shirleyliu
New Contributor I
1,826 Views

hello, I have uploaded the two configuration files, the dataset(2 images) and the model, could you take a look at the problem in accuracy check and pot

my two commad line is 

accuracy checker -c AC-SR-FP32-fsrcnn.yml

pot -c SR-fsrcnn.json

shirleyliu_2-1623767630008.png

 

shirleyliu_0-1623767567264.png

shirleyliu_1-1623767567286.png

 

0 Kudos
Zulkifli_Intel
Moderator
1,811 Views

Hello Liu Xue Chun.


Thank you for sharing the model and configuration files with us. It seems like the information you provided is not complete, the label and annotation files are missing. Please share the missing file with us.


Regards,

Zulkifli


0 Kudos
shirleyliu
New Contributor I
1,805 Views

hello, I have no labels and annotation files because it's the super-resolution task. and when you mentioned annotation files, do you mean "SR_fsrcnn_x4.pickle". Actually it's generated from the command "accuracy checker -c AC-SR-FP32-fsrcnn.yml"

0 Kudos
shirleyliu
New Contributor I
1,799 Views

The super-resolution task is to perform super-resolution on the image which is downsampled by four times on the original image, and then after the super-resolution, we calculate the PSNR between the super-resolution result and the original image to measure the performance of super-resolution. Therefore, the label of each downsampled image (lr_0801.png) is the original image (hr_0801.png).

0 Kudos
Zulkifli_Intel
Moderator
1,799 Views

Hello Liu Xue Chun.


Thank you for your sharing all the information with us. We might need time to replicate and investigate this issue.


Regards,

Zulkifli


0 Kudos
Zulkifli_Intel
Moderator
1,775 Views

Hello Liu Xue Chun.

 

We have successfully replicated the case and managed to obtain an accurate PSNR.

You are unable to obtain the correct value due to the .yml file created did not specify the Pre-processing and Post-processing configurations with some missing parameters in the Adapter section.

 

The attached file is the updated and validated version of your .yml file.

 

Regards,

Zulkifli

 

0 Kudos
shirleyliu
New Contributor I
1,766 Views

hello,thank you very much!I know the reason. it's because our model's input layer is in range [0,1] but my input image is in range [0,255]

and besides do you have any ideas for this problem when I run "pot -c SR-fsrcnn.json"

Screenshot.PNG

 

0 Kudos
Zulkifli_Intel
Moderator
1,744 Views

Hello Liu Xue Chun,

 

Since the initial issue has been resolved. Will it be okay if we close this case and focus the second question on this thread?


Regards,

Zulkifli 


0 Kudos
shirleyliu
New Contributor I
1,742 Views
0 Kudos
Zulkifli_Intel
Moderator
1,717 Views

Hello Liu Xue Chun.


 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Zulkifli


0 Kudos
Reply