Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

a problem in accuracy checker

shirleyliu
New Contributor I
968 Views


hello, I encounter a problem when I try to use accuracy checker tools. the error said that my input dataset size(1,3,339,510) doesn't match my model's input size(1,1,339,510). But actually, my input image size is in 8 bits(only one channel) not 24bits(RGB channel). I don't know why this happened? and there is also one problem that even if I changed my lr_suffix and hr_suffix in the confiuration file(yml), the lr_suffix and hr_suffix in CMD_PROMPT is not changed

20210608-142535(WeLinkPC).png

Screenshot.PNG

0 Kudos
1 Solution
Zulkifli_Intel
Moderator
810 Views

Hello Liu Xue Chun.

 

We have successfully replicated the case and managed to obtain an accurate PSNR.

You are unable to obtain the correct value due to the .yml file created did not specify the Pre-processing and Post-processing configurations with some missing parameters in the Adapter section.

 

The attached file is the updated and validated version of your .yml file.

 

Regards,

Zulkifli

 

View solution in original post

15 Replies
shirleyliu
New Contributor I
961 Views

I try to add the two lines in my configuration file, I don't know if it's correct to use "gray" flag. my input image is the Y channel which converted from RGB. now I successfully run the accuracy checker. but the PSNR I got is not correct. the PSNR should be around 28-30dB I suppose。

type: opencv_imread
reading_flag: gray

the PSNR got from the accuracy_checker.

  Screenshot.PNG

the dataset I use only contain two y channel images 

Screenshot.PNG

Zulkifli_Intel
Moderator
942 Views

Hello Liu Xue Chun,


Greetings to you.


Please share with us your model, command to use, and the necessary files (such as dataset, configuration files, etc) for us to reproduce the issue.


Regards,

Zulkifli



shirleyliu
New Contributor I
908 Views

hello, I will try to describe my necessary files later(some restrictions for uploading). and BTW when I tried to quantize my model I also meet this problem

Screenshot.PNG

my commad line is 

pot -c SR-fsrcnn.json

my json configuration file is here 

{
/* Model parameters */

"model": {
"model_name": "withoutd_s8_x4_fp32", // Model name
"model": "withoutd_s8_x4_fp32.xml", // Path to model (.xml format)
"weights": "withoutd_s8_x4_fp32.bin" // Path to weights (.bin format)
},

/* Parameters of the engine used for model inference */

"engine": {
"config": "AC-SR-FP32-fsrcnn.yml"
},

/* Optimization hyperparameters */

"compression": {
"target_device": "GPU", // Target device, the specificity of which will be taken
// into account during optimization
"algorithms": [
{
"name": "DefaultQuantization", // Optimization algorithm name
"params": {
"preset": "performance", // Preset [performance, mixed, accuracy] which control the quantization
// mode (symmetric, mixed (weights symmetric and activations asymmetric)
// and fully asymmetric respectively)

"stat_subset_size": 1 // Size of subset to calculate activations statistics that can be used
// for quantization parameters calculation
}
}
]
}
}

my yml configuration file is 

models:
- name: SR-fsrcnn-x4-fp32
launchers:
- framework: dlsdk
model: withoutd_s8_x4_fp32.xml
weights: withoutd_s8_x4_fp32.bin

device: GPU
adapter:
type: super_resolution
reverse_channels: True
datasets:
- name: SR_fsrcnn_x4
reader:
type: opencv_imread
reading_flag: gray
data_source: dataset_x4
annotation_conversion:
converter: super_resolution
data_dir: dataset_x4
lr_suffix: lr_y_
#upsample_suffix: upsample_x4
hr_suffix: hr_y_
#two_streams: True
annotation: SR_fsrcnn_x4.pickle
metrics:
- type: psnr
scale_border: 0
presenter: print_vector
- type: ssim
presenter: print_vector

 

Zulkifli_Intel
Moderator
883 Views

Hello Liu Xue Chun.


Can you try to upload the model, dataset, and all the configuration files on google drive and send the link for us to download?


Regards,

Zulkifli


shirleyliu
New Contributor I
862 Views

hello, I have uploaded the two configuration files, the dataset(2 images) and the model, could you take a look at the problem in accuracy check and pot

my two commad line is 

accuracy checker -c AC-SR-FP32-fsrcnn.yml

pot -c SR-fsrcnn.json

shirleyliu_2-1623767630008.png

 

shirleyliu_0-1623767567264.png

shirleyliu_1-1623767567286.png

 

Zulkifli_Intel
Moderator
847 Views

Hello Liu Xue Chun.


Thank you for sharing the model and configuration files with us. It seems like the information you provided is not complete, the label and annotation files are missing. Please share the missing file with us.


Regards,

Zulkifli


shirleyliu
New Contributor I
841 Views

hello, I have no labels and annotation files because it's the super-resolution task. and when you mentioned annotation files, do you mean "SR_fsrcnn_x4.pickle". Actually it's generated from the command "accuracy checker -c AC-SR-FP32-fsrcnn.yml"

shirleyliu
New Contributor I
835 Views

The super-resolution task is to perform super-resolution on the image which is downsampled by four times on the original image, and then after the super-resolution, we calculate the PSNR between the super-resolution result and the original image to measure the performance of super-resolution. Therefore, the label of each downsampled image (lr_0801.png) is the original image (hr_0801.png).

Zulkifli_Intel
Moderator
835 Views

Hello Liu Xue Chun.


Thank you for your sharing all the information with us. We might need time to replicate and investigate this issue.


Regards,

Zulkifli


Zulkifli_Intel
Moderator
811 Views

Hello Liu Xue Chun.

 

We have successfully replicated the case and managed to obtain an accurate PSNR.

You are unable to obtain the correct value due to the .yml file created did not specify the Pre-processing and Post-processing configurations with some missing parameters in the Adapter section.

 

The attached file is the updated and validated version of your .yml file.

 

Regards,

Zulkifli

 

shirleyliu
New Contributor I
802 Views

hello,thank you very much!I know the reason. it's because our model's input layer is in range [0,1] but my input image is in range [0,255]

and besides do you have any ideas for this problem when I run "pot -c SR-fsrcnn.json"

Screenshot.PNG

 

Zulkifli_Intel
Moderator
780 Views

Hello Liu Xue Chun,

 

Since the initial issue has been resolved. Will it be okay if we close this case and focus the second question on this thread?


Regards,

Zulkifli 


shirleyliu
New Contributor I
778 Views
Zulkifli_Intel
Moderator
753 Views

Hello Liu Xue Chun.


 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.


Regards,

Zulkifli


Reply