cannot run validation_app openvino

[Nunez-Yanez__Jose]

For some reason I am unable to run the validation app:

I launch with:

validation_app.exe -t C -i C:\Users\eejlny\Downloads\train_small  -d CPU -m ./alexnet/alexnet.xml

train_small contains as an example three directories and each directory contains some jpeg files.

the output is always "no images processed" as below. I have tried with a directory with 1000 subdirectories to match the 1000 classes of alexnet but the result is the same like if "no images" were seen. I have followed the instructions in https://software.intel.com/en-us/articles/OpenVINO-IE-Samples#preparing-dataset-validation

[ INFO ] Device: CPU
[ INFO ] Collecting labels
[ INFO ] Starting inference

[ INFO ] Inference report:
        Network load time: 364.385ms
        Model: ./alexnet/alexnet.xml
        Model Precision: FP32
        Batch size: 1
        Validation dataset: C:\Users\eejlny\Downloads\train_small
        Validation approach: Classification network
[ WARNING ] No images processed
 

[Shubha_R_Intel]

Here is the documentation for the validation app:

https://docs.openvinotoolkit.org/R5/_samples_validation_app_README.html

Try just one directory level for your images. In other words, no sub-folders. 

Thanks,

Shubha

[Nunez-Yanez__Jose]

Thanks but those instructions are the ones that I have followed. They clearly show an structure with subfolders apron, collie, Siamese in the example.

|-- <path>/dataset
    |-- apron
        |-- apron1.bmp
        |-- apron2.bmp
    |-- collie
        |-- a_big_dog.jpg
    |-- coral reef
        |-- reef.bmp
    |-- Siamese
        |-- cat3.jpg

Is this incorrect ?

It makes sense so the code can identify the correct class without a labels file. 

The other option seems to be using a folder with all images and a label files which is not my case since I am using imagenet. In any case I have tried this and it does not find anything like if there was no images but I can see them there.

The other example classification_sample_async.exe -i C:\Users\eejlny\Downloads\train_small\fish\ -d CPU -m ./resenet50fp32/caffe/resnet-50.xml works just fine.

[torazurey__vadim]

I running into the same issue with validation_app on Ubuntu 16.04 / Openvino R5
I downloaded eight Imagenet groom class jpeg pictures from here http://169.44.201.108:7002/imagenet/train/n10148035/ and put them to /OpenVINO_dataset_0_1_1/dataset/groom folder
here are the steps which I used for FP32:
1. python3 mo.py --input_model /OpenVINO_common_1/ResNet50/ResNet50.caffemodel --input_proto /OpenVINO_common_1/ResNet50/ResNet50.prototxt --output_dir /OpenVINO_common_1/ResNet50
2. validation_app -m /OpenVINO_common_1/ResNet50/ResNet50.xml -d CPU -t C -i /OpenVINO_dataset_0_1_1/dataset

here is the output:
[ INFO ] Inference report:
     Network load time: 49.9196ms
     Model: /OpenVINO_common_1/ResNet50/ResNet50.xml
     Model Precision: FP32
     Batch size: 1
     Validation dataset: /OpenVINO_dataset_0_1_1/dataset
     Validation approach: Classification network
[ WARNING ] No images processed

my directory structure is:
/OpenVINO_dataset_0_1_1/dataset/groom
n10148035_18604.JPEG
n10148035_18648.JPEG
next six images...

I also tried to point "-i /OpenVINO_dataset_0_1_1/dataset/groom" folder when run validation_app but results is the same: [ WARNING ] No images processed
Could you please help to fix the issue?
Thanks!

[torazurey__vadim]

I running into the same issue with validation_app on Ubuntu 16.04 / Openvino R5
I downloaded eight Imagenet groom class jpeg pictures from here http://169.44.201.108:7002/imagenet/train/n10148035/ and put them to /OpenVINO_dataset_0_1_1/dataset/groom folder
here are the steps which I used for FP32:
1. python3 mo.py --input_model /OpenVINO_common_1/ResNet50/ResNet50.caffemodel --input_proto /OpenVINO_common_1/ResNet50/ResNet50.prototxt --output_dir /OpenVINO_common_1/ResNet50
2. validation_app -m /OpenVINO_common_1/ResNet50/ResNet50.xml -d CPU -t C -i /OpenVINO_dataset_0_1_1/dataset

here is the output:
[ INFO ] Inference report:
     Network load time: 49.9196ms
     Model: /OpenVINO_common_1/ResNet50/ResNet50.xml
     Model Precision: FP32
     Batch size: 1
     Validation dataset: /OpenVINO_dataset_0_1_1/dataset
     Validation approach: Classification network
[ WARNING ] No images processed

my directory structure is:
/OpenVINO_dataset_0_1_1/dataset/groom
n10148035_18604.JPEG
n10148035_18648.JPEG
next six images...

I also tried to point "-i /OpenVINO_dataset_0_1_1/dataset/groom" folder when run validation_app but results is the same: [ WARNING ] No images processed
Could you please help to fix the issue?
Thanks!

[torazurey__vadim]

Hello!

I tried Shubha's way with one directory level for images and the "validation_app" found images successfully but it produced the weird results:
[ INFO ] Average infer time (ms): 6.36 (157.17 images per second with batch size = 1)
Top1 accuracy: 0.00% (0 of 10 images were detected correctly, top class is correct)
Top5 accuracy: 0.00% (0 of 10 images were detected correctly, top five classes contain required class)
where the correct output should be like this:
Top1 accuracy: 70.00% (7 of 10 images were detected correctly, top class is correct)
Top5 accuracy: 80.00% (8 of 10 images were detected correctly, top five classes contain required class)

Could you please point me what I am doing wrong? Why did I receive 0 of 10 images instead 8 of 10 images or similar?

Here is the my run conditions:
OS: 

• Ubuntu 16.04 

App version:

• Openvino R5

Trained Model:

• ResNet50
• I also tried to use this trained model: https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet but it produces the same result - "0 of 10 images".

Validation Datset:

• 10 Imagenet jpeg pictures with class ID 449 were downloaded from http://169.44.201.108:7002/imagenet/val/ and placed to /dataset folder along with labels.txt

Here is the contents of the my labels.txt file:

   ILSVRC2012_val_00000293.JPEG 449
   ILSVRC2012_val_00002138.JPEG 449
   ILSVRC2012_val_00003014.JPEG 449
   ILSVRC2012_val_00006697.JPEG 449
   ILSVRC2012_val_00007197.JPEG 449
   ILSVRC2012_val_00009111.JPEG 449
   ILSVRC2012_val_00009191.JPEG 449
   ILSVRC2012_val_00009346.JPEG 449
   ILSVRC2012_val_00009379.JPEG 449
   ILSVRC2012_val_00009396.JPEG 449

Here are the steps which I used to run "validation_app":

1. python3 mo.py --input_model /ResNet50/ResNet50.caffemodel --input_proto /ResNet50/ResNet50.prototxt --output_dir /ResNet50
2. validation_app -m /ResNet50/ResNet50.xml -d CPU -t C -i /dataset/labels.txt

Here is the full output with weird results in the end:
- Path to the Input Model: /ResNet50/ResNet50.caffemodel
- Path for generated IR: /ResNet50
- IR output name: ResNet50
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: Not specified, inherited from the model
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: False
- Reverse input channels: False
Caffe specific parameters:
- Enable resnet optimization: True
- Path to the Input prototxt: /ResNet50/ResNet50.prototxt
- Path to CustomLayersMapping.xml: Default
- Path to a mean file: Not specified
- Offsets for a mean file: Not specified
Model Optimizer version: 1.5.12.49d067a0

[ SUCCESS ] Generated IR model.
[ SUCCESS ] XML file: /ResNet50/ResNet50.xml
[ SUCCESS ] BIN file: /ResNet50/ResNet50.bin
[ SUCCESS ] Total execution time: 7.31 seconds.

[ INFO ] InferenceEngine:
     API version ............ 1.4
     Build .................. 19154
[ INFO ] Parsing input parameters
[ INFO ] Loading plugin

     API version ............ 1.5
     Build .................. lnx_20181004
     Description ....... MKLDNNPlugin
[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Device: CPU
[ INFO ] Collecting labels
[ INFO ] Starting inference
Progress: [....................] 100.00% done
[ INFO ] Inference report:
     Network load time: 58.59ms
     Model: /ResNet50/ResNet50.xml
     Model Precision: FP32
     Batch size: 1
     Validation dataset: /dataset/labels.txt
     Validation approach: Classification network
[ INFO ] Average infer time (ms): 6.36 (157.17 images per second with batch size = 1)
Top1 accuracy: 0.00% (0 of 10 images were detected correctly, top class is correct)
Top5 accuracy: 0.00% (0 of 10 images were detected correctly, top five classes contain required class)