Missing latest.pth file

[Afiq]

Hi, i've been facing a problem where the latest.pth file does not exist in the output folder. In the face detection step 4.c, it is said the checkpoint was saved to folder outputs and ${SNAPSHOT}=outputs/latest.pth. when i run step 3.b, the error was saying that the outputs/latest.pth is not a checkpoint file. How do i solve this problem and why it is not checkpoint or it is not exist in the mentioned folder?

[Vladimir_Dudnik]

@Afiq it seems your question related to some of OpenVINO sample or demo, is not it? Could you please specify what OpenVINO version do you use, what sample/demo you gave troubles with, what exact steps causing this issue?

[Afiq]

the Openvino version i am using is 2021.3.394. i was doing the face detection training extension, https://github.com/openvinotoolkit/training_extensions/tree/develop/models/object_detection/model_templates/face-detection. The step that will have an issue after i run code on step 4.c is step 3.b and 4.b, where it will says OSError: outputs/latest.pth is not a checkpoint file

[ilya-krylov]

Hi @Afiq Could you provide logs and error messages if it is possible?

[Afiq]

Traceback (most recent call last):
  File "/home/itxotic/Desktop/training_extensions/external/mmdetection/tools/test.py", line 260, in <module>
    main()
  File "/home/itxotic/Desktop/training_extensions/external/mmdetection/tools/test.py", line 207, in main
    checkpoint = load_checkpoint(model, args.checkpoint, map_location='cpu')
  File "/home/itxotic/Desktop/training_extensions/models/object_detection/venv/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 529, in load_checkpoint
    checkpoint = _load_checkpoint(filename, map_location, logger)
  File "/home/itxotic/Desktop/training_extensions/models/object_detection/venv/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 467, in _load_checkpoint
    return CheckpointLoader.load_checkpoint(filename, map_location, logger)
  File "/home/itxotic/Desktop/training_extensions/models/object_detection/venv/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 244, in load_checkpoint
    return checkpoint_loader(filename, map_location)
  File "/home/itxotic/Desktop/training_extensions/models/object_detection/venv/lib/python3.8/site-packages/mmcv/runner/checkpoint.py", line 260, in load_from_local
    raise IOError(f'{filename} is not a checkpoint file')
OSError: outputs/latest.pth is not a checkpoint file
Traceback (most recent call last):
  File "eval.py", line 46, in <module>
    main()
  File "eval.py", line 42, in main
    evaluator(**eval_args)
  File "/home/itxotic/Desktop/training_extensions/ote/ote/modules/evaluators/base.py", line 38, in __call__
    self._evaluate_internal(config, snapshot, out, update_config, metrics_functions, **kwargs)
  File "/home/itxotic/Desktop/training_extensions/ote/ote/modules/evaluators/base.py", line 62, in _evaluate_internal
    metrics.extend(func(**metric_args))
  File "/home/itxotic/Desktop/training_extensions/ote/ote/metrics/detection/common.py", line 98, in coco_ap_eval_det
    return coco_ap_eval(config_path, work_dir, snapshot, update_config, show_dir)
  File "/home/itxotic/Desktop/training_extensions/ote/ote/metrics/detection/common.py", line 92, in coco_ap_eval
    update_outputs(outputs, metric_keys, metric_names, average_precision)
  File "/home/itxotic/Desktop/training_extensions/ote/ote/metrics/detection/common.py", line 38, in update_outputs
    assert len(metric_values) == len(metric_names) == len(metric_keys), \
AssertionError: [] vs ('AP @ [IoU=0.50:0.95]',) vs ['bbox']

[Syamimi_Intel]

Hi Afiq,

Please install mmcv version 1.3.0 by using the following command:

pip install mmcv==1.3.0

pip install mmcv-full==1.3.0

After that, you may try to rerun the training_extensions again. If you are facing any issue please come back to us.

Regards,

Syamimi

[ilya-krylov]

this issue has been resolved on github https://github.com/openvinotoolkit/training_extensions/issues/568

[Syamimi_Intel]

Hi Afiq,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Syamimi