TF custom mobilenets – raise ValueError("No variables to save")

idata · ‎06-27-2018

Hi,

I'm trying to compile mobilenet tensorflow classifier on custom data, mobilenet_v1_1.0_128. I've tried using tensorflow 1.7 as well as 1.6 with no success.

I'm using latest ncsdk 1 and the Makefile for the mobilenets from the ncappzoo, the master branch, changed it in order to use my own training output.

The tensorflow training outputs 3 files data-00000-of-00001, index and meta, just like the files the example uses from the mobilenet_v1_1.0_128_2017_06_14.tar.gz

I encounter the following:

Freezing model for inference...
(cd model; python3 /home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py \
    --input_graph=mobilenet_v1_1.0_128.pb \
    --input_binary=true \
    --input_checkpoint=mobilenet_v1_1.0_128.ckpt \
    --output_graph=mobilenet_v1_1.0_128_frozen.pb \
    --output_node_name=MobilenetV1/Predictions/Reshape_1;)
/home/ubuntu/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
/home/ubuntu/anaconda3/lib/python3.6/site-packages/matplotlib/__init__.py:1067: UserWarning: Duplicate key in file "/home/ubuntu/.config/matplotlib/matplotlibrc", line #2
  (fname, cnt))
/home/ubuntu/anaconda3/lib/python3.6/site-packages/matplotlib/__init__.py:1067: UserWarning: Duplicate key in file "/home/ubuntu/.config/matplotlib/matplotlibrc", line #3
  (fname, cnt))
WARNING:tensorflow:From /home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
2018-06-27 12:19:29.932559: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Traceback (most recent call last):
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 382, in <module>
    run_main()
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 379, in run_main
    app.run(main=my_main, argv=[sys.argv[0]] + unparsed)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 378, in <lambda>
    my_main = lambda unused_args: main(unused_args, flags)
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 272, in main
    flags.saved_model_tags, checkpoint_version)
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 254, in freeze_graph
    checkpoint_version=checkpoint_version)
  File "/home/ubuntu/ncappzoo/tensorflow/mobilenets/../tf_src/tensorflow/tensorflow/python/tools/freeze_graph.py", line 128, in freeze_graph_with_def_protos
    var_list=var_list, write_version=checkpoint_version)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1311, in __init__
    self.build()
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1320, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1345, in _build
    raise ValueError("No variables to save")
ValueError: No variables to save
Makefile:80: recipe for target 'freeze' failed
make: *** [freeze] Error 1

Any advice?

Thanks,

filip.

idata · ‎06-27-2018

@fpopa Can you provide your files for issue reproduction? I came across a similar issue here.

idata · ‎06-28-2018

Hi @Tome_at_Intel,

sure, thanks for helping.

https://drive.google.com/drive/folders/1W8AF0xbD2v9DLvHT7118q9TtLyur42B3

idata · ‎07-01-2018

Any luck @Tome_at_Intel ?

idata · ‎07-02-2018

@fpopa I tried compiling your model, but ran into some errors. I think there may have been some issues when you retrained your model. How did you retrain your model? Please double check and visit this link and this link for possible solutions.

idata · ‎07-04-2018

I've retrained the model using this retraining script from the tensorflow hub.

this is how I run the training:

python retrain.py \
   --image_dir ~/human_class \
   --tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v1_100_128/classification/1

I've tried doing the same with feature_vector instead of classification, but I'm encountering the same problem.

I couldn't find anything related to from_detection_checkpoint for mobilenet classification.