Re: Error Converting BERT Onnx Model

Ray_Lo_Intel · ‎10-07-2020

I'm trying to convert the BERT model based on this documentation ...

https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX.html

python3 mo.py --input_model <INPUT_MODEL>.onnx
 
So, apparently the BERT link is broken, 
 
https://github.com/mlperf/inference/tree/master/v0.7/language/bert
 
Then, I visited 
 
https://github.com/mlperf/inference/tree/master/language/bert
 
and linked me to onnx model here. 
 
https://zenodo.org/record/3733910#.X35TI5NKg40
 
Finally, I ran the command and then it got this. Any ideas? Thank you. 
rlo@rlo-mac01 Desktop % python3 /opt/intel/openvino_2021.1.110/deployment_tools/model_optimizer/mo.py --input_model ~/Downloads/model.onnx 

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/Users/rlo/Downloads/model.onnx
	- Path for generated IR: 	/Users/rlo/Desktop/.
	- IR output name: 	model
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	None
	- Reverse input channels: 	False
ONNX specific parameters:
Model Optimizer version: 	2021.1.0-1237-bece22ac675-releases/2021/1
[libprotobuf ERROR google/protobuf/descriptor_database.cc:394] Invalid file descriptor data passed to EncodedDescriptorDatabase::Add().
[libprotobuf FATAL google/protobuf/descriptor.cc:1356] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
zsh: abort      python3 /opt/intel/openvino_2021.1.110/deployment_tools/model_optimizer/mo.py

Ray_Lo_Intel · ‎10-07-2020

Ok I made some progress, and apparently I have to uninstall tensorflow ...

pip3 uninstall tensorflow

I'm on Mac OSX 10.15.7 and Python 3.7.9

And then I can convert the model based on this now.

https://docs.openvinotoolkit.org/latest/omz_demos_python_demos_bert_question_answering_demo_README.html

But with the tensorflow installed, this always happens! and I can run

python3 -c 'import tensorflow' without triggering any errors.

rlo@rlo-mac01 Desktop % python3  /opt/intel/openvino_2021.1.110/deployment_tools/model_optimizer/mo.py -m ~/Downloads/bert_large_v1_1_fake_quant.onnx --input "input_ids,attention_mask,token_type_ids" --input_shape "[1,384],[1,384],[1,384]" --keep_shape_ops

[ WARNING ]  Use of removed cli option '--keep_shape_ops' detected. The option is ignored. 
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/Users/rlo/Downloads/bert_large_v1_1_fake_quant.onnx
	- Path for generated IR: 	/Users/rlo/Desktop/.
	- IR output name: 	bert_large_v1_1_fake_quant
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	input_ids,attention_mask,token_type_ids
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	[1,384],[1,384],[1,384]
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	None
	- Reverse input channels: 	False
ONNX specific parameters:
Model Optimizer version: 	2021.1.0-1237-bece22ac675-releases/2021/1
[libprotobuf ERROR google/protobuf/descriptor_database.cc:394] Invalid file descriptor data passed to EncodedDescriptorDatabase::Add().
[libprotobuf FATAL google/protobuf/descriptor.cc:1356] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException: CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size): 
zsh: abort      python3 /opt/intel/openvino_2021.1.110/deployment_tools/model_optimizer/mo.py

JesusE_Intel · ‎10-08-2020

Hi @Ray_Lo_Intel

What version of protobuf are you using? Using the same model optimizer command, I was able to convert this model successfully with OpenVINO toolkit 2021.1 on MacOS 10.14.6, Python 3.6.5 and Protobuf 3.13.0. Try re-installing protobuf and see if that helps.

Regards,

Jesus

IntelSupport · ‎10-21-2020

Hi Raymond,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Adli