Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Downloader.py cannot download BERT

Ray_Lo_Intel
Employee
720 Views

I think the binary is broken? It always fails as the Downloaded file is shorter*.

 

rlo@rlo-mac01 omz_demos_build % /opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name "bert*" --output_dir ~/Documents/openvino_models
...
========== Downloading /Users/rlo/Documents/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.bin
... 80%, 1053849 KB, 2988 KB/s, 352 seconds passed
########## Error: Downloaded file is shorter (1079141619 B) than expected (1335853368 B)

0 Kudos
1 Solution
Munesh_Intel
Moderator
681 Views

Hi Raymond,

 

Thanks for reaching out to us. I've just validated the download link, and managed to download the model using the following command:

./downloader.py --name bert* --num_attempts 5

 

I've attached the snapshot for your reference.

 

Regards,

Munesh

 

View solution in original post

5 Replies
Vladimir_Dudnik
Employee
700 Views

Hi Raymond,

BERT large model has more than 1 Gb in size, so downloader might experience networking issues. To make downloading more stable please use option to increase number of attempts downloader will try to load the model, as described in Model downloader usage section of OMZ tools downloader README.md file

By default, the script will attempt to download each file only once. You can use the --num_attempts option to change that and increase the robustness of the download process:

./downloader.py --all --num_attempts 5 # attempt each download five times

Regards,
  Vladimir

Ray_Lo_Intel
Employee
690 Views

Can you test this command on your side? It seems to fail at the same location everytime. 

/opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name "bert*" --output_dir ~/Documents/openvino_models

Munesh_Intel
Moderator
682 Views

Hi Raymond,

 

Thanks for reaching out to us. I've just validated the download link, and managed to download the model using the following command:

./downloader.py --name bert* --num_attempts 5

 

I've attached the snapshot for your reference.

 

Regards,

Munesh

 

Ray_Lo_Intel
Employee
672 Views

Got it.

I see that you also have the same 'drop off' at approximately the 1GB mark.

Apparently that attempt flag is a 'must' for downloading large files or it will always fail at the similar place. I think there is a size limitation somewhere in the script and it got cut every time. It is always around the 1GB zone. Why don't we just have that auto-attempt feature by default? 

 

 

 

rlo@rlo-mac01 Documents % /opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name="bert*" --num_attempts 5 -o ~/Documents

################|| Downloading models ||################

========== Downloading /Users/rlo/Documents/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.xml
... 100%, 934 KB, 43981 KB/s, 0 seconds passed

========== Downloading /Users/rlo/Documents/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.bin
... 81%, 1056867 KB, 9966 KB/s, 106 seconds passed
########## Error: Downloaded file is shorter (1082232051 B) than expected (1335853368 B)
Will retry in 10 seconds...
... 100%, 1304544 KB, 42845 KB/s, 30 seconds passedd

 

 

 

 

 

 

Munesh_Intel
Moderator
659 Views

Hi Raymond,

We'll forward your improvement suggestion to the development team.

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 


Regards,

Munesh


Reply