Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Raymond
Employee
449 Views

Downloader.py cannot download BERT

Jump to solution

I think the binary is broken? It always fails as the Downloaded file is shorter*.

 

rlo@rlo-mac01 omz_demos_build % /opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name "bert*" --output_dir ~/Documents/openvino_models
...
========== Downloading /Users/rlo/Documents/openvino_models/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.bin
... 80%, 1053849 KB, 2988 KB/s, 352 seconds passed
########## Error: Downloaded file is shorter (1079141619 B) than expected (1335853368 B)

0 Kudos
1 Solution
Munesh_Intel
Moderator
410 Views

Hi Raymond,

 

Thanks for reaching out to us. I've just validated the download link, and managed to download the model using the following command:

./downloader.py --name bert* --num_attempts 5

 

I've attached the snapshot for your reference.

 

Regards,

Munesh

 

View solution in original post

5 Replies
Vladimir_Dudnik
Employee
429 Views

Hi Raymond,

BERT large model has more than 1 Gb in size, so downloader might experience networking issues. To make downloading more stable please use option to increase number of attempts downloader will try to load the model, as described in Model downloader usage section of OMZ tools downloader README.md file

By default, the script will attempt to download each file only once. You can use the --num_attempts option to change that and increase the robustness of the download process:

./downloader.py --all --num_attempts 5 # attempt each download five times

Regards,
  Vladimir

Raymond
Employee
419 Views

Can you test this command on your side? It seems to fail at the same location everytime. 

/opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name "bert*" --output_dir ~/Documents/openvino_models

Munesh_Intel
Moderator
411 Views

Hi Raymond,

 

Thanks for reaching out to us. I've just validated the download link, and managed to download the model using the following command:

./downloader.py --name bert* --num_attempts 5

 

I've attached the snapshot for your reference.

 

Regards,

Munesh

 

View solution in original post

Raymond
Employee
401 Views

Got it.

I see that you also have the same 'drop off' at approximately the 1GB mark.

Apparently that attempt flag is a 'must' for downloading large files or it will always fail at the similar place. I think there is a size limitation somewhere in the script and it got cut every time. It is always around the 1GB zone. Why don't we just have that auto-attempt feature by default? 

 

 

 

rlo@rlo-mac01 Documents % /opt/intel/openvino_2020.4.287/deployment_tools/open_model_zoo/tools/downloader/downloader.py --name="bert*" --num_attempts 5 -o ~/Documents

################|| Downloading models ||################

========== Downloading /Users/rlo/Documents/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.xml
... 100%, 934 KB, 43981 KB/s, 0 seconds passed

========== Downloading /Users/rlo/Documents/intel/bert-large-uncased-whole-word-masking-squad-fp32-0001/FP32/bert-large-uncased-whole-word-masking-squad-fp32-0001.bin
... 81%, 1056867 KB, 9966 KB/s, 106 seconds passed
########## Error: Downloaded file is shorter (1082232051 B) than expected (1335853368 B)
Will retry in 10 seconds...
... 100%, 1304544 KB, 42845 KB/s, 30 seconds passedd

 

 

 

 

 

 

Munesh_Intel
Moderator
388 Views

Hi Raymond,

We'll forward your improvement suggestion to the development team.

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 


Regards,

Munesh


Reply