Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6503 Discussions

openvino python API ie.load_network() is too slow

zihao
Beginner
1,665 Views

Hi, 

 

I'm using the latest version of openvino (2021.3) to do inference with python API.

I found that loading the IR model onto CPU is too slow. Here is the code:

net = ie.read_network(model=path_to_xml, weights=path_to_bin)
exec_net = ie.load_network(network=net, device_name="CPU")
res = exec_net.infer(inputs=data)

 Any methods to speed up? 

Thanks.

0 Kudos
4 Replies
IntelSupport
Community Manager
1,642 Views

Hi Zihao,

Thanks for reaching out.

Your code seems fine. Could you share more information about your model? Did you load and read the network more than once for the inference?

 

Regards,

Aznie


0 Kudos
zihao
Beginner
1,627 Views

Thanks for reply.

Yes, I did load the network more than once. My input shape is dynamic so I have to do the net.reshape() first and then reload it to CPU.  The code is something like this:

 

 

 

for img in all_imgs:
    h_new, w_new = img.shape[:-1]
    net.reshape({input_name: [n, c, h_new, w_new]})
    exec_net = ie.load_network(network=net, device_name="CPU")
    res = exec_net.infer(inputs=img)
 
    

 

 

 

I printed out the time cost by ie.load_network() every cycle and it was always around 220ms while the time cost by exec_net.infer() was only around 30ms. 

Any ideas to optimize?

0 Kudos
IntelSupport
Community Manager
1,604 Views

Hi Zihao,

Sorry for the delay in replying to you. You could try referring to Model Optimization techniques to accelerate the inference. Meanwhile, the slowdown might be also due to the layers of your network topology. Check out the layers that relevant to be executed on CPU below:

https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_CPU.html

 

Regards,

Aznie

 

0 Kudos
IntelSupport
Community Manager
1,582 Views

Hi Zihao,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.


Regards,

Aznie


0 Kudos
Reply