- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
In OpenVINO R3, I can see that there is an option to change the batch size dynamically (C++). This can be done just before loading the network into the plugin.
if (enable_dynamic_batch) { config[PluginConfigParams::KEY_DYN_BATCH_ENABLED] = PluginConfigParams::YES; }
I want to know if there is any similar functionality available in Python API. Could you please let me know the correct way to do it in python?
Thanks
Indrajit
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Indrajit,
It looks like we should have, if you see the following document:
Plugin load method, we do have a config dictionary as an input parameters. I would expect this should include the parameter you need.
I check the python sample and other document and I didn't find it. So let me check internally.
Mark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Mark
I tried setting it in my python script like this:
exec_net = plugin.load(network=net,config={'KEY_DYN_BATCH_ENABLED':'YES'})
But I got the following error:
RuntimeError: [NOT_FOUND] Unsupported property KEY_DYN_BATCH_ENABLED by CPU plugin
I want to know if that is the correct way to set it in python.
Thanks
Indrajit
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Indrajit,
To set this config property for the plugin please pass the key without 'KEY_' prefix like this
exec_net = plugin.load(network=net, config={'DYN_BATCH_ENABLED':'YES'})
Other option to change batch size is to use batch_size property of the IENetwork instance or use reshape function (which is preferable in some cases like object detection networks) before loading network to the plugin:
# Setting batch using batch_size net.batch_size = 8 # Setting batch using reshape example_shape = (8, 3, 224, 224) net.reshape({"input_layer_name" : example_shape})
Regards,
Mikhail
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Mikhail
Thanks for your response. Now there is no issue loading the network into the plugin.
For my application, I would load the model once and run inference multiple times with different batch size each time. So I want to be able to change the batch size after the loading the network to the plugin.
With your suggestion I'm able to load the network without any issue but I get the this error when I send a batch of 2 images instead of one:
ValueError: could not broadcast input array from shape (2,3,72,72) into shape (1,3,72,72)
Shouldn't the config property "'DYN_BATCH_ENABLED'
:
'YES'
" allow the network to accept images of any batch size?
Please clarify on this.
Thanks
Indrajit
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Mikhail,
Do we have a document to describe the Python API?
This is the only document I found:
https://software.intel.com/en-us/articles/OpenVINO-InferEngine
Mark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Was the 'DYN_BATCH_ENABLED' config name changed in R5?
Using the code described above I now get the following error when using MYRIAD:
RuntimeError: [NOT_FOUND] DYN_BATCH_ENABLED key does not exist
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Itai!
Myriad doesn't support dynamic batch. It's supported by CPU and GPU plugins.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Got it. Thanks Nikolay!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Running into the same issue as original poster, how do you enable dynamic batching in 2019_R2 please?
I've tried to set `config={'DYN_BATCH_ENABLED':'YES'}` when loading the network but getting an error "ValueError: could not broadcast input array from shape (2,3,227,227) into shape (1,3,227,227)"
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page