Initially, I used a keras hd5 file for inference and it took around 54 sec to produce the result on a 5 sec 30fps video input file. Then I used openvino tool kit for optimization and inference now inference is taking around 50 sec, and I haven't gained much. Since I am new to openvino I don't know much about optimization, please help
It really depends on the network you're using. Maybe it's already as optimised as it can be. Here are some explanations of how OpenVINO optimises a model.
Other alternatives are: use another model, or modify your current network to make it lighter while reducing some accuracy.
one more thing, I saw in a couple of examples that we can use CPU extension like sse4, avx2. According to the latest release, these extensions are moved to plugin. I don't know how to use CPU plugins either, can you help me with this?
CPU extensions are moved to plugin in the latest version of OpenVINO. The extensions are loaded automatically while loading the plugin.
You can refer here for our latest documentation: