Exe built using OpenVino and pyinstaller does not work with R3

[selvakumar__chandra1]

I have taken the face-detection-retail-0005 model from open model zoo and ran a simple python script to load the model. This script works and it prints "Network loaded" when executed as a python program by setting the environment path on the terminal.
I built a simple exe using pyinstaller for this program and copied the exe to another location along with the following DLLs from openvino - cpu_extension_avx2.dll, inference_engine.dll, mkl_tiny_tbb.dll, MKLDNNPlugin.dll, tbb.dll and tbbmalloc.dll. I also copied the 'openvino' folder from "<path_to_openvino>/python/python3.6". Now, on running the exe the model doesn't load and it gets stuck in this line - "exec_net = plugin.load(network=net)" and just exits. I want the exe to work as the python program. I am using Intel Openvino 2019 R3.1 version. It works with the R1 version of Intel OpenVino, but does not work with R3. The application I am building requires version R3 for compatibility issues. Can you help me solve this. 

Following is the code that I used:-

import numpy
from openvino.inference_engine import IENetwork, IEPlugin

def load():
    global plugin
    fd_model_bin = "D:\\test_openvino\\test_models\\face-detection-retail-0005.bin"
    fd_model_xml = "D:\\test_openvino\\test_models\\face-detection-retail-0005.xml"

    intel_plugin_dir = "C:\\Intel\\openvino_2019.3.379\\inference_engine\\bin\\intel64\\Release"

    plugin = IEPlugin(device='CPU', plugin_dirs=intel_plugin_dir)
    plugin.add_cpu_extension('cpu_extension_avx2.dll')

    net = IENetwork(model=fd_model_xml, weights=fd_model_bin)
    supported_layers = plugin.get_supported_layers(net)
    not_supported_layers = [l for l in net.layers.keys() if l not in supported_layers]
    if(len(not_supported_layers)!=0):
        print("Layers not supported are = ",not_supported_layers)
        exit()
    print("before network loading")
    exec_net = plugin.load(network=net)
    print("loaded net = ",exec_net)
    print("Network loaded")
load()

[Luis_at_Intel]

Hi selvakumar, chandrakanth,

May I ask what is the error that you receive and share the log? Looking at the documentation, the Python* API for Inference Engine seems to load the network slightly different than what you have in the program (see below). Take a look at the Instance Methods here.

exec_net = plugin.load_network(network=net, device_name="CPU", num_requsts=2)

I believe the API changed from R1 to R3, which could be the reason why your program works on the previous release and not on R3. I'd suggest to take a look at the release notes for more information.

Regards,

Luis

[Joyce]

HI Luis,

I encounter exactly the same problem as selvakumar, chandrakanth, using version R3. The program executes good in python, but the .exe export from pyinstaller returns without any extra message(include error message & warning). And it is sure to stuck in line 21(when calling IEPlugin.load()) and never goes to line 22 according to selvakumar, chandrakanth's code. 

Is there any other suggestion or solution so far? Thank you!

Regards,

Joyce

[Srujana]

Hi @selvakumar__chandra1 , @Joyce  , @Luis_at_Intel 

I was able to run the sample that loads the network using OpenVINO 2020.3 on windows. Here is the code.

import numpy
import logging as log
from openvino.inference_engine import IECore, IENetwork

def load():
    ie = IECore()
    model_bin = "C:\\Users\\abc\\intel\\face-detection-retail-0005\\FP32\\face-detection-retail-0005.bin"
    model_xml = "C:\\Users\\abc\\intel\\face-detection-retail-0005\\FP32\\face-detection-retail-0005.xml"
    net = ie.read_network(model=model_xml, weights=model_bin)
    log.info("Device info: CPU")
    device="CPU"
    log.info("Loading model to the device")
    print("Loading model to the device")
    exec_net = ie.load_network(network=net, device_name=device)
    print(exec_net)
    print("model loaded")

load()

Created the .exe using  the following statement which includes the underlying dependencies for the smooth run of the .exe file.

pyinstaller --onefile --add-data "C:\Program Files (x86)\IntelSWTools\openvino_2020.3.194\deployment_tools\inference_engine\bin\intel64\Release\plugins.xml;." demo_sample.py

Here is the output:

Regards,

Srujana K

[Igor_F_Intel]

Hi Srujana, 

   I couldn't run it on my environment (Windows10, Openvino basically all versions from 2020 to 2021.1 using conda env with different versions of Pyinstaller). 

   See that I installed the ffmpeg-download.ps1 to correct the MXF bug. 

   When I ran my application via Python it works accordingly, but not via Pyinstaller exe. 

   Which version of python, pyinstaller, numpy, opencv are you using? Do you know if the 2021.1 solved this issue? 

  I would suggest to Intel really make OpenVino support and work with PYinstaller mainly because it is the major platform to build PYthon applications.

 See below my error using Openvino 2020.4:

(pv) Z:\pv>pyinstaller pv.py -y --onefile --distpath ..\deploy-win --add-data openvino_deploy_package_win_2020-4\deployment_tools\inference_engine\bin\intel64\Release\plugins.xml;. --noconfirm --hidden-import matplotlib
154 INFO: PyInstaller: 4.1.dev0
154 INFO: Python: 3.6.12 (conda)
155 INFO: Platform: Windows-10-10.0.19041-SP0
157 INFO: wrote Z:\pv\pv.spec
164 INFO: UPX is not available.
169 INFO: Extending PYTHONPATH with paths
['Z:\\', 'Z:\\pv']
189 INFO: checking Analysis
352 INFO: Appending 'datas' from .spec
357 INFO: checking PYZ
417 INFO: checking PKG
494 INFO: Building because Z:\pv\build\portao_virtual_tracking\portao_virtual_tracking.exe.manifest changed
494 INFO: Building PKG (CArchive) PKG-00.pkg
158249 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
158393 INFO: Bootloader c:\users\pv\anaconda3\envs\pv\lib\site-packages\PyInstaller\bootloader\Windows-64bit\run.exe
158394 INFO: checking EXE
158416 INFO: Rebuilding EXE-00.toc because pv.exe missing
158416 INFO: Building EXE from EXE-00.toc
158418 INFO: Appending archive to EXE ..\deploy-win\pv.exe
164105 INFO: Building EXE from EXE-00.toc completed successfully.

(pv) Z:\portao-virtual>cd ..\deploy-win

(pv) Z:\deploy-win>pv.exe

When I try to run the pv.exe

(pv) Z:\deploy-win>pv.exe
Traceback (most recent call last):
File "pv.py", line 1, in <module>
ImportError: DLL load failed: Não foi possível encontrar o procedimento especificado.
[4668] Failed to execute script pv

best,

Igor 

[Igor_F_Intel]

Now I´m running without Anaconda, and I used pip to isntall the packages directly on the system and got the error reported in this thread:

Openvino 2020.4 (but it happens with 2020 all version and 2021.1)

altgraph==0.17
certifi==2020.6.20
cycler==0.10.0
future==0.18.2
kiwisolver==1.2.0
matplotlib==3.3.2
numpy==1.19.2
pefile==2019.4.18
Pillow==8.0.0
pygame==1.9.6
pyinstaller==4.0
pyinstaller-hooks-contrib==2020.9
pyparsing==2.4.7
PyQt5==5.15.1
PyQt5-sip==12.8.1
python-dateutil==2.8.1
python-engineio==3.13.2
python-socketio==4.6.0
pywin32-ctypes==0.2.0
scipy==1.5.2
six==1.15.0

Z:\deploy-win>pv.exe
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
File "pv.py", line 18, in <module>
File "c:\users\pv\appdata\local\programs\python\python37\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.__dict__)
File "pluginOpenVino.py", line 26, in <module>
File "c:\users\pv\appdata\local\programs\python\python37\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 493, in exec_module
exec(bytecode, module.__dict__)
File "openvino\inference_engine\__init__.py", line 1, in <module>
File "ie_api.pyx", line 24, in init openvino.inference_engine.ie_api
ModuleNotFoundError: No module named 'openvino.inference_engine.constants'
[3804] Failed to execute script pv

[Sanjay]

Hi,
I'm trying to convert my opencv inference engine based application to exe .So currently I'm passing the entire openvino path along with plugins.xml,but still not able to get the application running .Along with all this I'm running pyinstaller command inside the openvino environment so that pyinstaller is able to get the required modules.Can you please help me with this.

[Sanjay]

When I run my exe files ,it works fyn but when it's about to initialise the openvino detector using cv2.dnn.readfrommodeloptimiser ,it just breaks out and doesn't even print any message,just as shown in the image below.Basically after the line "Enter the file number to process ",it just breaks out.

[Vina_Alvionita]

hi, i have the same issues. i've tried this, file .exe successfully created. but, when i run .exe, i got an error 

Traceback (most recent call last):
File "demo_sample.py", line 3, in <module>
ModuleNotFoundError: No module named 'openvino'
[14088] Failed to execute script demo_sample

when i tried to set up openvino environment first in cmd before build exe using pyinstaller, it run well. can you help me to solve this problem? 

[MH8940]

Hi, @Srujana 

I followed your suggestion and added plugins.xml to my pyinstaller cmd argument,

It can produce .exe file, however when I execute it, I got following error,

how can I fix it?

(my pip install openvino version is 2021.4.2

and execute following command on terminal to produce the exe file:

pyinstaller --noconfirm --onedir --windowed --icon "C:/Users/USER/Desktop/lab5/media/ultrasound_icon.ico" --debug "all" --add-data "C:/Users/USER/Desktop/lab5/media;media/" --add-data "C:/Users/USER/Desktop/AIGO/lab5/model_IR;model_IR/" --add-data "C:/Program Files (x86)/Intel/openvino_2021.4.752/inference_engine/bin/intel64/Release/plugins.xml;." "C:/Users/USER/Desktop/lab5/UI.py"

in the "lab5" folder, file structure as below: