Error message when running command line on POT

[DarkHorse]

Hi,

I was able to run the pot.exe until recently I did some update in my system and I am having compilation error listed below:

Have you guys seen this error message before? 

(openvino_env) F:\openvino_env\Scripts>pot.exe -c "F:\openvino_env\Scripts\accuracy\mobilenet_v2_pytorch_int8.json" -e --output-dir "F:\IR_Models\int8"
C:\Program Files (x86)\Intel\openvino_2023\python\python3.8\openvino\offline_transformations\__init__.py:10: FutureWarning: The module is private and following namespace `offline_transformations` will be removed in the future.
warnings.warn(
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
INFO:openvino.tools.pot.app.run:Output log dir: F:\IR_Models\int8\mobilenet-v2-pytorch_DefaultQuantization\2023-09-11_23-34-37
INFO:openvino.tools.pot.app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : mixed
stat_subset_size : 300
target_device : ANY
model_type : None
dump_intermediate_model : False
inplace_statistics : True
exec_log_dir : F:\IR_Models\int8\mobilenet-v2-pytorch_DefaultQuantization\2023-09-11_23-34-37
===========================================================================
IE version: 2023.0.1-11005-fa1c41994f3-releases/2023/0
Loaded CPU plugin version:
CPU - openvino_intel_cpu_plugin: 2023.0.2023.0.1-11005-fa1c41994f3-releases/2023/0
Annotation conversion for classification_dataset dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: F:\ImageNet\val.txt
Annotation conversion for classification_dataset dataset has been finished
INFO:openvino.tools.pot.pipeline.pipeline:Inference Engine version: 2023.0.1-11005-fa1c41994f3-releases/2023/0
INFO:openvino.tools.pot.pipeline.pipeline:Model Optimizer version: 2023.0.0-10926-b4452d56304-releases/2023/0
INFO:openvino.tools.pot.pipeline.pipeline:Post-Training Optimization Tool version: 2023.0.0-10926-b4452d56304-releases/2023/0
INFO:openvino.tools.pot.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:openvino.tools.pot.statistics.collector:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "F:\openvino_env\Scripts\pot.exe\__main__.py", line 7, in <module>
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 26, in main
app(sys.argv[1:])
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 59, in app
metrics = optimize(config)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 122, in optimize
compressed_model = pipeline.run(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\pipeline\pipeline.py", line 52, in run
result = self.collect_statistics_and_run(model, current_algo_seq)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\pipeline\pipeline.py", line 62, in collect_statistics_and_run
model = algo.run(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\algorithms\quantization\default\algorithm.py", line 80, in run
self.algorithms[0].algo_collector.compute_statistics(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\statistics\collector.py", line 75, in compute_statistics
_, stats_ = self._engine.predict(combined_stats, sampler, stat_aliases_)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 131, in predict
self.set_model(model_with_stat_op)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 80, in set_model
stdout_redirect(_set_model, self._tmp_dir.name)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\utils\logger.py", line 120, in stdout_redirect
res = fn(*args, **kwargs)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 76, in _set_model
paths = save_model(model, path, tmp_model_name, for_stat_collection=True)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\model_utils.py", line 34, in save_model
model_paths = model.save(save_path, model_name=model_name, for_stat_collection=for_stat_collection)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\nx_model.py", line 115, in save
stdout_redirect(save_graph, model_dict['model'], save_path, name)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\utils\logger.py", line 120, in stdout_redirect
res = fn(*args, **kwargs)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\graph_utils.py", line 86, in save_graph
save_restored_graph(graph=graph_copy, path=save_path, meta_data=graph.meta_data,
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\utils\ir_reader\restore_graph.py", line 91, in save_restored_graph
prepare_emit_ir(graph, data_type, path, name, meta_info=meta_data, rename_results=rename_results)
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\pipeline\common.py", line 247, in prepare_emit_ir
serialize_constants(graph, bin_file)
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\back\ie_ir_ver_2\emitter.py", line 41, in serialize_constants
with open(bin_file_name, 'wb') as bin_file:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmp_9brkjzt\\tmp_model.bin'
Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 613, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmp_9brkjzt\\tmp_model.bin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 803, in onerror
_os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmp_9brkjzt\\tmp_model.bin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\weakref.py", line 642, in _exitfunc
f()
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\weakref.py", line 566, in __call__
return info.func(*info.args, **(info.kwargs or {}))
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 818, in _cleanup
cls._rmtree(name)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 814, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 737, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 615, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 806, in onerror
cls._rmtree(path)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 814, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 737, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 596, in _rmtree_unsafe
onerror(os.scandir, path, sys.exc_info())
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 593, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmp_9brkjzt\\tmp_model.bin'

[Aznie_Intel]

Hi DarkHorse,

Thanks for reaching out.

Are you using the same OpenVINO version before? For your information, the Post-training Optimization Tool has been deprecated since OpenVINO 2023.0. Neural Network Compression Framework (NNCF) is recommended for the post-training quantization instead.

Regards,

Aznie

[DarkHorse]

Hi Aznie,

Yes i am using the same OpenVINO version 2023.0.1

So you guys dont recommend to use POT tools for optimization from now onwards?

Thanks.

[Aznie_Intel]

Hi DarkHorse,

It is recommended to use  Neural Network Compression Framework (NNCF) post-training quantization. You may refer to the release notes for the latest update of OpenVINO.

However, it seems like the POT usage on Windows platform is a known issue caused by accessing the same files in read and write modes. Refer to this Known Issues. You may try following this guide to disable the default switch to the MMap allocator. The issue is fixed for POT quantization on Windows, see the PR in the OpenVINO repository for more information.

Hope this helps.

Regards,

Aznie

[DarkHorse]

Hello Aznie,

Thanks I managed to find the files and add in the workaround.

But seems like I am still getting the same error messages... any idea?

(openvino_env) F:\openvino_env\Scripts>pot.exe -c "F:\openvino_env\Scripts\accuracy\mobilenet_v2_pytorch_int8.json" -e --output-dir "F:\IR_Models\int8"
C:\Program Files (x86)\Intel\openvino_2023\python\python3.8\openvino\offline_transformations\__init__.py:10: FutureWarning: The module is private and following namespace `offline_transformations` will be removed in the future.
warnings.warn(
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
[ DEBUG ] Creating converter from 7 to 5
[ DEBUG ] Creating converter from 5 to 7
Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
Post-training Optimization Tool is deprecated and will be removed in the future. Please use Neural Network Compression Framework instead: https://github.com/openvinotoolkit/nncf
INFO:openvino.tools.pot.app.run:Output log dir: F:\IR_Models\int8\mobilenet-v2-pytorch_DefaultQuantization\2023-09-20_10-45-24
INFO:openvino.tools.pot.app.run:Creating pipeline:
Algorithm: DefaultQuantization
Parameters:
preset : mixed
stat_subset_size : 300
target_device : ANY
model_type : None
dump_intermediate_model : False
inplace_statistics : True
exec_log_dir : F:\IR_Models\int8\mobilenet-v2-pytorch_DefaultQuantization\2023-09-20_10-45-24
===========================================================================
IE version: 2023.0.1-11005-fa1c41994f3-releases/2023/0
Loaded CPU plugin version:
CPU - openvino_intel_cpu_plugin: 2023.0.2023.0.1-11005-fa1c41994f3-releases/2023/0
Annotation conversion for classification_dataset dataset has been started
Parameters to be used for conversion:
converter: imagenet
annotation_file: F:\ImageNet\val.txt
Annotation conversion for classification_dataset dataset has been finished
INFO:openvino.tools.pot.pipeline.pipeline:Inference Engine version: 2023.0.1-11005-fa1c41994f3-releases/2023/0
INFO:openvino.tools.pot.pipeline.pipeline:Model Optimizer version: 2023.0.0-10926-b4452d56304-releases/2023/0
INFO:openvino.tools.pot.pipeline.pipeline:Post-Training Optimization Tool version: 2023.0.0-10926-b4452d56304-releases/2023/0
INFO:openvino.tools.pot.statistics.collector:Start computing statistics for algorithms : DefaultQuantization
INFO:openvino.tools.pot.statistics.collector:Computing statistics finished
INFO:openvino.tools.pot.pipeline.pipeline:Start algorithm: DefaultQuantization
INFO:openvino.tools.pot.algorithms.quantization.default.algorithm:Start computing statistics for algorithm : ActivationChannelAlignment
Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "F:\openvino_env\Scripts\pot.exe\__main__.py", line 7, in <module>
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 26, in main
app(sys.argv[1:])
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 59, in app
metrics = optimize(config)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\app\run.py", line 122, in optimize
compressed_model = pipeline.run(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\pipeline\pipeline.py", line 52, in run
result = self.collect_statistics_and_run(model, current_algo_seq)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\pipeline\pipeline.py", line 62, in collect_statistics_and_run
model = algo.run(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\algorithms\quantization\default\algorithm.py", line 80, in run
self.algorithms[0].algo_collector.compute_statistics(model)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\statistics\collector.py", line 75, in compute_statistics
_, stats_ = self._engine.predict(combined_stats, sampler, stat_aliases_)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 131, in predict
self.set_model(model_with_stat_op)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 80, in set_model
stdout_redirect(_set_model, self._tmp_dir.name)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\utils\logger.py", line 120, in stdout_redirect
res = fn(*args, **kwargs)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\engines\ac_engine.py", line 76, in _set_model
paths = save_model(model, path, tmp_model_name, for_stat_collection=True)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\model_utils.py", line 34, in save_model
model_paths = model.save(save_path, model_name=model_name, for_stat_collection=for_stat_collection)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\nx_model.py", line 115, in save
stdout_redirect(save_graph, model_dict['model'], save_path, name)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\utils\logger.py", line 120, in stdout_redirect
res = fn(*args, **kwargs)
File "f:\openvino_env\lib\site-packages\openvino\tools\pot\graph\graph_utils.py", line 87, in save_graph
save_restored_graph(graph=graph_copy, path=save_path, meta_data=graph.meta_data,
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\utils\ir_reader\restore_graph.py", line 91, in save_restored_graph
prepare_emit_ir(graph, data_type, path, name, meta_info=meta_data, rename_results=rename_results)
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\pipeline\common.py", line 247, in prepare_emit_ir
serialize_constants(graph, bin_file)
File "f:\openvino_env\lib\site-packages\openvino\tools\mo\back\ie_ir_ver_2\emitter.py", line 41, in serialize_constants
with open(bin_file_name, 'wb') as bin_file:
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmpft8f8n0t\\tmp_model.bin'
Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 613, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmpft8f8n0t\\tmp_model.bin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 803, in onerror
_os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmpft8f8n0t\\tmp_model.bin'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\weakref.py", line 642, in _exitfunc
f()
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\weakref.py", line 566, in __call__
return info.func(*info.args, **(info.kwargs or {}))
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 818, in _cleanup
cls._rmtree(name)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 814, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 737, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 615, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 806, in onerror
cls._rmtree(path)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\tempfile.py", line 814, in _rmtree
_shutil.rmtree(name, onerror=onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 737, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 596, in _rmtree_unsafe
onerror(os.scandir, path, sys.exc_info())
File "C:\Users\allensen\AppData\Local\Programs\Python\Python38\lib\shutil.py", line 593, in _rmtree_unsafe
with os.scandir(path) as scandir_it:
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\allensen\\AppData\\Local\\Temp\\tmpft8f8n0t\\tmp_model.bin'

Thanks for the help.

[DarkHorse]

Hello Aznie,

I saw the workaround is stated as 

• Weights mapping - memory mapping (using mmap) has been introduced as the default way to work with weights. Currently, this feature is supported by the IR frontend. Mapping may be switched by specifying the ov::enable_mmap(BOOL) property for the ov::Core. Because of its “memory-on-demand” nature, there is no need to store all weights in RAM. Storing just the data that is needed at the moment lowers the amount of memory required for compilation. Moreover, mmap provides extensive memory sharing, so the consecutive compilation of the same model will fetch the information already stored in RAM instead of reading it one more time from storage.

By the way I look at the github fix but I dont seems to be able to find the files location

Right now I am running pot.exe function from openvino_env\Scripts\pot.exe 

May I know where i can locate these files?

tools/pot/openvino/tools/pot/engines/ie_engine.py

tools/pot/openvino/tools/pot/graph/graph_utils.py

tools/pot/openvino/tools/pot/utils/launcher.py

[Aznie_Intel]

Hi DarkHorse,

You may find the files in your pot directory: <path-to-venv>Lib\site-packages\openvino\tools\pot\graph. All files are available in your pot directory. If you are installing OpenVINO from PyPI, your OpenVINO directory is depends to your Python location: <path-to-venv>/site-packages/openvino.

Regards,

Aznie

[Aznie_Intel]

Hi Darkhouse,

What update did you do with your system before? Can you try using an older version of OpenVINO and also using a different file to test the POT?

Regards,

Aznie

[Aznie_Intel]

Hi Darkhouse,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored. 

Regards,

Aznie