- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Help!
Hello, I want to use OpenVino to convert the pytorch model to the ov model. The reference textbook is this link tutorial, but I failed and encountered some errors:
Traceback (most recent call last):
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 535, in _convert
raise e
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 478, in _convert
ov_model = driver(argv, {"conversion_parameters": non_default_params})
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 230, in driver
ov_model = moc_emit_ir(prepare_ir(argv), argv)
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 180, in prepare_ir
ov_model = moc_pipeline(argv, moc_front_end)
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\moc_frontend\pipeline.py", line 247, in moc_pipeline
ov_model = moc_front_end.convert(input_model)
File "D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\frontend.py", line 18, in convert
converted_model = super().convert(model)
openvino._pyopenvino.OpConversionFailure: Check 'is_conversion_successful' failed at src\frontends\pytorch\src\frontend.cpp:140:
FrontEnd API failed with OpConversionFailure:
Model wasn't fully converted. Failed operations detailed log:
-- ov::align_types with a message:
This is internal operation for type alignment and should be removed at normalization step. It can't be removed if types can't be resolved.
-- prim::Constant with a message:
Exception happened during conversion of operation prim::Constant_115 with schema (no schema)
TypeError: Tensor.numpy() takes no keyword arguments
At:
D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\utils.py(70): torch_tensor_to_ov_const
D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\utils.py(87): ivalue_to_constant
D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\ts_decoder.py(319): as_constant
D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\ts_decoder.py(228): visit_subgraph
D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\frontend.py(18): convert
D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\moc_frontend\pipeline.py(247): moc_pipeline
D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(180): prepare_ir
D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(230): driver
D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(478): _convert
D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert.py(101): convert_model
D:\code\Multi_HASS\Openvino_ptorch_test\t1.py(83): <module>
D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py(18): execfile
D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(1496): _exec
D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(1489): run
D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(2177): main
D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(2195): <module>
Summary:
-- Conversion is failed for: ov::align_types, prim::Constant
python-BaseException
The following is the code I implemented.
import requests
from pathlib import Path
from PIL import Image
import torchvision
import torch
import numpy as np
from scipy.special import softmax
import openvino as ov
MODEL_DIR = Path("model")
DATA_DIR = Path("data")
MODEL_DIR.mkdir(exist_ok=True)
DATA_DIR.mkdir(exist_ok=True)
MODEL_NAME = "regnet_y_800mf"
image = Image.open(requests.get("https://farm9.staticflickr.com/8225/8511402100_fea15da1c5_z.jpg", stream=True).raw)
labels_file = DATA_DIR / "imagenet_2012.txt"
if not labels_file.exists():
resp = requests.get("https://raw.githubusercontent.com/openvinotoolkit/open_model_zoo/master/data/dataset_classes/imagenet_2012.txt")
with labels_file.open("wb") as f:
f.write(resp.content)
imagenet_classes = labels_file.open("r").read().splitlines()
# get default weights using available weights Enum for model
weights = torchvision.models.RegNet_Y_800MF_Weights.DEFAULT
# create model topology and load weights
model = torchvision.models.regnet_y_800mf(weights=weights)
# switch model to inference mode
model.eval()
# Initialize the Weight Transforms
preprocess = weights.transforms()
# Apply it to the input image
img_transformed = preprocess(image)
# Add batch dimension to image tensor
input_tensor = img_transformed.unsqueeze(0)
# Perform model inference on input tensor
result = model(input_tensor)
# Postprocessing function for getting results in the same way for both PyTorch model inference and OpenVINO
def postprocess_result(output_tensor:np.ndarray, top_k:int = 5):
"""
Posprocess model results. This function applied sofrmax on output tensor and returns specified top_k number of labels with highest probability
Parameters:
output_tensor (np.ndarray): model output tensor with probabilities
top_k (int, *optional*, default 5): number of labels with highest probability for return
Returns:
topk_labels: label ids for selected top_k scores
topk_scores: selected top_k highest scores predicted by model
"""
softmaxed_scores = softmax(output_tensor, -1)[0]
topk_labels = np.argsort(softmaxed_scores)[-top_k:][::-1]
topk_scores = softmaxed_scores[topk_labels]
return topk_labels, topk_scores
# Postprocess results
top_labels, top_scores = postprocess_result(result.detach().numpy())
# Show results
for idx, (label, score) in enumerate(zip(top_labels, top_scores)):
_, predicted_label = imagenet_classes[label].split(" ", 1)
print(f"{idx + 1}: {predicted_label} - {score * 100 :.2f}%")
model(input_tensor)
core = ov.Core()
# Convert model to openvino.runtime.Model object
ov_model = ov.convert_model(model)
# Save openvino.runtime.Model object on disk
ov.save_model(ov_model, MODEL_DIR / f"{MODEL_NAME}_dynamic.xml")
Here are my requirements.txt
certifi==2023.11.17
charset-normalizer==3.3.2
idna==3.6
numpy==1.26.2
opencv-python==4.8.1.78
openvino==2023.2.0
openvino-telemetry==2023.2.1
Pillow==10.1.0
requests==2.31.0
scipy==1.11.4
torch==1.12.0
torchaudio==0.12.0
torchvision==0.13.0
typing_extensions==4.9.0
urllib3==2.1.0
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Fanzhihao,
Thanks for reaching out.
To convert a PyTorch model into Intermediate Representation (IR) format, you need to convert PyTorch model to ONNX, then to OpenVINO IR. You may follow the steps in Exporting a PyTorch Model to ONNX Format.
Regards,
Aznie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aznie,
Thank you very much for your reply!
I also used pytorch to onnx and then OpenVINO IR format before. I just found out that the latest version seems to support directly converting pytorch mods to OpenVINO IR, so I wanted to try it.
BR,
Fanzhihao
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Fanzhihao,
As mentioned in Convert PyTorch Model to OpenVINO Intermediate Representation direct support for PyTorch model conversion is an experimental feature. Model coverage will be increased in the next releases. For cases, when PyTorch model conversion failed, you still can try to export the model to ONNX format. Please refer to this tutorial which explains how to convert PyTorch model to ONNX, then to OpenVINO.
Meanwhile, I have run your code and can see the output.
Regards,
Aznie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Fanzhihao,
This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question.
Regards,
Aznie
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I meet the same problem. On conclusion, there exists some problem when directly converting pytorch models to OpenVINO IR format.
As the link of the Moderator failed to be open, I post the alternative solution here.
1, firstly convert pytorch model to onnx
import torch
torch.onnx.export(mymodel,(input_tensor,),'./data/model.onnx')
2, convert the onnx model to openvino
import openvino as ov
core = ov.Core()
ov_model = core.read_model('data/model.onnx')
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page