Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6498 Discussions

The function "convert_model" cannot be used

fanzhihao
Beginner
2,688 Views

Help!

 

Hello, I want to use OpenVino to convert the pytorch model to the ov model. The reference textbook is this link tutorial, but I failed and encountered some errors:

 

 

Traceback (most recent call last):
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 535, in _convert
    raise e
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 478, in _convert
    ov_model = driver(argv, {"conversion_parameters": non_default_params})
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 230, in driver
    ov_model = moc_emit_ir(prepare_ir(argv), argv)
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py", line 180, in prepare_ir
    ov_model = moc_pipeline(argv, moc_front_end)
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\moc_frontend\pipeline.py", line 247, in moc_pipeline
    ov_model = moc_front_end.convert(input_model)
  File "D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\frontend.py", line 18, in convert
    converted_model = super().convert(model)
openvino._pyopenvino.OpConversionFailure: Check 'is_conversion_successful' failed at src\frontends\pytorch\src\frontend.cpp:140:
FrontEnd API failed with OpConversionFailure:
Model wasn't fully converted. Failed operations detailed log:
-- ov::align_types with a message:
This is internal operation for type alignment and should be removed at normalization step. It can't be removed if types can't be resolved.
-- prim::Constant with a message:
Exception happened during conversion of operation prim::Constant_115 with schema (no schema)
TypeError: Tensor.numpy() takes no keyword arguments

At:
  D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\utils.py(70): torch_tensor_to_ov_const
  D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\utils.py(87): ivalue_to_constant
  D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\ts_decoder.py(319): as_constant
  D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\pytorch\ts_decoder.py(228): visit_subgraph
  D:\Anaconda\envs\py310\lib\site-packages\openvino\frontend\frontend.py(18): convert
  D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\moc_frontend\pipeline.py(247): moc_pipeline
  D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(180): prepare_ir
  D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(230): driver
  D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert_impl.py(478): _convert
  D:\Anaconda\envs\py310\lib\site-packages\openvino\tools\ovc\convert.py(101): convert_model
  D:\code\Multi_HASS\Openvino_ptorch_test\t1.py(83): <module>
  D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py(18): execfile
  D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(1496): _exec
  D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(1489): run
  D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(2177): main
  D:\pycharm\PyCharm Community Edition 2022.2.3\plugins\python-ce\helpers\pydev\pydevd.py(2195): <module>

Summary:
-- Conversion is failed for: ov::align_types, prim::Constant

python-BaseException

 

 

 

The following is the code I implemented.

 

import requests
from pathlib import Path
from PIL import Image
import torchvision
import torch

import numpy as np
from scipy.special import softmax
import openvino as ov


MODEL_DIR = Path("model")
DATA_DIR = Path("data")

MODEL_DIR.mkdir(exist_ok=True)
DATA_DIR.mkdir(exist_ok=True)
MODEL_NAME = "regnet_y_800mf"

image = Image.open(requests.get("https://farm9.staticflickr.com/8225/8511402100_fea15da1c5_z.jpg", stream=True).raw)

labels_file = DATA_DIR / "imagenet_2012.txt"

if not labels_file.exists():
    resp = requests.get("https://raw.githubusercontent.com/openvinotoolkit/open_model_zoo/master/data/dataset_classes/imagenet_2012.txt")
    with labels_file.open("wb") as f:
        f.write(resp.content)

imagenet_classes = labels_file.open("r").read().splitlines()


# get default weights using available weights Enum for model
weights = torchvision.models.RegNet_Y_800MF_Weights.DEFAULT

# create model topology and load weights
model = torchvision.models.regnet_y_800mf(weights=weights)

# switch model to inference mode
model.eval()

# Initialize the Weight Transforms
preprocess = weights.transforms()

# Apply it to the input image
img_transformed = preprocess(image)

# Add batch dimension to image tensor
input_tensor = img_transformed.unsqueeze(0)


# Perform model inference on input tensor
result = model(input_tensor)


# Postprocessing function for getting results in the same way for both PyTorch model inference and OpenVINO
def postprocess_result(output_tensor:np.ndarray, top_k:int = 5):
    """
    Posprocess model results. This function applied sofrmax on output tensor and returns specified top_k number of labels with highest probability
    Parameters:
      output_tensor (np.ndarray): model output tensor with probabilities
      top_k (int, *optional*, default 5): number of labels with highest probability for return
    Returns:
      topk_labels: label ids for selected top_k scores
      topk_scores: selected top_k highest scores predicted by model
    """
    softmaxed_scores = softmax(output_tensor, -1)[0]
    topk_labels = np.argsort(softmaxed_scores)[-top_k:][::-1]
    topk_scores = softmaxed_scores[topk_labels]
    return topk_labels, topk_scores

# Postprocess results
top_labels, top_scores = postprocess_result(result.detach().numpy())

# Show results
for idx, (label, score) in enumerate(zip(top_labels, top_scores)):
    _, predicted_label = imagenet_classes[label].split(" ", 1)
    print(f"{idx + 1}: {predicted_label} - {score * 100 :.2f}%")


model(input_tensor)
core = ov.Core()

# Convert model to openvino.runtime.Model object
ov_model = ov.convert_model(model)

# Save openvino.runtime.Model object on disk
ov.save_model(ov_model, MODEL_DIR / f"{MODEL_NAME}_dynamic.xml")

 

 

 

Here are my requirements.txt

 

certifi==2023.11.17
charset-normalizer==3.3.2
idna==3.6
numpy==1.26.2
opencv-python==4.8.1.78
openvino==2023.2.0
openvino-telemetry==2023.2.1
Pillow==10.1.0
requests==2.31.0
scipy==1.11.4
torch==1.12.0
torchaudio==0.12.0
torchvision==0.13.0
typing_extensions==4.9.0
urllib3==2.1.0

 

 

0 Kudos
6 Replies
Aznie_Intel
Moderator
2,663 Views

Hi Fanzhihao,

 

Thanks for reaching out.

 

To convert a PyTorch model into Intermediate Representation (IR) format, you need to convert PyTorch model to ONNX, then to OpenVINO IR. You may follow the steps in Exporting a PyTorch Model to ONNX Format.

 

 

Regards,

Aznie


0 Kudos
fanzhihao
Beginner
2,649 Views

Hi Aznie,

 

Thank you very much for your reply!

 

I also used pytorch to onnx and then OpenVINO IR format before. I just found out that the latest version seems to support directly converting pytorch mods to OpenVINO IR, so I wanted to try it.

 

BR,

Fanzhihao

0 Kudos
Aznie_Intel
Moderator
2,604 Views

Hi Fanzhihao,

 

As mentioned in Convert PyTorch Model to OpenVINO Intermediate Representation direct support for PyTorch model conversion is an experimental feature. Model coverage will be increased in the next releases. For cases, when PyTorch model conversion failed, you still can try to export the model to ONNX format. Please refer to this tutorial which explains how to convert PyTorch model to ONNX, then to OpenVINO.

 

Meanwhile, I have run your code and can see the output.

Fanzhihao.JPG 

 

 

Regards,

Aznie

 

0 Kudos
fanzhihao
Beginner
2,590 Views

Hi Aznie,

 

OK, thanks!

 

BR,

Fanzhihao

0 Kudos
Aznie_Intel
Moderator
2,559 Views

Hi Fanzhihao,


This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question. 



Regards,

Aznie


0 Kudos
IMly
Beginner
1,769 Views

I meet the same problem. On conclusion, there exists some problem when directly converting pytorch models to OpenVINO IR format.

As the link of the Moderator failed to be open, I post the alternative solution here.

1, firstly convert pytorch model to onnx

import torch

torch.onnx.export(mymodel,(input_tensor,),'./data/model.onnx')

2, convert the onnx model to openvino

import openvino as ov
core = ov.Core()
ov_model = core.read_model('data/model.onnx')
0 Kudos
Reply