Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

HRNetV2C1 Conversion from .pth to IR

Norrie
Beginner
1,078 Views

Hello, I am using the HRNetV2C1 model for inference on a ROS2 Humble AMR with Intel NUC13 (OS: Ubuntu 22.04).

I am using ROS2 Openvino Toolkit (-b ros2) to load the model in the .xml format, and it has been working without errors.

However, after training the model on Google Colab in the PyTorch framework and converting the trained model, A) directly into the IR format, or B) first into ONNX and then into the IR format, I get the same error when loading the model (conversion process itself completes without errors):

[pipeline_with_params-5] [ ERROR ] Exception from src/inference/src/core.cpp:99:

[pipeline_with_params-5] Check '(axis_range_min <= axis) && (axis <= axis_range_max)' failed at src/core/src/validation_util.cpp:1121:

[pipeline_with_params-5] Gather Parameter axis -6260240599907491525 out of the tensor rank range [-1, 0].

 

Given that this error appears in both A) and B) scenarios, I suspect that the conversion process from .pth to .onnx has some flaws.

Could you identify potential causes for this in the following script?

Thank you very much!

 

 

A) Direct conversion:

from google.colab import drive
drive.mount('/content/drive')
# Install openvino package
%pip install -q "openvino>=2023.3.0" onnx
 
import torch
import sys
import openvino as ov
from pathlib import Path

# Adjust sys.path to include the path to your HRNetV2C1 model implementation
sys.path.append('/content/drive/Shareddrives/PSOL/open_model_zoo/models/public/hrnet-v2-c1-segmentation')
sys.path.append('/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation')
sys.path.append('/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/mit_semseg')

from model import HrnetV2C1

# Initialize the model
model = HrnetV2C1(encoder_weights='/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/ckpt/encoder_epoch_30.pth',
decoder_weights='/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/ckpt/decoder_epoch_30.pth')

# Load the trained weights
weights_path = "/content/drive/Shareddrives/PSOL/preprocessed/best_model_weights.pth"
model.load_state_dict(torch.load(weights_path, map_location=torch.device('cpu')))

# Set the model to evaluation mode
model.eval()

# Define a dummy input tensor. The size might need to be adjusted based on your model's input.
example_input = torch.randn(1, 3, 320, 320)
ov_model = ov.convert_model(model, example_input=example_input)

ov_ir_path = Path("/content/drive/Shareddrives/PSOL/preprocessed/")
ov_ir_path.mkdir(parents=True, exist_ok=True)

# Save the OpenVINO model
ov.save_model(ov_model, ov_ir_path / "hrnet-v2-c1.xml")

print(f"OpenVINO model was saved successfully to {ov_ir_path}")
 
 
 
B) Into ONNX, then into IR:
from google.colab import drive
drive.mount('/content/drive')
 
# Install openvino package
%pip install -q "openvino>=2023.3.0" onnx
 
import time
import os
import sys
import warnings
from pathlib import Path
import cv2
import numpy as np
import openvino as ov
import torch
import onnx
 
IMAGE_WIDTH = 320
IMAGE_HEIGHT = 320
weights_path = "/content/drive/Shareddrives/PSOL/preprocessed/best_model_weights.pth"

# Paths where ONNX and OpenVINO IR models will be stored.
onnx_path = '/content/drive/Shareddrives/PSOL/preprocessed/model.onnx'
ir_path = "/content/drive/Shareddrives/PSOL/preprocessed/hrnet-v2-c1-segmentation.xml"
 
# Model weights
sys.path.append('/content/drive/Shareddrives/PSOL/open_model_zoo/models/public/hrnet-v2-c1-segmentation')

# Model network
sys.path.append('/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/mit_semseg')
sys.path.append('/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation')

from model import HrnetV2C1
model = HrnetV2C1(encoder_weights='/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/ckpt/encoder_epoch_30.pth',
decoder_weights='/content/drive/Shareddrives/PSOL/hrnet-models/hrnet-v2-c1-segmentation/ckpt/decoder_epoch_30.pth')

# Load the trained weights into the model
model.load_state_dict(torch.load(weights_path, map_location=torch.device('cpu')))

# Move the model to the GPU if available
model.eval()
 
with warnings.catch_warnings():
    warnings.filterwarnings("ignore")
    dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
    torch.onnx.export(
    model,
    dummy_input,
    onnx_path,
    )
    print(f"ONNX model exported to {onnx_path}.")
 
# Load the ONNX model
onnx_model = onnx.load(onnx_path)

# Check the model
onnx.checker.check_model(onnx_model)
print("The model is checked and valid.")
 
if os.path.exists(onnx_path) and os.path.getsize(onnx_path) > 0:
    print("ONNX model file exists and is not empty.")
else:
    print("ONNX model file does not exist or is empty.")

ov_model = ov.convert_model(onnx_path)
ov.save_model(ov_model, ir_path)
0 Kudos
5 Replies
Wan_Intel
Moderator
1,018 Views

Hi Norrie,

Thanks for reaching out to us.

 

Could you please share the custom HRNetV2C1 model with us so that we can further investigate the issue from our end?

 

 

Regards,

Wan

 

0 Kudos
Norrie
Beginner
984 Views

Hello Wan,

 

Sorry, it was my fundamental mistake.

In the param file of ROS2 OpenVINO Toolkit, the model file is specified by referring only to the .xml file.

That is why I kept updating only the .xml file and caused mismatch between that and the .bin file in the same folder, which is apparently automatically found and read by the ROS2 node.

After replacing both the .xml and .bin files, the trained model was normally loaded without any error.

Please close the issue, and I'm sorry for disturbing you with such a blunder.

 

Best regards,

Norrie

Wan_Intel
Moderator
961 Views

Hi Norrie,

Thanks for the information.

We're glad to know that you have solved the issue. If you need additional information from Intel, please submit a new question and we'll be happy to help.

 

 

Regards,

Wan

 

0 Kudos
Hance
Beginner
353 Views
In Ubuntu22.04, how can ROS2 Humble be connected to OpenVINO so that ROS2 can be used to call the motion model
0 Kudos
Hance
Beginner
314 Views
In Ubuntu22.04, how can ROS2 Humble be connected to OpenVINO so that ROS2 can be used to call the motion model
0 Kudos
Reply