From my side, I don't see any issue converting & inferencing both the original YOLOv7 and your model (model named best)
Conversion to ONNX using the script export.py
YOLOV7
BEST
Infer the ONNX model using OpenVINO benchmark_app (default infer without device specification means I'm using intel CPU)
YOLOV7
BEST
Convert ONNX into OpenVINO IR with FP16 precision
YOLOV7
BEST
Infer the IR with NCS2/MYRIAD
YOLOV7
BEST
Therefore, from OpenVINO perspective, I don't see any issue with the model.
For implementing those YOLO models (training/infer/etc) in Google Colab you'll need to think out of the box and figure it out by reading the Google Colab documentation. There's a functionality in that collab where you could import your own script and utilize it.
This part is beyond OpenVINO utilization hence it's irrelevant to be answered in this forum.
Note: one of the main purpose of using Intel OpenVINO is to enable inferencing on Intel GPU.
If you want to use CUDA/NVDIA, you might as well use other Deep Learning Platforms e.g. Tensorflow & OpenCV.
Sincerely,
Iffa
Link Copied
Hi,
Thank you for your patience. After doing some clarification, this training extension is validated to be run with Ubuntu 18.04 / 20.04 only. ( I tried this on both windows 10 and ubuntu 20 & only ubuntu works.)
As of now, there's no plan of enabling this on Windows 10 yet.
Hope you are aware of these prerequisites (which are stated in the official doc):
- Ubuntu 18.04 / 20.04
- Python 3.8+
- CUDA Toolkit 11.1 - for training on GPU In order to get started with OpenVINO™ Training Extensions click here.
As I mentioned above, I managed to run the OTE in my Ubuntu 20.04:
I just followed the official documentation or the GitHub quick start page.
You could try to use the older version if you want to, ( same as in the video) but you need to have the same older environment.
Prerequisites
- Ubuntu 18.04 / 20.04
- Python 3.6+
- OpenVINO™ - for exporting and running models
- CUDA Toolkit 10.2 - for training on GPU
- Tensorflow 1.13.1
Older training_extension
Older training_extension_models
If your main objective is just to run a custom model on NCS2, I suggest you just train your model in your native platform (eg: tensorflow) and then convert that trained model into IR with FP16 model precision.
Run that trained using OpenVINO benchmark app with parameters -d MYRIAD and see whether it could infer.
I believe this would be easier for you compared to re-training your model using the OpenVINO Training Extension since you are already familiar with your native development platform.
Sincerely,
Iffa
here is a link to the weights file :
https://drive.google.com/file/d/100BIULHH_FTV4y6WPNmJGE03EwHpZYy8/view?usp=share_link
The weights model works fine in yolov7, it gives me the output i am seeking. The problem is just that yolov7 only runs on my CPU and it is slow. I have 2x NCS2 sticks and I would like to get a custom model running on my NCS2.
here is a demo image that the model should run detection on :
i'm not sure what additional info i can post.
here is the yoloV7 command i used to generate the weights file "best.pt" in google colab:
!python train.py --device 0 --batch-size 16 --epochs 300 --img 640 640 --data data/coco.yaml --hyp data/hyp.scratch.custom.yaml --cfg cfg/training/yolov7.yaml --weights yolov7.pt --name goodies
the python code i used to convert the model was lifted and modified from this reference
Convert a PyTorch Model to ONNX and OpenVINO™ IR — OpenVINO™ documentation — Version(latest)
here is the conversion code i used :
device = 'cpu'
ie_core = Core()
sys.path.append("./utils")
import notebook_utils as utils
# from notebook_utils import CityScapesSegmentation, segmentation_map_to_image, viz_result_image # pip install nbutils
# settings :
IMAGE_WIDTH = 640 # Suggested values: 2048, 1024 or 512. The minimum width is 512.
# Set IMAGE_HEIGHT manually for custom input sizes. Minimum height is 512.
IMAGE_HEIGHT = 640 # if IMAGE_WIDTH == 2048 else 512
DIRECTORY_NAME = "mymodels"
BASE_MODEL_NAME = DIRECTORY_NAME + f"/best"
# Paths where PyTorch, ONNX and OpenVINO IR models will be stored.
model_path = Path(BASE_MODEL_NAME).with_suffix(".pth")
print(f"modelpath: {model_path}")
onnx_path = model_path.with_suffix(".onnx")
ir_path = model_path.with_suffix(".xml")
model = attempt_load( model_path.with_suffix(".pt"), map_location=device) # load model
model.train(False)
# #----------------------------------------------------------------------
# Convert PyTorch model to ONNX
if not onnx_path.exists():
x = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH, requires_grad=True)
#
# # For the Fastseg model, setting do_constant_folding to False is required
# # for PyTorch>1.5.1
torch.onnx.export(
model,
x,
onnx_path,
opset_version=11
# do_constant_folding=False,
)
print(f"ONNX model exported to {onnx_path}.")
else:
print(f"ONNX model {onnx_path} already exists.")
#
#
# Convert ONNX Model to OpenVINO IR Format
# Construct the command for Model Optimizer.
mo_command = f"""mo
--input_model "{onnx_path}"
--input_shape "[1,3, {IMAGE_HEIGHT}, {IMAGE_WIDTH}]"
--mean_values="[123.675, 116.28 , 103.53]"
--scale_values="[58.395, 57.12 , 57.375]"
--data_type FP16
--output_dir "{model_path.parent}"
"""
mo_command = " ".join(mo_command.split())
print("Model Optimizer command to convert the ONNX model to OpenVINO:")
display(Markdown(f"`{mo_command}`"))
if not ir_path.exists():
print("Exporting ONNX model to IR... This may take a few minutes.")
# mo_result = %sx $mo_command
mo_result = os.system(mo_command)
print("created onnx model")
else:
print(f"IR model {ir_path} already exists.")from here i have my best.bin & best.xml file, but i do not know if they are working properly. I use code lifted & modified from this page openvino_notebooks/notebooks/004-hello-detection at main · openvinotoolkit/openvino_notebooks · GitHub
to attempt detection and classification, but this code fails. I'm not good at python. Should I also re-post the code I used to attempt inference detection and classification?
here it is :
ie = Core()
classification_model_xml = ir_path
model = ie.read_model(model=classification_model_xml)
print(f"model any_name : {model.input(0).any_name}")
input_layer = model.input(0)
print(f"input precision: {input_layer.element_type}")
print(f"input shape: {input_layer.shape}")
output_layer = model.output(0)
print(f"output precision: {output_layer.element_type}")
print(f"output shape: {output_layer.shape}")
compiled_model = ie.compile_model(model=model, device_name="CPU")
input_layer = compiled_model.input(0)
output_layer = compiled_model.output(0)
image_filename = "assets/demoimage.jpg"
image = cv2.imread(image_filename)
print(f"image shape {image.shape}")
N, C, H, W = input_layer.shape
resized_image = cv2.resize(src=image, dsize=(W, H))
print(f"resized image shape : {resized_image.shape}")
input_data = np.expand_dims(np.transpose(resized_image, (2, 0, 1)), 0).astype(np.float32)
print(f"input data shape : {input_data.shape}")
result = compiled_model([input_data])[output_layer]
result_index = np.argmax(result)
print(f"result index : {result_index}")
boxes = result
boxes = boxes[~np.all(boxes == 0, axis=2)]
# For each detection, the description is in the [x_min, y_min, x_max, y_max, conf] format:
# The image passed here is in BGR format with changed width and height. To display it in colors expected by matplotlib, use cvtColor function
def convert_result_to_image(bgr_image, resized_image, boxes, threshold=0.3, conf_labels=True):
# Define colors for boxes and descriptions.
colors = {"red": (255, 0, 0), "green": (0, 255, 0)}
# Fetch the image shapes to calculate a ratio.
(real_y, real_x), (resized_y, resized_x) = bgr_image.shape[:2], resized_image.shape[:2]
ratio_x, ratio_y = real_x / resized_x, real_y / resized_y
# Convert the base image from BGR to RGB format.
rgb_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB)
# Iterate through non-zero boxes.
# for box in boxes:
# Pick a confidence factor from the last place in an array.
# conf = box[-1]
# if conf > threshold:
# Convert float to int and multiply corner position of each box by x and y ratio.
# If the bounding box is found at the top of the image,
# position the upper box bar little lower to make it visible on the image.
# (x_min, y_min, x_max, y_max) = [
# int(max(corner_position * ratio_y, 10)) if idx % 2
# else int(corner_position * ratio_x)
# for idx, corner_position in enumerate(box[:-1])
# ]
# Draw a box based on the position, parameters in rectangle function are: image, start_point, end_point, color, thickness.
# rgb_image = cv2.rectangle(rgb_image, (x_min, y_min), (x_max, y_max), colors["green"], 3)
# Add text to the image based on position and confidence.
# Parameters in text function are: image, text, bottom-left_corner_textfield, font, font_scale, color, thickness, line_type.
# if conf_labels:
# rgb_image = cv2.putText(
# rgb_image,
# f"{conf:.2f}",
# (x_min, y_min - 10),
# cv2.FONT_HERSHEY_SIMPLEX,
# 0.8,
# colors["red"],
# 1,
# cv2.LINE_AA,
# )
return rgb_image
cv2.imshow("image",convert_result_to_image(image, resized_image, boxes));
cv2.waitKey(0)
cv2.destroyAllWindows()