Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

How can I share weights between two models loading on same device?

Farhadi
Beginner
744 Views

Hello,

Is it possible to share weights between two models running on the same device?

Also, can we change the model input size after loading it in the device (not before loading same as "Shape Inference")? I want to change the model input size while it's running.

Thanks,

 

0 Kudos
1 Solution
Iffa_Intel
Moderator
705 Views

The page https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_ShapeInference.html (the ones that you are referring to) provides all available methods for runtime model reshaping. We don't have options of model reshaping other than mentioned there as of now.


Also, regarding your question for weights sharing, model weights are stored within .bin file of the resulted IR model. We don't have an embedded option to implement this, however, you could simply use multiple instances of the same .bin file.


Sincerely,

Iffa


View solution in original post

0 Kudos
4 Replies
Iffa_Intel
Moderator
729 Views

Greetings,


You may refer to this: https://www.youtube.com/watch?v=Ga8j0lgi-OQ

This is about utilizing a model optimizer to adjust the model size


Sincerely,

Iffa


0 Kudos
Farhadi
Beginner
719 Views

Thanks,

I am looking to reshape the model while the model is running on the device, not before loading on the device. I follow the provided instruction (https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_ShapeInference.html), but it is for prior loading the model on the device. Actually, I found out there is no way to reshape the model after loading the device.

I also need to load several models on the device, just the weights for these models are same, but it seems OpenVino doesn't support weight sharing.     

0 Kudos
Iffa_Intel
Moderator
706 Views

The page https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_ShapeInference.html (the ones that you are referring to) provides all available methods for runtime model reshaping. We don't have options of model reshaping other than mentioned there as of now.


Also, regarding your question for weights sharing, model weights are stored within .bin file of the resulted IR model. We don't have an embedded option to implement this, however, you could simply use multiple instances of the same .bin file.


Sincerely,

Iffa


0 Kudos
Iffa_Intel
Moderator
699 Views

Greetings,


Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 


Sincerely,

Iffa


0 Kudos
Reply