- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi,
I am running a OVModelForCausalLM sample code which results in a RuntimeError while compiling.
My code:
def ovConvertAlt(optm_dir, model_id, device):
'''OpenVINO Convert Alternate - Conduct model optimization via Intel OpenVINO.
Args:
optm_dir:= path to save OpenVINO model
model_id:= model repo identifier
device:= one of "CPU", "GPU"
Return: None
'''
from optimum.intel.openvino import OVModelForCausalLM
model_ov = OVModelForCausalLM.from_pretrained(model_id, export=True, device=device)
model_ov.save_pretrained(optm_dir)
device is set to 'GPU'
optm_dir is set to the path to save the model
model_id is set to "databricks/dolly-v2-3b"
The execution ends in RuntimeError: with being identical to that of that listed at
https://docs.openvino.ai/2023.0/notebooks/240-dolly-2-instruction-following-with-output.html
Saving external data to one file...
Compiling the model...
Set CACHE_DIR to /tmp/tmpdi3xa6vi/model_cache
Traceback (most recent call last):
...
File "/home/jellybean/workspace/optimum_hf/src/./ovino_optimum.py", line 115, in ovConvertAlt
model_ov = OVModelForCausalLM.from_pretrained(model_id, export=True, device=device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
File "/home/jellybean/.virtualenvs/optimum_hf/lib64/python3.11/site-packages/openvino/runtime/ie_api.py", line 398, in compile_model
super().compile_model(model, device_name, {} if config is None else config),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Check 'false' failed at src/inference/src/core.cpp:114:
clEnqueueMapBuffer
Can someone help decipher this issue?
Thanks
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hairul,
Thanks for the link, I had been on that page before, but somehow I missed the Dolly v2 notebook on it.
I ran the code to confirm that it does work. I am however still having issues with the same code running as a Python module in my venv rather than as a notebook.
Checking thru installed packages there are different versions for torch, cuda, and some others. I will try to realign the packages and try again therafter.
For the time being lets consider the issue closed.
Thanks,
Berdy
コピーされたリンク
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Berdy,
Thank you for reaching out to us.
We're investigating this issue and will update you on any findings as soon as possible.
Regards,
Hairul
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Hairul_Intel,
Thanks for the reply. Perhaps the following provides further info for the investigation.
I attempted to first convert databricks/dolly-v2-3b to ONNX, and then convert from ONNX to OpenVINO with mo from openvino.tools. I never get to create the OpenVINO model since the conversion to ONNX fails. ONNX conversion code:
def onnxModel(onnx_file_path, model_id):
'''ONNX Model - Convert HF Pytorch to ONNX model
Args:
onnx_file_path:= path to onnx model file name
model_id:= HF model repo identifier
Return: None
'''
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.onnx import export, FeaturesManager
tokenizer = AutoTokenizer.from_pretrained(model_id)
hf_model = AutoModelForCausalLM.from_pretrained(model_id)
model_kind, model_onnx_config = FeaturesManager.check_supported_model_or_raise(hf_model, feature="text-generation")
onnx_config = model_onnx_config(hf_model.config)
# export to onnx format
export(preprocessor=tokenizer, model=hf_model, config=onnx_config, opset=onnx_config.default_onnx_opset, output=onnx_file_path)
This results in KeyError: "gpt-neox is not supported yet..., at line
model_features = FeaturesManager.get_supported_features_for_model_type(model_type, model_name=model_name)
Indicates that gpt-neo is supported, but not gpt-neox which is the basis for Dolly-v2.
The strange thing is that with the previously posted ovConvertAlt() function, which uses OVModelForCausalLM.from_pretrained
shows the conversion to ONNX being completed. That code fails during compile, not during ONNX conversion. Perhaps the compile fails because model.onnx is configured incorrectly.
Best,
Berdy
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Berdy,
We've escalated this issue to the relevant teams for further investigation and will update you on any findings as soon as possible.
Regards,
Hairul
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Berdy,
Thank for your patience.
We've received feedback from the Engineering team regarding this issue. This issue is no longer observed on the Master branch. Here is the result from running the notebook from the Master branch:
Please have a try using the Master branch and let us know if you require any further assistance.
Regards,
Hairul
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Hairul,
Thanks for the reply.
Can you kindly post the link for the notebook on the Master branch? I know that I should be able to find it but somehow I can't. I would like to verify the code there against mine since I am still getting the same issue so far.
Thank you,
Berdy
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Berdy,
The current Master branch would be the "main" branch.
Running the default git clone command should get you the main branch:
git clone --depth=1 https://github.com/openvinotoolkit/openvino_notebooks.git
Regards,
Hairul
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hairul,
Thanks for the link, I had been on that page before, but somehow I missed the Dolly v2 notebook on it.
I ran the code to confirm that it does work. I am however still having issues with the same code running as a Python module in my venv rather than as a notebook.
Checking thru installed packages there are different versions for torch, cuda, and some others. I will try to realign the packages and try again therafter.
For the time being lets consider the issue closed.
Thanks,
Berdy
- 新着としてマーク
- ブックマーク
- 購読
- ミュート
- RSS フィードを購読する
- ハイライト
- 印刷
- 不適切なコンテンツを報告
Hi Berdy,
This thread will no longer be monitored. If you need any additional information from Intel, please submit a new question.
Regards,
Hairul
