- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I tried to use mkl-dnn accelerate the inference of NER(Named Entity Recognition) model, however, some errors happened:
dnnl_verbose,exec,cpu,reorder,simple:any,undef,src_f32::blocked:a:f0 dst_f32::blocked:a:f0,,,5,0.0090332
Traceback (most recent call last):
File "bert_for_token_classification_example_fast_dnnmkl.py", line 161, in <module>
torch_outputs_ = model_(input_ids_.to(device), attention_mask_.to(device), token_type_ids_.to(device))
File "/opt/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/miniconda3/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 1701, in forward
return_dict=return_dict,
File "/opt/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/opt/miniconda3/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 944, in forward
extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape, device)
File "/opt/miniconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 289, in get_extended_attention_mask
extended_attention_mask = attention_mask[:, None, None, :]
RuntimeError: opaque tensors do not have strides
I use the transformers(hugging face) with 4.6.1, can someone help to point out the problem in it? Thanks a lot.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
We have not heard back you. As mentioned please try with the next release of ipex. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel
Regards
Gopika
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for posting in Intel Communities. We are trying to reproduce the issue from our end, we will get back to you as soon as we get an update.
Regards
Gopika
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Could you please give us the following information so that we can try reproducing the issue from our end?
1. The pytorch version
2. Whether you are using quantized model or not
3. Steps to reproduce the issue
4. The model you are using
Regards
Gopika
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, Gopika,
thank you for your reply!
1. The pytorch version
Reply:The pytorch version is 1.9.0
2. Whether you are using quantized model or not
Reply:not quantized model, just a model finetuned in BERT-Base, Chinese
3. Steps to reproduce the issue
Reply:
device = torch.device("cpu")
model_id = "bert-base-chinese"
config = PretrainedConfig.from_pretrained(model_id)
config.num_labels = 5
model = BertForTokenClassification.from_pretrained(model_id, config=config).to(device)
model.load_state_dict(torch.load("model-epoch49.pt", map_location=torch.device(device)))
model.eval()
tokenizer = BertTokenizer.from_pretrained(model_id)
sentence = "海底捞"
result = tokenizer(sentence, padding='max_length', max_length=512, truncation=True, return_tensors='pt').data
input_ids, token_type_ids, attention_mask = result.get('input_ids'), result.get('token_type_ids'), result.get('attention_mask')
input_ids_ = input_ids.float().to_mkldnn()
token_type_ids_ = token_type_ids.float().to_mkldnn()
attention_mask_ = attention_mask.float().to_mkldnn()
model_ = mkldnn.to_mkldnn(model)
torch_outputs_ = model_(input_ids_.to(device), attention_mask_.to(device), token_type_ids_.to(device))
torch_outputs = torch_outputs_.to_dense()
4. The model you are using
Reply:The "model-epoch49.pt" is 388M and exceed the maximum limit, so i can not attach here
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
The problem is that the opaque tensors do not have strides and it is not supported by the current version of ipex. Please try with the next release of ipex. Please refer: https://github.com/intel/intel-extension-for-pytorch for more information on ipex.
Regards
Gopika
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Has the solution provided helped? Please let us know whether your query is resolved.
Regards
Gopika
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi
We have not heard back you. As mentioned please try with the next release of ipex. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel
Regards
Gopika
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page