Intel® Optimized AI Frameworks
Get community support for questions related to PyTorch* and TensorFlow* frameworks.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.
56 Discussions

Can MKL_DNN used for NER model with a bert?

Hap_Zhang
Beginner
940 Views

Hi, 

 

I tried to use mkl-dnn accelerate the inference of NER(Named Entity Recognition) model, however, some errors happened:

dnnl_verbose,exec,cpu,reorder,simple:any,undef,src_f32::blocked:a:f0 dst_f32::blocked:a:f0,,,5,0.0090332
Traceback (most recent call last):
  File "bert_for_token_classification_example_fast_dnnmkl.py", line 161, in <module>
    torch_outputs_ = model_(input_ids_.to(device), attention_mask_.to(device), token_type_ids_.to(device))
  File "/opt/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 1701, in forward
    return_dict=return_dict,
  File "/opt/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/opt/miniconda3/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 944, in forward
    extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape, device)
  File "/opt/miniconda3/lib/python3.7/site-packages/transformers/modeling_utils.py", line 289, in get_extended_attention_mask
    extended_attention_mask = attention_mask[:, None, None, :]
RuntimeError: opaque tensors do not have strides

I use the transformers(hugging face) with 4.6.1,  can someone help to point out the problem in it? Thanks a lot.

 

0 Kudos
1 Solution
Gopika_Intel
Moderator
763 Views

Hi

We have not heard back you. As mentioned please try with the next release of ipex. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel

Regards

Gopika


View solution in original post

6 Replies
Gopika_Intel
Moderator
904 Views

Hi,

Thank you for posting in Intel Communities. We are trying to reproduce the issue from our end, we will get back to you as soon as we get an update.

Regards

Gopika


Gopika_Intel
Moderator
895 Views

Hi,

Could you please give us the following information so that we can try reproducing the issue from our end?

1. The pytorch version

2. Whether you are using quantized model or not

3. Steps to reproduce the issue

4. The model you are using

Regards

Gopika


Hap_Zhang
Beginner
886 Views

Hi, Gopika,

thank you for your reply!

1. The pytorch version

Reply:The pytorch version is 1.9.0

2. Whether you are using quantized model or not

Reply:not quantized model, just a model finetuned in BERT-Base, Chinese

3. Steps to reproduce the issue

Reply:

device = torch.device("cpu")
model_id = "bert-base-chinese"
config = PretrainedConfig.from_pretrained(model_id)
config.num_labels = 5
model = BertForTokenClassification.from_pretrained(model_id, config=config).to(device)
model.load_state_dict(torch.load("model-epoch49.pt", map_location=torch.device(device)))
model.eval()
tokenizer = BertTokenizer.from_pretrained(model_id)
sentence = "海底捞"
result = tokenizer(sentence, padding='max_length', max_length=512, truncation=True, return_tensors='pt').data
input_ids, token_type_ids, attention_mask = result.get('input_ids'), result.get('token_type_ids'), result.get('attention_mask')
input_ids_ = input_ids.float().to_mkldnn()
token_type_ids_ = token_type_ids.float().to_mkldnn()
attention_mask_ = attention_mask.float().to_mkldnn()
model_ = mkldnn.to_mkldnn(model)
torch_outputs_ = model_(input_ids_.to(device), attention_mask_.to(device), token_type_ids_.to(device))
torch_outputs = torch_outputs_.to_dense()

4. The model you are using

Reply:The "model-epoch49.pt" is 388M and exceed the maximum limit, so i can not attach here

Gopika_Intel
Moderator
874 Views

Hi,

The problem is that the opaque tensors do not have strides and it is not supported by the current version of ipex. Please try with the next release of ipex. Please refer: https://github.com/intel/intel-extension-for-pytorch for more information on ipex.

Regards

Gopika


Gopika_Intel
Moderator
817 Views

Hi,

Has the solution provided helped? Please let us know whether your query is resolved.

Regards

Gopika

 

Gopika_Intel
Moderator
764 Views

Hi

We have not heard back you. As mentioned please try with the next release of ipex. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel

Regards

Gopika


Reply