Optimizing Dolly 2

[Berdy]

I attended today's (6/28/2023) OpenVino 2023.00 workshop. It was indicated that there is a sample code implementation using the Dolly-2 LLM implementation at github's openvino_notebboks.

While there is a link to it, there is actually no code available there. There is only an explanation of what Dooly is about. Could anyone provide any sample code for optimizing/quantizing NLP models, being Dolly, T5, or any other autoregressive model?

Thanks,

Berdy

[Aznie_Intel · Accepted solution]

Hi Berdy,

Thanks for reaching out.

You may refer the tutorial for using Databricks Dolly 2.0 and OpenVINO from OpenVINO notebooks documentation or 240-dolly-2-instruction-following.ipynb.

Regards,

Aznie

[Berdy]

Aznie, thanks very much for the links.

[Aznie_Intel]

Hi Berdy,

This thread will no longer be monitored since we have provided the information. If you need any additional information from Intel, please submit a new question. 

Regards,

Aznie