Intel® Liftoff member, Selecton Technologies is a generative AI startup dedicated to exploring and researching AI technologies to reimagine the buyer experience for human users. They believe that AI has the potential to assist humans by communicating, listening, and learning each person’s preferences and habits. And their vision is to perfect and simplify human interaction with AI, enabling it to effectively aid human decision-making in everyday life.
As part of the Intel® Liftoff Hackathon, Selecton developed Selecta Expert, a dialogue machine capable of answering complex questions about games by extracting knowledge from textual databases. The system consists of a customized LLM model, an embedding model for semantic search, and a knowledge base stored in databases. This knowledge base contains comprehensive texts related to the domain of video games.
The Problem Selecton Set Out to Solve
Getting an LLM to provide comprehensive knowledge on a niche topic in the mode of free dialogue is a real challenge. To tackle this problem, Selecton chose to investigate the possibility of fine-tuning an LLM to answer questions specifically about video games.
They knew they would have to train their embedding model for semantic search, and develop the complete pipeline for question answering.
Here’s how they did it:
- Created a knowledge base on video games domain from Wikipedia articles.
- Made a custom embedding model to represent semantic similarity between users’ posts and articles about games.
- Fine-tuned LLM model for question-answering tasks.
- Used Langchain i to make the solution integrating semantic search, context-aware question answering and the dialogue.
- Implemented the solution as a microservice with an endpoint for question answering.
They used the following hardware and software:
The Results: New Vistas for Selecton’s Customer-Centric Vision
Selecton’s goal was to deploy this solution (Selecta Expert) as a microservice for answering questions, receiving POST requests and sending answers in JSON format.
Here are their performance indicators from the hackathon:
- Training: IPEX float32 XPU - 400 it /20 h, ~ 5h/100 it
- Training :Stock pytorch (estimate)== 11h /100 it (edited)
- Inference CPU: IPEX bfloat16 ~32 sec
- Inference CPU: Stock pytorch ~ 47 sec
Through a one-day training session, they experienced what Lead AI & ML Researcher at Selecton Technologies Dimitri Nowicki called “the remarkable capabilities of Intel® resources”, with promising results:
- An exceptional ability to train and fine-tune the LLM: during the project they fine-tuned the Dolly LLM with 12B weights using LoRA
- The training protocol was based on the LoRA-Alpaca training script.
- They observed around 2.5 times speed compared to other CPUs.
Beyond the Intel® Liftoff Hackathon: What’s Next for Selecton Technologies?
In the team’s words, this impressive training speed opened new horizons for their vision. By harnessing the immense computational power offered by the GPU, they achieved results that surpassed their expectations, allowing them to explore previously uncharted territories in terms of computational capacity.
Selecton intends to productize Selecta Expert as part of their main product, thanks to its ability to provide knowledge by answering the questions in a natural form. This will make their product more engaging for gamers in their audience, help them make more precise purchase decisions, and increase overall customer satisfaction.
Intel® Liftoff Accelerator: the Right Choice for AI Startups
Selecton Technologies’ results show how Intel® accelerated computing solutions make the impossible possible for ambitious startups seeking to make the next big breakthrough in AI. Apply to our tech startup accelerator to turn your great idea into groundbreaking reality.
I'm a proud team member of the Intel® Liftoff for Startups, an innovative, free virtual program dedicated to accelerating the growth of early-stage AI startups.