The Intel® Liftoff team and AI startups has recently completed the second development sprint on Intel® Developer Cloud (read the report about the first sprint here). During this sprint, the Intel® Liftoff program team and startups focused on customized, LLM-powered feature enablement for their products. Leveraging Intel’s AI stack and support, hackathon participants are setting new benchmarks in AI application development across business sectors.
Opening Doors for Intel® Liftoff Members
The virtual event provided AI startups with access to the Intel Developer Cloud, including 4th Gen Intel® Xeon® Scalable processors and Intel® Data Center GPU Max Series 1550. The hackathon participants were tasked with exploring the potential of next-generation LLM-based applications.
The Top Four Innovations
Out of the teams participating, the four most innovative were selected for the final showcase:
 |  |  |  |
Customized LLMs for business | Controlled & compliant LLM outputs for business | AI automated business intelligence for HR | Automated marketing business intelligence |
In a first for Intel® Liftoff, all hackathon participants were given access to multiple GPUs and were able to distribute training (data parallel) runs over them simultaneously. With the larger Data Center GPU Max 1550 model available with 128 GB of VRAM, this meant 512 GB of VRAM in a 4-way server, allowing our hackathon participants to fine-tune 3 to 13 billion parameter LLMs in a matter of hours. The models for all four of these applications were fine-tuned on Intel GPUs using the Intel® oneAPI software stack, Intel® oneAPI Base Toolkit, Intel® oneAPI Deep Neural Network (oneDNN), Intel® oneAPI DPC++/C++ Compiler with SYCL* runtime, and Intel® Extension for PyTorch.
Dowork Technologies: LLM Chatbots
Dowork Technologies synthesized a fine-tuning dataset and utilized it to customize OpenLLaMA-13B with 4 PVC 1550s using LoRA. Their platform enables companies to securely utilize internal data to construct LLM chatbots and other applications, serving as dynamic, conversational institutional memories for employees—essentially, a Private ChatGPT!
“We’ve been using Intel Hardware for fine-tuning our 13 billion-parameter model. The results have been encouraging, providing us with the computational power needed for such an extensive model. However, during inference, we noticed a slight delay in text generation. As we continue to push our AI models to new heights, we are eager to collaborate with Intel to overcome this challenge and unlock even greater performance in future solutions," stated Mushegh Gevorgyan, founder, and CEO of Dowork Technologies.
The Mango Jelly: SQL Queries for Business Analytics
The Mango Jelly's application required a new feature for generating SQL queries to automate business analytics for marketers. This integral feature, central to their business strategy, was developed from the ground up during this Intel Liftoff development sprint, yielding impressive outcomes. Leveraging Intel GPUs, the team fine-tuned OpenLLaMA-3B using actual customer data, enabling it to generate well-structured queries in response to marketing inquiries formulated in plain English.
"As part of Intel® Liftoff, we were able to fine-tune an open-source LLM on exceptionally powerful hardware. The speed and performance of the Intel® XPU were astonishing to witness. This partnership grants us greater control over customizability, fine-tuning, and usage limits using open-source models, supported by efficient and cutting-edge hardware. It also positions our solution as ready and optimal for enterprise use cases," says Divya Upadhyay, co-founder, and CEO of The Mango Jelly.
Terrain Analytics: Better Staffing Decisions
Terrain Analytics offers a platform designed to enhance companies' staffing decisions. Prior to the sprint, Terrain had developed a feature to parse job listings using OpenAI’s Ada API; however, they encountered challenges related to cost and throughput. During the Intel® Liftoff sprint, they successfully fine-tuned an LLM for this specific use case, utilizing Intel® Data Center GPU Max (Ponte Vecchio) for training and 4th Generation Intel® Xeon® Scalable Processor (Sapphire Rapids) for inference. The resulting model outperformed the generic Ada API, exhibiting significantly higher throughput and delivering substantial cost savings.
With the integration of Intel® hardware, Terrain can now seamlessly scale Deep Learning and Language Learning Models without encountering computational limitations. According to Terrain Analytics’ Nathan Berkley, Software Engineer at Terrain Analytics, and Riley Kinser, co-founder and Head of Product: “Both of the models we created demonstrated superior success metrics compared to the ones generated using OpenAI’s Ada model, and they were 15 times faster in terms of processing speed”.
Prediction Guard: Making LLMs Business-friendly
Prediction Guard specializes in specializes in facilitating the integration of LLMs into business operations, prioritizing security and feasibility. The outputs produced by LLMs introduce potential challenges related to compliance and reputation, often exhibiting an unstructured nature. Prediction Guard's platform offers solutions to address these issues. Leveraging data from two paying customers, they fine-tuned Camel-5B and Dolly-3B models, demonstrating their capability to refine LLM outputs for enhanced business applicability.
“After benchmarking LLM data extraction on Intel DCGM, Prediction Guard was able to demonstrate how the client could reduce their current OpenAI-based transcription processing time from 20 minutes to under one minute. Their Pivoting project for future clients has the potential to save $2M annually in operational expenses," says Daniel Withenack, founder, and CEO of Prediction Guard.
Looking Forward to the Intel Innovation Event
As we race towards the Intel® Innovation event this September, these accomplishments illustrate the potential that AI startups can unlock with the Liftoff for Startups program. Leveraging Intel’s AI stack and support, our program members are setting new benchmarks in AI application development.
Read more about Intel® Liftoff for Startups: https://developer.intel.com/liftoff
Get in touch with our program team Ralph de Wargny, Rahul Unnikrishnan Nair, Ryan Metz and Eugenie Wirz: intel.liftoff@intel.com
Learn more about Intel® Developer Cloud.
*The benchmarking referenced in this article was conducted by the startups highlighted here.
*Other names and brands may be claimed as the property of others. SYCL is a trademark of the Khronos Group Inc.
I'm a proud team member of the Intel® Liftoff for Startups, an innovative, free virtual program dedicated to accelerating the growth of early-stage AI startups.