Artificial Intelligence (AI)
Discuss current events in AI and technological innovations with Intel® employees
491 Discussions

Intel Collaboration With Deci Boosts AI Performance on Intel Hardware

1 0 5,715

Scott Bair is a key voice at Intel Labs, sharing insights into innovative research for inventing tomorrow’s technology.

A collaboration between Intel Labs and Deci, a start-up company focused on deep learning development, is accelerating the journey toward more scalable AI. By combining Deci’s algorithmic acceleration technology with Intel chip architectures, the two companies have successfully--and dramatically--optimized deep learning inferencing. Thanks in no small part to this collaboration, deep learning can now be used to build, optimize, and deploy more accurate and efficient models for cloud environments.

Intel’s journey with Deci began in 2019 when the two joined forces to optimize deep learning inference on Intel Architecture (IA) CPUs. A year later in 2020, Deci submitted inference results achieved with its proprietary neural architecture search engine called AutoNAC (Automated Neural Architecture Construction) and 2nd Gen Intel® Xeon® Scalable processors to the open division of the MLPerf v0.7 benchmark suite. The submission used the Intel® Distribution of the OpenVINOtoolkit to cut the inference latency and boost the throughput of the ResNet-50 deep neural network model. The results showed a whopping 11.8x reduction in latency and an 11x increase in throughput, all while preserving the model's accuracy within 1%.  (Additional details about the submission can be found on Intel Communities Blog.)

"This was a major win for hardware-aware acceleration of computer vision models with neural architecture search and 8-bit quantization running on Intel hardware,” said Guy Boudoukh, a deep learning researcher within Intel’s AI Lab. Guy is also a mentor to Deci under Intel® Ignite, a global corporate program for early-stage start-ups, launched in 2019 by Intel.

Deci was one of the first companies to join Intel Ignite, a program designed to help turn the cutting-edge ideas of ambitious entrepreneurs into industry-disrupting technologies.The program is focused on deep tech innovations. Working with leading mentors and industry experts, each Ignite start-up receives a tailor-made program to meet its specific business needs. The program covers a wide range of fields, including founders’ dynamics, go-to-market strategy, marketing and sales, product development and management, recruitment, and more. DECI is a batch 1 alumni of the Intel Ignite Tel Aviv program.


Deci and Intel Boost Computer Vision (CV) and Natural Language Processing (NLP) Models

Over the last two years, the Deci/Intel collaboration has continued, resulting in another compelling MLPerf submission, this time for both Computer Vision (CV) and Natural Language Processing (NLP) inference models.  

The CV submission accelerated ResNet50 with Deci’s AutoNAC to generate DeciNets deep neural networks. The optimized DeciNets models were submitted in the ResNet50 category in an offline scenario on two different hardware platforms: a 12-core 2nd Gen Intel® Xeon® processor and two 3rd Gen Intel Xeon Scalable processors with different configurations of 4 and 32-cores. The models were optimized on a batch size of 32 and quantized to INT8 using OpenVINO. Compared to the 8-bit ResNet50 model, Deci delivered a+1.74% improvement in accuracy and a 4x improvement in throughput.  The submission demonstrated a 37% improvement in throughput performance, as well as a notable improvement in accuracy, compared to their previous submission in 2020.

The NLP submission accelerated the Bert-Large model by implementing Deci’s Automated Neural Architecture Construction (AutoNAC) to generate the Intel-optimized DeciBERT models. The Deci Bert models were quantized with OpenVINO to 8-bit precision. Benchmarked on 32-core  3rd Gen Intel Xeon Scalable processors and 16-core 2nd Gen Intel Xeon C processors, the .results demonstrated up to a 5x acceleration in throughput performance and +1.03 improvement in accuracy.  

"These recent MLPerf submissions mark another significant milestone in our collaboration with Intel to deliver efficient deep learning inference on Intel CPUs. The major increase delivered both in accuracy and throughput directly translates into a larger number of the resource-intensive tasks and applications that can now be carried out on Intel CPUs,” said Yonatan Geifman, Deci CEO and Co-founder.

More details can be found about the NLP and CV submissions in these articles, published on Medium:


The Intel-Deci collaboration continues. Just recently Deci joined the Intel Disruptor Program and currently, Deci is working with Intel to demonstrate AutoNAC technology performance on 4thGen Intel Scalable processors, codenamed Sapphire Rapids.


About Deci

Founded in 2019 and based in Tel Aviv, Deci has been capturing headlines in AI news sources since October 2020 when it announced that it had raised $9.1 million in seed funding.  Deci has also scored a total of $55.1 million in Series A and Series B financing rounds led by Insight Partners, a global venture capital and private equity firm that invests in high-growth technology and software companies.

Deci is also one of a select number of companies that have been accepted into both Intel Ignite and Intel Disrupter initiatives.

Tags (1)
About the Author
Scott Bair is a Senior Technical Creative Director for Intel Labs, chartered with growing awareness for Intel’s leading-edge research activities, like AI, Neuromorphic Computing and Quantum Computing. Scott is responsible for driving marketing strategy, messaging, and asset creation for Intel Labs and its joint-research activities. In addition to his work at Intel, he has a passion for audio technology and is an active father of 5 children. Scott has over 23 years of experience in the computing industry bringing new products and technology to market. During his 15 years at Intel, he has worked in a variety of roles from R&D, architecture, strategic planning, product marketing, and technology evangelism. Scott has an undergraduate degree in Electrical and Computer Engineering and a Masters of Business Administration from Brigham Young University.