Microsoft Office brings the power of AI to your local PC

Co-authors: Ken Koyanagi, Charlie Hewett, Bryan Hobbs

   The team behind Microsoft Office has been delivering AI features to their suite of productivity tools for more than 5 years, and is continuing to collaborate with Intel and ONNX Runtime to bring AI features on-device. Expanding deployment of AI inference from servers to include Windows PCs improves responsiveness, maintains data on the PC to bolster privacy, and makes AI tooling more versatile by eliminating the need for an internet connection. These improvements continue to power Office features such as text prediction, ink shape detection, and neural grammar checker. 

The Role of Software and Optimization

   Intel and Microsoft are accelerating efforts to bring AI features to life in Microsoft Office for Windows platforms, a product of their deep engagement and 20+ years of collaboration. A part of this success comes from using ONNX Runtime, which allows machine learning models to scale across different operating systems and hardware combinations. ONNX runtime is continually optimized by the open-source community, Intel, and Microsoft—its usage here improves the performance of Microsoft Office AI models on Intel platforms.

   Delivering the best on-device AI user experience requires continuous effort through hardware and software optimization, orchestrated by experienced AI-skilled engineers. The AI engine of Microsoft Office is frequently updated with the latest features of ONNX Runtime, ensuring peak performance and smooth execution of AI models directly on client devices.

   Microsoft Office and Intel have utilized quantization of individual AI models, a process of optimizing these models to use smaller datatypes while maintaining accuracy. "By continuing to focus on our customers, Microsoft Office's collaboration with Intel on a myriad of inference projects has achieved significant reductions in memory consumption, improved performance, and increased parallelization all while maintaining accuracy” said Joshua Burkholder, Principal Software Engineer of the Microsoft’s Office AI Platform.  This approach minimizes the on-device memory footprint, which in turn further minimizes latency via Intel’s DL Boost, a set of specialized hardware instruction sets. ONNX Runtime has been optimized for Intel’s hybrid CPU architecture, consisting of Performance-cores and Efficiency-cores. This is taken a step further with Intel Thread Director, which uses machine learning to schedule tasks on the right core, ensuring they work together to optimize performance-per-watt.

   Additionally, Intel and Microsoft are collaborating to provide on-device AI support for Office web-based experiences. This is made possible through ONNX Runtime Web, which enables AI feature support directly in web applications, such as Microsoft Designer. 

Finding the Balance Between Cloud and On-device

   The era of AI PCs, including devices with the new Intel® Core™ Ultra processor, is enabling more workloads to migrate from the cloud to client devices. By combining CPU (Core), GPU (graphics), and NPU (neural processor), Intel Core Ultra processors provide complimentary AI compute capabilities that can be used to deliver the best user experience when paired with software and model optimizations.

   While the AI PC provides new capabilities for running AI tasks on client devices, each model must be evaluated individually to determine if running locally is a good fit.  The future of AI compute may be a hybrid model where many AI models are executed on client devices, and the cloud is leveraged for more complex processing needs. Intel AI PC development aims to help address this, working with the Office team to identify use cases best suited to run on clients with the Intel Core Ultra processor.

   Intel and Microsoft's ongoing collaboration is anchored in a shared vision of an optimized AI experience that bridges cloud and on-device with solutions like AI PC. As the availability of client compute for AI workloads continues to increase with future generations of Intel processors, we can expect critical tools like Microsoft Office to be designed to deliver a great user experience using the best of cloud and client technologies. Joshua added "Microsoft Office looks forward to our on-going work with Intel to further optimize inference for our customers."

To learn more, check out AI PC with the Intel Core Ultra Processor and explore the tools for development.