Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6501 Discussions

Newest release of OpenVINO™ 2024.4 is now available!

Luis_at_Intel
Moderator
‎09-20-2024 10:27 AM
904 Views

More Gen AI coverage and frameworks integrations to minimize code changes

  • Support for GLM-4-9B Chat, MiniCPM-1B, Llama 3 and 3.1, Phi-3-Mini, Phi-3-Medium and YOLOX-s models.
  • Noteworthy notebooks added: Florence-2, NuExtract-tiny Structure Extraction, Flux.1 Image Generation, PixArt-α: Photorealistic Text-to-Image Synthesis, and Phi-3-Vision Visual Language Assistant.

Broader LLM model support and more model compression techniques

  • OpenVINO™ Runtime optimized for Intel® Xe Matrix Extensions (Intel® XMX) systolic arrays on built-in GPUs for efficient matrix multiplication resulting in significant LLM performance boost with improved 1st and 2nd token latency, as well as a smaller memory footprint on Intel® Core™ Ultra Processors (Series 2).
  • Memory sharing enabled for NPUs on Intel® Core™ Ultra Processors (Series 2) for efficient pipeline integration without memory copy overhead.
  • Addition of the PagedAttention feature for discrete GPUs enables a significant boost in throughput for parallel inferencing when serving LLMs on Intel® Arc™ Graphics or Intel® Data Center GPU Flex Series.

More portability and performance to run AI at the edge, in the cloud or locally

  • Support for Intel® Core Ultra Processors Series 2 on Windows.
  • OpenVINO™ Model Server now comes with production-quality support for OpenAI-compatible API which enables significantly higher throughput for parallel inferencing on Intel® Xeon® processors when serving LLMs to many concurrent users.
  • Improved performance and memory consumption with prefix caching, KV cache compression, and other optimizations for serving LLMs using OpenVINO™ Model Server.
  • Support for Python 3.12.
  • Support for Red Hat* Enterprise Linux* (RHEL) version 9.3 - 9.4.

 

Download the 2024.4 Release 
Download Latest Release Now

 

Get all the details 
See 2024.4 release notes 

 

NNCF RELEASE

Check out the new NNCF release

 

Helpful Links

NOTE: Links open in a new window.

0 Kudos

Link Copied

0 Replies
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.