As we realize AI is everywhere, a natural question arises – how does the data move “everywhere?” The answer shouldn’t be surprising – it’s Ethernet. Ethernet has been moving data worldwide for 50 years, ever-evolving to meet modern workloads, including AI.
Why is Ethernet so prevalent? It’s an open industry standard with a broad array of interoperable products. Because many vendors offer Ethernet products, costs are generally less than proprietary products only available from a single vendor. Since Ethernet has been in use for a long time, you would be hard-pressed to find a data center manager who isn’t familiar with the technology. I often hear the phrase, “Everyone knows Ethernet.”
Moving AI to the edge with Ethernet
One fundamental component of AI is AI inference, and it’s trending to move toward the edge to reduce latency and deliver better customer experience. Traditionally, an edge device sends data to a server, which then runs the AI model and sends the results back to the edge device. Instead, we can run the AI model on the edge device to reduce latency. The edge device will communicate with a server for further model fine-tuning and data storage. These results are sent via a wireless or lower-speed wired Ethernet connection, making AI at the edge possible.
With AI at the edge, an edge device runs the AI model and provides real-time results, reducing latency and delivering better customer experience.
Ethernet for AI training on small clusters
AI training is another component of AI that covers a variety of usages. Some training models, such as deep learning, are complex and extensive, requiring a large cluster and accelerators to train the model in a reasonable timeframe. However, some training models only need a small cluster to deliver good time-to-train results. In other instances, models are pre-trained and fine-tuned with curated data sets on small clusters.
Today, Ethernet is an excellent technology for AI training that occurs on small clusters – fine-tuning or small models. All of Intel’s recent MLPerf submissions for 4th Gen Intel® Xeon® Scalable processors were made using Intel® Ethernet, demonstrating that Intel® Xeon® Scalable processors with Intel® Ethernet Network Adapters provide enterprises out-of-the-box capabilities to deploy AI on general-purpose systems, avoiding the cost and complexity of introducing dedicated AI systems.
Hyperscalers also offer cloud environments for AI training, often implemented using Ethernet. A good example of this is the Google A3 Supercomputer. This supercomputer uses the Intel® Infrastructure Processing Unit (IPU) E2000, which contains a 200Gb Ethernet controller and is the first device to incorporate the Google Falcon transport. Falcon is a low-latency, reliable Ethernet transport that Google recently made available through the Open Compute Project.
What about complex AI training on large clusters? Today, this is typically delivered using closed, proprietary technology, such as InfiniBand. The industry recognizes the need for an open, standards-based alternative and is aligning around Ethernet. Industry leaders assembled the Ultra Ethernet Consortium earlier this year to evolve Ethernet into the interconnect of choice for large clusters. Intel is proud to be a founding member of this organization.
Ethernet connects the different uses of AI together
Ethernet has been evolving over the last half-century, with the industry consistently working to make it better, make it cheaper, scale up and scale out faster than proprietary technologies and standardize it. As AI permeates every aspect of our lives and businesses, Ethernet is the natural networking technology to connect the many different AI usages together, helping bring AI everywhere from the edge to the cloud.
Thomas is VP/GM Networking Solutions Group in the Network and Edge Cloud Connectivity Group, responsible for defining and selling innovative connectivity solutions for cloud centric deployments, optimized for performance, security, and TCO.
In his previous role, Thomas was Vice President, Product Management, for Cisco’s Data Center Networking business group. He led the consolidation and growth of the Nexus portfolio into a multi-billion business with a large recurring software component. Thomas has 25 years of experience in the networking industry working with cloud providers, enterprise customers, and partners in various roles for Cisco, Juniper, and Insieme Networks. His main areas of expertise are the datacenter and optical interconnect technologies. In the past, Thomas served on the Board of Directors of The Ethernet Alliance., was a Board Observer at Innovium (acquired by Marvell), and a Board of Directors member for the Inspur-Cisco JV. Thomas is a frequent speaker at data center industry events and published blogs on technology trends. Before his first stint at Cisco, Thomas started his career in Germany working as a management consultant at McKinsey & Company.
Thomas holds an MSEE from the Technical University Chemnitz (Germany) and an MBA from the Haas School of Business, U.C. Berkeley.