Cloud
Examine critical components of Cloud computing with Intel® software experts
119 Discussions

Comparing HPC Workload performance between Intel and AMD on AWS (Part 3 of 3)

Mohan_Potheri
Employee
0 0 58.5K

In part 1 of the blog series we looked at the HPC landscape and the tools available on AWS for HPC deployment. In part 2 of the blog series we compared the performance between Intel and AMD HPC instances on AWS for GROMACS. In this final part 3, we will compare the performance between Intel and AMD HPC instances on AWS for OpenFoam.

OpenFoam:

OpenFOAM (Open Field Operation and Manipulation) is an open-source computational fluid dynamics (CFD) software package. It provides a comprehensive set of tools for solving complex fluid flow and heat transfer problems using numerical simulation methods. OpenFOAM is widely used in academia, industry, and research institutions for a variety of applications, including aerospace, automotive, energy, and environmental engineering.

Here are some key features and aspects of OpenFOAM:

  1. Open-Source Nature: OpenFOAM is released under the GNU General Public License (GPL) and is freely available for use, modification, and distribution. Its open-source nature encourages collaboration, knowledge sharing, and community contributions, allowing users to customize and extend the software to meet their specific needs.
  2. Finite Volume Method: OpenFOAM utilizes the finite volume method, a numerical technique for solving partial differential equations that describe fluid flow and heat transfer phenomena. This approach discretizes the domain into finite control volumes and solves the governing equations based on conservation principles for mass, momentum, and energy.
  3. Wide Range of Solvers: OpenFOAM offers a comprehensive suite of solvers that can handle various types of flow problems. It includes solvers for incompressible and compressible flows, laminar and turbulent flows, multiphase flows, reacting flows, and heat transfer problems. These solvers are designed to accurately simulate complex flow phenomena using robust numerical algorithms.
  4. Meshing Capabilities: OpenFOAM provides meshing tools for generating high-quality computational grids. It supports structured, unstructured, and hybrid meshing methods, allowing users to generate meshes that suit their specific simulation requirements. OpenFOAM also includes mesh manipulation and conversion utilities, which facilitate the preprocessing stage of CFD simulations.
  5. Boundary Conditions and Models: OpenFOAM offers a wide range of boundary conditions and physical models to represent various flow conditions and phenomena. It provides options for specifying velocity inlets, pressure outlets, walls, symmetry planes, and other boundary conditions. Additionally, OpenFOAM includes turbulence models, such as the popular k-epsilon and k-omega models, as well as models for heat transfer, combustion, multiphase flows, and more.
  6. Parallel Computing: OpenFOAM supports parallel computing, allowing users to distribute computational tasks across multiple cores or nodes for faster simulations. It utilizes Message Passing Interface (MPI) for parallelization and can take advantage of high-performance computing (HPC) clusters or multi-core workstations to speed up simulations.
  7. Extensibility and Customization: OpenFOAM is designed to be highly extensible and customizable. Users can add their own solvers, models, and utilities to the software to address specific simulation requirements or research objectives. This flexibility allows for the development of new capabilities and the adaptation of OpenFOAM to emerging computational fluid dynamics challenges.

OpenFOAM provides a powerful and versatile platform for performing CFD simulations, enabling engineers, scientists, and researchers to study and analyze a wide range of fluid flow and heat transfer problems. Its open-source nature, robust solvers, and extensive capabilities make it a popular choice for those seeking a flexible and customizable CFD software package.

Performance testing for OpenFoam:

The motorbike Simulation Test Case:

This benchmark offers a free comparison of various CPU processors. The test case is a well-known motorBike tutorial[i] which is a standard part of OpenFOAM installation. The case settings are intentionally left default. The test case can be run on any computer. Three different mesh settings of with three different sizes were used. The base mesh (size S) is about 8.6 * 10^6 cells (the tutorial one is 3.2 * 10^5) # of cells. Appendix A shows the configuration used.

The total simulation time calculated was for the blockMesh, decomposePar, snappyHexMesh, potentialFoam and simpleFoam processes combined.

 

Mohan_Potheri_0-1689703577649.png

Figure 3: The Motorbike Simulation Test case

The tests were performed in June 2023 on AWS ParallelCluster in Amazon region us-east-2. All Head Nodes and Compute nodes for the testing used general Purpose SSD gp3 storage as local storage. The benchmarks were run on 6 nodes of Intel based Amazon EC2 hpc6id instances and compared with the run of 4 nodes of AMD based Amazon EC2 hpc6a instances. The goal was to match the number of total cores in each cluster and so 6 nodes of hpc6id were compared with 4 nodes of hpc6a for a total of 384 cores. The config files and the scripts used for the testing are shown in Appendix A. The following IAAS instance configuration was used for the comparison in performance between Intel (HPC6id) and AMD (HPC6a) instances.

 

Category

Attribute

Config1

Config2

Run Info

 

 

 

 

Cumulus Run ID

N/A

N/A

 

Benchmarks

OpenFOAM Motorbike L, XL

OpenFOAM Motorbike L, XL

 

Date

June 2023

June 2023

 

Test by

Intel

Intel

CSP and VM Config

 

 

 

 

Cloud

 AWS

 AWS

 

Region

 

 

 

Instance Type

HPC6a in AWS ParallelCluster

HPC6id in AWS ParallelCluster

 

CPU(s)

 384

 384

 

Microarchitecture

 AWS Nitro

 AWS Nitro

 

Instance Cost

 

 

 

Number of Instances or VMs (if cluster)

 

 

 

Iterations and result choice (median, average, min, max)

 

 

Memory

 

 

 

 

Memory

1534 GB

6144 GB

 

DIMM Config

 

 

 

Memory Capacity / Instance

 

 

Network Info

 

 

 

 

Network BW / Instance

100 GBPS EFA

 200 GBPS EFA

 

NIC Summary

 

 

Storage Info

 

 

 

 

Storage: Network Storage

FSX Lustre 1.2 TB

FSX Lustre 1.2 TB

Table 4: Instance and Benchmark Details for OpenFOAM

Multiple runs of the small, medium, and large use cases were run for HPC6id (Intel) an HPC6a (AMD) with a configuration to leverage a total of 384 cores. The results are shown below.

 

Large

HPC6id (Intel)

HPC6a (AMD)

Run 1

252

369

Run 2

252

368

Run 3

256

369

Average

253

369

   

X-Large

  

Run 1

886

1472

Run 2

884

1479

Run 3

881

1467

Average

884

1473

   
   

Runtime

HPC6id (Intel)

HPC6a (AMD)

Large (42 million Cells)

253

369

X-Large (145 million Cells)

884

1473

  

 

 

HPC6id (Intel)

HPC6a (AMD)

Large (42 million Cells)

145.53%

100.00%

X-Large (145 million Cells)

166.69%

100.00%

Table 5: Raw Data from OpenFOAM Testing

 

Mohan_Potheri_1-1689703577654.png

Figure 4: Chart comparing absolute performance in runtimes for motorbike OpenFOAM benchmark between Intel and AMD instance types.

 

Mohan_Potheri_2-1689703577658.png

Figure 5: Performance comparison by percentage for OpenFOAM benchmarks

OpenFoam Results:

The results clearly show that Intel based HPC6id instances perform 46-67% faster than AMD based HPC6a instances. Intel is the established player in the HPC market with many SW optimizations for HPC workloads that could further improve performance for its instances over that of AMD. Any potential cost differences between the instances and cluster cost are easily compensated for with the improved performance shown in the results.

Conclusion:

Intel has been the leader and innovator for HPC over the past few decades. Intel based cloud instances on Amazon EC2 provide the agility and scalability of the Cloud for HPC workloads. Intel works with all major HPC workloads to optimize them for performance and scalability on its processors running on-premises and in the Cloud. Intel contributes to many libraries and open-source projects for HPC such as MPI and with Intel oneAPI HPC toolkit[ii]. Our head-to-head comparison between Intel and AMD based instances on Amazon EC2 for common HPC Workloads such as GROMACS and OpenFoam, clearly showcases Intel superiority in performance and scalability.  Superior performance for Intel more than compensates for any differences in cost with AMD for HPC workloads.        

 

Bibliography:

[i] Amazon Elastic Compute Cloud (Amazon EC2) Hpc6id[1] instances, powered by 3rd Generation Intel Xeon Scalable processors, offer cost-effective price performance for memory-bound and data-intensive high-performance computing (HPC) workloads in Amazon EC2. 

[ii] Amazon EC2 AMD based HPC6a instances: AMD is claiming that Amazon EC2 Hpc6a [1]instances offer the best price performance for compute-intensive high-performance computing (HPC) workloads in Amazon EC2. Most of the cost savings claims can be attributed to increased core count for AMD over Intel per socket.

[iii] The PEP (Performance Evaluation Project) benchmark for GROMACS is a widely recognized benchmark used to assess the performance of computer systems and architectures in running molecular dynamics simulations using GROMACS.

[iv] Satellite Tobacco Mosaic Virus (STMV) is a small, icosahedral plant virus that worsens the symptoms of infection by Tobacco Mosaic Virus (TMV).

[v] This benchmark offers a free comparison of various CPU processors. The test case is a well-known Motor Bike tutorial which is a standard part of OpenFOAM installation.

[vi] The Intel® oneAPI Base Toolkit includes powerful data-centric libraries, advanced analysis tools, and Intel® Distribution for Python* for near-native code performance of core Python numerical, scientific, and machine learning packages.

Disclosure text:

Tests were performed April-June 2023 on AWS in region us-east-1. Full configuration details are shown in table 1 and 4. The total number of cores of the AWS ParallelCluster configs were matched for Intel and AMD at 384 cores for the comparison.

 

Instance Size

Physical Cores

Memory (GiB)

EFA Network Bandwidth (Gbps)

Network Bandwidth (Gbps)*

hpc6id.32xlarge

64

1024

200

25

6 x HPC6id instances were used in an AWS ParallelCluster with the following configuration:

The Amazon Elastic Compute Cloud (Amazon EC2) Hpc6id instances, powered by 3rd Generation Intel Xeon Scalable processors, offer cost-effective price performance for memory-bound and data-intensive high performance computing (HPC) workloads in Amazon EC2. 

 

Instance Size

Physical Cores

Memory (GiB)

EFA Network Bandwidth (Gbps)

Network Bandwidth (Gbps)*

Hpc6a.48xlarge

96

384

100

25

4 x HPC6a instances were used in an AWS ParallelCluster with the following configuration:

The Amazon EC2 Hpc6a instance features the 3rd generation AMD EPYC 7003 series processors with up to 3.6 GHz all-core turbo frequency built on a 7nm process node for increased efficiency. 

 

DB Client machine details:  For the client machine, we used the EC2 instance type: c6i.4xlarge with 16vCPU (8 core), with 32 GB Memory, 75 GB GP2 Storage volume with 12.5GB Network bandwidth powered by 3rd Generation Intel Xeon Scalable processors. The client machines use the following Software Image (AMI) with Canonical, Ubuntu, 20.04 LTS, amd64 focal image build on 2022-09-14 & ami-0149b2da6ceec4bb0. All Instances, as well as the client Instances were run in US-EAST-1 region. Benchmarking Software:

 

FSX Shared storage details:

  • File system type: Lustre
  • Deployment type: Persistent 2
  • Data compression type: NONE
  • Storage type: SSD
  • Storage capacity: 1.2 TiB
  • Throughput per unit of storage: 125 MB/s/TiB
  • Total throughput: 150 MB/s
  • Root Squash: Disabled
  • Lustre version: 2.12

Notices & Disclaimers:

Performance varies by use, configuration, and other factors. Learn more on the Performance Index site

Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates.  See backup for configuration details.  No product or component can be absolutely secure. 

Your costs and results may vary.  For further information please refer to  Legal Notices and Disclaimers.

Intel technologies may require enabled hardware, software, or service activation.

© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.

 

 

About the Author
Mohan Potheri is a Cloud Solutions Architect with more than 20 years in IT infrastructure, with in depth experience on Cloud architecture. He currently focuses on educating customers and partners on Intel capabilities and optimizations available on Amazon AWS. He is actively engaged with the Intel and AWS Partner communities to develop compelling solutions with Intel and AWS. He is a VMware vExpert (VCDX#98) with extensive knowledge on premises and hybrid cloud. He also has extensive experience with business critical applications such as SAP, Oracle, SQL and Java across UNIX, Linux and Windows environments. Mohan Potheri is an expert on AI/ML, HPC and has been a speaker in multiple conferences such as VMWorld, GTC, ISC and other Partner events.