Improve Latency of NGINX in AWS using Intel’s QAT Engine for OpenSSL - Part 3 of a 5 Part Series

This is Part 3 of a 5 Part Series

Welcome to our 3rd installment of the series. In this post we will cover:

• Building Intel integrated performance primitives' library
• Building the Intel multi-buffer crypto for IPsec library
• Building the QAT Engine for OpenSSL for software acceleration only <crypto-ni>
• Testing the QAT Engine for software acceleration only <crypto-ni>

You can get caught up on where we are by reading Part 1 and Part 2.

The software acceleration support in the Intel QAT Engine for OpenSSL depends on the following two libraries. They must be built first, however, they may be built in any order:

Intel® Integrated Performance Primitives Cryptography
Intel® Multi-Buffer Crypto for IPsec Library

Once these libraries are installed, you can build the Intel Quick Assist Technology OpenSSL Engine.

Building Intel® Integrated Performance Primitives Cryptography

Intel® Integrated Performance Primitives (Intel® IPP) is a software library that provides a broad range of functionality, including general signal and image processing, computer vision, data compression, and string manipulation.

SSH into Machine-2 EC2 as "ubuntu" user and run the following commands.

First, checkout the source code respository from GitHub:*

Ensure you are building against a fixed release of the code, and not the development branch. At the time of this writing, the latest release was ippcp_2021.6:

You only need to build the multi-buffer portion of the Intel IPP package, so change to the multi-buffer crypto library sub-directory. Then, prepare the build by running cmake:

This will configure the library to install into /usr. To perform the full build, run:

This will put the shared library in /usr/lib, which means we won't need to set LD_LIBRARY_PATH.

Building the Intel Multi-Buffer Crypto for IPsec Library

The library provides software crypto acceleration primarily targeting packet processing applications. It can be used for application such as: IPsec, TLS, Wireless (RAN), Cable or MPEG DRM.

First, checkout the source code repository from GitHub:

Ensure you are building against a fixed release of the code, and not a development branch. At the time of this writing, the latest release was v1.2:

There is no configuration step. Build the library using:

To install:

This will place the shared libraries in /usr/lib, which again means no LD_LIBRARY_PATH modificiations.

Building the Intel Quick Assist Technology (Intel QAT) Engine for OpenSSL

Checkout the software repository from GitHub:

Next, ensure you are building against a fixed release of the code, and not a development branch. At the time of this writing, the latest release was v0.6.15:

To configure the Intel QAT Engine for OpenSSL for all software acceleration features:

Then build and install: 

Since this makes use of the distribution-provided OpenSSL installation, we won't need to modify LD_LIBRARY_PATH to use the engine.

After the installation has completed, you should see the engine present in OpenSSL's engine directory.

Below is an example of using OpenSSL 1.1; In Ubuntu 20.04, this is in /usr/lib/x86_64-linux-gnu/engines-1.1.  Please use your supported OpennSSL version: For non-standard builds, the engine directory can be obtained by running the "openssl version" command.

Verify that the engine is present by running "ls." You should see qatengine.so in the directory list:

Testing the Engine

Proper testing at this stage ensures that the QAT Engine is installed and running correctly. The proper working of the QAT Engine is important for the performance improvement of the NGINX web server.

Once the engine is in place, you can proceed with functionality tests. The first test is to ensure the Intel QAT Engine loads correctly.

If the above command returns errors such as the following, then you need to check the path where the Intel® Multi-Buffer Crypto for IPsec Library and the Intel IPP CryptoMB Library are both installed.

The error may look like this:

We recommend using the distribution provided OpenSSL from your OS. In this example, we have used the distribution provided OpenSSL. The instructions are based on this Intel article. 

Note: If you are using the distribution-provided OpenSSL:

Make sure the Intel® Multi-Buffer Crypto for IPsec Library and the Intel IPP CryptoMB Library are both installed into /usr/lib. If you did not set a prefix for the former, it will not install into /usr/local, and you’ll need to set LD_LIBRARY_PATH in your environment.

Testing with 2048-bit keys

This will do an OpenSSL speed test for 2048-bit encryption key. First, we will do a baseline test with regular OpenSSL without the QAT engine. Next, we will do the same test with QAT engine enabled. We will be able to see the performance improvements comaring the sign and verify operations per second.

Baseline

Testing with Intel QAT Engine for OpenSSL

From the results, you should be able to see that the speed test with the -engine qatengine flag produces improved metrics. This is the result of the performance gain that is coming from the underlying Intel Crypto NI instructions from the CPU.

Now that we have set up the QAT Engine for OpenSSL and tested the engine, we are ready to move on to Part 4: Building NGINX async mode from source code and optimize settings.