Cloud
Examine critical components of Cloud computing with Intel® software experts
112 Discussions

Improve Latency of NGINX in AWS using Intel’s QAT Engine for OpenSSL - Part 3 of a 5 Part Series

RajivMandal
Employee
0 0 3,420

This is Part 3 of a 5 Part Series

Welcome to our 3rd installment of the series. In this post we will cover:

  • Building Intel integrated performance primitives' library
  • Building the Intel multi-buffer crypto for IPsec library
  • Building the QAT Engine for OpenSSL for software acceleration only <crypto-ni>
  • Testing the QAT Engine for software acceleration only <crypto-ni>

You can get caught up on where we are by reading Part 1 and Part 2.

The software acceleration support in the Intel QAT Engine for OpenSSL depends on the following two libraries. They must be built first, however, they may be built in any order:

Intel® Integrated Performance Primitives Cryptography
Intel® Multi-Buffer Crypto for IPsec Library

Once these libraries are installed, you can build the Intel Quick Assist Technology OpenSSL Engine.

Building Intel® Integrated Performance Primitives Cryptography

Intel® Integrated Performance Primitives (Intel® IPP) is a software library that provides a broad range of functionality, including general signal and image processing, computer vision, data compression, and string manipulation.

SSH into Machine-2 EC2 as "ubuntu" user and run the following commands.

First, checkout the source code respository from GitHub:*

git clone https://github.com/intel/ipp-crypto.git
cd ipp-crypto

 

Ensure you are building against a fixed release of the code, and not the development branch. At the time of this writing, the latest release was ippcp_2021.6:

git checkout ippcp_2021.6

 

You only need to build the multi-buffer portion of the Intel IPP package, so change to the multi-buffer crypto library sub-directory. Then, prepare the build by running cmake:

cd sources/ippcp/crypto_mb
cmake . -Bbuild -DCMAKE_INSTALL_PREFIX=/usr

 

This will configure the library to install into /usr. To perform the full build, run:

cd build
make -j
sudo make install

 

This will put the shared library in /usr/lib, which means we won't need to set LD_LIBRARY_PATH.

Building the Intel Multi-Buffer Crypto for IPsec Library

The library provides software crypto acceleration primarily targeting packet processing applications. It can be used for application such as: IPsec, TLS, Wireless (RAN), Cable or MPEG DRM.

First, checkout the source code repository from GitHub:

git clone https://github.com/intel/intel-ipsec-mb.git
cd intel-ipsec-mb

 

Ensure you are building against a fixed release of the code, and not a development branch. At the time of this writing, the latest release was v1.2:

git checkout v1.2

 

There is no configuration step. Build the library using:

make -j

 

To install:

sudo make install NOLDCONFIG=y

This will place the shared libraries in /usr/lib, which again means no LD_LIBRARY_PATH modificiations.

Building the Intel Quick Assist Technology (Intel QAT) Engine for OpenSSL

Checkout the software repository from GitHub:

git clone https://github.com/intel/QAT_Engine.git
cd QAT_Engine

 

Next, ensure you are building against a fixed release of the code, and not a development branch. At the time of this writing, the latest release was v0.6.15:

git checkout v0.6.15

 

To configure the Intel QAT Engine for OpenSSL for all software acceleration features:

./autogen.sh
./configure --enable-qat_sw

 

Then build and install: 

make -j
sudo make install

 

Since this makes use of the distribution-provided OpenSSL installation, we won't need to modify LD_LIBRARY_PATH to use the engine.

After the installation has completed, you should see the engine present in OpenSSL's engine directory.

Below is an example of using OpenSSL 1.1; In Ubuntu 20.04, this is in /usr/lib/x86_64-linux-gnu/engines-1.1.  Please use your supported OpennSSL version: For non-standard builds, the engine directory can be obtained by running the "openssl version" command.

openssl version -e
# Results looks like below
# ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"

 

Verify that the engine is present by running "ls." You should see qatengine.so in the directory list:

ls -l /usr/lib/x86_64-linux-gnu/engines-1.1
# Results look like below
# total 396
# -rw-r--r-- 1 root root 23104 Apr 20 2020 afalg.so
# -rw-r--r-- 1 root root 14120 Apr 20 2020 capi.so
# -rw-r--r-- 1 root root 26688 Apr 20 2020 padlock.so
# -rwxr-xr-x 1 root root 334160 Sep 28 13:27 qatengine.so

 

Testing the Engine

Proper testing at this stage ensures that the QAT Engine is installed and running correctly. The proper working of the QAT Engine is important for the performance improvement of the NGINX web server.

Once the engine is in place, you can proceed with functionality tests. The first test is to ensure the Intel QAT Engine loads correctly.

openssl engine -v qatengine
# Results looks like below
#(qatengine) Reference implementation of QAT crypto engine(qat_sw) v0.6.15
# ENABLE_EXTERNAL_POLLING, POLL, ENABLE_HEURISTIC_POLLING,
# GET_NUM_REQUESTS-IN-FLIGHT, INIT_ENGINE, SW_ALGO_BITMAP

 

If the above command returns errors such as the following, then you need to check the path where the Intel® Multi-Buffer Crypto for IPsec Library and the Intel IPP CryptoMB Library are both installed.

The error may look like this:

#139667965596992:error:25066067:DSO support routines:dlfcn_load:could not load
the shared library:../crypto/dso/dso_dlfcn.c:118:filename(/usr/lib/x86_64-linux-
gnu/engines-1.1/qatengine.so): libcrypto_mb.so: cannot open shared object file:
No such file or directory
#139667965596992:error:25070067:DSO support routines:DSO_load: could not load the
shared library:../crypto/dso/dso_lib.c:162:
#139667965596992:error:260B6084:engine routines:dynamic_load:dso not
found:../crypto/engine/eng_dyn.c:414:
#139667965596992:error:2606A074:engine routines: ENGINE_by_id:no such
engine:../crypto/engine/eng_list.c:334:id=qatengine

 

We recommend using the distribution provided OpenSSL from your OS. In this example, we have used the distribution provided OpenSSL. The instructions are based on this Intel article

Note: If you are using the distribution-provided OpenSSL:

Make sure the Intel® Multi-Buffer Crypto for IPsec Library and the Intel IPP CryptoMB Library are both installed into /usr/lib. If you did not set a prefix for the former, it will not install into /usr/local, and you’ll need to set LD_LIBRARY_PATH in your environment.

Testing with 2048-bit keys

This will do an OpenSSL speed test for 2048-bit encryption key. First, we will do a baseline test with regular OpenSSL without the QAT engine. Next, we will do the same test with QAT engine enabled. We will be able to see the performance improvements comaring the sign and verify operations per second.

Baseline

taskset 0x1 openssl speed rsa2048

 

Testing with Intel QAT Engine for OpenSSL

taskset 0x1 openssl speed -engine qatengine -async_jobs 8 rsa2048

 

From the results, you should be able to see that the speed test with the -engine qatengine flag produces improved metrics. This is the result of the performance gain that is coming from the underlying Intel Crypto NI instructions from the CPU.

Now that we have set up the QAT Engine for OpenSSL and tested the engine, we are ready to move on to Part 4: Building NGINX async mode from source code and optimize settings.