Intel aes-ni not available in RHEL 9 on Intel® Xeon® Processor

[Toby-GE-Aerospace]

It doesn't appear the the Intel AES NI hardware acceleration is working on Intel® Xeon® Processor D-2733NT or D-1718T running RHEL 9.1.

I installed the QAT driver QAT20.L.1.0.10-00005 using the "./configure --enable-qat-lkcf" to register with RHEL crypto modules.

A few observations:

1) the qat service status says  qat_service[20983]: There is 0 QAT acceleration device(s) in the system

2) openssl engine  - does not show aes-ni present....   Only shows the following
    (rdrand) Intel RDRAND engine

3) there is no kernel AES NI module per command

sort -u /proc/crypto | grep module
module : crc32c_intel
module : crc32_pclmul
module : crct10dif_pclmul
module : ghash_clmulni_intel
module : kernel

4) modprobe intel-aesni
modprobe: FATAL: Module intel-aesni not found in directory /lib/modules/5.14.0-162.6.1.el9_1.x86_64

5) mpicalc --print-config    - this shows the intel-aesni available

hwflist:intel-cpu:intel-bmi2:intel-ssse3:intel-sse4.1:intel-pclmul:intel-aesni:intel-rdrand:intel-avx:intel-avx2:intel-fast-vpgather:intel-rdtsc:intel-shaext:intel-vaes-vpclmul:
fips-mode:n:::
rng-type:standard:1:3030000:1:
compliance:::

Per the cryptsetup benchmark below, maybe the intel-aes-ni instructions are being executed as the aes benchmakrs far exceed the serpent and twofish benchmark.

cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1 1820444 iterations per second for 256-bit key
PBKDF2-sha256 3172695 iterations per second for 256-bit key
PBKDF2-sha512 1365333 iterations per second for 256-bit key
PBKDF2-ripemd160 783689 iterations per second for 256-bit key
PBKDF2-whirlpool 596459 iterations per second for 256-bit key
argon2i 10 iterations, 1048576 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
argon2id 10 iterations, 1048576 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 1345.8 MiB/s 4858.5 MiB/s
serpent-cbc 128b 88.2 MiB/s 592.9 MiB/s
twofish-cbc 128b 208.4 MiB/s 386.1 MiB/s
aes-cbc 256b 1029.4 MiB/s 3917.9 MiB/s
serpent-cbc 256b 91.3 MiB/s 595.8 MiB/s
twofish-cbc 256b 212.5 MiB/s 386.3 MiB/s
aes-xts 256b 3975.3 MiB/s 3986.6 MiB/s
serpent-xts 256b 564.2 MiB/s 539.8 MiB/s
twofish-xts 256b 356.9 MiB/s 366.4 MiB/s
aes-xts 512b 3499.8 MiB/s 3500.7 MiB/s
serpent-xts 512b 570.1 MiB/s 538.8 MiB/s
twofish-xts 512b 359.5 MiB/s 365.2 MiB/s

So are there instructions on how to invoke the intel-AESNI hardware acceleration for openssl in RHEL 9.1 ?

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Thank you for posting on the Intel Community Support forum.

I received your inquiry regarding Intel QAT, and I will be assisting you with this.

Please allow us to review the information provided and I will contact you back as soon as possible or in case more information is required.

Best regards,

Bruce C.

Intel Customer Support Technician

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Thank you for waiting.

Please review the following and let us know if you have any questions.

• The AES-NI kernel module is independent from the QAT driver. Regardless of the QAT driver been installed or not, a "aesni_intel" module should be registered.
• AES-NI should be in both processors (D-2733NT and D-1718T). Chances are AES-NI is disabled in BIOS, so our recommendation is to check if AES-NI is enabled. Since this community is focused only on the QAT scope, if additional help with the Xeon D processors themselves is needed, we recommend posting in the Intel Embedded Forum to get assistance with the processor settings.
• The fact that there are 0 QAT devices available is interesting. Was this seen from the D-1718T or from the D-2733NT? The D-1718T does not support QAT, hence it's expected to see the message of no QAT devices available, but the D-2733NT does support QAT.
• For reference, this is the output you should see when running the command sort -u /proc/crypto | grep module. (Note: This output was from a different system with a Xeon Scalable Processor, but it should be similar for a Xeon-D platform):
• Without QAT installed:
  • module    : aesni_intel <---- Note AES-NI is here because it's enabled in BIOS
    • module    : kernel
  • module    : crc32c_intel
  • module    : crc32_pclmul
  • module    : crct10dif_pclmul
  • module    : ghash_clmulni_intel
• With QAT installed using --enable-qat-lkcf:
  • module    : aesni_intel <------ AES-NI still there (no QAT related)
    • module    : kernel
  • module    : crc32c_intel
  • module    : crc32_pclmul
  • module    : crct10dif_pclmul
  • module    : ghash_clmulni_intel
  • module    : intel_qat <------ New QAT module registered in LKCF
• Lastly, there is a newer QAT driver available (4.21.0). Although not impacting anything in the setup, it's always recommended to use the latest version available: https://www.intel.com/content/www/us/en/download/19734/intel-quickassist-technology-driver-for-linux-hw-version-1-7.html

I will follow up March 28th in case additional time is required.

Regards,

Bruce C.

Intel Customer Support Technician

[Toby-GE-Aerospace]

Bruce,

This was seen on the D-2733NT and the AES-NI is enabled in the Super Micro BIOS. 

The Super Micro hardware we are using is: https://www.supermicro.com/en/products/motherboard/x12sdv-8c-spt8f

The following commands show no QAT devices and no  crypto module named aesni_intel

systemctl status qat.service
Loaded: loaded (/usr/lib/systemd/system/qat.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2023-03-28 12:34:00 EDT; 20min ago
TriggeredBy: ● qat.timer
Process: 2379 ExecStart=/etc/init.d/qat_service start (code=exited, status=0/SUCCESS)
Main PID: 2379 (code=exited, status=0/SUCCESS)
CPU: 236ms

Mar 28 12:34:00 D2700-175 systemd[1]: Starting QAT service...
Mar 28 12:34:00 D2700-175 qat_service[2431]: Restarting all devices.
Mar 28 12:34:00 D2700-175 qat_service[2433]: Checking status of all devices.
Mar 28 12:34:00 D2700-175 qat_service[2433]: There is 0 QAT acceleration device(s) in the system:
Mar 28 12:34:00 D2700-175 systemd[1]: Finished QAT service.

sort -u /proc/crypto | grep module
module : crc32c_intel
module : crc32_pclmul
module : crct10dif_pclmul
module : ghash_clmulni_intel
module : kernel

I will install the latest QAT driver 4.21.0, but that probably won't make a difference.

Do have any other items I should look at ?

Thank You,

-Toby

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Good day,

Please allow us to further review and I will get back to you soon.

Regards,

Bruce C.

Intel Customer Support Technician

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Thank you for your time.

Please share with us the output of the command lspci -nn | egrep -e '8086:'. It will list all Intel PCI devices in the system.

I will follow up March 31st in case you need additional time.

Regards,

Bruce C.

Intel Customer Support Technician

[Toby-GE-Aerospace]

Bruce,

We want to purchase a D-1747 mother board that supports QAT.

https://www.intel.com/content/www/us/en/products/sku/226116/intel-xeon-d1747nte-processor-15m-cache-up-to-3-50-ghz/specifications.html

Can you recommend a mother board / evaluation board that has the  D-1747 ?

Following is output of the command you requested from the D-2733NT.

[root@D2700-175 ~]# lspci -nn | egrep -e '8086:'
00:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
00:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
00:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
00:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
00:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
00:01.0 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.1 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.2 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.3 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.4 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.5 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.6 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.7 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:02.0 System peripheral [0880]: Intel Corporation Ice Lake MSM [8086:09a6]
00:02.1 System peripheral [0880]: Intel Corporation Ice Lake PMON MSM [8086:09a7]
00:02.4 Non-Essential Instrumentation [1300]: Intel Corporation Ice Lake NorthPeak [8086:3456] (rev 01)
00:08.0 SATA controller [0106]: Intel Corporation Device [8086:18c2] (rev 11)
00:09.0 PCI bridge [0604]: Intel Corporation Device [8086:18a4] (rev 11)
00:0b.0 PCI bridge [0604]: Intel Corporation Device [8086:18a6] (rev 11)
00:0f.0 System peripheral [0880]: Intel Corporation Device [8086:18ac] (rev 11)
00:12.0 PCI bridge [0604]: Intel Corporation Device [8086:18aa] (rev 11)
00:14.0 PCI bridge [0604]: Intel Corporation Device [8086:18ad] (rev 11)
00:16.0 PCI bridge [0604]: Intel Corporation Device [8086:18af] (rev 11)
00:18.0 Communication controller [0780]: Intel Corporation Device [8086:18d3] (rev 11)
00:18.1 Communication controller [0780]: Intel Corporation Device [8086:18d4] (rev 11)
00:18.4 Communication controller [0780]: Intel Corporation Device [8086:18d6] (rev 11)
00:1a.0 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.1 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.2 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.3 Unassigned class [ff00]: Intel Corporation Device [8086:18d9] (rev 11)
00:1c.0 SD Host controller [0805]: Intel Corporation Device [8086:18db] (rev 11)
00:1d.0 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
00:1e.0 USB controller [0c03]: Intel Corporation Device [8086:18d0] (rev 11)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:18dc] (rev 11)
00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:18df] (rev 11)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Device [8086:18e0] (rev 11)
00:1f.7 Non-Essential Instrumentation [1300]: Intel Corporation Device [8086:18e1] (rev 11)
02:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X550 [8086:1563] (rev 01)
02:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X550 [8086:1563] (rev 01)
04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.2 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.3 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
15:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
15:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
15:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
15:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
15:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
84:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
84:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
84:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
84:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
84:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
84:04.0 PCI bridge [0604]: Intel Corporation Device [8086:18d1]
85:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection E823-C for SFP [8086:188c]
85:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection E823-C for SFP [8086:188c]
8c:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
8c:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
8c:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
8c:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
8c:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
8c:02.0 PCI bridge [0604]: Intel Corporation Device [8086:347a] (rev 06)
8c:03.0 PCI bridge [0604]: Intel Corporation Device [8086:347b] (rev 06)
8c:04.0 PCI bridge [0604]: Intel Corporation Device [8086:347c] (rev 06)
8c:05.0 PCI bridge [0604]: Intel Corporation Device [8086:347d] (rev 06)
fe:00.0 System peripheral [0880]: Intel Corporation Device [8086:3450]
fe:00.1 System peripheral [0880]: Intel Corporation Device [8086:3451]
fe:00.2 System peripheral [0880]: Intel Corporation Device [8086:3452]
fe:00.3 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
fe:00.5 System peripheral [0880]: Intel Corporation Device [8086:3455]
fe:0b.0 System peripheral [0880]: Intel Corporation Device [8086:3448]
fe:0b.1 System peripheral [0880]: Intel Corporation Device [8086:3448]
fe:0b.2 System peripheral [0880]: Intel Corporation Device [8086:344b]
fe:0c.0 Performance counters [1101]: Intel Corporation Device [8086:344a]
fe:0d.0 Performance counters [1101]: Intel Corporation Device [8086:344a]
fe:1a.0 Performance counters [1101]: Intel Corporation Device [8086:2880]
fe:1b.0 Performance counters [1101]: Intel Corporation Device [8086:2880]
ff:00.0 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.1 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.2 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.3 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.4 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.5 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.6 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.7 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:01.0 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:01.1 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:01.2 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:01.3 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:0a.0 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.1 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.2 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.3 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.4 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.5 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.6 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.7 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0b.0 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0b.1 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0b.2 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0b.3 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:1d.0 System peripheral [0880]: Intel Corporation Device [8086:344f]
ff:1d.1 System peripheral [0880]: Intel Corporation Device [8086:3457]
ff:1e.0 System peripheral [0880]: Intel Corporation Device [8086:3458] (rev 01)
ff:1e.1 System peripheral [0880]: Intel Corporation Device [8086:3459] (rev 01)
ff:1e.2 System peripheral [0880]: Intel Corporation Device [8086:345a] (rev 01)
ff:1e.3 System peripheral [0880]: Intel Corporation Device [8086:345b] (rev 01)
ff:1e.4 System peripheral [0880]: Intel Corporation Device [8086:345c] (rev 01)
ff:1e.5 System peripheral [0880]: Intel Corporation Device [8086:345d] (rev 01)
ff:1e.6 System peripheral [0880]: Intel Corporation Device [8086:345e] (rev 01)
ff:1e.7 System peripheral [0880]: Intel Corporation Device [8086:345f] (rev 01)
[root@D2700-175 ~]#

[Toby-GE-Aerospace]

Bruce,

Please disregard the comment regarding we  want to purchase a D-1747.

Here's the output of the  lspci -nn | egrep -e '8086:' command on our Intel(R) Xeon(R) D-1747NTE CPU @ 2.50GHz 

login as: toby
toby@10.32.164.189's password:
Activate the web console with: systemctl enable --now cockpit.socket

Register this system with Red Hat Insights: insights-client --register
Create an account or view all your systems at https://red.ht/insights-dashboard
Last login: Tue Mar 28 15:29:04 2023 from 10.80.111.253
[toby@HSDTS-189-D1700 ~]$ lspci -nn | egrep -e '8086:'
00:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
00:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
00:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
00:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
00:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
00:01.0 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.1 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.2 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.3 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.4 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.5 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.6 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:01.7 System peripheral [0880]: Intel Corporation Ice Lake CBDMA [QuickData Technology] [8086:0b00]
00:02.0 System peripheral [0880]: Intel Corporation Ice Lake MSM [8086:09a6]
00:02.1 System peripheral [0880]: Intel Corporation Ice Lake PMON MSM [8086:09a7]
00:02.4 Non-Essential Instrumentation [1300]: Intel Corporation Ice Lake NorthPeak [8086:3456] (rev 01)
00:06.0 PCI bridge [0604]: Intel Corporation Device [8086:18da] (rev 11)
00:07.0 SATA controller [0106]: Intel Corporation Device [8086:18b2] (rev 11)
00:09.0 PCI bridge [0604]: Intel Corporation Device [8086:18a4] (rev 11)
00:0a.0 PCI bridge [0604]: Intel Corporation Device [8086:18a5] (rev 11)
00:0b.0 PCI bridge [0604]: Intel Corporation Device [8086:18a6] (rev 11)
00:0f.0 System peripheral [0880]: Intel Corporation Device [8086:18ac] (rev 11)
00:10.0 PCI bridge [0604]: Intel Corporation Device [8086:18a8] (rev 11)
00:12.0 PCI bridge [0604]: Intel Corporation Device [8086:18aa] (rev 11)
00:14.0 PCI bridge [0604]: Intel Corporation Device [8086:18ad] (rev 11)
00:16.0 PCI bridge [0604]: Intel Corporation Device [8086:18af] (rev 11)
00:18.0 Communication controller [0780]: Intel Corporation Device [8086:18d3] (rev 11)
00:18.1 Communication controller [0780]: Intel Corporation Device [8086:18d4] (rev 11)
00:18.4 Communication controller [0780]: Intel Corporation Device [8086:18d6] (rev 11)
00:1a.0 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.1 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.2 Serial controller [0700]: Intel Corporation Device [8086:18d8] (rev 11)
00:1a.3 Unassigned class [ff00]: Intel Corporation Device [8086:18d9] (rev 11)
00:1c.0 SD Host controller [0805]: Intel Corporation Device [8086:18db] (rev 11)
00:1d.0 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
00:1e.0 USB controller [0c03]: Intel Corporation Device [8086:18d0] (rev 11)
00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:18dc] (rev 11)
00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:18df] (rev 11)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Device [8086:18e0] (rev 11)
00:1f.7 Non-Essential Instrumentation [1300]: Intel Corporation Device [8086:18e1] (rev 11)
01:00.0 Co-processor [0b40]: Intel Corporation 200xx Series QAT [8086:18ee] (rev 11)
02:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
03:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)
04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
14:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
14:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
14:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
14:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
14:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
f3:00.0 System peripheral [0880]: Intel Corporation Ice Lake Memory Map/VT-d [8086:09a2] (rev 04)
f3:00.1 System peripheral [0880]: Intel Corporation Ice Lake Mesh 2 PCIe [8086:09a4] (rev 04)
f3:00.2 System peripheral [0880]: Intel Corporation Ice Lake RAS [8086:09a3] (rev 04)
f3:00.3 System peripheral [0880]: Intel Corporation Device [8086:09a5] (rev 04)
f3:00.4 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
f3:04.0 PCI bridge [0604]: Intel Corporation Device [8086:18d1]
f4:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection E823-L for SFP [8086:124d]
f4:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection E823-L for SFP [8086:124d]
fe:00.0 System peripheral [0880]: Intel Corporation Device [8086:3450]
fe:00.1 System peripheral [0880]: Intel Corporation Device [8086:3451]
fe:00.2 System peripheral [0880]: Intel Corporation Device [8086:3452]
fe:00.3 Host bridge [0600]: Intel Corporation Ice Lake IEH [8086:0998]
fe:00.5 System peripheral [0880]: Intel Corporation Device [8086:3455]
fe:0b.0 System peripheral [0880]: Intel Corporation Device [8086:3448]
fe:0b.1 System peripheral [0880]: Intel Corporation Device [8086:3448]
fe:0b.2 System peripheral [0880]: Intel Corporation Device [8086:344b]
fe:0c.0 Performance counters [1101]: Intel Corporation Device [8086:344a]
fe:1a.0 Performance counters [1101]: Intel Corporation Device [8086:2880]
ff:00.0 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.1 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.2 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.3 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.4 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:00.5 System peripheral [0880]: Intel Corporation Device [8086:344c]
ff:0a.0 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.1 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.2 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.3 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.4 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:0a.5 System peripheral [0880]: Intel Corporation Device [8086:344d]
ff:1d.0 System peripheral [0880]: Intel Corporation Device [8086:344f]
ff:1d.1 System peripheral [0880]: Intel Corporation Device [8086:3457]
ff:1e.0 System peripheral [0880]: Intel Corporation Device [8086:3458] (rev 01)
ff:1e.1 System peripheral [0880]: Intel Corporation Device [8086:3459] (rev 01)
ff:1e.2 System peripheral [0880]: Intel Corporation Device [8086:345a] (rev 01)
ff:1e.3 System peripheral [0880]: Intel Corporation Device [8086:345b] (rev 01)
ff:1e.4 System peripheral [0880]: Intel Corporation Device [8086:345c] (rev 01)
ff:1e.5 System peripheral [0880]: Intel Corporation Device [8086:345d] (rev 01)
ff:1e.6 System peripheral [0880]: Intel Corporation Device [8086:345e] (rev 01)
ff:1e.7 System peripheral [0880]: Intel Corporation Device [8086:345f] (rev 01)
[toby@HSDTS-189-D1700 ~]$ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 57 bits virtual
Byte Order: Little Endian
CPU(s): 20
On-line CPU(s) list: 0-19
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) D-1747NTE CPU @ 2.50GHz
CPU family: 6
Model: 108
Thread(s) per core: 2
Core(s) per socket: 10
Socket(s): 1
Stepping: 1
Frequency boost: enabled
CPU max MHz: 2501.0000
CPU min MHz: 800.0000
BogoMIPS: 5000.00
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe s
yscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf t
sc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2
apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_s
ingle intel_ppin ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1
avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_
ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect w
bnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme
avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities
Virtualization features:
Virtualization: VT-x
Caches (sum of all):
L1d: 480 KiB (10 instances)
L1i: 320 KiB (10 instances)
L2: 12.5 MiB (10 instances)
L3: 15 MiB (1 instance)
NUMA:
NUMA node(s): 1
NUMA node0 CPU(s): 0-19
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Retbleed: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Srbds: Not affected
Tsx async abort: Not affected
[toby@HSDTS-189-D1700 ~]$ ~

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Good day and thank you for the information.

I will contact you back soon.

Regards,

Bruce C.

Intel Customer Support Technician

[Toby-GE-Aerospace]

Bruce,

I upgraded to the new driver and I'm seeing the QAT device.

The only thing is now the performance using crypsetup benchmark is down about 40%.

Is that expected ?

[root@HSDTS-189-D1700 QAT.4.21.0]# systemctl status qat
● qat.service - QAT service
Loaded: loaded (/usr/lib/systemd/system/qat.service; enabled; vendor preset: disabled)
Active: active (exited) since Wed 2023-03-29 15:29:23 EDT; 25min ago
Process: 85560 ExecStart=/etc/init.d/qat_service start (code=exited, status=0/SUCCESS)
Main PID: 85560 (code=exited, status=0/SUCCESS)
CPU: 115ms

Mar 29 15:29:22 HSDTS-189-D1700 systemd[1]: Starting QAT service...
Mar 29 15:29:22 HSDTS-189-D1700 qat_service[85607]: Restarting all devices.
Mar 29 15:29:23 HSDTS-189-D1700 qat_service[85607]: Processing /etc/200xx_dev0.conf
Mar 29 15:29:23 HSDTS-189-D1700 qat_service[85627]: Checking status of all devices.
Mar 29 15:29:23 HSDTS-189-D1700 qat_service[85627]: There is 1 QAT acceleration device(s) in the system:
Mar 29 15:29:23 HSDTS-189-D1700 qat_service[85627]: qat_dev0 - type: 200xx, inst_id: 0, node_id: 0, bsf: 0000:01:00.0, #accel: 3 #eng>
Mar 29 15:29:23 HSDTS-189-D1700 systemd[1]: Finished QAT service.

[root@HSDTS-189-D1700 QAT.4.21.0]# sort -u /proc/crypto | grep module
module : crc32c_intel
module : crc32_pclmul
module : crct10dif_pclmul
module : ghash_clmulni_intel
module : intel_qat
module : kernel
module : serpent_avx2
module : serpent_avx_x86_64
module : serpent_generic
module : serpent_sse2_x86_64
module : twofish_avx_x86_64
module : twofish_generic
module : twofish_x86_64
module : twofish_x86_64_3way
[root@HSDTS-189-D1700 QAT.4.21.0]# lsmod | grep qat
qat_200xx 20480 0
intel_qat 372736 2 usdm_drv,qat_200xx
uio 24576 1 intel_qat
[root@HSDTS-189-D1700 QAT.4.21.0]# mpicalc --print-config
version:1.10.0-unknown:10a00:1.42-unknown:12a00:
cc:110201:gcc:11.2.1 20220127 (Red Hat 11.2.1-9):
ciphers:arcfour:blowfish:cast5:des:aes:twofish:serpent:rfc2268:seed:camellia:idea:salsa20:gost28147:chacha20:
pubkeys:dsa:elgamal:rsa:ecc:
digests:crc:gostr3411-94:md4:md5:rmd160:sha1:sha256:sha512:sha3:tiger:whirlpool:stribog:blake2:
rnd-mod:getentropy:
cpu-arch:x86:
mpi-asm:amd64/mpih-add1.S:amd64/mpih-sub1.S:amd64/mpih-mul1.S:amd64/mpih-mul2.S:amd64/mpih-mul3.S:amd64/mpih-lshift.S:amd64/mpih-rshift.S:
hwflist:intel-cpu:intel-bmi2:intel-ssse3:intel-sse4.1:intel-pclmul:intel-aesni:intel-rdrand:intel-avx:intel-avx2:intel-fast-vpgather:intel-rdtsc:intel-shaext:intel-vaes-vpclmul:
fips-mode:n:::
rng-type:standard:1:3030000:1:
compliance:::
[root@HSDTS-189-D1700 QAT.4.21.0]#

cryptsetup benchmark Before QAT Driver Install 

crypsetup benchmark after QAT driver install

Why is the performance degraded now ?

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Good day,

I'm glad to hear there is some progress with the new driver.

Now, regarding the performance, Cryptsetup is an open-source utility used to conveniently set up disk encryption based on the dm-crypt kernel module, however, QAT is not used for dm-crypt and dm-integrity, due to how dm-integrity allocates memory, and we don’t have any performance numbers between QAT and cryptsetup for this reason.

On this scenario, our only recommendation would be to make sure you are using kernel 5.19 or greater, and check the performance results again; QAT should not be used during that test.

Please let me know if you have any additional questions or concerns, and I will follow up on April 3rd in case you would like additional time for testing.

Regards,

Bruce C.

Intel Customer Support Technician

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Good day,

I wanted to follow up on this thread in case you had any questions or concerns.

The thread will be closed on April 6th if there is no confirmation about further assistance being required.

Regards,

Bruce C.

Intel Customer Support Technician

[BrusC_Intel]

Hello, Toby-GE-Aerospace.

Good day,

This message is to inform you about the closure of this support thread.

I previously wanted to know if you had further question or concerns, but there was no response afte rtrying to follow up.

The thread will be closed right now, and it will no longer be monitored by Intel support, but if you require any type of assistance from Intel in the future, just open a new thread, or contact us using any of the available support methods:

- https://www.intel.com/content/www/us/en/support/contact-intel.html

Best regards,

Bruce C.

Intel Customer Support Technician