topic Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N in Intel® QuickAssist Technology (Intel® QAT)

Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Wed, 18 Jun 2025 06:03:29 GMT

Hello. I used QAT Engine based on in-tree QAT driver and library, which was built by source to offload AES-GCM encryption at QAT_HW.

I tested it by using async_nginx but it failed to connect. The baseline nginx and QAT Engine uses QAT_SW to run AES-GCM. I wonder that Xeon Gold 6438N does not support HW offloading for AES-GCM.

OS: Ubuntu 24.04, Linux kernel 6.8.0-86-generic

Processor: Intel(R) Xeon(R) Gold 6438N

QAT driver: in-tree firmware(1.1.20)

Re:Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

Ronny_G_Intel — Thu, 19 Jun 2025 20:09:42 GMT

Hi dykim1101,

Can you please provide me with more information (configuration settings) during the in-tree QAT driver installation?

Do you see a directory ICP_ROOT/quickassist/utilities/release-files/debug_tool/? If this directory was generated during the package installation, you will find an icp_dump.sh script file, please provide the output of this script for better understanding of your QAT deployment.

Regards,

Ronny G

Re: Re:Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Fri, 20 Jun 2025 04:38:42 GMT

Hi Ronny,

I uses `QAT20.L.1.2.30-00090.tar.gz` as QAT driver installation to get `debug_tool/icp_dump.sh`.

Here are the configuration result

adf_ctl_status.txt

Checking status of all devices. There is 4 QAT acceleration device(s) in the system: qat_dev0 - type: 4xxx, inst_id: 0, node_id: 0, bsf: 0000:76:00.0, #accel: 1 #engines: 9 state: up qat_dev1 - type: 4xxx, inst_id: 1, node_id: 0, bsf: 0000:7a:00.0, #accel: 1 #engines: 9 state: up qat_dev2 - type: 4xxx, inst_id: 2, node_id: 1, bsf: 0000:f3:00.0, #accel: 1 #engines: 9 state: up qat_dev3 - type: 4xxx, inst_id: 3, node_id: 1, bsf: 0000:f7:00.0, #accel: 1 #engines: 9 state: up

BIOS.txt

# dmidecode 3.5 Getting SMBIOS data from sysfs. SMBIOS 3.6.0 present. # SMBIOS implementations newer than version 3.5.0 are not # fully supported by this version of dmidecode. Handle 0x0000, DMI type 0, 26 bytes BIOS Information Vendor: American Megatrends International, LLC. Version: 2.2 Release Date: 03/01/2024 Address: 0xF0000 Runtime Size: 64 kB ROM Size: 32 MB Characteristics: PCI is supported BIOS is upgradeable BIOS shadowing is allowed Boot from CD is supported Selectable boot is supported BIOS ROM is socketed EDD is supported Japanese floppy for NEC 9800 1.2 MB is supported (int 13h) Japanese floppy for Toshiba 1.2 MB is supported (int 13h) 5.25"/360 kB floppy services are supported (int 13h) 5.25"/1.2 MB floppy services are supported (int 13h) 3.5"/720 kB floppy services are supported (int 13h) 3.5"/2.88 MB floppy services are supported (int 13h) Print screen service is supported (int 5h) Serial services are supported (int 14h) Printer services are supported (int 17h) CGA/mono video services are supported (int 10h) ACPI is supported USB legacy is supported BIOS boot specification is supported Targeted content distribution is supported UEFI is supported BIOS Revision: 5.32 Handle 0x0068, DMI type 13, 22 bytes BIOS Language Information Language Description Format: Long Installable Languages: 1 en|US|iso8859-1 Currently Installed Language: en|US|iso8859-1

cmdline.txt

BOOT_IMAGE=/boot/vmlinuz-6.8.0-60-generic root=UUID=1b58c53f-5c30-44c5-8c83-9cf8432355e6 ro intel_iommu=on vfio-pci.ids=8086:4943 ipv6.disable=1

environment.txt

SHELL=/bin/bash SUDO_GID=836842799 SUDO_COMMAND=/usr/bin/su SUDO_USER=ubuntu PWD=/QAT/quickassist/utilities/debug_tool LOGNAME=root HOME=/root LANG=en_US.UTF-8 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90: LESSCLOSE=/usr/bin/lesspipe %s %s TERM=screen LESSOPEN=| /usr/bin/lesspipe %s USER=root SHLVL=2 ICP_ROOT=/QAT PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin SUDO_UID=836842799 MAIL=/var/mail/root OLDPWD=/QAT _=/usr/bin/env

Sku_details.txt

********** SKU Details ********** qat_dev0 is 4xxx BDF=0000:76:00.0 SKU=0x-0x qat_dev1 is 4xxx BDF=0000:7a:00.0 SKU=0x-0x qat_dev2 is 4xxx BDF=0000:f3:00.0 SKU=0x-0x qat_dev3 is 4xxx BDF=0000:f7:00.0 SKU=0x-0x

config_files/4xxx_dev0.conf

[GENERAL] ServicesEnabled = asym;dc ConfigVersion = 2 #Default values for number of concurrent requests*/ CyNumConcurrentSymRequests = 512 CyNumConcurrentAsymRequests = 64 #Statistics, valid values: 1,0 statsGeneral = 1 statsDh = 1 statsDrbg = 1 statsDsa = 1 statsEcc = 1 statsKeyGen = 1 statsDc = 1 statsLn = 1 statsPrime = 1 statsRsa = 1 statsSym = 1 # This flag enables dc chaining # when set to a value of 1 (default is 0). ServiceChainingEnabled = 0 # Default heartbeat timer is 1s HeartbeatTimer = 1000 # This flag is to enable SSF features StorageEnabled = 0 # services by specifying a value of 1 (default is 0) PkeServiceDisabled = 0 # This flag is to enable device auto reset on heartbeat error AutoResetOnError = 0 # This flag is to enable power management idle support PmIdleSupport = 1 # This flag is to enable key protection technology KptEnabled = 1 # Define the maximum SWK count per function can have # Default value is 1, the maximum value is 128 KptMaxSWKPerFn = 1 # Define the maximum SWK count per pasid can have # Default value is 1, the maximum value is 128 KptMaxSWKPerPASID = 1 # Define the maximum SWK lifetime in second # Default value is 0 (eternal of life) # The maximum value is 31536000 (one year) KptMaxSWKLifetime = 31536000 # Flag to define whether to allow SWK to be shared among processes # Default value is 0 (shared mode is off) KptSWKShared = 0 ############################################## # Kernel Instances Section ############################################## [KERNEL] NumberCyInstances = 0 NumberDcInstances = 0 ############################################## # ADI Section for Scalable IOV ############################################## [SIOV] NumberAdis = 0 ############################################## # User Process Instance Section ############################################## [SSL] NumberCyInstances = 2 NumberDcInstances = 2 NumProcesses = 1 LimitDevAccess = 0 Cy0Name = "SSL0" Cy0IsPolled = 1 # List of core affinities Cy0CoreAffinity = 1 Cy1Name = "SSL1" Cy1IsPolled = 1 # List of core affinities Cy1CoreAffinity = 2 # Data Compression - User instance #0 Dc0Name = "Dc0" Dc0IsPolled = 1 # List of core affinities Dc0CoreAffinity = 3 # Data Compression - User instance #1 Dc1Name = "Dc1" Dc1IsPolled = 1 # List of core affinities Dc1CoreAffinity = 4

qat_4xxx_0000:76:00.0/pm_status

----------- PM Fuse info --------- enable_pm: 0x0 enable_pm_idle: 0x1 enable_deep_pm_idle: 0x1 max_pwrreq: 0x1c min_pwrreq: 0x8 ------------ PM Info ------------ power_level: min cpm_pm_state: 0x4 pending_wp: 0x8 current_wp: 0x8 idle_enable: 0x1 idle_filter: 0x6 min_pwr_ack: 0x0 min_pwr_ack_pending: 0x0 thr_value: 0x7 pm_mode: STATIC ----------- SSM_PM Info ---------- ssm_pm_enable: 0x0 active_constraint: 0x0 domain_power_gated: 0xf ath_active_count: 0x0 cph_active_count: 0x0 pke_active_count: 0x0 cpr_active_count: 0x0 dcpr_active_count: 0x0 ucs_active_count: 0x0 xlt_active_count: 0x0 wat_active_count: 0x0 wcp_active_count: 0x0 ath_managed_count: 0x0 cph_managed_count: 0x0 pke_managed_count: 0x0 cpr_managed_count: 0x0 dcpr_managed_count: 0x0 ucs_managed_count: 0x0 xlt_managed_count: 0x0 wat_managed_count: 0x0 wcp_managed_count: 0x0 ------------- PM Log ------------- host_msg_event_count: 0x1 sys_pm_event_count: 0x2 ssm_event_count: 0x0 timer_event_count: 0x0 unknown_event_count: 0x0 event0: 0x0 event1: 0x0 event2: 0x0 event3: 0x0 event4: 0x0 event5: 0x1c01 event6: 0x20000003 event7: 0x80f idle_irq_cnt: 0x1 host_ack_counter: 0x1 host_nack_counter: 0x0 fw_irq_cnt: 0x0 throttle_irq_cnt: 0x0 ----------- HW PM CSRs ----------- CPM_PM_FW_INIT: 0x37 CPM_PM_STATUS: 0x4404700 CPM_PM_MAIN_FW: 0x70 CPM_PM_PWRREQ: 0x8 CPM_PM_HOST_MSG: 0x2 CPM_PM_INTERRUPT: 0x140700

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 01:53:25 GMT

Hi Ronny,

Here are results of the script

adf_ctl_status.txt

BIOS.txt

environment.txt

Sku_details.txt

config_files/4xxx_dev0.conf

qat_4xxx_0000:76:00.0/dev_cfg

# cat qat_4xxx_0000\:76\:00.0/dev_cfg [GENERAL] AutoResetOnError = 0 HeartbeatTimer = 1000 KptEnabled = 1 KptMaxSWKLifetime = 31536000 KptMaxSWKPerFn = 1 KptMaxSWKPerPASID = 1 KptSWKShared = 0 PkeServiceDisabled = 0 PmIdleSupport = 1 ServiceChainingEnabled = 0 ServicesEnabled = asym;dc StorageEnabled = 0 statsDc = 1 statsDh = 1 statsDrbg = 1 statsDsa = 1 statsEcc = 1 statsGeneral = 1 statsKeyGen = 1 statsLn = 1 statsPrime = 1 statsRsa = 1 statsSym = 1 Device_Max_Banks = 64 Device_PkgId = 0 Device_NodeId = 0 Device_Max_Rings_Per_Bank = 2 Firmware_UofVer = 1.2.70 HW_RevId = 64 Cipher_Capabilities_Mask = 0x3500f2 Hash_Capabilities_Mask = 0xcd73f0 Asym_Capabilities_Mask = 0x1fc Firmware_MmpVer = 6.0.0 SVMEnabled = 0 ATEnabled = 0 UQEnabled = 0 Device_Capabilities_Mask = 0x3843a22 Device_DcExtendedFeatures = 0x60001b05 [KERNEL] NumberCyInstances = 0 NumberDcInstances = 0 [SIOV] NumberAdis = 0 [SSL] LimitDevAccess = 0 [SSL_INT_0] Cy0IsPolled = 1 Cy0Name = SSL0 Cy1IsPolled = 1 Cy1Name = SSL1 Dc0IsPolled = 1 Dc0Name = Dc0 Dc1IsPolled = 1 Dc1Name = Dc1 NumberCyInstances = 2 NumberDcInstances = 2 Dc0CoreAffinity = 3 Dc0BankNumber = 2 Dc0RingTx = 0 Dc0RingRx = 1 Dc0NumConcurrentRequests = 512 Dc1CoreAffinity = 4 Dc1BankNumber = 3 Dc1RingTx = 0 Dc1RingRx = 1 Dc1NumConcurrentRequests = 512 Cy0CoreAffinity = 1 Cy0BankNumberAsym = 0 Cy0RingAsymTx = 0 Cy0RingAsymRx = 1 Cy0NumConcurrentAsymRequests = 64 Cy1CoreAffinity = 2 Cy1BankNumberAsym = 1 Cy1RingAsymTx = 0 Cy1RingAsymRx = 1 Cy1NumConcurrentAsymRequests = 64 [Accelerator0] Bank0InterruptCoalescingEnabled = 1 Bank0InterruptCoalescingTimerNs = 1000 Bank0InterruptCoalescingNumResponses = 0 Bank0CoreAffinity = 255 Bank1InterruptCoalescingEnabled = 1 Bank1InterruptCoalescingTimerNs = 1000 Bank1InterruptCoalescingNumResponses = 0 Bank1CoreAffinity = 255 ... Bank63InterruptCoalescingEnabled = 1 Bank63InterruptCoalescingTimerNs = 1000 Bank63InterruptCoalescingNumResponses = 0 Bank63CoreAffinity = 255

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:21:51 GMT

Hi Ronny,

Since Intel Community blocks the results of script, I send you the partial of the result.

BIOS.txt

environment.txt

SHELL=/bin/bash SUDO_GID=836842799 SUDO_COMMAND=/usr/bin/su SUDO_USER=user PWD=/QAT/quickassist/utilities/debug_tool LOGNAME=root HOME=/root LANG=en_US.UTF-8 LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90: LESSCLOSE=/usr/bin/lesspipe %s %s TERM=screen LESSOPEN=| /usr/bin/lesspipe %s USER=root SHLVL=2 ICP_ROOT=/QAT PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin SUDO_UID=836842799 MAIL=/var/mail/root OLDPWD=/QAT _=/usr/bin/env

Sku_details.txt

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:24:53 GMT

qat_4xxx_0000:76:00.0/dev_cfg

[GENERAL] AutoResetOnError = 0 HeartbeatTimer = 1000 KptEnabled = 1 KptMaxSWKLifetime = 31536000 KptMaxSWKPerFn = 1 KptMaxSWKPerPASID = 1 KptSWKShared = 0 PkeServiceDisabled = 0 PmIdleSupport = 1 ServiceChainingEnabled = 0 ServicesEnabled = asym;dc StorageEnabled = 0 statsDc = 1 statsDh = 1 statsDrbg = 1 statsDsa = 1 statsEcc = 1 statsGeneral = 1 statsKeyGen = 1 statsLn = 1 statsPrime = 1 statsRsa = 1 statsSym = 1 Device_Max_Banks = 64 Device_PkgId = 0 Device_NodeId = 0 Device_Max_Rings_Per_Bank = 2 Firmware_UofVer = 1.2.70 HW_RevId = 64 Cipher_Capabilities_Mask = 0x3500f2 Hash_Capabilities_Mask = 0xcd73f0 Asym_Capabilities_Mask = 0x1fc Firmware_MmpVer = 6.0.0 SVMEnabled = 0 ATEnabled = 0 UQEnabled = 0 Device_Capabilities_Mask = 0x3843a22 Device_DcExtendedFeatures = 0x60001b05 [KERNEL] NumberCyInstances = 0 NumberDcInstances = 0 [SIOV] NumberAdis = 0 [SSL] LimitDevAccess = 0 [SSL_INT_0] Cy0IsPolled = 1 Cy0Name = SSL0 Cy1IsPolled = 1 Cy1Name = SSL1 Dc0IsPolled = 1 Dc0Name = Dc0 Dc1IsPolled = 1 Dc1Name = Dc1 NumberCyInstances = 2 NumberDcInstances = 2 Dc0CoreAffinity = 3 Dc0BankNumber = 2 Dc0RingTx = 0 Dc0RingRx = 1 Dc0NumConcurrentRequests = 512 Dc1CoreAffinity = 4 Dc1BankNumber = 3 Dc1RingTx = 0 Dc1RingRx = 1 Dc1NumConcurrentRequests = 512 Cy0CoreAffinity = 1 Cy0BankNumberAsym = 0 Cy0RingAsymTx = 0 Cy0RingAsymRx = 1 Cy0NumConcurrentAsymRequests = 64 Cy1CoreAffinity = 2 Cy1BankNumberAsym = 1 Cy1RingAsymTx = 0 Cy1RingAsymRx = 1 Cy1NumConcurrentAsymRequests = 64 [Accelerator0] Bank0InterruptCoalescingEnabled = 1 Bank0InterruptCoalescingTimerNs = 1000 Bank0InterruptCoalescingNumResponses = 0 Bank0CoreAffinity = 255 ... Bank63InterruptCoalescingEnabled = 1 Bank63InterruptCoalescingTimerNs = 1000 Bank63InterruptCoalescingNumResponses = 0 Bank63CoreAffinity = 255

config_files/dev_4xxx0.conf

################################################################ # This file is provided under a dual BSD/GPLv2 license. When using or # redistributing this file, you may do so under either license. # # GPL LICENSE SUMMARY # # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. # # This program is free software; you can redistribute it and/or modify # it under the terms of version 2 of the GNU General Public License as # published by the Free Software Foundation. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. # The full GNU General Public License is included in this distribution # in the file called LICENSE.GPL. # # Contact Information: # Intel Corporation # # BSD LICENSE # # Copyright(c) 2007-2023 Intel Corporation. All rights reserved. # All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # * Neither the name of Intel Corporation nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # version: QAT20.L.1.2.30-00090 ################################################################ [GENERAL] ServicesEnabled = asym;dc ConfigVersion = 2 #Default values for number of concurrent requests*/ CyNumConcurrentSymRequests = 512 CyNumConcurrentAsymRequests = 64 #Statistics, valid values: 1,0 statsGeneral = 1 statsDh = 1 statsDrbg = 1 statsDsa = 1 statsEcc = 1 statsKeyGen = 1 statsDc = 1 statsLn = 1 statsPrime = 1 statsRsa = 1 statsSym = 1 # This flag enables dc chaining # when set to a value of 1 (default is 0). ServiceChainingEnabled = 0 # Default heartbeat timer is 1s HeartbeatTimer = 1000 # This flag is to enable SSF features StorageEnabled = 0 # services by specifying a value of 1 (default is 0) PkeServiceDisabled = 0 # This flag is to enable device auto reset on heartbeat error AutoResetOnError = 0 # This flag is to enable power management idle support PmIdleSupport = 1 # This flag is to enable key protection technology KptEnabled = 1 # Define the maximum SWK count per function can have # Default value is 1, the maximum value is 128 KptMaxSWKPerFn = 1 # Define the maximum SWK count per pasid can have # Default value is 1, the maximum value is 128 KptMaxSWKPerPASID = 1 # Define the maximum SWK lifetime in second # Default value is 0 (eternal of life) # The maximum value is 31536000 (one year) KptMaxSWKLifetime = 31536000 # Flag to define whether to allow SWK to be shared among processes # Default value is 0 (shared mode is off) KptSWKShared = 0 ############################################## # Kernel Instances Section ############################################## [KERNEL] NumberCyInstances = 0 NumberDcInstances = 0 ############################################## # ADI Section for Scalable IOV ############################################## [SIOV] NumberAdis = 0 ############################################## # User Process Instance Section ############################################## [SSL] NumberCyInstances = 2 NumberDcInstances = 2 NumProcesses = 1 LimitDevAccess = 0 Cy0Name = "SSL0" Cy0IsPolled = 1 # List of core affinities Cy0CoreAffinity = 1 Cy1Name = "SSL1" Cy1IsPolled = 1 # List of core affinities Cy1CoreAffinity = 2 # Data Compression - User instance #0 Dc0Name = "Dc0" Dc0IsPolled = 1 # List of core affinities Dc0CoreAffinity = 3 # Data Compression - User instance #1 Dc1Name = "Dc1" Dc1IsPolled = 1 # List of core affinities Dc1CoreAffinity = 4

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:29:17 GMT

Here is the configuration.

qat_4xxx_0000\:76\:00.0/dev_cfg

[GENERAL]
AutoResetOnError = 0
HeartbeatTimer = 1000
KptEnabled = 1
KptMaxSWKLifetime = 31536000
KptMaxSWKPerFn = 1
KptMaxSWKPerPASID = 1
KptSWKShared = 0
PkeServiceDisabled = 0
PmIdleSupport = 1
ServiceChainingEnabled = 0
ServicesEnabled = asym;dc
StorageEnabled = 0
statsDc = 1
statsDh = 1
statsDrbg = 1
statsDsa = 1
statsEcc = 1
statsGeneral = 1
statsKeyGen = 1
statsLn = 1
statsPrime = 1
statsRsa = 1
statsSym = 1
Device_Max_Banks = 64
Device_PkgId = 0
Device_NodeId = 0
Device_Max_Rings_Per_Bank = 2
Firmware_UofVer = 1.2.70
HW_RevId = 64
Cipher_Capabilities_Mask = 0x3500f2
Hash_Capabilities_Mask = 0xcd73f0
Asym_Capabilities_Mask = 0x1fc
Firmware_MmpVer = 6.0.0
SVMEnabled = 0
ATEnabled = 0
UQEnabled = 0
Device_Capabilities_Mask = 0x3843a22
Device_DcExtendedFeatures = 0x60001b05
[KERNEL]
NumberCyInstances = 0
NumberDcInstances = 0
[SIOV]
NumberAdis = 0
[SSL]
LimitDevAccess = 0
[SSL_INT_0]
Cy0IsPolled = 1
Cy0Name = SSL0
Cy1IsPolled = 1
Cy1Name = SSL1
Dc0IsPolled = 1
Dc0Name = Dc0
Dc1IsPolled = 1
Dc1Name = Dc1
NumberCyInstances = 2
NumberDcInstances = 2
Dc0CoreAffinity = 3
Dc0BankNumber = 2
Dc0RingTx = 0
Dc0RingRx = 1
Dc0NumConcurrentRequests = 512
Dc1CoreAffinity = 4
Dc1BankNumber = 3
Dc1RingTx = 0
Dc1RingRx = 1
Dc1NumConcurrentRequests = 512
Cy0CoreAffinity = 1
Cy0BankNumberAsym = 0
Cy0RingAsymTx = 0
Cy0RingAsymRx = 1
Cy0NumConcurrentAsymRequests = 64
Cy1CoreAffinity = 2
Cy1BankNumberAsym = 1
Cy1RingAsymTx = 0
Cy1RingAsymRx = 1
Cy1NumConcurrentAsymRequests = 64
[Accelerator0]
Bank0InterruptCoalescingEnabled = 1
Bank0InterruptCoalescingTimerNs = 1000
Bank0InterruptCoalescingNumResponses = 0
Bank0CoreAffinity = 255
...
Bank63InterruptCoalescingEnabled = 1
Bank63InterruptCoalescingTimerNs = 1000
Bank63InterruptCoalescingNumResponses = 0
Bank63CoreAffinity = 255

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:30:28 GMT

Sku_details.txt
********** SKU Details **********
qat_dev0 is 4xxx
BDF=0000:76:00.0
SKU=0x-0x

qat_dev1 is 4xxx
BDF=0000:7a:00.0
SKU=0x-0x

qat_dev2 is 4xxx
BDF=0000:f3:00.0
SKU=0x-0x

qat_dev3 is 4xxx
BDF=0000:f7:00.0
SKU=0x-0x

BIOS.txt
# dmidecode 3.5
Getting SMBIOS data from sysfs.
SMBIOS 3.6.0 present.
# SMBIOS implementations newer than version 3.5.0 are not
# fully supported by this version of dmidecode.

Handle 0x0000, DMI type 0, 26 bytes
BIOS Information
Vendor: American Megatrends International, LLC.
Version: 2.2
Release Date: 03/01/2024
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 32 MB
Characteristics:
PCI is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
BIOS ROM is socketed
EDD is supported
Japanese floppy for NEC 9800 1.2 MB is supported (int 13h)
Japanese floppy for Toshiba 1.2 MB is supported (int 13h)
5.25"/360 kB floppy services are supported (int 13h)
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
CGA/mono video services are supported (int 10h)
ACPI is supported
USB legacy is supported
BIOS boot specification is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 5.32

Handle 0x0068, DMI type 13, 22 bytes
BIOS Language Information
Language Description Format: Long
Installable Languages: 1
en|US|iso8859-1
Currently Installed Language: en|US|iso8859-1

environment.txt
SHELL=/bin/bash
SUDO_GID=836842799
SUDO_COMMAND=/usr/bin/su
PWD=/QAT/quickassist/utilities/debug_tool
LOGNAME=root
HOME=/root
LANG=en_US.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:
LESSCLOSE=/usr/bin/lesspipe %s %s
TERM=screen
LESSOPEN=| /usr/bin/lesspipe %s
USER=root
SHLVL=2
ICP_ROOT=/QAT
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SUDO_UID=836842799
MAIL=/var/mail/root
OLDPWD=/QAT
_=/usr/bin/env

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:34:45 GMT

I don't know the reason but some result would be blocked by Community policy. Here are others.

Sku_details.txt
********** SKU Details **********
qat_dev0 is 4xxx
BDF=0000:76:00.0
SKU=0x-0x

qat_dev1 is 4xxx
BDF=0000:7a:00.0
SKU=0x-0x

qat_dev2 is 4xxx
BDF=0000:f3:00.0
SKU=0x-0x

qat_dev3 is 4xxx
BDF=0000:f7:00.0
SKU=0x-0x

BIOS.txt
# dmidecode 3.5
Getting SMBIOS data from sysfs.
SMBIOS 3.6.0 present.
# SMBIOS implementations newer than version 3.5.0 are not
# fully supported by this version of dmidecode.

Handle 0x0068, DMI type 13, 22 bytes
BIOS Language Information
Language Description Format: Long
Installable Languages: 1
en|US|iso8859-1
Currently Installed Language: en|US|iso8859-1

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

dykim1101 — Mon, 23 Jun 2025 02:37:47 GMT

I don't know the reason but some result would be blocked by Community policy.

Sku_details.txt
********** SKU Details **********
qat_dev0 is 4xxx
BDF=0000:76:00.0
SKU=0x-0x

qat_dev1 is 4xxx
BDF=0000:7a:00.0
SKU=0x-0x

qat_dev2 is 4xxx
BDF=0000:f3:00.0
SKU=0x-0x

qat_dev3 is 4xxx
BDF=0000:f7:00.0
SKU=0x-0x

I wonder which results are important, then I'll upload the result as file.

Re:Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

Ronny_G_Intel — Fri, 27 Jun 2025 21:39:31 GMT

Hi dykim1101,

I am currently looking into this issue, and I will get back to you as soon as possible.

Regards,

Ronny G

Re: Intel QAT HW offloading at Intel(R) Xeon(R) Gold 6438N

DiegoV_Intel — Tue, 22 Jul 2025 23:23:49 GMT

Hi,

I may be able to provide some help here. I'd like first to clarify the setup your are trying to run.

Are you using the Intel® QAT in-tree or out-of-tree driver? If you are attempting to use the in-tree driver, it's better not to combine in-tree and out-of-tree setups as there might be conflicts.

Going back to your original description, were you able to test that QAT_Engine was using QAT_HW? I mean, before adding the NGINX layer to the equation, did you test QAT_Engine with QAT_HW? A good guide to confirm that is available at Intel® QuickAssist Technology(QAT) OpenSSL* Engine. At the bottom there are some commands to verify QAT_Engine is enabled and working.

Regards,

Diego V.