Problems building/installing QAT 1.X Linux drivers

[rolypoly]

Ubuntu 22.04 in vm and on e5 v4 metal: builds, installs, sample code runs, but i needed to patch 
quickassist/qat/compat/qat_compat.h

513c513
< #if (KERNEL_VERSION(6, 3, 0) > LINUX_VERSION_CODE)
---
> #if (KERNEL_VERSION(6, 1, 29) > LINUX_VERSION_CODE && (KERNEL_VERSION(6, 1, 0) < LINUX_VERSION_CODE)) || (KERNEL_VERSION(5, 15, 112) > LINUX_VERSION_CODE) 

Debian 12 in vm and on e5 v4 metal: build error, something about no rule to make vmlinux.o needed by modpost

Clear linux on kernel 6.7 / 6.8 in vm: ditto debian 12, patched some stuff (dont know if it works, but the corresponding errors went away)
cpuinfo_x86.phys_proc_id got renamed to cpuinfo_x86.topo.pkg_id
https://github.com/torvalds/linux/commit/eb55307e6716b1a02f7db05e27d60e8ca2289c03#diff-8505bcf7b64fc29e6ba57bbd574d01d5e3bd3f0e8badefdd4a01c654de26fdccR62 

/opt/intel/QAT/quickassist/qat/drivers/crypto/qat/qat_common/adf_ctl_drv.c

		struct cpuinfo_topology *ct; 
		
		node_id = 0;

		/* if there is only one physical processor don't need
		 * to do any further calculations
		 */
		if (ct->pkg_id == 0) {
			node_id = 0;
		} else {
			bus_per_cpu = MAX_PCI_BUS / (ct->pkg_id + 1);

 strlcpy went away, replaced it with strscpy (sd is like sed but written in rust)

sd -F 'strlcpy' 'strscpy' $(grep -R strlcpy | awk -F: '{ print $1}' | sort -u)

something about prototypes idk i gave up and went back to ubuntu shortly afterwards. 

quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv_utils.c

int handle_other_ioctls(uint32_t cmd);

Rocky 9 had a broken symlink in /lib/modules/$(uname -r) for /build, couldn't even ./configure

End goal has been to test out QAT for ZFS, but it hung after a single 16M? compression call, but I was using SRIOV host mode, and i wanted to see if physical mode worked any better, because if I understand correctly, there are 3 modes of operation (sriov host, sriov guest, and physical)

(people that say they got it working in debian 11) 

[Ronny_G_Intel]

Hi rolypoly,

Please see below and help me answer my questions:

• Ubuntu 22.04 in vm and on e5 v4 metal, running Intel® QAT Driver Version 1.x is working okay.
• Debian 12 in vm and on e5 v4 metal, giving you errors about no rule to make vmlinux.o needed by modpost, you patched it and the errors got fixed. If you still need help with this, can you please provide the exact error message?
• Rocky 9, has a broken symlink in /lib/modules/$(uname -r) for /build, couldn't even ./configure.
• Do you mean that you have a link that points to a file or directory that no longer exists and cant run ./configure command? Did you create the lib/modules directory manually? Probably the drivers wont load, you may use modprobe to load them but this is more of Linux configuration issue than QAT.
• Your end goal is to test out QAT for ZFS, but it hangs after a single 16M using SRIOV host mode.
• Can you please provide more details? Platform, Operating System and steps that you are trying to get this configured?

Regards,

Ronny G

[rolypoly]

Repro steps for clear linux in a vm

Linux clearlinuxvm 6.8.1-1418.native #1 SMP Mon Mar 18 07:13:47 PDT 2024 x86_64 GNU/Linux

swupd bundle-add devpkg-libgudev devpkg-libnl kernel-native-dkms go-basic-dev rust-basic wget
mkdir -p /opt/intel/QAT && cd /opt/intel/QAT
wget https://downloadmirror.intel.com/795697/QAT.L.4.24.0-00005.tar.gz
tar -xf QAT.L.4.24.0-00005.tar.gz
./configure && make -j 

Clear linux and debian 12 (both kernel 6+) have the same error below

make[4]: Entering directory '/opt/intel/QAT/quickassist/lookaside/access_layer/src/qat_direct/src/uio'
make[4]: Entering directory '/opt/intel/QAT/quickassist/lookaside/access_layer/src/qat_direct/src/common'
Creating output directory
Creating output directory
make[6]: *** No rule to make target 'vmlinux.o', needed by 'modpost'.  Stop.
make[5]: *** [Makefile:240: __sub-make] Error 2
make[5]: Leaving directory '/usr/lib/modules/6.8.1-1418.native/build'
Copying outputs
mv: cannot stat 'OsalServices.o': No such file or directory
mv: cannot stat 'OsalSemaphore.o': No such file or directory
mv: cannot stat 'OsalSpinLock.o': No such file or directory
mv: cannot stat 'OsalThread.o': No such file or directory
mv: cannot stat 'OsalAtomic.o': No such file or directory
mv: cannot stat 'OsalMemMgmt.o': No such file or directory
mv: cannot stat 'OsalIOMMUMgmt.o': No such file or directory
mv: cannot stat 'OsalUsrKrlProxy.o': No such file or directory
mv: cannot stat 'OsalCryptoInterface.o': No such file or directory
make[4]: Leaving directory '/opt/intel/QAT/quickassist/utilities/osal/src/linux/kernel_space'
make[3]: Leaving directory '/opt/intel/QAT/quickassist/utilities/osal/src'

I'll try to reproduce some of the other errors on the xeon e5 v4 bare metal on tuesday.
Ideally, I would like to be able to get ZFS QAT offloads working in clear linux on bare metal, but it's looking like ubuntu 22.04 lts is more realistic for now.

[Ronny_G_Intel]

Hi rolypoly,

Ubuntu works well but Debian 12 and Clearlinux are giving you errors.

When I search our internal debug table, the same error you have is reported as "QAT compile error due to kernel mismatch/unsupported kernel" and the resolution is to try the latest QAT driver package which you already have.

I would recommend the following:

• Check if these errors are related to iommu_map function
• Check if there are previous error messages, if there are, please share.
• Refer to the Getting Started Guide - Customer Enabling Release - https://www.intel.com/content/www/us/en/content-details/710059/intel-quickassist-technology-software-for-linux-getting-started-guide-customer-enabling-release.html
• Ensure that you have all necessary software packages installed, see section 3.2.1
• Which compilation flag are you using? See table 4
• Please check the following article: https://github.com/spdk/spdk/issues/887 it looks very similar to the issue that you are describing.

Let me know your comments.

Regards,

Ronny G 

[rolypoly]

I got QAT+ZFS working together on Ubuntu 22.04 LTS. 
Some caveats: I needed to disable SRIOV and VT-d in UEFI; I needed to patch qat_compat.h (see above), I also needed to resolve the GPL/CDDL licensing issue. 

• Check if these errors are related to iommu_map function.

I'll pull at that thread this week.

• Ensure that you have all necessary software packages installed, see section 3.2.1

./configure doesn't complain, so it seems like I have all the dependencies?

• Which compilation flag are you using?

For clearlinux, I only used this option:

export KERNEL_SOURCE_ROOT=/lib/modules/$(kernelWithTheBuildDirPresent)

I used these compilation flags for Ubuntu:

./configure --enable-kapi --enable-icp-thread-specific-usdm  --enable-128k-slab --enable-icp-log-syslog 

• Check if there are previous error messages, if there are, please share.

After following this guide, and using:

 ./configure KERNEL_SOURCE_ROOT=/root/clearlinux/packages/linux/linux-6.8.1

I get these errors on clearlinux LTS (6.6.x?) on Xeon E5 v4 metal :

Building qat_direct for user space
make[3]: Entering directory '/opt/intel/QAT/quickassist/lookaside/access_layer/src/qat_direct/src'
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_utils.o
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv.o
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv_utils.o
In file included from ./include/linux/errno.h:5,
                 from ./include/linux/mm.h:5,
                 from /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_utils.c:71:
./include/uapi/linux/errno.h:1:10: fatal error: asm/errno.h: No such file or directory
    1 | #include <asm/errno.h>
      |          ^~~~~~~~~~~~~
compilation terminated.
make[7]: *** [scripts/Makefile.build:243: /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_utils.o] Error 1
make[7]: *** Waiting for unfinished jobs....
In file included from ./include/linux/array_size.h:5,
                 from ./include/linux/string.h:5,
                 from ./arch/x86/include/asm/io.h:38,
                 from /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv.c:72:
./include/linux/compiler.h:251:10: fatal error: asm/rwonce.h: No such file or directory
  251 | #include <asm/rwonce.h>
      |          ^~~~~~~~~~~~~~
compilation terminated.
make[7]: *** [scripts/Makefile.build:243: /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv.o] Error 1
In file included from ./include/linux/array_size.h:5,
                 from ./include/linux/kernel.h:16,
                 from /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv_utils.c:72:
./include/linux/compiler.h:251:10: fatal error: asm/rwonce.h: No such file or directory
  251 | #include <asm/rwonce.h>
      |          ^~~~~~~~~~~~~~
compilation terminated.

and these errors on clearlinux rolling (6.8.1) in a vm:

Building qat_direct for user space
make[3]: Entering directory '/opt/intel/QAT/quickassist/lookaside/access_layer/src/qat_direct/src'
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_utils.o
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv.o
  CC [M]  /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv_utils.o
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
    5 | #include <generated/autoconf.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[7]: *** [scripts/Makefile.build:243: /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_utils.o] Error 1
make[7]: *** Waiting for unfinished jobs....
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
    5 | #include <generated/autoconf.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[7]: *** [scripts/Makefile.build:243: /opt/intel/QAT/quickassist/utilities/libusdm_drv/linux/kernel_space/qae_mem_drv.o] Error 1
In file included from <command-line>:
././include/linux/kconfig.h:5:10: fatal error: generated/autoconf.h: No such file or directory
    5 | #include <generated/autoconf.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I'm going to try some of these fixes tomorrow.

• Ensure that you have all necessary software packages installed, see section 3.2.1

That documentation is actually missing some dependencies On ubuntu 22.04 LTS, ./configure complained and I fixed it with

apt install -y nasm libnl-genl-3-dev linux-headers-generic build-essential

• Please check the following article: https://github.com/spdk/spdk/issues/887 it looks very similar to the issue that you are describing.

Following the spdk issue, it seems like they were having the same issue with SRIOV and VT-d. I will look into reproducing the hang/crash on ubuntu 22.04 LTS, and hopefully I will get logs of some sort too, but I would like to get QAT+ZFS on clearlinux working first.

Does the internal debug table mention which kernels are known working? I tried building it on centos7 with the latest driver, but I recall running into problems with it earlier than I did with ubuntu 22.04 LTS.

[Ronny_G_Intel]

Hi rolypoly,

The error message "fatal error: generated/autoconf.h: No such file or directory" typically occurs during the compilation of a Linux kernel or a kernel module, and it indicates that the build system is unable to find the autoconf.h header file. This file is generated during the configuration step of the kernel build process and contains macro definitions that are used to enable or disable features in the kernel.

Here are some steps to resolve this issue:

Ensure Kernel Configuration: Before compiling the kernel or a kernel module, you need to configure the kernel source tree. This is usually done with one of the following commands, which should be run from the root of the kernel source directory:

This step will generate the .config file and the include/generated/autoconf.h file.

Run Make Prepare: After configuring the kernel, you should run the following command to prepare the kernel build environment, which will generate necessary header files including autoconf.h:

Check the Include Path: If you're compiling a kernel module outside of the kernel source tree, ensure that the module's Makefile correctly points to the kernel headers, including the directory where autoconf.h is located. The path to the kernel headers is often specified with the -I option in the Makefile.

Install Kernel Headers: If you're building a module for a running kernel, make sure you have the appropriate kernel headers installed for your running kernel version. On Debian-based systems, you can install them with:

I dont have any specific reference for Clear Linux but as an example for Red Hat-based systems, use:

Use the Correct Kernel Source: If you're building a module, ensure that you're using the correct kernel source or headers that match the version of the kernel you're running or targeting. Mismatches can lead to the absence of the autoconf.h file.

Check for File Existence: Verify that autoconf.h actually exists in the expected directory within the kernel source tree (include/generated/ or include/linux/ depending on the kernel version). If it's missing, there might have been an issue during the configuration or preparation steps.

Clean the Build Environment: If you've previously attempted to build the kernel or module and encountered errors, it might be helpful to clean the build environment before trying again:

Note that make mrproper will remove the .config file and other generated files, so you should back up your .config file if you want to preserve your configuration.

Re-run Configuration and Build: After cleaning, re-run the kernel configuration and preparation steps, then attempt to build the kernel or module again.

By following these steps, you should be able to resolve the "fatal error: generated/autoconf.h: No such file or directory" issue and successfully compile the kernel or kernel module.

#make clean2make mrproper3

#sudo yum install kernel-devel-$(uname -r)2

#sudo apt-get install linux-headers-$(uname -r)2

#make prepare2

#make menuconfig  # Text-based user interface2make xconfig   # Graphical user interface (requires Qt)3make gconfig   # Graphical user interface (requires GTK+)4make oldconfig  # Use an old .config file as a base5

Regards,

Ronny G

[Ronny_G_Intel]

Hi rolypoly,

I am just following up on this request, please let me know if you have any update regarding this case.

Thanks,

Ronny G

[Ronny_G_Intel]

Hi rolypoly,

I am closing this ticket due to no response but please feel free to reopen if you still need assistance.

Regards,

Ronny G