Issue with Passthrough and Configuration of IAA Devices in QEMU Virtual Machine

[zzz10]

In my host machine, I can successfully use Intel In-Memory Analytics Accelerator (IAA) in ​​dedicated mode​​ (shared mode is unavailable). However, when attempting to use IAA in a QEMU virtual machine (VM), I encountered the following issues:

1. ​​Passthrough Setup:​​

   • I passed through both IAA and DSA hardware devices to the VM by unbinding them from the idxd driver and rebinding them to the vfio-pci driver.
   • The VM successfully recognizes these devices via lspci, but ​​only one IAA device can be initialized in dedicated mode​​, while the other fails to start. On the host, both IAA devices work properly.

2. ​​Test Failure in VM:​​

   • When running the iaa-test utility (provided by the idxd driver) in the VM, the test fails.
   • The dmesg log shows the error: ​​"can not turn on SVA feature"​​.

​​Potential Root Cause:​​

• I suspect the issue might be due to missing dependencies (e.g., additional devices or subsystems required by IAA that were not passed through to the VM).

​​Request:​​

• Please assist in diagnosing why the second IAA device cannot initialize and why the SVA feature fails to enable in the VM.
• Are there specific hardware/software dependencies or configuration steps missing for IAA passthrough in virtualized environments?

​​Environment:​​

• Host: Dedicated mode for IAA, Ubuntu OS.
• VM: QEMU/KVM with device passthrough via vfio-pci.

Thank you for your support!

[TheExpertGuy]

...hey mate... you correctly unbound the IAA and DSA devices from the idxd driver and rebound them to vfio-pci for passthrough, which is the appropriate first step. It is promising that the VM detects both IAA devices via lspci, confirming successful enumeration at the PCI level.

This behavior might be related to resource mapping constraints within the VM, particularly around MMIO aperture or interrupt remapping limitations. Some accelerators, including IAA, may require access to system-wide resources or BIOS-level configurations not exposed within the virtualized environment. Confirm that your host CPU and platform support ENQCMD and PASID, and that Intel VT-d with scalable mode and SVA support is enabled in the BIOS.

You are running a recent QEMU version (6.1 or higher recommended).

1. Kernel is compiled with:

   • CONFIG_INTEL_IOMMU_SVM=y

   • CONFIG_IOMMU_SUPPORT=y

2. Launch QEMU with SVA-related flags and IOMMU emulation (intel-iommu=on,device-iotlb=on).

IAA devices may rely on additional system or support devices (like ACPI entries, system DMA controller, etc.). Consider testing full idxd subsystem passthrough if not already done.

• If your hardware supports SR-IOV, verify if virtual functions are involved or can be leveraged.

• Testing in Shared Mode: You said that shared mode is not available on the host. Confirm whether IAA shared mode support is enabled in the firmware, kernel driver, and BIOS. It might provide a workaround if passthrough fails in dedicated mode

...hope it helps?