Intel® ARC™ Graphics
Get answers to your questions or issues when gaming on the world’s best discrete video cards with the latest news surrounding Intel® ARC™ Graphics
1593 Discussions

A380 won't passthrough to VM

CaptainCoder
Beginner
‎12-16-2022 01:37 AM
1,520 Views

I bought an A380 to play Eve Online with better visuals than my iGPU. However I can't get it to work. Since I run Linux my initial plan was to use gpu passthrough to a Windows 10 VM. After setting it up I get error 43 on the driver in windows. Since I don't know how to get more diagnostic data on the driver in windows, I also tried gpu passthrough to a Linux VM and looking at the dmesg there. I've played around with the various QEMU settings and options but nothing seems to work.

After a day of fiddling with qemu/kvm/vfio I thought sod it, i'll live with the graphics glitches and run Eve directly on Linux through Lutris/Wine. I couldn't even start my desktop on the A380. But since my distro is not Ubuntu I guess we'll leave this setup as a bonus round.

The details:

Hardware: Asus P10S WS (bios up to date), Xeon E3-1275v6, Asrock A380

Host: Linux 6.0.12 (arch default kernel), qemu 7.1.0 (edk-ovmf 202211-3)

Guest: Windows 10 (22H2 build 19045.2364), Graphics driver 31.0.101.3959

 

IOMMU Group 1:
	00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 05)
	01:00.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa1] (rev 01)
	02:01.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa4]
	02:04.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa4]
	03:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05)
	04:00.0 Audio device [0403]: Intel Corporation Device [8086:4f92]

01:00.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa1] (rev 01)
	Kernel driver in use: pcieport
02:01.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa4]
	Subsystem: Intel Corporation Device [8086:4fa4]
02:04.0 PCI bridge [0604]: Intel Corporation Device [8086:4fa4]
	Subsystem: Intel Corporation Device [8086:0000]
03:00.0 VGA compatible controller [0300]: Intel Corporation DG2 [Arc A380] [8086:56a5] (rev 05)
	Subsystem: ASRock Incorporation Device [1849:6004]
	Kernel driver in use: vfio-pci
	Kernel modules: i915
04:00.0 Audio device [0403]: Intel Corporation Device [8086:4f92]
	Subsystem: ASRock Incorporation Device [1849:6004]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

 

 

Vfio driver binding seems to have worked. Lets see if they are actually isolated: $ dmesg | egrep "vfio|DMAR"

 

 

[    0.007703] ACPI: DMAR 0x000000007A1D8408 0000B0 (v01 INTEL  KBL      00000001 INTL 00000001)
[    0.007743] ACPI: Reserving DMAR table memory at [mem 0x7a1d8408-0x7a1d84b7]
[    0.051875] DMAR: IOMMU enabled
[    0.122232] DMAR: Host address width 39
[    0.122233] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.122237] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.122239] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.122241] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.122243] DMAR: RMRR base: 0x00000079100000 end: 0x0000007911ffff
[    0.122244] DMAR: RMRR base: 0x0000007b800000 end: 0x0000007fffffff
[    0.122246] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.122247] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.122248] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.123754] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.291026] pci 0000:03:00.0: [8086:56a5] type 00 class 0x030000
[    0.291053] pci 0000:03:00.0: reg 0x10: [mem 0xdd000000-0xddffffff 64bit]
[    0.291069] pci 0000:03:00.0: reg 0x18: [mem 0xc0000000-0xcfffffff 64bit pref]
[    0.291095] pci 0000:03:00.0: reg 0x30: [mem 0xde000000-0xde1fffff pref]
[    0.291117] pci 0000:03:00.0: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[    0.291174] pci 0000:03:00.0: PME# supported from D0 D3hot
[    0.335515] pci 0000:03:00.0: vgaarb: bridge control possible
[    0.335515] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    0.351814] DMAR: [Firmware Bug]: RMRR entry for device 09:00.0 is broken - applying workaround
[    0.351816] DMAR: No ATSR found
[    0.351816] DMAR: No SATC found
[    0.351817] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.351818] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.351819] DMAR: IOMMU feature nwfs inconsistent
[    0.351819] DMAR: IOMMU feature pasid inconsistent
[    0.351820] DMAR: IOMMU feature eafs inconsistent
[    0.351821] DMAR: IOMMU feature prs inconsistent
[    0.351821] DMAR: IOMMU feature nest inconsistent
[    0.351822] DMAR: IOMMU feature mts inconsistent
[    0.351822] DMAR: IOMMU feature sc_support inconsistent
[    0.351823] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.351824] DMAR: dmar0: Using Queued invalidation
[    0.351826] DMAR: dmar1: Using Queued invalidation
[    0.352255] pci 0000:03:00.0: Adding to iommu group 1
[    0.352450] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    1.074606] vfio_pci: add [8086:4fa1[ffffffff:ffffffff]] class 0x000000/00000000
[    1.074758] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    1.074999] vfio_pci: add [8086:56a5[ffffffff:ffffffff]] class 0x000000/00000000
[    1.075023] vfio_pci: add [8086:4f92[ffffffff:ffffffff]] class 0x000000/00000000
[    1.325821] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[  301.622303] vfio-pci 0000:03:00.0: enabling device (0000 -> 0002)
[  302.055672] vfio-pci 0000:04:00.0: enabling device (0000 -> 0002)

 

 

That looks ok, the last lines are when qemu is started.

Qemu command line: 

 

 

qemu-system-x86_64 --enable-kvm -M q35 -m 12288 -drive file=/dev/sde,format=raw -drive if=pflash,format=raw,readonly=on,file=/usr/share/ovmf/x64/OVMF_CODE.fd -drive if=pflash,format=raw,file=/root/OVMF_VARS.fd -smp sockets=2,cores=2 -cpu host,kvm=off -device vfio-pci,host=03:00.0 -display sdl -vga qxl

 

 

Things I've tried:

  • With/Without passing the audio along, since they might depend on each other. No change.
  • Without/with -M argument, since the q35 machine has an actual pcie port I thought that could make a difference. If I start it as the default i440fx then the hosting X session suddenly has it's sample format reversed as soon as the Windows Vm initializes graphics. Leading to... funky colours. I think the VM is altering some MSR's that it shouldn't. And crashes the host as soon as qemu exits, using q35 solved this.
  • Since the CPU/iGPU was somehow accessible inside the VM I first focused on various combinations of -cpu host,... with other vendor_id's, kvm on/off, max and default. No effect.
  • With a separate pcie root port ( -device pcie-root-port ) and link the card into there. No effect.
  • Using a chipset PCI port on the host instead of the CPU/PEG port. The only difference is that the whole device tree is suddenly broken up over multiple IOMMU groups. Passing them individually did not make a difference. 
  • There seems to be some native pcie bridge/switch embedded on the gpu (pci device id's 8086:4fa1 & 8086:4fa4) I thought this might do something more than simple connectivity. So I tried to pass this whole tree of devices to windows, but I can't seem to get vfio to bind to the root of this device tree. (error 22 EINVAL on vfio probe).
  • Passing through to a Linux VM and looked at the i915 dmesg, to get more information about why error 43 is happening on windows. I get "Timed out waiting for forcewake ack to clear" for render, dboxes and eboxes. That doesn't mean much to me, and google was no help on this subject (especially related to ARC).
  • I thought this might be some IRQ issue so I also tried with kernel_irqchip=on/off/split. No dice.

The for the bonus round I removed the vfio binding and tried to directly run the card in Linux. I unplugged from the iGPU and into the A380. After i915 is initialized i get a console on frame buffer. I can see the dri device nodes (card1/renderD129). If I try to start X and force it to use card1 (instead of card0 from the iGPU) the Xorg log tells me, without any other errors, that there are "No Devices". I have mesa 22.2 and xf86-video-intel up to date.

(And i've been waiting 2 years on a fix for this as well for Gvt-g )

It's getting hard to stay positive here. So, how do I get to a working setup for playing Eve Online?

0 Kudos

Link Copied

0 Replies
Reply

Community support is provided during standard business hours (Monday to Friday 7AM - 5PM PST). Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.