Hi
I'm having a problem: the GPU, even if the monitor isn't connected, unloads the model after a while. Even though it shouldn't. It looks like the GPU is being disabled (powered off) when there is no API activity (no model being generated).
All power-saving options are disabled in the BIOS (ASPM, Disabling unused PCIe portsor, any other). PCIe power saving options are also disabled in Windows.
The main problem I had after the break was trying to access the model again via the API — it takes a very long time to load... — and it happens in gigabyte increments over a long period of time. Sometimes it loads instantly, but I don't know what determines this.
If I load a model from scratch, it loads very quickly from RAM.
All I've come up with so far is a stupid script that polls the API every 30 seconds to keep things active.
$ServerUrl="http://localhost:8080"; $IntervalSec=30
while($true){
$ts=Get-Date -f "HH:mm:ss"
$body='{"prompt":"ping","n_predict":1,"temperature":0.1}'
try{
Invoke-RestMethod -Uri "$ServerUrl/completion" -Method Post -Body $body -ContentType "application/json" -TimeoutSec 10 | Out-Null
Write-Host "[$ts] OK" -ForegroundColor Green
}catch{
Write-Host "[$ts] ERROR" -ForegroundColor Red
}
Start-Sleep $IntervalSec
}
Windows 11
Driver 8626
llama.cpp
A770 (x16 gen3)
C612 (Z10PE-D8 WS), Xeon 2699v3 x2
I'm really desperate, I don't know how to fix this annoying problem. I have 4x A770s.
Link Copied
Important note.
After a period of time, GPUs that aren't connected to displays or aren't under load become unmanageable in Intel Graphics Software. Their REBAR availability also changes.
And until I launch hwiNFO® 64 and restart the program (Intel Graphics Software), it is impossible to configure the GPUs.
Right now, the model is loading back into the GPU memory very quickly, despite being unloaded. 🤷
Unfortunately, I can't figure out why this is happening. I haven't restarted the computer, installed, or uninstalled anything new.
ps 100% HDD utilization - this is the defragmentation process running in the background.
Hello, @Nikhil_Intel
I reinstalled the driver with the latest 8724 via DDU, but the problem still remains
Hello, Nikhil
I reinstalled the drivers and reset the BIOS. I also have a problem with the GPUs not going into power-saving mode—they simply "turn off" instead of consuming 1-8W when idle. I contacted Asrock support, but so far, no luck. I think some update failed to flash the GPU firmware correctly, and it's now constantly affecting their performance. I saw that during the driver installation, it indicated that the firmware was being installed, but I'm not sure that helps, even though there were no error messages during the installation.
A long time ago, I managed to enable low power mode in Windows, but that only happened once and never worked again. Currently, low power mode only works on Ubuntu with the out-of-tree driver, but then there's no display and the computer turns into a regular server without a monitor.
I don't know if it's important or not.
If possible, please run the same workload on 2 GPUs, and keep the others disabled. Let us know if the issue still occurs in this reduced setup.
I've tried with 1 or 2 or 3, and 4 GPUs. The problem still persists. I've tried disabling the GPU in Device Manager, and I've tried disabling it in the BIOS. The only thing I haven't tried in a while is physically removing the GPU from the PCIe ports—but don't even suggest it—it won't change anything. I've also tried resetting the CMOS (disconnecting the battery and disconnecting the power for an extended period).
I also opened an issue:
https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1434
And here is a report:
The Arc GPUs require specific PCIe power management and Resizable BAR handshakes that your older chipset does not support. When the GPUs sit idle, they attempt to enter low-power sleep states that the legacy hardware cannot properly manage, causing the devices to disconnect or become unmanageable. Because this is a hardware-level incompatibility, it cannot be resolved via driver or BIOS updates.
Hello, Nikhil
My motherboard supports ReBAR. Here's a screenshot. And Intel Graphics Software confirms this.
Thanks for the workaround!
I have another option for anyone who encounters a similar problem. You need to run the console in the background with this command:
for /l %i in () do @(cls & xpu-smi discovery & timeout /t 20 >nul)
The model is no longer unloaded from memory!
