Link Copied
First of all i don't want to open another thread for the same issue, so forgive my long answer. I hope it is still helpful!
The whole USB C state issue appears both in Linux and Windows, so it seems like a hardware problem, not driver or software related.
Your ~ 20 W idle power consumptions sounds really nice.
The best I got was ~ 32 W in Linux, ~ 35 W in Windows but fluctuating between 32 to 38 W. But my GPU always uses a minimum of around 10 W at idle, even when not utilized at all. So that means 22 W in best case for me for all other components.
I don't know how efficient are my PSU and the internal voltage regulators on that low power consumption though. Cannot say where the 22 W are "wasted" exactly. CPU Package shows less than 5 W on idle, got even down to 3 W according HWinfo. NVMe drives can easly get much lower than 1 W power consumption too when idle. I have one HDD always spinning though. This can easily be 1 to 5 W probably. Cannot see DRAM power either. Each PCIE Slot itself can waste 1 W when something is inserted too. The PCH Chipset itself needs power too, probably ~ 1 W ballpark.
My setup is very similar but CPU is older:
Asus B760 ProArt D4 (DDR4)
Intel i9 12900 K
Nvidia GTX 1060
Problematic devices (running at USB 1.1 aka "full speed" according usbtreeview
Logitech Wireless Dongle for keyboard
Roccat Kone XTD mouse
Problematic usb ports:
All back panel ports (USB 2 and 3)
I have two USB 3.1 front panel ports which are fine. I can connect everything there, even mouse and keyboard, and power consumption is unaffected.
On the motherboard i have Intel AX 210 PCIE card where the Bluetooth side is connected via USB directly to the USB 2.0 motherboard pins. Again, no problem with that USB connection regarding power consumption.
Here is the rest of my kind of thread
Intro
C states are really important for laptops, especially with Modern Standby. While many desktop users don't care much about C states, they can lower RAM temperature and interact with ASPM (power saving) of various components like NVMe, Network Cards etc..
Latency issues are very rare because of mechanisms like undemotion/demotion of C states. Core parking affects latency more.
Sometimes there is a drastic difference in total watts from wall adapter when CPU is in Package C6 state or only C2/C3. The Package C states interacts with ASPM over PCI express and directly affect DRAM power consumption too. Even the voltage regulator and power phases to CPU can get more efficient with lower Package C states.
For example my 12900 K uses only 5 Watts when not moving mouse, while browsing or typing in Word only uses less than 10 Watts. The whole computer stays under 50 Watts as long as GPU doesn't go into highest power mode. Chipset temperature is only 40 C. No performance loss measurable. This is despite low utilization of Package C6 state.
Some Bios settings needed to be adjusted, like Native ASPM set to disabled, PCIE Clock Gating set to enabled, and all other ASPM related settings manually at maximum possible value (L1, Enabled). This is easily possible on Asus, but on other vendors some settings may be hidden or missing at all. In ideal case the OS should fully control ASPM but this doesn't always work properly with older hardware like some GPUs.
Issue: USB ports affecting C states
Normally USB devices connected to ports should not affect Package C states when idle. Example: an USB drive inserted but not accessed via explorer / virus scan.
When an mouse (especially 1000 Hz or higher) is connected, moving the cursors causes interrupts which can lower Package C state usage. The effect should be minimal because on modern USB host controllers these are MSI interrupts (whole other topic too big to explain here). Basically this means these MSI interrupts don't "interrupt" the whole computer anymore but are handled smartly via CPU an can be routed to unused cores.
When you dont move the cursor, the mouse doesn't send any data to computer and doesn't cause MSI interrupts. So the Package C state usage should be unaffected.
The problem on chipset B760 (and similar) motherboards:
USB devices like many mice or keyboards operating in slower Full Speed mode (USB 1.x) seems to hinder Package C state usage, even when idle. This at least affects all USB back panel ports as long as they appear as USB hubs in device manager. The front USB ports usually are connected directly to host controller and don't affect C states. In device manager the connected devices appear under Generic USB hubs.
Preparation
For testing the affect of connected USB devices and overall usage of Package C states, you need some software and correct Bios settings.
Software prequisites
- HWinfo64
- usbtreeview (by Uwe Siebert, not the older version from Windows WDK)
Bios settings (mostly Asus)
Without the correct Bios settings you won't even get Package C3 if you are unlucky. To get at least C6 you need ASPM settings on.
Intel: Under ASPM everything needs to be selected, no setting at Auto.
- Turn on PCI Express Native Power Management.
- Native ASPM disabled
(In my case i get Package C6 only when Native ASPM disabled) - Under the various other subsettings choose Enabled and L1 if possible.
L0s is not enough. L0sL1 sometimes causes WHEA errors on some devices. - PCI Express Clock Gating must be on!
Testing
You can watch what happens with Package C-states with HWinfo. You don't need to close every program in Task Manager, just be sure no audio is playing and CPU utilization is very low. Wait for a minute and watch the values, don't move the mouse at all! Every Package C state residency are the key values here to watch
The values like CPU Package power, System Agent power and Rest-of-Chip (DRAM mostly) power are directly affected by Package C states. But other components on motherboard are directly or indirectly affected by Package C states too which don't show up in HWinfo. Ideally you have an wall outlet adapter to measure power consumption. Less than 40 W for whole computer on idle with monitor on is very good. I once got 32 W with high usage of Package C6 state. For a desktop computer with an i9 CPU and 4K 60 Hz display this is probably the lowest consumption i can get while idle. My GPU uses 10 W at idle though, so that means roughly 22 W for the rest.
First of all i disconnected all usb devices at back panel and monitored HWinfo a minute at least. Then i connected the wireless keyboard to front port, confirming that the Package C state usage quickly recovery to the previous value. Finally i took a screenshot.
Nothing connected at any back panel USB ports, only Bluetooth and Aura LED over USB internally
The Package C6 residency is around 65 percent, while C2 and C3 are almost unused. This is a good sign that C states work properly.
My 12900 K doesn't support Package C7 state. The next lower states are Package C8 and C10 are supported too but i don't know how to even get there, maybe you need Modern Standby and physically disconnected Nvidia GPU (?)
This is from the 12th Generation Intel® Core™ Processors Datasheet, Volume 1 of 2:
So it seems i am out of luck, when i don't even get lower than C7 Core state (neccesary for Package C8). But nonetheless the step from Package C3 to C6 is significant enough according to the datasheet for 12th gen Intel. I couldn't find datasheets for C states relating 13th or 14th Intel yet.
Another interesting fact is that Package C2 and C3 cannot be requested by Software, while Package C6 and lower can. So when Package C2 and C3 have high utilization while C6 has none or low utilization, this must be hardware related by motherboard because of the USB hubs itself.
Next we check what happens when a mouse or/and keyboard is connected via USB at any back panel port.
Lower Package C state residency when device connected back panel USB port
When even one device like mouse or keyboard is inserted in any back panel USB port, the C state usage in percent drops immediately. This is when the devices operates in slower full speed mode only. Sadly I don't have an mouse which operates in high speed mode for testing.
Mouse and wireless keyboard dongle connected at any back panel USB ports
I hardly get 30 percent Package C6. Definitely much worse than before. The usage of Package C2 is much higher. The Core C states usage is always deep but don't matter much because to get sub 5 Watts you need definitely at least Package C6.
For comparison:
When nothing is connected at back usb port I get 65 percent Package C6 residency while package C2/C3 almost not used. And USB 2 or 3 flash drive can be connected though without negative effect, as long as the device operated in High Speed mode. So the shallow states C2 and C3 are mostly skipped which is a more efficient way.
Common observation
What is interesting is that this seems to only happen when an USB device is in Full Speed mode (this is like USB 1.1). Most mice and wireless keyboard dongles use USB 1.1 only despite sometimes even supporting High Speed (USB 2.0 standard, 480 MBit/s).
Again, when an USB 3.0 capable USB flash drive is inserted in any USB port, the C state usage doesn't change at all. This is on my Asus B760 ProArt D4 and should affect similar motherboard with the same chipset. Maybe even all Asus motherboards?
This can easily be verified by usbtreeview (by Uwe Sieber). You can see any ports and devices, including the maximum and current transfer speed.
Setup which affects Package C state residency
The back and front panel USB ports are all driven by Intel USB 3.20 xHCI Host Controller. But the most important difference is that the back panel ports are grouped into three separate USB hubs (Genesys Logic USB 2.0, ASMedia USB 2.1 and ASMedia USB 3.0).
USB hubs (back panel ports) marked orange, front panel device (directly connected to Host controller) marked blue
The Logitech device is connected to Generic USB 2.1 hub (back panel USB 2.0 port), this is the wireless keyboard dongle. The Roccat device is my mouse, and while it supports High Speed, it only operates in Full Speed (proably enough for 125 up to 1000 Hz polling rate). I don't know how to force an USB device into maximum possible speed as a workaround.
I don't have any third party driver installed for mouse and keyboard. Both work without issues and with all functionality.
When one device (old Taiwan OEM usb drive, High speed capable) is attached to any of the front ports, it doesn't appear under a USB hub. The same for Aura LED Controller und Bluetooth. These devices don't affect the C states either, probably because they are directly connected to Host Controller.
My theory
The whole issue only seems to happen when a device is in full speed mode, while it is connected to an USB hub in device manager / usbtreeview). This may (or may not) be an issue with wrong handles interrupts (?), very speculative though. Very weird that not moving mouse at all does lower C state usage too, so this probably contradicts an interrupt issue. LatencyMon doesn't show unusual intterupts either.
Validation
- When connecting both mouse and wireless keyboard dongle to both front ports, C state usage is unaffected, like when no USB device is connected at all.
- An USB 3.0 capable flash drive doesn't affect C states either, because on USB 2.0 ports it works in High Speed mode (480 MBit/s), and NOT in Full Speed mode which seems to be the issue.
- I can reach Package C state 6 in both cases, but the C6 usage is higher when no mouse or keyboard is attached to the back.
Further testing (with a second mouse connected at front port)
- Disabling USB devices and/or the corresponding hubs in device manager but keep them physically attached at the ports.
Already tested, not always possible to disable in device manager.
No effect on C states at all. - Eject aka "safely remove" mouse, keyboard device via usbtreeview like a flash drive.
Already tested, yes it has the same effect as physically disconnecting. - Using Linux and powertop to monitor C states.
For some reason my Nvidia GPU prevents C6, probably driver issue in Linux
Additional notes
- Beware that Package C state usage depend on GPU too. Nvidia, Intel or AMD graphics heavily affect Package C states.
Especially Intel Integrated Graphics can even prevent Package C6. - On Nvidia Package C state 6 only works for me when GPU is in a lower power mode. With HDR and 4K even some mouse movement can force GPU into highest power mode which kicks the CPU out of C6, especially on weaker GPUs for HDR (like GTX 1060).
- Linux sometimes has problems with ASPM which can prevent Package C state 6 or lower.
- When connecting both mouse and wireless keyboard dongle to both front ports, C state usage is unaffected, like when no USB device is connected at all.
Are you certain? The results from my thread were obtained when the devices were plugged into ports directly connected to the chipset. Obviously the results were worse with hubs.
I may have found something interesting which could be related to the whole issue.
My ASMmedia USB 2.1 Hub which powers 4 separate USB 2.1 ports at back panel of motherboard is running in High-Speed mode (USB 2.0 standard), which can be verified with USB Device Tree Viewer easily.
USB Hub Connection details
My Roccat Kone mouse reports only Full-Speed (USB 1.1) which is different to the USB hub connection.
Mouse connected to back panel USB hub - Connection details
When you go to the USB Hub again and scroll all the way down, you see that Link power Management is not supported. The Intel Wireless Bluetooth device shows Link power Management enabled.
So as i understand every single USB port restricts package C10 usage? This is similar on my machine but with C6 state.
C6 drops from 70 % to barely 30 %. Doesn't matter if mouse or wirless keyboard dongle, I get the same exact result. When i move mouse slightly it still stays above 50 %.
The yellow marked USB 3.2 connector powers my two unproblematic front USB 3 ports. The blue marked USB 2 connector is connected with Intel Bluetooth, again unproblematic.
B760 motherboard layout
- yellow: internal USB 3 connected with front ports
- blue: internal USB 2 port (blue)
But my two front USB ports don't exhibit this problem at all! The same as my Bluetooth controller, both are directly connected to my motherboard pin sockets. So they should go into the Chipset too. Maybe the back USB ports go into CPU directly? This seems unlikely though.
I don't know how the back ports are routed but at least a row of ports share a common power source.
