AX210 system crash after 'iwlwifi Queue stuck' error

[chrisx]

Hi Intel Community,

I'm experiencing every other day system crashes on my system with an Asus X670E-E motherboard and an Intel AX210 Wi-Fi/BT adapter running NixOS. The crashes are consistently associated with `iwlwifi` error messages and a subsequent frozen system that often auto-reboots after several minutes.

Key error messages include `iwlwifi 0000:08:00.0: Queue 4 is stuck 20395 20402` and `rcu: INFO: rcu_preempt detected stalls on CPUs/tasks`.

```

$ hostnamectl
Operating System: NixOS 24.05 (Uakari)             
         Kernel: Linux 6.6.2
   Architecture: x86-64
Hardware Vendor: ASUS
 Hardware Model: ROG STRIX X670E-E GAMING WIFI
Firmware Version: 1602
  Firmware Date: Tue 2023-08-15
   Firmware Age: 3month 3w
```

For an in-depth description of the issue including system logs, troubleshooting steps, kernel version, and driver details, please see my bug report on the kernel bug tracker: https://bugzilla.kernel.org/show_bug.cgi?id=217990.

This issue has persisted since I first purchased the motherboard, so there isn't a previous 'working' version to revert to. I haven't been able to identify any specific circumstances that trigger the bug, making it seemingly random and not directly reproducible.

My system, including the BIOS, kernel, and firmware (integrated into the kernel), is all updated to the latest releases.

I would greatly appreciate any insights or suggestions from this community that could help resolve this problem.

Thank you for your time and assistance!

Chris

[IsaacQ_Intel]

Hello @chrisx

Thank you for posting on the Intel️® communities. We would be more than glad to help you with your issue.

In order to have a better understanding of your issue, please provide me with the following:

1.      Is this the original adapter that came pre-installed in your system, or have you changed/installed a new adapter recently?

2.     Please download and scan your system using our tool, Intel® System Support Utility so we can help check your system info first. Once finished downloading, open the tool, click on scan 'Everything', click next, then save the system information to a .txt file. Please provide the .txt file.

This document will be necessary to check your entire system, in order to find possible errors or incompatibilities.

Best regards,

Isaac Q. 

Intel Customer Support Technician

[chrisx]

Hi Isaac,

Thank you for your willingness to assist with this issue.

1. The Intel AX210 adapter came preinstalled with the motherboard. No hardware changes have been made since purchase. The system is not overclocked, as the goal is to ensure maximum stability with settings as close to default as possible.

2. I'm more than happy to provide all the necessary information. However, the diagnostic tool you referred to is designed for Windows systems, and I'm using Linux (NixOS), which is not compatible with that software. Moreover, as part of open-source best practices and for security reasons, running proprietary binaries is generally avoided.

I've compiled extensive technical details about my system in the kernel.org bug report #217990 (linked in my initial post). Here's some specific information about the adapter from my Linux environment, which could be equivalent to what your Windows tool would gather:

```
$ lspci -vvv

08:00.0 Network controller: Intel Corporation Wi-Fi 6 AX210/AX211/AX411 160MHz (rev 1a)
       Subsystem: Intel Corporation Wi-Fi 6 AX210 160MHz
       Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
       Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
       Latency: 0, Cache Line Size: 64 bytes
       Interrupt: pin A routed to IRQ 24
       IOMMU group: 14
       Region 0: Memory at fc700000 (64-bit, non-prefetchable) [size=16K]
       Capabilities: <access denied>
       Kernel driver in use: iwlwifi
       Kernel modules: iwlwif

```

If there is any additional system information you require that typically comes from the Intel® System Support Utility, please let me know, and I'll do my best to provide equivalent information using Linux-native tools.

Looking forward to your guidance on this issue.

Best regards,
Chris

[chrisx]

Hi Isaac,

Upon further investigation, I found a Linux-compatible version of the Intel® System Support Utility, and that it was a `bash` script, hence not binaries. I have downloaded and executed the tool on my system and generated the relevant system report. Please find the generated `nixos.txt` file attached with this message.

However, I noticed that the report might not include comprehensive details about the Wi-Fi adapter, specifically indicating `Interface wlp8s0: "Not Available"`. This likely stems from the fact that my network is configured with `systemd-networkd`, which may not be entirely compatible with the tools utilized by the script.

Despite this, I have attached the `nixos.txt` report. It may still contain useful insights into my system configuration that could aid in troubleshooting the issue.

I hope this report provides the information necessary to help us diagnose and resolve the issue with the Intel AX210 Wi-Fi adapter. Should you require any additional information or further details, please let me know and I will be glad to assist.

Thank you once again for your assistance.

Best regards,
Chris

[IsaacQ_Intel]

Hello @chrisx

Thank you for your reply.

I will proceed to check the issue internally and post back soon with more details.

Best regards,

Isaac Q. 

Intel Customer Support Technician

[IsaacQ_Intel]

Hello chrisx

Thank you for your response.

We suggest attempting the solution with the latest stable kernel, version 6.6.5. Please inform us of any improvements or issues.

Best regards,

Isaac Q.

Intel Customer Support Technician

[chrisx]

Hello Isaac,

Thank you for your follow-up and suggestions.

I have updated my system to kernel version 6.6.5 as you recommended. So far, I have not experienced any crashes within the first 24 hours post-update.

Given the past pattern of intermittent crashes, gauging the effectiveness of the kernel update will require further observation. If a crash occurs, I will report it to you promptly. Conversely, if system stability is maintained, I suggest I wait up to two months to confirm the resolution.

Thank you once again for your assistance.

Best regards,
Chris

[IsaacQ_Intel · Accepted solution]

Hello @chrisx

Thank you for your response.

We're pleased to know that the reported issue appear to be resolved following the Kernel 6.6.5 update. We anticipate that this troubleshooting measure will provide a lasting solution.

Regarding your suggestion to keep the thread open for two months, we regret to inform you that, in adherence to community guidelines, we are unable to maintain the thread open for that extended duration. However, we can extend the follow-up period for an additional 7 days. If the issue does not resurface during this time frame, we will consider it resolved. In the event of a recurrence, we encourage you to initiate a new thread for further assistance.

Best regards,

Isaac Q.

Intel Customer Support Technician

[chrisx]

Hello Isaac,

Thank you for the follow-up.

It seems I still have some degree of error. A few minutes back I had the following logged in `systemctl`:

```

iwlwifi 0000:08:00.0: Queue 4 is stuck 27958 27984
iwlwifi 0000:08:00.0: Microcode SW error detected. Restarting 0x0.
iwlwifi 0000:08:00.0: Start IWL Error Log Dump:
iwlwifi 0000:08:00.0: Transport status: 0x0000004A, valid: 6
iwlwifi 0000:08:00.0: Loaded firmware version: 83.e8f84e98.0 ty-a0-gf-a0-83.ucode
iwlwifi 0000:08:00.0: 0x00000084 | NMI_INTERRUPT_UNKNOWN
iwlwifi 0000:08:00.0: 0x00808210 | trm_hw_status0
iwlwifi 0000:08:00.0: 0x00000000 | trm_hw_status1

```

So there still is the `Queue is stuck` part, but no longer (for now), `rcu: INFO: rcu_preempt detected stalls on CPUs/tasks`, and instead `Microcode SW error detected. Restarting 0x0.`, seemingly indicating the system is taking heed of the problem and acting accordingly through a restart of the wifi adapter.

I have pasted a longer version of the log on the bugzilla.kernel.org link given in the first post.

I agree to close this thread. Should the issue resurface or require further attention, I will start a new thread as suggested.

Once again, thanks for the assistance in this matter.

Best regards,
Chris

[IsaacQ_Intel]

Hello @chrisx,

Thank you for getting back to us.

Since you've agreed to close this thread, we'll go ahead and proceed with the closure. Should you require further information, please feel free to submit a new question, as this thread will no longer be actively monitored.

I trust that my recommendations were helpful.

Best regards,

Isaac Q.

Intel Customer Support Technician