dot11ax clients connecting to low data rates even with good signal

[JPavonM]

Hi Intel engineers,

I'm seeing from the last months lots of clients connecting to a Cisco AP (C9130 running 17.6.3 code) and they are using the lower rate on 5GHz band, even when they have good signal strength and quality.

I have rates lower than 24 Mbps disabled on the AP profile, and wonder if the clients obey the rates been advertised on the AP beacons and AP probe responses, or they select whatever rate they choose to use as that parameters from AP are only for the AP and not to avoid the clients from using them.

The problem is that this is creating a performance issue, not only on the affected devices, but also on the other devices using the same AP radio and others using the same channel.

This is the ouput from the WLC and from Win10:

gbrdcwc03#show wireless client mac aaaa.bbbb.cccc detail | inc Client (MAC Add|IPv4|Username)|AP Name|Signal|Channel :|Current Rate|Protocol Capability|Time source 
Time source is NTP, 15:39:18.652 UTC Tue Oct 4 2022
Client MAC Address : aaaa.bbbb.cccc
Client IPv4 Address : 1.2.3.4
Client Username : host2
AP Name: AP1
Channel : 64
Current Rate : 6.0
Radio Signal Strength Indicator : -49 dBm
Signal to Noise Ratio : 51 dB
Max Client Protocol Capability: Wi-Fi6 (802.11ax)
gbrdcwc03#show wireless client mac 0000.1111.2222 detail | inc Client (MAC Add|IPv4|Username)|AP Name|Signal|Channel :|Current Rate|Protocol Capability|Time source
Time source is NTP, 15:39:30.210 UTC Tue Oct 4 2022
Client MAC Address : 0000.1111.2222
Client IPv4 Address : 5.6.7.8
Client Username : host1
AP Name: AP1
Channel : 64
Current Rate : 6.0
Radio Signal Strength Indicator : -46 dBm
Signal to Noise Ratio : 54 dB
Max Client Protocol Capability: Wi-Fi6 (802.11ax)
gbrdcwc03#
!
!
!
C:\Users\user\Downloads\PSTools>psexec \\host1 netsh wlan show driver
Interface name: Wi-Fi

Driver : Intel(R) Wi-Fi 6 AX201 160MHz
Vendor : Intel Corporation
Provider : Intel
Date : 24/05/2022
Version : 22.150.0.3
INF file : oem293.inf
Type : Native Wi-Fi Driver
Radio types supported : 802.11b 802.11g 802.11n 802.11a 802.11ac 802.11ax
FIPS 140-2 mode supported : Yes
802.11w Management Frame Protection supported : Yes
Hosted network supported : No
Authentication and cipher supported in infrastructure mode:
Open None
Open WEP-40bit
Open WEP-104bit
Open WEP
WPA-Enterprise TKIP
WPA-Enterprise CCMP
WPA-Personal TKIP
WPA-Personal CCMP
WPA2-Enterprise TKIP
WPA2-Enterprise CCMP
WPA2-Personal TKIP
WPA2-Personal CCMP
Open Vendor defined
WPA3-Personal CCMP
Vendor defined Vendor defined
WPA3-Enterprise GCMP-256
OWE CCMP
IHV service present : Yes
IHV adapter OUI : [00 00 00], type: [00]
IHV extensibility DLL path: C:\WINDOWS\system32\IntelIHVRouter10.dll
IHV UI extensibility ClSID: {00000000-0000-0000-0000-000000000000}
IHV diagnostics CLSID : {00000000-0000-0000-0000-000000000000}
Wireless Display Supported: Yes (Graphics Driver: Yes, Wi-Fi Driver: Yes)

C:\Users\user\Downloads\PSTools>psexec \\host2 netsh wlan show driver
Interface name: Wi-Fi

Driver : Intel(R) Wi-Fi 6 AX200 160MHz
Vendor : Intel Corporation
Provider : Intel
Date : 4/26/2022
Version : 22.140.0.3
INF file : oem246.inf
Type : Native Wi-Fi Driver
Radio types supported : 802.11b 802.11g 802.11n 802.11a 802.11ac 802.11ax
FIPS 140-2 mode supported : Yes
802.11w Management Frame Protection supported : Yes
Hosted network supported : No
Authentication and cipher supported in infrastructure mode:
Open None
Open WEP-40bit
Open WEP-104bit
Open WEP
WPA-Enterprise TKIP
WPA-Enterprise CCMP
WPA-Personal TKIP
WPA-Personal CCMP
WPA2-Enterprise TKIP
WPA2-Enterprise CCMP
WPA2-Personal TKIP
WPA2-Personal CCMP
Open Vendor defined
WPA3-Personal CCMP
Vendor defined Vendor defined
WPA3-Enterprise GCMP-256
OWE CCMP
IHV service present : Yes
IHV adapter OUI : [00 00 00], type: [00]
IHV extensibility DLL path: C:\WINDOWS\system32\IntelIHVRouter10.dll
IHV UI extensibility ClSID: {00000000-0000-0000-0000-000000000000}
IHV diagnostics CLSID : {00000000-0000-0000-0000-000000000000}
Wireless Display Supported: Yes (Graphics Driver: Yes, Wi-Fi Driver: Yes) 

 Regards 

Jesus

[Alberto_R_Intel]

JPavonM, Thank you for posting in the Intel® Communities Support.

In order for us to provide the most accurate assistance on this scenario, we just wanted to confirm a few details about your system:

Please provide the model of at least one of the laptops/motherboards showing the problem.

What is the model of the Router?

Were you noticing this problem with those devices before?

If not, when did the issue start?

Did you make any recent hardware/software changes that might cause this issue?

The wireless cards, were purchased separately or did they come installed on the computers?

Does the problem happen at home or in the work environment?

Please attach the SSU report, at least of one of those machines, so we can verify further details about the components in your platform, check all the options in the report including the one that says "3rd party software logs":

https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91600

Any questions, please let me know.

[JPavonM]

Hi @Alberto_R_Intel ,

As initially described, this is an enterprise setup with Cisco access points running on IOS-XE 17.6.3 WLC. I'm seeing the issue is impacting different Cisco AP models so not attached to any special, but they are all from modern models (Aironet 3800/4800 series and Catalyst 9100 series). I'm also seeing this issue not only on 5GHz band but also some of them on 2.4GHz band (less used as corporate acess is only been broadcasted on 5GHz).

Regarding the laptops with issues, these are different models from Lenovo with embedded Intel chipsets (AC8265, AC9260, AC9560, AX210) but the most representatives are AX200 and AX201.

I have not noticed the issue before due to COVID19 restrictions, so I don't know if it was there before, but now that all users are back into the office, they are massively using WLAN specially for Zoom/Teams conferencing, but the performance issue from using such lower data rates is there for any application, and some users started complaining a month ago (no changes made on WLAN config or drivers been used).

The drivers been used for All AX adapters are from 22.70 to most recent. the drivers for AC adapters is not controlled at this time so there could be drivers from 2 years to the most recent.

My concern is that if this is normal behaviour from the Intel adapters or not when using multirate to select a lower-than-announced-supported-rate on beacon.

I'm sending you the SSU report and other data from access points on a DM.

Regards

[Alberto_R_Intel]

Hi JPavonM, Thank you very much for providing that information and the SSU report.

It is important to mention that, like in this case, we always recommend to install the graphics driver provided by the manufacturer of the computer, since that driver was heavily customized by them to work with your specific platform. According to the information shown in the SSU document, the wireless driver version currently installed in that specific computer is: 22.140.0.3, which is provided by Lenovo and should be the proper driver for that device:

https://pcsupport.lenovo.com/us/en/products/laptops-and-netbooks/thinkpad-t-series-laptops/thinkpad-t490s-type-20nx-20ny/downloads/driver-list/component?name=Networking%3A%20Wireless%20LAN

You mentioned that there are no changes made to the WLAN config or drivers being used. Before we can go further with the research on this matter, there are some things that we wanted to confirm if it is possible for you to try:

Since the problem remains with the driver version provided by Lenovo, even though the Intel® wireless drivers are generic, for testing purposes, could you please try a clean installation of Intel® wireless driver version 22.160.0.4 on that machine following the instructions in the links below:

https://www.intel.com/content/www/us/en/download/19351/windows-10-and-windows-11-wi-fi-drivers-for-intel-wireless-adapters.html?wapkw=ax201

https://www.intel.com/content/www/us/en/support/articles/000022173/wireless.html

In reference to the Intel® wireless cards settings, in the link below you will see the "Advanced Intel® Wireless Adapter Settings" that contain some suggestions to try on the wireless card to improve the performance of the system, such as changing the "Channel width" from "Auto" to "20 MHZ" and then test the connection:

https://www.intel.com/content/www/us/en/support/articles/000005585/wireless/legacy-intel-wireless-products.html

Also, we recommend to get in contact, for this specific equipment, with Lenovo Support, in order to verify that the latest BIOS version is installed or to gather the instructions on how to update it:

https://support.lenovo.com/us/en/

Once you get the chance, please let us know if you are able to try those troubleshooting steps and the outcome of doing that so, if necessary, we can continue with further technical assistance on this scenario.

Regards,

Albert R.

Intel Customer Support Technician

[JPavonM]

@Alberto_R_Intel how can benefit to improve data rate in use to set channel width to legacy 20MHz and not using what it's been configured on the WLAN infrastructure? Why the wNIC is associating to 6.0 Mbps and not using a higher one?

I understand that disabling Auto negotiation could reduce retries if sending data over the bonded channel is not been ACKed, but from Wi-Fi 4 standard there is the possibility to bond channels so to increase data link and throughput. Is there any known issue from Intel side about the use of Auto config? Or is it only a best practice to avoid investigating if channel bonding is the problem?

I would suggest the Windows infrastructure team to upgrade BIOS in some of the impacted devices to check how they behave.

Regards.

[Alberto_R_Intel]

JPavonM, Thank you very much for sharing those details and comments.

It is important to mention that the suggestions provided previously are standard troubleshooting steps that are used in order to try to isolate the issue as much as possible and confirm how the wireless card behaves after doing those changes.

There are no know issues reported, based on that, we will do further research on this matter. As soon as I get any updates, I will post all the details on this thread.

Regards,

Albert R.

Intel Customer Support Technician

[Alberto_R_Intel]

Hello JPavonM, I just received an update on this matter.

In order for us to be able to continue with our research, we just wanted to check:

How many systems are affected?

Could you please confirm if the issues occur on different APs non-cisco?

Did you open a ticket with cisco support? If yes, what is the ticket number?

Please try a clean installation of Intel® wireless driver version 22.170.0 and once you get the chance, let us know the results:

https://www.intel.com/content/www/us/en/download/19351/windows-10-and-windows-11-wi-fi-drivers-for-intel-wireless-adapters.html

https://www.intel.com/content/www/us/en/support/articles/000022173/wireless.html

If the problem remains, please provide the details requested in the files attached to this post:

Kernel memory dump

Sys Event and WLAN Autoconfig

Wireless ETW Log

Regards,

Albert R.

Intel Customer Support Technician

[JPavonM]

@Alberto_R_Intel here are the answers you are requesting:

How many systems are affected? IT is not possible to know as we are not monitoring all devices individually for connectivity impairments.

Could you please confirm if the issues occur on different APs non-cisco? There is no way to monitor this in a non-Cisco deployment at the company, nor this is possible to check when users are connected in another remote places.

Did you open a ticket with cisco support? If yes, what is the ticket number? Yes I did, ticket number is SR694158296 and this has been escalated to BU for further investigation.

But my question to Intel is, if it's the client device the one which selects the Tx data rate to the AP, and the client device is just 4-5 meters from the AP with really high signal and SNR values, why client device is selecting to transmit at such really low data rate? Is the Tx rate selected from the supported rates been announced in the beacon/probe response from the AP?, or is the client device selecting whatever supported data rate in the standard even if they are disabled on the WLAN infrastructure?

Regards,

Jesus

[Alberto_R_Intel]

Hi JPavonM, Thank you very much for clarifying those aspects.

Even though the information from the files requested previously is needed, I will check on your inquiries and provide the response as soon as it becomes available.

Regards,

Albert R.

Intel Customer Support Technician

[Alberto_R_Intel]

Hello JPavonM, I just received an update on this matter.

While we are still working on this matter, we just wanted to confirm:

Did you upgrade the Cisco firmware on APs WCL respectively?

When did this issue start? Any recent changes in your network environment?

What encryption protocol are you using and which EAP authentication type is being used?

Is it possible to test one of the affected systems outside your network to check if the issue happens or not?

Are these systems running Win 10 or 11?

Have you tried using different channel numbers to avoid overlapping?

Any difference between onsite users vs VPN users (if used)?

Regards,

Albert R.

Intel Customer Support Technician

[JPavonM]

Regarding your questions:

Did you upgrade the Cisco firmware on APs WCL respectively? Yes I did. Because this is a production environment I cannot upgrade/downgrade to a different code to check how is it like there.

When did this issue start? I don't know the exact date it all started, I noticed it because clients started to complaint about poor performance. Any recent changes in your network environment? No changes in the last 12 months.

What encryption protocol are you using and which EAP authentication type is being used? I'm using EAP PEAP and EAP-TLS with AES

Is it possible to test one of the affected systems outside your network to check if the issue happens or not? This is hard to check as the issue is randomly happening to users and time, and it would take too much time to try to reproduce it, apart from building a parallel WLAN infrastructure to test to have all parameters uner control.

Are these systems running Win 10 or 11? Win 10

Have you tried using different channel numbers to avoid overlapping? I have noticed the issue in almost all 5-GHz band channels.

Any difference between onsite users vs VPN users (if used)? onsite users are connected to Cisco infrastructure, VPN users are using any infrastructure (Teleworker, SoHo, retail,...) so not possible to check the behaviour under such uncontrolled scenarios.

I would like to have an answer to my previous question, are Intel adapters using ONLY data rates that are advertized on the beacons/probe responses/association responses? Or are they also using any 802.11 standard supported rate for OFDM independently from the ones advertized by the AP? This is important to know not only for this case, but for any possible data rate impairment scenario that could happen.

SIDE NOTE: On the TAC case I'm working with Cisco we have seen the AP with a defect sending supported rate 6.0 Mbps when this is disabled, so that could be the cause about why the Intel adapter connects to 6.0 Mbps but, why is it doing it if the laptop is just below the AP with huge signal and SNR values? This question concerns Intel and the algorithm the device uses to select one rate or another during connection.

[Alberto_R_Intel]

Hello JPavonM, Thank you very much for providing those facts.

Based on that and on your questions, we will continue with our research on this matter, as soon as I get more details, I will post all the information on this thread.

Regards,

Albert R.

Intel Customer Support Technician

[Alberto_R_Intel]

Hello JPavonM, I just received an update on this matter.

After reviewing the case, it is important to mention that the information you are looking for is not publicly available. Based on that, please visit, log in and submit your inquiries on our Intel® RDC website so they can further assist you:

https://www.intel.com/content/www/us/en/resources-documentation/developer.html

For other third-party details please get in contact directly with your respective AP OEM for additional assistance on that topic.

Regards,

Albert R.

Intel Customer Support Technician

[JPavonM]

Hi @Alberto_R_Intel and Intel communinities colleagues.

After continuing troubleshotting this scenario, I have seen that all Intel AX wNICs using whatever driver version from 120 to 180 are sending support to 6.0Mbps on the association-request frame to the Cisco APs (now I'm testing with another vendor to verify if this happen in other scenarios). This behaviour I cannot see it if I use other laptops such as Lenovos with embedded Mediatek MT7921 or Realtek RTL88852AE, not even using MacBooks.

There is a double issue here:

1. Intel device sending assoc-request to AP with supported data rate 6.0Mbps, plus the configured rates 24.0Mbps and above (in the office I have seen that behaviour). This 6.0Mbps rate is not announced in the beacon from Cisco AP.
2. Cisco AP accepting the Intel device supporting a rate which is not configured, and sometimes turning that rate enabled in the beacons after listening for the Intel devices (I have not found the trigger why this is happening sometimes and not all times). this is leading to devices connected at 6.0Mbps.

I have tested that behaviour with Cisco IOS-XE 17.6 train, and with Intel AX adaptersrunning  drivers from 22.120 to 22.180.

After talking to Cisco BU engineers thay have confirmed this scenario, and they are going to talk to Intel developers about this to be fixed, as the root source of the issue seems to be the Intel driver which sends unsupported rates in the association request to the AP (diving into technical documentation, that shouldn't happen as the connecting station should request to use only announced rates and features found in the beacons or probe responses)