Re: Hyper-V vSwitches do not work with 2.x drivers on a I225-V - Page 2

Elu · ‎10-02-2023

I have a system with an I225-V running driver version 1.1.3.28, with a Hyper-V vSwitch configured against it and multiple management OS vNICs (with both VLAN and not). This works correctly.

When any driver from the 2.x branch is installed, e.g. 2.1.3.15, this fails. The vNICs appear to stop receiving any network traffic (packets received count stops incrementing).

This also occurs with a clean setup without VLANs: even with a basic vSwitch, the default management vNIC does not appear to receive any packets.

NIC: (3) I225-V

Working driver version: 1.1.3.28, any other 1.x

Failing driver versions: 2.1.3.15, any other 2.x (e.g. 2.1.1.7)

OS: Windows 11 22H2 Build 22621.2283

Motherboard: Asus Maximus Z690 Hero

BIOS: 2703 (latest)

Intel ME: 16.1.27.2176 (latest)

Elu · ‎11-01-2023

First we set up Hyper-V vNICs as described in https://community.intel.com/t5/Ethernet-Products/Hyper-V-vSwitches-do-not-work-with-2-x-drivers-on-a-I225-V/m-p/1531012/highlight/true#M34821

From there, with a 2.x driver installed, Windows will show 0 packets received in the adapter settings (via `ncpa.cpl` control panel) on the VLAN-access vNIC:

This same information can be seen in Powershell via the command `Get-NetAdapterStatistics 'vEthernet (VLAN1)'`

On my test setup I have DHCP configured on this network, so I expect to receive DHCP replies immediately. But this can be tested with any traffic, e.g. with a static address the same result happens when attempting to ping, with ICMP packets being sent but nothing received on the vNIC.

These same tests perform correctly when using the 1.x driver.

CGGX · ‎11-02-2023

Hi @Irwan_Intel, in my case, the virtual ethernet adapters with VLAN are used by the Hyper-V virtual machines. For instance, here is one using VLAN ID:

Now I just did a test and did clear the traffic counters on the router for the VLAN 12 bridge interface of the router and booted the virtual machine.

What I could observe on the router is that for the whole duration of the test, the router received no packets for VLAN 12 (Rx Bytes and Rx Packets fields of the screenshot above), while the router did regularly send packets on the line, because it was advertising its present, both for IPv6 and IPv4.

The network interface of the virtual machine (a Windows installation) however received none of those packets. But it did send out packets, probably for DHCP.

When I rolled back the driver on the host machine to 1.1.4.42, everything works again, and we have normal traffic recorded both by the VM and on the router:

ABCa · ‎11-06-2023

Hello,

You can reproduce this issue on any Windows 11 22H2 and 23H2 using the latest I225-V drivers from release 28.2.1.

This presumes Hyper-V is installed and a virtual switch is created WITHOUT a VLAN ID for the management OS.

Using Powershell, verify your setup:

PS C:\Users\Usager> (Get-WindowsOptionalFeature -Online -FeatureName *hyper-v* | select DisplayName, FeatureName, State) | ft * -Autosize

DisplayName FeatureName State
----------- ----------- -----
Hyper-V Microsoft-Hyper-V-All Enabled
Plateforme Hyper-V Microsoft-Hyper-V Enabled
Outils d’administration Hyper-V Microsoft-Hyper-V-Tools-All Enabled
Module Hyper-V pour Windows PowerShell Microsoft-Hyper-V-Management-PowerShell Enabled
Hyperviseur Hyper-V Microsoft-Hyper-V-Hypervisor Enabled
Services Hyper-V Microsoft-Hyper-V-Services Enabled
Outils de gestion de l’interface graphique utilisateur Hyper-V Microsoft-Hyper-V-Management-Clients Enabled

PS C:\Users\Usager> Get-NetAdapter | Get-NetAdapterBinding | Where { ($_.Enabled -eq $true) -and ($_.ComponentID -eq "vms_pp") }

Name DisplayName ComponentID Enabled
---- ----------- ----------- -------
Ethernet 3C-08-7A Commutateur virtuel extensible Hyper-V vms_pp True

Now, you can verify with pktmon that trafic is flowing normally:

PS C:\Users\Usager> pktmon start -c

Paramètres de l'enregistreur d'événements :
    Nom de l'enregistreur d'événements :        PktMon
    Mode de journalisation :       circulaire
    Fichier journal :           C:\Users\Usager\PktMon.etl
    Taille maximale du fichier :      512 Mo
    Mémoire utilisée :        512 Mo

Données collectées :
    Compteurs de paquets, capture de paquets

Type de capture :
    Tous les paquets

Composants analysés :
    Tous

Filtres de paquet :
    Aucun(e)
PS C:\Users\Usager> pktmon counters

VSwitch: TestSwitch
 ID Nom                        Compteur Direction     Paquets         Octets | Direction     Paquets         Octets
 -- ---                        -------- ---------     -------         ------ | ---------     -------         ------
142 Intel(R) Ethernet Contr... Lower    Rx                707        182 278 | Tx                612         85 202
143 Intel(R) Ethernet Contr... Ignore   Rx                706        182 178 | Tx                  0              0
                               Upper    Rx                707        182 278 | Tx                612         85 202
                                                                             |
144 WFP Native Filter          Lower    Rx                707        182 278 | Tx                612         85 202
                               Upper    Rx                707        182 278 | Tx                612         85 202
                                                                             |
158 VMS Protocol               Lower    Rx                707        182 278 | Tx                612         85 202
149 MSLLDP                     Lower    Rx                  1            100 | Tx                  0              0
   -     -     -     -     -     -     -     -     -     -     -     -     - |   -     -     -     -     -     -
176 vEthernet (TestSwitch)     Ignore   Sortie             26          1 759 | Entrée              0              0
176 Host vNic #2 (NDIS)        Upper    Rx                686        180 978 | Tx                612         85 202
                                                                             |
161 WFP Native Filter          Lower    Rx                686        180 978 | Tx                612         85 202
                               Upper    Rx                686        180 978 | Tx                612         85 202
                                                                             |
171 TCPIP6                     Ignore   Rx                 42         19 455 | Tx                  5              0
                               Lower    Rx                 87         47 840 | Tx                 67         20 649
168 TCPIP                      Ignore   Rx                192         57 934 | Tx                 34              0
                               Lower    Rx                594        132 679 | Tx                545         64 553

PS C:\Users\Usager> pktmon stop
Journaux de vidage...
Fusion des métadonnées...
Fichier journal : C:\Users\Usager\PktMon.etl (aucun événement perdu)

Now assign a VLAN ID to the management OS (using the Hyper-V GUI or powershell) and repeat the above "pktmon start -c", "pktmon counters" and "pktmon stop" sequence: no packets will be capturred.

It does not matter that the VLAN ID used for testing exists or not in your physical switch.

You can reproduce this at will on any Windows 11 22H2/23H2 using an I225-V. For this test, I used a Intel NUC model NUC12WSK (12th Gen Intel(R) Core(TM) i7-1260P 2.10 GHz, 64GB).

B_Y · ‎11-07-2023

Hello @Elu,

Thank you for your patience. As per our investigation, we would suggest to utilize NDIS driver instead of NetAdapterCx and monitor if the issue persists? From our complete driver pack, the driver is located at PRO2500\Winx64\NDIS68.

Thank you.

Best regards,

BY_Intel

Intel Customer Support

Elu · ‎11-07-2023

Hi,

At this point we are well aware that the NDIS drivers do work, as noted by many previous posts in this topic. Specifically, the NDIS drivers are version 1.1.4.42, which is the working 1.x branch.

However, this does not solve the biggest problem: the broken 2.x branch was being served to us via Windows Update with no easy way to opt out.

Additionally, it looks like the 2.x branch is the primary supported version for Windows 11. While the 1.x old drivers may currently work, do we have any guarantee that they will continue working with future versions of Windows 11, even ignoring the issues with Windows Update forcibly updating them?

Could you please tell us whether this issue will be worked on and fixed in a future version of the 2.x NetAdapterCx drivers, as long as they are considered the supported option for Windows 11? Will this be properly fixed before any date where the 1.x drivers may be considered unsupported? Will this be documented as a known issue in your release notes?

ABCa · ‎11-07-2023

Hello,

If I add a Realtek USB GbE adapter that strictly relies on Microsoft's distributed drivers, I can create a virtual switch and assign a VLAN to the management OS part. See exact bindings below

This adapter relies on netadaptercx.sys just as the latest 28.2.1 Intel released drivers and VLAN trafic flows through. The sample below shows both IPv4 and IPv6 addresses for the management OS NIC named "vEthernet (TestSwitch)".

As this issue been escalated to a full-fledge bug ? And what is the next release of these drivers expected ?

PS C:\Users\Usager> pktmon list -a
[ ... ]

VSwitch: TestSwitch
    ID: 144

    Carte réseau: Realtek USB GbE Family Controller
        ID: 165
        Pilote: rtucx22x64.sys
        Adresse MAC: A0-CE-C8-7A-AF-46
        ifIndex: 4

        ID: 2 (NDIS)
        Pilote: netadaptercx.sys

        Pilotes de filtre :
            ID Pilote      Nom
            -- ------      ---
            22 wfplwfs.sys WFP Native Filter

        Protocoles :
            ID Pilote       Nom          EtherType
            -- ------       ---          ---------
            58 mslldp.sys   MSLLDP       LLDP
            57 vmswitch.sys VMS Protocol * (Tous)

    Carte réseau: vEthernet (TestSwitch)
        ID: 146
        Pilote: vmswitch.sys
        Adresse MAC: A0-CE-C8-7A-AF-46
        ifIndex: 51

        Pilotes de filtre :
            ID Pilote      Nom
            -- ------      ---
            21 wfplwfs.sys WFP Native Filter
            20 pacer.sys   QoS Packet Scheduler
            19 wfplwfs.sys WFP 802.3 Filter

        Protocoles :
            ID Pilote      Nom     EtherType
            -- ------      ---     ---------
            54 tcpip.sys   TCPIP6  IPv6
            55 tcpip.sys   TCPIP   IPv4, ARP
            53 mslldp.sys  MSLLDP  LLDP
            52 rspndr.sys  RSPNDR  VLAN, LLTD
            51 lltdio.sys  LLTDIO  * (Tous)
            50 ndisuio.sys NDISUIO VLAN, 802.1X, 802.11i

        Protocoles d’application :
             ID Pilote   Nom  Adresse IP
             -- ------   ---  ----------
            167 http.sys HTTP fdfa:fe0:50aa:389d:bc69:6030:d02c:6d4d
            166 http.sys HTTP fdfa:fe0:50aa:389d:1bd9:a05b:5c81:d9ba
            162 http.sys HTTP fe80::d13a:4c95:3100:d520
            154 http.sys HTTP 192.168.166.207

[ ... ]

Carte réseau: Intel(R) Ethernet Controller (3) I225-V
    ID: 164
    Pilote: e2fn.sys
    Adresse MAC: 48-21-0B-3C-08-7A
    ifIndex: 18

    ID: 4 (NDIS)
    Pilote: netadaptercx.sys

    Pilotes de filtre :
        ID Pilote      Nom
        -- ------      ---
        26 wfplwfs.sys WFP Native Filter
        25 pacer.sys   QoS Packet Scheduler
        24 wfplwfs.sys WFP 802.3 Filter

    Protocoles :
        ID Pilote      Nom     EtherType
        -- ------      ---     ---------
        64 tcpip.sys   TCPIP6  IPv6
        65 tcpip.sys   TCPIP   IPv4, ARP
        63 mslldp.sys  MSLLDP  LLDP
        62 rspndr.sys  RSPNDR  VLAN, LLTD
        61 lltdio.sys  LLTDIO  * (Tous)
        60 ndisuio.sys NDISUIO VLAN, 802.1X, 802.11i

    Protocoles d’application :
         ID Pilote   Nom  Adresse IP
         -- ------   ---  ----------
        157 http.sys HTTP fe80::4f9:8aef:6a5b:5440
        149 http.sys HTTP 192.168.18.31

[... ]

B_Y · ‎11-07-2023

Hello @Elu,

Thank you for the feedback. Please give us some time to investigate and get back to you with recommendations.

We appreciate your patience.

Best regards,

BY_Intel

Intel Customer Support

B_Y · ‎11-08-2023

Hello @Elu,

Further investigation reveals that NDIS is compatible with Windows 10 and Windows 11, and you are correct about the Netadaptercx driver—a new version is exclusive to Windows 11.

Rest assured, that the Intel engineering team is still working to find a solution for Windows' constant updates. As you may know, the update for Windows 11, version 22H2 22621.2283, was released on September 12, 2023. Microsoft has already released the most recent update for 22621.2506 and 22631.2506 as of October 31, 2023.

Our advice is to disable auto update if the system is in production by taking the following actions:

https://www.techadvisor.com/article/745607/how-to-turn-off-automatic-updates-in-windows-11.html

Best regards,

BY_Intel

Intel Customer Support

Elu · ‎11-08-2023

> Further investigation reveals that NDIS is compatible with Windows 10 and Windows 11, and you are correct about the Netadaptercx driver—a new version is exclusive to Windows 11.

We know the NDIS drivers work. What is unclear is whether they are a supported option, since Intel's driver download page for Windows 11 only provides the NetAdapterCx driver.

> Rest assured, that the Intel engineering team is still working to find a solution for Windows' constant updates. As you may know, the update for Windows 11, version 23H2 22621.2283, was released on September 12, 2023. Microsoft has already released the most recent update for 22621.2506 and 22631.2506 as of October 31, 2023.

If by "working to find a solution" you mean you are investigating this specific issue of VLANs not working, and are working towards fixing it, that is good news. But if that is the case could you please say so directly?

I do not think the fact that 23H2 is available is relevant considering we are reporting this issue against the 22H2 version that has been out for a whole year. The problem isn't the Windows version. The problem is that Intel's 2.x NetAdapterCx drivers are broken. As others have noted, Realtek's NetAdapterCx drivers work just fine. So it's not that the new driver model is fundamentally incompatible with Hyper-V or VLANs.

> Our advice is to disable auto update if the system is in production by taking the following actions:

I'm sorry, but this is not a viable solution. Obviously Windows patches are required to remain secure in the modern threat environment; to suggest otherwise is just plain irresponsible. My comment previously about the update being unavoidable is precisely because in modern Windows there is no easy way to ignore one specific update (the updated/broken Intel driver) while still receiving all other updates.

---

Let me reiterate the two problems that need to be solved for Intel to be a viable choice for future NICs:

Most immediately, this 2.x driver should not be distributed/forced via Windows Update if it has a significant regression in functionality. You can probably tell by the increasing number of people posting in this thread that this is affecting more and more people as the update has been pushed down to them. This will only get worse.
In the long term, there needs to be a supported driver for Windows 11 that supports Hyper-V with VLANs. Preferably the 2.x driver is properly fixed to support this case. If not, we need some kind of assurance that the NDIS drivers are actually the officially supported solution and will remain supported.

Elu · ‎11-08-2023

I appear to be unable to post my originally intended reply here, since the message disappears shortly after submitting it. I will try a shortened version:

---

Re: 23H2 being available, we've got confirmations of this bug existing within both 22H2 and 23H2. 22H2 has been out for over a year. Please don't use the Windows update cadence as an excuse for this bug.

Our advice is to disable auto update if the system is in production by taking the following actions:

I'm sorry, but this is not a viable solution. Obviously Windows patches are required to remain secure in the modern threat environment; to suggest otherwise is just plain irresponsible. My comment previously about the update being unavoidable is precisely because in modern Windows there is no easy way to ignore one specific update (the updated/broken Intel driver) while still receiving all other updates.

---

Let me reiterate the two problems that need to be solved for Intel to be a viable choice for future NICs:

Most immediately, this 2.x driver should not be distributed/forced via Windows Update if it has a significant regression in functionality. You can probably tell by the increasing number of people posting in this thread that this is affecting more and more people as the update has been pushed down to them. This will only get worse.
In the long term, there needs to be a supported driver for Windows 11 that supports Hyper-V with VLANs. Preferably the 2.x driver is properly fixed to support this case. If not, we need some kind of assurance that the NDIS drivers are actually the officially supported solution and will remain supported.

B_Y · ‎11-08-2023

Hello @Elu,

Thank you again for your thorough explanation and feedback. We understand where you are coming from.

We sincerely apologize; there is no estimated time for the solution that the Intel engineering team is still working on.

We recommended stopping the Windows update, which should only be done temporarily, as there is a temporary fix that involves utilizing NDIS or older drivers. Unless Windows update comes with its own driver or from OEM.

I hope this explanation clarifies your concern.

Best regards,

BY_Intel

Intel Customer Support

Elu · ‎11-09-2023

I'm sorry if I have not been clear enough, but I am not asking you for an estimated time. I am asking you to do these two things:

1. You really should be withdrawing the nonfunctional driver versions from Windows Update. This is something Intel should do, to stop the versions with the functionality regression from further rolling out to unsuspecting users. This is not the job of the user, who has no control over specific updates.

Additionally, your advice to "temporarily" stop Windows Updates globally is an absolute non-starter unless you can guarantee that Intel will release a proper fixed driver to supersede the broken version within a month. Because every month, if not more often, Windows releases important security updates, and not having these updates installed can very quickly become a critical security issue.

In case I've not made it clear enough: it is not safe to disable Windows Updates. That is not a reasonable workaround for one vendor (Intel) releasing broken drivers.

2. Please confirm that this specific issue, of the 2.x drivers not working with Hyper-V VLANs, is now recognised as an issue by Intel and will be worked on. At this point we still don't have positive confirmation that this is even known, never mind any estimated time frame.

Given the runaround and passing of blame (to the OEM, to Microsoft, to ...) in earlier posts, I really want this to be crystal clear:

Can you answer "yes" to this specific statement? "Intel acknowledges that the Intel wired network adapter driver versions 2.x including 2.1.3.15 do not function correctly with Hyper-V virtual NICs and VLANs, and will commit to providing a fix in a future version."

Once again, I am not asking for a specific timeframe. I am asking for a clear acknowledgement that this is an issue with the drivers, and will be fixed at some point down the track. Are you able to confirm this? If not, why not?

B_Y · ‎11-09-2023

Hello @Elu,

Thank you for your concern.

Intel is pleased to inform that we have identified and resolved the issue. To ensure optimal performance, we recommend all affected users work with motherboard vendors to update NVM version to 2.22 and Windows 11 driver to 2.1.3.15 or Windows 10 driver to 1.1.4.42. These updates have undergone extensive testing and have demonstrated stability. However, Intel is unable to predict when Windows or OEM will release the latest update on NVM upgrade.

After applying these updates, it is safe to manually enable "Energy Efficient Ethernet (EEE)" mode in the Windows driver.

If you have any questions, concerns, or feedback regarding these updates, please contact us.

Best regards,

BY_Intel

Intel Customer Support

Elu · ‎11-10-2023

Hi,

Are you sure you posted this to the right thread?

"Energy Efficient Ethernet" has nothing to do with the VLAN issue reported here. Looks like you intended to address the issue over at https://community.intel.com/t5/Ethernet-Products/Intel-Communication-Intel-Ethernet-Controller-I226-Series-Random/m-p/1464539.

Please note that the driver version 2.1.3.15 you recommend is one of the FAILING driver versions reported in this thread. It is also the one being forced down via Windows Updates and causing issues there.

I am still waiting for a positive confirmation that Intel is aware of and working on a resolution specifically for the Hyper-V VLAN issues as raised in THIS thread and as I asked for in this post: https://community.intel.com/t5/Ethernet-Products/Hyper-V-vSwitches-do-not-work-with-2-x-drivers-on-a-I225-V/m-p/1542549/highlight/true#M35304

Elu · ‎11-10-2023

Hi @B_Y,

Could you please confirm that the VLAN issue is still under active investigation, and that your previous message saying you've "resolved the issue" was not intended for this thread? The version numbers and suggestion to enable "Energy Efficient Ethernet" do not appear to be relevant.

It really should not be so hard to confirm such a simple thing.

If instead Intel intends to ignore the VLAN issue, then please tell me now so I can stop wasting my and your time here and go by a Realtek card or something.

Elu · ‎11-13-2023

Still waiting for a response to whether you intended to post this message to this thread.

Still also waiting for a response to whether the VLAN issue is under active investigation by Intel.

Elu · ‎11-14-2023

@B_Y

Could you please respond to the previous questions? Namely, could you please clarify your post talking about an "Energy Efficient Ethernet" that appears irrelevant to this Hyper-V VLAN issue, and could you please tell me whether the VLAN issue remains open and under investigation?

B_Y · ‎11-15-2023

Hi @Elu ,

Good day, we apologies for the delay in response and also please disregard the Energy Efficient Ethernet (EEE).

I'm working closely with the team and still waiting for an accurate response from the Intel Engineering team. I'm unable to provide an answer on your inquiries at this time. As far as we know, there could be issues with the Driver version 2.1.3.15 when the Windows update took place. As a result, we've also taken your input seriously.

Please allow some time to get back to you.

Best regards,

BY_Intel

Intel Customer Support

Elu · ‎11-15-2023

Just a quick note that as per the response in https://community.intel.com/t5/Ethernet-Products/Please-confirm-whether-Hyper-V-VLAN-issues-in-driver-2-1-3-15/m-p/1544116 the post earlier about a 'fixed' issue (EEE) actually was not intended here.

From my end, I've also found an additional (3) I225-V PCIe card I have installed in a Dell Optiplex 7040 system and reproduced the issue there. Windows Update only brought the version from inbox 1.0.2.8 up to 1.1.3.28, so perhaps at least the Windows Update issue has been corrected? Manually installing 2.1.3.15 with the PCIe card still reproduces the VLAN issue on that system, matching others' experiences with PCIe cards in this thread.

L_Gk · ‎11-15-2023

I'm having the same issue on an Asus board with a I226-v, I've rolled back the driver to 2.1.3.3. VLANs aren't working with this NIC. I can throw an older Intel 1GB NIC and the VLANs do work. Problem with that is I need the 2.5gb speed of the onboard NIC

L_Gk · ‎11-15-2023

I can also confirm if I use the WS2022 driver from the Release 28.2.1 it it passes VLAN traffic.