Intel NIC cards bleeding tagged vLANs into non-tagged vLANs (IPv6)

[idata]

Hi Guys,

First time i'm logging a case on this forum.

We have an interesting issue that i'd like to flag with Intel, which can be summarised as follows. Note, this issue is only seen on Windows machines, not Apple or Linux.

1. We have Windows machines (XP or Vista or 7) connected to Cisco switches which have a standard config attached to it. Here the switchport has both an access vlan (data) and a voice VLAN as shown below:

switchport access vlan 84

switchport mode access switchport nonegotiate switchport voice vlan 729 switchport port-security maximum 3 switchport port-security maximum 2 vlan access switchport port-security maximum 1 vlan voice switchport port-security switchport port-security aging time 1 switchport port-security violation restrict switchport port-security aging type inactivity ipv6 traffic-filter RA-FILTER in no mdix auto storm-control broadcast level 1.00 spanning-tree portfast spanning-tree bpduguard enable

For the most part, we have thousands of Cisco IP phones deployed across the network, such that the Phone is inline with the PC - no issue here.

The issue we see is when a PC is directly attached to the switchport. Here, when IPv6 is enabled on the NIC card, the driver rather than discarding the tagged Voice vLAN traffic, presents the IPv6 router advertisments from the Voice vLAN to the data vLAN. As such the RA packets bleed into the data vLAN.

This causes the TCP/IP stack in microsoft to install 2 IPv6 addresses, one for the data vLAN and one for the voice vLAN! IPv6 breaks as a result.

The other related issue is that we've tried lowering the voice vLANs router priority and prefix lifetime such that the NIC card preferences the data vLAN over the voice vLAN. This doesn't work either.

I've tried downloading the most current driver for my Intel NIC card to no avail. A ticket to troubleshoot this issue was logged with Microsoft, however they've pointed out that this is a driver issue that needs to be looked at by Intel.

Have you seen such a problem before? or have a driver that resolves both issues?

My NIC card is: Intel 82567LM Gigabit Network Connection, running 10.1.9.0 (4/7/2010).

Let me know if you have any questions.

thanks

Sheldon

[idata]

Is the second IPv6 address the link-local IPv6 address?

Can you copy and paste the output of "ipconfig /all" into this thread? It would be nice to have it from a system connected to the switch, and a system connected to a phone.

Are you actually using IPv6 in your environment?

What is the model of Cisco phone you are using?

Have you taken Wireshark traces from both scenarios (connected to switch and connected to phone) and analyzed them?

[idata]

Another question, since I'm reading up on this...

How do you have your voice vlan trunk setup, isl or dot1q?

Or is it not trunked and the data vlan is trunked?

[idata]

Hi Freddy,

Comments in line below:

>>> Is the second IPv6 address the link-local IPv6 address?

The second IPv6 address is not the link-local addess, it is based on the IPv6 prefix for the Voice VLAN (e.g. 2001:388:608c:28da::/64) as seen below in my ipconfig /all output.

>>> Can you copy and paste the output of "ipconfig /all" into this thread? It would be nice to have it from a system connected to the switch, and a system connected to a phone.

This is the output when connected to the switchport directly:

Ethernet adapter Local Area Connection 4:

Connection-specific DNS Suffix . : its.monash.edu.au

Description . . . . . . . . . . . : Intel(R) 82566DM Gigabit Network Connection Physical Address. . . . . . . . . : 00-19-BB-D7-7E-34 Dhcp Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes IP Address. . . . . . . . . . . . : 130.194.85.158 Subnet Mask . . . . . . . . . . . : 255.255.254.0 IP Address. . . . . . . . . . . . : 2001:388:608c:28da:7dd9:bb14:60fa:a1aa IP Address. . . . . . . . . . . . : 2001:388:608c:28da:219:bbff:fed7:7e34 IP Address. . . . . . . . . . . . : 2001:388:608c:4882:7dd9:bb14:60fa:a1aa IP Address. . . . . . . . . . . . : 2001:388:608c:4882:44e4:d1ea:3e51:27b5 IP Address. . . . . . . . . . . . : 2001:388:608c:4882:219:bbff:fed7:7e34 IP Address. . . . . . . . . . . . : fe80::219:bbff:fed7:7e34%20 Default Gateway . . . . . . . . . : 130.194.85.254 fe80::5:73ff:fea0:6%20 DHCP Server . . . . . . . . . . . : 130.194.15.1 DNS Servers . . . . . . . . . . . : 130.194.1.99 130.194.7.99 fec0:0:0:ffff::1%1 fec0:0:0:ffff::2%1 fec0:0:0:ffff::3%1 Lease Obtained. . . . . . . . . . : Thursday, September 16, 2010 11:03:39 AM Lease Expires . . . . . . . . . . : Thursday, September 16, 2010 11:03:39 PMThe output when connected directly to the phone is the same as above, except, without the '2001:388:608c:28da::/64' addresses. FYI, the following prefix 2001:388:608c:4882::/64 is for the data VLAN.

>>> Are you actually using IPv6 in your environment?

Yes we use it within our intranet and also when connecting to certain external sites (e.g. Google)

>>> What is the model of Cisco phone you are using?

Cisco IP Phone 7945

>>> Have you taken Wireshark traces from both scenarios (connected to switch and connected to phone) and analyzed them?

Yes have done this. When looking at a trace when the machine is directly connected to the port, we observe RA messages from both the VOice and Data VLANs, however this isn't the case when the machine is connected via the phone. Only the Data VLAN RA messages are seen as the phone filters the voice RA messages from the directly attached PC. Further, all the RA messages are sourced from the router with the same Link local adress: ( as an example fe80::5:73ff:fea0:6). All SVIs on the router use the same Link local address - the destination here is to all devices (ff02::1). SO the only way to distinguish the traffic is based on the fact that the data VLAN is not tagged, but the Voice VLAN traffic is. The NIC card should therefore ignore the Voice VLAN packets, and only present the data VLAN to the machine, as it happens under Ubuntu or MAC for instance.

The other issue is that the NIC card driver does not honor the router priority set...which we've set to low for the Voice VLAN, and also set the prefix lifetime to be lower than the data VLAN.

>>> Another question, since I'm reading up on this...how do you have your voice vlan trunk setup, isl or dot1q?Or is it not trunked and the data vlan is trunked?The switchport is setup as follows. As such, the Data VLAN is not tagged, but the voice VLAN is tagged (dot1q)

switchport access vlan 84 <<<<<<<<<<<<<<<<<<<<< Data VLAN<p> 

switchport mode accessswitchport nonegotiateswitchport voice vlan 750 <<<<<<<<<<<<<<<<<<<<< VOice VLAN<p> switchport port-security maximum 3switchport port-security maximum 2 vlan accessswitchport port-security maximum 1 vlan voiceswitchport port-securityswitchport port-security aging time 1switchport port-security violation restrictswitchport port-security aging type inactivityipv6 traffic-filter RA-FILTER inno mdix autostorm-control broadcast level 1.00spanning-tree portfastspanning-tree bpduguard enableLet me know if you have any other questions.

thanks

Sheldon

[idata]

Sheldon,

Did you ever resolve this? We are having the exact same problem. Windows 7 machines are self-configuring IPv6 addresses on both the user and VoIP VLANs and causing connectivity problems.

-Michael

[idata]

Hi Sparks,

We couldn't get anywhere with Microsoft so we decided to tackle this from the switch end. At the moment we have suppresed the RAs on the VoIP subnets using:

ipv6 nd ra suppress

..since the VoIP phones at this stage don't support IPv6. Separately I successfully tested the use of smartport macros on the switch to only present the Voice VLAN whenever a phone is connected to the switchport, and just the standard data port when there isn't a phone connected.

Hope this helps.

Cheers

Sheldon

[idata]

Well I have a workaround involving the drivers. Make sure you have the current Intel PROset driver installed (this has VLAN functions the standard driver does not). Then from Device Manager, configure the physical adapter to create a virtual adpater on the untagged VLAN. One catch: you have to configure a tagged VLAN virtual adapter before you can create an untagged VLAN virtual adapter, so just create one on any VLAN (it won't be used), create the untagged virtual adapter, and then disable the tagged one. If you delete it you lose them both. Not really something I want to do across hundres of machines...

This works with Intel adapters, but the same approach doesn't work with Broadcom adapters and their BASP utility. For Broadcom I've found you have to select to disable VLAN in the adpater properties and also set a registry value to drop tagged packets. You have to do both.

This is definitely a Microsoft problem with the way they handle VLAN tagged traffic. I don't see this on my Mac which has the same Broadcom chip as some of my Dells that do have the problem. However, it happens with ethernet chipsets from multiple vendors on my Windows boxes.

[idata]

Here's an update. Some guys in our central networking group talked with both Microsoft and Broadcom and figured out that this is intended behavior per Microsoft's NDIS specs. If the driver isn't configured for a VLAN, then incoming VLAN tagged traffic is to be converted to untagged. Makes no sense to me since Windows will receive the de-tagged traffic and have no way of properly replying to it on the correct VLAN. Seems to me if the driver isn't configured for a tagged VLAN then tagged traffic should just be ignored (as sane OSs like OSX and Linux already do).

Configuring NIC drivers as I described in my first reply was going to be too much work for all our machines and varying NICs. Since this is a purely Windows problem, I created a Windows Group Policy setting for the Windows firewall to drop unsolicited RAs. The specific filter is for ICMPv6 RA addressed to the all-nodes multicast address (ff02::1). Windows still gets the unicast RAs in reply to Router Solicitations so IPv6 self-addressing still works fine. Since SLAAC still works and I don't know of any other need Windows would have for those dropped RAs, I think this is a safe thing to do. It's working for us.

I'm not sure if the RA suppression on the VLAN as you describe suppresses both multicast and unicast RAs, but if there's a way to suppress just the multicast RAs at the network level that would also probably be a fair solution, as long any IPv6 device on your network issues RSs. I think I prefer the Windows firewall rule though, since this is really a Windows-only problem and there's no need to break multicast RAs for the whole network if you can do it just for Windows.

Best,

Michael

Message was edited by: Michael Sparks (after re-reading Sheldon's earlier reply and rethinking my comments about RA suppression)

[Mark_H_Intel]

Hi Sheldon,

Thank you for updating the thread.

Mark H

[idata]

Turns out relying on unicast RAs is insufficient. It works for SLAAC, but without the multicast RA's Windows loses its default route. So it's back to blocking RAs from the specific VoIP VLAN interfaces, configuring the NIC drivers for the untagged VLAN, or waiting for Microsoft to fix the default handling of VLAN tagged packets. Or Sheldon's smartport macro solution, which seems to be the next best solution to Microsoft fixing it.