Ethernet Products
Determine ramifications of Intel® Ethernet products and technologies
4811 Discussions

How to do NIC teaming (LACP) and VLANs on Windows Server 2008 R2 Hyper-V?

RNiss
Beginner
6,153 Views

First a bit about my environment:

  • Dell PowerEdge 2970
  • Intel Quad-port I340-T4 10/100/1000 Ethernet HBA
  • Intel driver version: 12.7.28.0 (installed drivers and PROSet using PROWinx64.exe, v. 18.3)
  • Microsoft Windows Server 2008 R2 Datacenter w/SP1 (Core, with Hyper-V role)
  • Cisco (Linksys) SRW2016 16-port full GbE (10/100/1000) switch

I used PROSetCL.exe and the SRW2016 web management interface to configure a 3-port LAG (in LACP mode). The NIC team comes up fine and I'm able to use it for "normal" network stuff. The teamed interface is used by the virtual servers / guests for network connectivity. I'm not doing anything with VMDq yet, although I'm interested in exploring it after I get VLANs working with teaming. Here is the output for PROSetcl.exe Team_GetTeamInfo 1:

1) TEAM : VMNetwork

Team Information:

Team GUID - '{305A7788-9AFC-4DD4-9B47-2CF0CF70412B}'

Team MAC - '001B21AEB9A5'

Team Mode - 'IEEE 802.3ad Dynamic Link Aggregation'

Team Name - 'VMNetwork'

And the output for PROSetcl.exe Team_EnumerateAdaptersInTeam 1:

1) TEAM : VMNetwork

Number of adapters currently present: 3

4) TEAM : VMNetwork - Intel(R) Ethernet Server Adapter I340-T4

3) TEAM : VMNetwork - Intel(R) Ethernet Server Adapter I340-T4 # 4

2) TEAM : VMNetwork - Intel(R) Ethernet Server Adapter I340-T4 # 3

On the SRW2016, the LAG corresponding to these 3 network interfaces is configured for VLAN trunking, untagged in VLAN 1 and tagged in all other VLANs. Somewhere in my explorations, I stumbled on a suggestion to also configure individual ports for VLANs, but when I tried this on the SRW2016, it wouldn't let me add the ports to a LAG. So, I can configure VLANs on a LAG, but the individual ports (apparently in trunk mode) are excluded from all VLANs.

I haven't set any VLAN configuration (using PROSetCL.exe) on the team (or individual NICs) in Hyper-V, and the related documentation I've read suggests that not only is it unnecessary, doing so is undesirable. My understanding is that if I set any VLAN configuration at the Hyper-V host level, then I must configure all VLANs I wish to be used by Hyper-V guests.

See:

http://www.intel.com/support/network/sb/cs-030993.htm http://www.intel.com/support/network/sb/cs-030993.htm

http://social.technet.microsoft.com/wiki/contents/articles/151.hyper-v-virtual-networking-survival-guide.aspx# Understanding_Hyper-V_VLANs http://social.technet.microsoft.com/wiki/contents/articles/151.hyper-v-virtual-networking-survival-guide.aspx# Understanding_Hyper-V_VLANs

http://www.aidanfinn.com/?p=10164 http://www.aidanfinn.com/?p=10164

For Hyper-V host management, I'm using a separate / dedicated NIC (not the teamed NIC interface) not involving VLANs. So, in Hyper-V's virtual switch connected to the teamed network interface, the option Allow management operating system to share this network adapter is unchecked, and there are no VLAN settings at that level.

One Hyper-V guest (running Windows Server 2008 R2 Standard) is multi-homed (two hybrid virtual network adapters). One of the adapters is configured for VLAN identification (in Hyper-V's settings, not inside the guest OS). Inside the guest OS, an appropriate IP address is assigned to the adapter, but the guest cannot communicate with other hosts on the VLAN it is configured for.

I'm just now beginning to think I should be using VMLB instead of LACP...? Am I correct in thinking that with VMLB, teaming is one-sided, meaning the physical switch knows nothing of the team configuration, and physical switch ports are configured individually? Or do I need a physical switch that understands / supports VMLB?

I would appreciate any suggestions on troubleshooting or getting this to work.

Thank you!

0 Kudos
9 Replies
RNiss
Beginner
4,094 Views

Changing the team to VMLB was less painful than expected. On the Core Server with the Hyper-V role:

PROSetCL.exe Team_ModifyTeamInfo 1 mode VMLB

In SRW2016 web management, I removed the LAG. The ports are now individually managed on the physical switch. This made no difference with VLAN functionality, even after configuring the physical switch ports for VLAN trunking and tagging the VLANs I want to be able to access from my virtual servers / guests.

I confirmed my VLAN configuration up to the SRW2016 by configuring a laptop for one of my tagged VLANs and connecting it to a free port on the SRW2016. I was able to ping several systems on that VLAN as expected, but wasn't able to ping one of the virtual servers / guests that has a network connection configured for that VLAN. Not sure what I'm missing...

0 Kudos
Mark_H_Intel
Employee
4,095 Views

Your last post happened while I was typing mine, so I didn't see it until after I replied. Are you still running with two virtual NICs in the VM? Are you configuring the VLAN in the VM virtual NIC only? Or are you configuring the VLAN on the virtual network and/or on the team? Just configuring the VLAN on the virtual NIC of the VM should be enough, but if you have more than one virtual NIC, maybe the VM doesn't know which one to use.

0 Kudos
RNiss
Beginner
4,095 Views

mark_h_@intel wrote:

Are you still running with two virtual NICs in the VM?

Yes. One connects to the default / untagged VLAN. The other is configured (in the Hyper-V VM settings) to "Enable virtual LAN identification", with the appropriate VLAN id in the text box provided.

mark_h_@intel wrote:

Are you configuring the VLAN in the VM virtual NIC only? Or are you configuring the VLAN on the virtual network and/or on the team?

In the VM virtual NIC only (in the Hyper-V network adapter settings for the VM in question). I'm not configuring the VLAN on the virtual network or on the team.

mark_h_@intel wrote:

Just configuring the VLAN on the virtual NIC of the VM should be enough, but if you have more than one virtual NIC, maybe the VM doesn't know which one to use.

I also thought that should be sufficient, but I'm obviously missing something. IP configuration is set statically on both network interfaces inside the VM, and there are no connectivity problems with the adapter connected to the default / untagged VLAN. The adapter for the tagged VLAN is the one with connectivity problems (can't ping other hosts on that VLAN, and vice versa).

0 Kudos
Mark_H_Intel
Employee
4,095 Views

I have heard from some of the other people at Intel who believe that the VM can't tell which interface belongs to which virtual network. So there might be a problem with the dual virtual NIC configuration. I would try removing the virtual NIC for the untagged VLAN to see if the tagged VLAN starts working.

Mark H

0 Kudos
RNiss
Beginner
4,095 Views

mark_h_@intel wrote:

I would try removing the virtual NIC for the untagged VLAN to see if the tagged VLAN starts working.

Mark H

I gave it a shot: shutdown the VM, removed the virtual NIC for the untagged VLAN, and booted the VM. No difference. I'm not exactly surprised, though. In my exploration, I stumbled on several references to configuring VMs with multiple network adapters / VLANs. http://www.virtualizationadmin.com/articles-tutorials/microsoft-hyper-v-articles/networking/introduction-vlan.html Here is one.

0 Kudos
RNiss
Beginner
4,095 Views

Possible clues...?

Notice the output for PROSetCL.exe Adapter_EnumerateSettings 1 (a non-teamed network interface):

1) Intel(R) Ethernet Server Adapter I340-T4 # 2

Settings:

LLIPorts -

NetworkAddress -

*ReceiveBuffers - 256

*RssBaseProcNumber - 0

*TransmitBuffers - 512

EnableLLI - Disabled

*FlowControl - Rx & Tx Enabled

*InterruptModeration - Enabled

*IPChecksumOffloadIPv4 - Rx & Tx Enabled

*JumboPacket - Disabled

*LsoV2IPv4 - Enabled

*LsoV2IPv6 - Enabled

*MaxRssProcessors - 8

*NumaNodeId - System Default

*PriorityVLANTag - Priority & VLAN Enabled

*RSS - Enabled

*SpeedDuplex - Auto Negotiation

*TCPChecksumOffloadIPv4 - Rx & Tx Enabled

*TCPChecksumOffloadIPv6 - Rx & Tx Enabled

*UDPChecksumOffloadIPv4 - Rx & Tx Enabled

*UDPChecksumOffloadIPv6 - Rx & Tx Enabled

*VMQ - Disabled

*WakeOnMagicPacket - Enabled

*WakeOnPattern - Enabled

EnablePME - Disabled

ITR - Adaptive

LogLinkStateEvent - Enabled

MasterSlave - Auto Detect

NumRssQueues - 1 Queue

ReduceSpeedOnPowerDown - Enabled

WaitAutoNegComplete - Auto Detect

WakeOnLink - Disabled

Here is the output for PROSetCL.exe Adapter_EnumerateSettings 2 (an interface in the VMLB team):

2) TEAM : VMNetwork - Intel(R) Ethernet Server Adapter I340-T4 # 3

Settings:

LLIPorts -

NetworkAddress - 001B21AEB9A5

*ReceiveBuffers - 256

*RssBaseProcNumber - 0

*TransmitBuffers - 512

EnableLLI - Disabled

*FlowControl - Rx & Tx Enabled

*InterruptModeration - Enabled

*IPChecksumOffloadIPv4 - Rx & Tx Enabled

*JumboPacket - Disabled

LSO - Enabled

*LsoV2IPv6 - Enabled

*MaxRssProcessors - 8

*NumaNodeId - System Default

*RSS - Enabled

*SpeedDuplex - Auto Negotiation

*TCPChecksumOffloadIPv4 - Rx & Tx Enabled

*TCPChecksumOffloadIPv6 - Rx & Tx Enabled

*UDPChecksumOffloadIPv4 - Rx & Tx Enabled

*UDPChecksumOffloadIPv6 - Rx & Tx Enabled

*VMQ - Disabled

ITR - Adaptive

LogLinkStateEvent - Enabled

MasterSlave - Auto Detect

NumRssQueues - 1 Queue

WaitAutoNegComplete - Auto Detect

Note the absence of Priority / VLAN settings in the enumerated settings for the teamed interface.

Finally, here is the output for PROSetCL.exe Team_EnumerateSettings 1:

1) TEAM : VMNetwork

Settings:

BalanceInterval - 10

CheckTime - 1000

MaxNumProbeRetry - 10

STForwardDelay - 0

FailbackEnabled - Enabled

ProbeEnabled - Enabled

ProbePacketType - Broadcast Probes

Again, no settings for Priority / VLAN.

As a test, I removed one of the teamed network interfaces from the VMLB team and created a new Hyper-V virtual network (external) connected to the interface removed from the team. On the VM with 2 virtual network adapters, I configured the tagged-VLAN adapter to connect to the newly created virtual network (instead of the virtual network associated with the VMLB team). After booting the VM, I successfully tested basic connectivity on the default (untagged) VLAN and the tagged VLAN.

Are VLANs unsupported on I340-T4 teamed interfaces? I haven't seen anything explicitly stating so, and have seen documentation, threads and posts suggesting the opposite. Well, maybe not for the I340-T4 in particular, but teamed Intel server nics in general.

0 Kudos
Mark_H_Intel
Employee
4,095 Views

Yes, VLANs are supported on a team interface. You can explicitly configure VLANs on the teamed interface using PROSetCL, but that is not supposed to be necessary. However, you might want to give it a try.

When you configure VLANs on a physical port, a hardware filter is created so that VLANs that do not match get blocked. I'm not sure how that differs on the teamed interface. Anyway, the point is, with filters setup using PROSetCL, you will have to make sure that you assign all the VLAN numbers on the trunk and an untagged VLAN. Make sure you configure all VLANs that are used by VMs connected to the HyperV network that uses the team. Any tagged VLAN traffic that does not have a match will get blocked and dropped by the team. And if you do not configure the untagged VLAN, any untagged traffic would also get dropped.

Mark H

0 Kudos
RNiss
Beginner
4,095 Views

Hi Mark,

Thanks for the follow-up. I had already tried adding VLANs to the team interface, and I was aware of the requirement to configure all VLANs (used by VMs) if any are defined at the interface level. Here is what happens when I try using PROSetCL.exe to add a 'default' (untagged) VLAN:

> PROSetCL.exe Team_CreateVlan 1 untagged default

1) TEAM : VMNetwork

Attempting to create VLAN 'default' ...

Failed to create VLAN

and if I try to add a VLAN for our IP phone VLAN:

> PROSetCL.exe Team_CreateVlan 1 5 IP_PHONES

1) TEAM : VMNetwork

Attempting to create VLAN 'IP_PHONES' ...

Failed to create VLAN

I checked Windows' event logs but didn't find anything additionally relevant. Recall from my previous post how priority / VLAN settings don't appear in Adapter_EnumerateSettings output for teamed network interfaces, and never appear for the team itself, in the output for Team_EnumerateSettings. Perhaps incorrectly, this suggested to me that (at least for my combination of hardware, drivers, etc.) prioritization and VLANs are unsupported on teams...

For now, I've reserved a non-teamed interface for our non-default (tagged) VLANs. This works, but isn't an "ideal" solution, and doesn't satisfy my curiosity.

0 Kudos
Mark_H_Intel
Employee
4,095 Views

The setup you are doing where you have an LAG team that you use for your virtual switch should work. You are correct that a VMLB team does not need switch support, but you shouldn't have to use that team type.

I am not familiar with having multiple virtual NICs for the VM. You might want to simplify the configuration for troubleshooting and only use one NIC to check your communication on the VLAN. You might want to try a static IP from the appropriate VLAN on the remaining virtual NIC. If everything works, then you could try adding the second virtual NIC.

Mark H

 

0 Kudos
Reply