Intel i350 NIC Teaming Question

[MHibb3]

Hi

I have a question surrounding the native NIC Teaming in Windows Server 2012 R2 with regards to the i350 Quad Port adapters.

In our Hyper-V implementation, we have 3 x quad port NICs per Hyper-V Cluster node.

With two of these adapters, we have balanced a Switch Independent / Dynamic (Sum of Queues mode) team across 4 of these ports for our VM Switch. Here's the thing... two of these ports are on one physical adapter and the other two ports are in the second physical adapter. Each port has VMQ enabled with suitable processor core allocations for a Sum of Queues team. VMQ is required as the team is connected to a VM Switch which is then supporting a reasonable number of Virtual Machines.

The remaining 4 ports in these two adapters (2 in one card and 2 in the other) are then allocated to iSCSI MPIO use for access to our SAN. These remaining ports have RSS enabled.

The questions I have are as follows:

A) What is Intels Stance when mixing RSS and VMQ modes on different ports, on the same physical adapter?

B) Is splitting a VMQ enabled NIC team between ports from physical NICs supported by Intel when using the native Windows NIC Teaming in Server 2012 R2?

We are seeing intermittent latency issues and packet loss from within virtual machines with VMQ enabled. Disabling VMQ does work around the issue, however this isn't really a solution.

Are there any known issues with these adapters when using Virtual Machine Queues?

Kind Regards

Matt

[MHibb3]

No takers at all?

[MHibb3]

Can I have an answer here please Intel? These cards were not cheap and we have purchased them specifically for their VMQ abilities. Being Intel cards, reliability is one thing that I would expect however I am disappointed thus far with both customer service and reliability.

We are seeing constant issues here with Virtual Machine Queues. The four adapters in the team are configured correctly. RSS is disabled and VMQ is enabled. As this is a Switch Independent team in Dynamic mode, Sum of Queues mode is in operation, therefore we have been careful not to overlap processors for each team member.

Even with the text book configuration, latest drivers and firmware, we still see problems with Virtual Machine Queues when they are enabled on the physical NICs and Microsoft Multiplex Team Adapter. Packet loss, high latency or complete loss of network connectivity for VMs is the order of the day. This mainly happens when a VM is migrated from host to host. Secondly, during a Live Migration, we see a longer than usual delay in network connectivity. Usually, we lose about a single ping during a Live Migration with VMQ turned off. With VMQ enabled, we see a good 5 or 6 lost ping requests before connectivity is restored. Clearly this isn't acceptable.

Answers please....

[vt5]

Hi MattJH, have you tried using VMLB teaming mode of Intel on your setup? kindly refer to the site below for details.

http://www.intel.com/support/network/sb/CS-030993.htm Network Connectivity — FAQs: VMDq, VLANs, and Teaming on Intel® Ethernet Adapters in Hyper-V*

hope this help.

[MHibb3]

That's not really a valid answer if I'm completely honest with you.

My questions were:

A) What is Intels Stance when mixing RSS and VMQ modes on different ports, on the same physical adapter?

B) Is splitting a VMQ enabled NIC team between ports from physical NICs supported by Intel when using the native Windows NIC Teaming in Server 2012 R2?

I guess now I should be asking whether or not these are adapters are even compatible at all with Windows NIC Teaming in 2012 R2???

Regards

Matt

[VincentT_T_Intel]

Hi MattJH, i'm currently checking on your 2 questions related to VMQ and RSS. For the mean time, would like to clarify if i captured your issue and setup correctly.

Issue:

Packet loss, high latency on VMs such as 5-6 lost pings request during live migration

Setup:

3 I350-T4 installed on 1 system and configured as one of the Hyper-V. 2 out of 3 I350-T4 configured as Native Win2k12 Teaming (Switch Independent/Dynamic distribution). Total of 4 ports in 1 team. 2 ports belongs to I350-T4 # 1 and 2 ports belongs to I350-T4 # 2

With VMQ enabled and processor core allocations. Remaining ports from I350-T4 1 and 2 were configured for ISCSI MPIO to access SAN with RSS enabled.

Troubleshooting done:

Turned off VMQ

Please share the output of the following

Get-NetAdapterVmq

Get-NetAdapterRss

[MHibb3]

Hi Vince

Yes that's correct. With VMQ enabled in conjunction with the native Windows Server 2012 R2 NIC Teaming, we see intermittent packet loss, lengthy timeouts when Live Migrating VMs. With VMQ disabled, things seem to behave as expected.

I can also confirm that out of the two quad port i350 adapters, two ports from each physical adapter are in a Switch Independent / Dynamic team (Sum-of-Queues mode) for the core VM Switch (4 interfaces), whilst the remaining 4 ports across the two physical adapters are ISCSI MPIO interfaces to the SAN. RSS is enabled on the ISCSI ports and VMQ is enabled on the VM Switch Team ports.

We are also allocating a single CPU core to each of the VMQ enabled VM Switch Team interfaces (separate cores for each interface with no overlap). Hyper-Threading is enabled on the CPUs, however HT Logical execution units are not being allocated to VMQ or RSS, only actual cores.

Here are the Get-NetAdapterRSS and Get-NetAdapterVMQ outputs as requested:

Kind Regards

Matt

[VincentT_T_Intel]

Hi MattJH,

Please try the version 17.4 driver, it should help with your issue.

https://downloadcenter.intel.com/Detail_Desc.aspx?DwnldID=21228 https://downloadcenter.intel.com/Detail_Desc.aspx?DwnldID=21228

Regards,

Vince

[HTank2]

Hey,

Can you help me with core allocation process. I have 4 pcie gen 2.0 slots. Each slot is having 2x 10G Flexi ports.

I am using E5-2680V2 (Ivy Bridge- EP) which has 10 cores and 2.8GHz.

I am familier with RSS but a little more description would be helpful.

Also want to mention we are using Linux.

Can i be able to implement VMQ on this platform in order to enhance the performance. I am not that much familier with VMQ.

Please help me with this situation ASAP.

Thanks,

Himanshu

connect: mailto:himanshutank75@gmail.com himanshutank75@gmail.com

skype: himanshu.tank3

[VincentT_T_Intel]

Hi Himanshu, RSS distributes the load of data processing to multiple CPUs, please share your linux platform and kernel version, also the model of Intel network adapter involved.

For VMQ, this is being implemented on Virtualized environment.

regards,

vince

[HTank2]

Can you get connected with me on skype or hangouts. It would be much better and fast to resolve the issue. As it is on high priority.

Skype: himanshu.tank3

mailto:himanshutank75@gmail.com himanshutank75@gmail.com

Thanks,

Himanshu

[VincentT_T_Intel]

Hi Himanshu, don't have skype or hangout apps on my system, for faster resolution, suggest to coordinate with your system manufacturer or Linux platform as they might have sample case studies / best practices using similar setup.

regards,

Vince