I217-LM - no RSS possible under Win 8.1 Pro x64 - driver setting without effect

[JJoha1]

Hi,

I'm using an http://www.asrockrack.com/general/productdetail.asp?Model=EPC612D8A-TB# Specifications ASRock Rack EPC612D8A-TB motherboard with two Intel onboard NICs (I210 (Ethernet 2) and I217-LM (Ethernet)), running Win 8.1 Pro x64 with all Windows Updates installed and the latest Intel Ethernet driver package (20.2.3001.0). My problem is with the I217-LM, according to the http://www.mouser.com/pdfdocs/i217brief.pdf Intel spec sheet it does support RSS (cp. figure on page 2).

In the I217-LM's advanced driver settings there is an option to enable and disable RSS. However Windows itself always says that the I217-LM is not RSS-capable.

The same option in the I210 driver options has the expected effect: With the PowerShell command get-SmbClientNetworkInterface you can see the RSS capability changing from True to False and vice versa.

Can anyone tell why the I217-LM is not getting the RSS feature?

Further system details:

CPU: Xeon E5-1620-V3

RAM: 2 x 16 GiB Crucial DDR4-2133 ECC

The motherboard does not have a later BIOS/UEFI release than the one installed.

Thank you very much for your help!

[AP16]

Well, it looks great. We have a NICs with useless RSS capability, because any real-life apps require something more. But some questions still present:

Yes, thats MSI, Intel NICs are already work in MSI mode - just install Intel driver. Considering Windows support for MSI, citing http://download.microsoft.com/download/0/0/B/00BE76AF-D340-4759-8ECD-C80BC53B6231/performance-tuning-guidelines-windows-server-2012.docx http://download.microsoft.com/download/0/0/B/00BE76AF-D340-4759-8ECD-C80BC53B6231/performance-tuning-guidelines-windows-… :

Message-Signaled Interrupts (MSI/MSI-X)

Network adapters that support MSI/MSI-X can target their interrupts to specific logical processors. If the adapters also support RSS, then a logical processor can be dedicated to servicing interrupts and deferred procedure calls (DPCs) for a given TCP connection. This preserves the cache locality of TCP structures and greatly improves performance.

Here is why MSIs are probably required by SMB-Multichannel, and that lost is in Intel drivers. For russians - youtube.com/watch?v=L5mrMR07Zkw - exactly my feelings.

[JJoha1]

Does the listing of the 82579LM in the MSI mode utility mean that this device should actually be capable of MSI?

[AP16]

IRQ number in Windows device manager definitely tells us about MSI mode:

[JJoha1]

Am I understanding it correctly that if a device's IRQ setting line has a negative number at the end it is running in MSI mode? I'm new at this and just read this:

http://forums.guru3d.com/showthread.php?t=378044 Windows: Line-Based vs. Message Signaled-Based Interrupts ... - Guru3D.com Forums

If so, I just checked a "RSS-light" I218 and it is actually running in MSI mode without being able to use RSS properly as described in this thread in length. MSI-X (only available with a PCIe 3.0 interface to the device?) doesn't seem necessary for this, I just checked my motherboard's specs and the I210 (where RSS is working properly) is only connected to the chipset via PCIe 1.0 x1.

Is that RSS-not-working-because-of-not-supporting-MSI answer technical nonsense?

This thread is a gift that keeps on giving

[AP16]

Am I understanding it correctly that if a device's IRQ setting line has a negative number at the end it is running in MSI mode?

Yep.

Is that RSS-not-working-because-of-not-supporting-MSI answer technical nonsense?

May be. There is a need of kinda link between MSI and RSS in NDIS driver, so the questions is: are Intel cut driver for desktop adapters or just nevermind to finish it properly?

[JJoha1]

Maybe ThomasHi can check this on his system and if his NIC is also running in MSI mode respond to his support case that the Intel technician seems to have something mixed up...

Why does there have to be a link between RSS and MSI in these cases where NICs that (should) support RSS but don't distribute the load in reality? I get that this might be a real requirement in general for RSS but all problematic NICs I can access at the moment actually already run in MSI mode.

(But by all means I'm not defending Intel's drivers)

[JHend8]

As far as I can tell in my research, enabling MSI just means that MS just puts the device in the MSI enabled bucket, interrupts without the extra message signalling come in like normal on their line interrupt. Then, it's up to the driver to signal that it wants more, and how many. Example, if you look at the datasheet on the I210/211, you will see that it can signal for 5 MSI based interrupts. Here is a machine with an I217-V (does not support MSI), and I211 (does support MSI), and a Broadcom 10G NIC. The I217 is set to enable MSI, but that doesn't do anything because it's just using the default (the catch all for it's line interrupt). The I211 registers for 5 additional MSI's, which is why it shows 6 total. Setting something for MSI just enables the possbiliity of it from the OS's level. If the device isn't capable of it (either because it was omitted from the driver, or because the chipset really doesn't include that feature, no matter what the driver does), then the net effect is nothing.

[JJoha1]

Thank you for that easy-to-follow explanation. Then let's take the lack of MSI stuff as "god-given" and come back to the question: In what scenario does RSS on these chips actually provide any advantage?

[JHend8]

For that question, I will provide my most helpful answer yet...

...

Basically, I struggle with that question myself. But, *I THINK* what it means is that there are multiple queues to receive data in so that, in between interrupt cycles, more data can be received. But, in my tests, I can't pin it to anything I can perceive empirically. So, you can handle more incoming data, but the same interrupt still services all of the queues. Once combined with MSI, you can have individual MSI's assosciated with each queue, which is where the big win comes in as different CPU's can handle the different interrupts.

[AP16]

Not so easy:http://www.intel.com/content/www/us/en/embedded/products/networking/82574l-gbe-controller-datasheet.html Intel® 82574 Gigabit Ethernet Controller Family: Datasheet

The 82574L supports the following interrupt modes: • PCI legacy interrupts • PCI MSI - Message Signaled Interrupts • PCI MSI-X - Extended Message Signaled Interrupts

Five MSI-X interrupt vectors are provided (calculated based on four vectors for queues and one vector for other causes). The requested number of vectors is loaded from the MSI_X_N fields in the EEPROM into the PCIe MSI-X capability structure of the function.

But:

[AP16]

Also, on page 2 of this thread we seen WORKING RSS on Relatek chip. Yes, Realtek supports MSI/X - http://www.realtek.com.tw/products/productsView.aspx?Langid=1&PFid=5&Level=5&Conn=4&ProdID=239 Realtek. But in device manager ThomasHi will likely see only one interrupt line, like shown in https://forum-en.msi.com/index.php?topic=161643.0 Z68A-GD65 (B3) ISR/DPC difference on LAN/Wireless example (server OS BTW).

[JJoha1]

I hope I'm not interpreting it wrong but your 82574L's cable doesn't seem to be plugged in or the connection is deactivated ("Ethernet 2" and "Speed: 0 bps"). In my cases where RSS is actually working the IndirectionTable is only NOT empty if a gigabit connection is established.

[JHend8]

Agree with JBBG... plug it into something so we have apples and oranges. Also, are you using anything else on that adapter? Teaming (Microsoft or Intel)? VLANs? Hyper-V switching? Just wanting to double check...

[JStud]

Hi folks:

I've been following this thread for awhile now and wanted to make sure it stays active. We have a room full of Windows 10 Pro workstations with X540-T2 NICs talking to a SuperMicro Windows Server 2012 R2 file server with an embedded 4 x X540-T2 NIC. The workstations (not the server) are demonstrating the *exact* same behavior described in this thread ("RSS Capable" shows as False in PowerShell even though it's enabled, empty indirection tables, very poor network throughput, etc.). We are using Intel's latest available drivers and Windows 10 has been patched and is up to date.

We spent a lot of money upgrading the workstations with these Intel NICs to use SMB3 multichannel over 10G via RSS. I have seen it work -- once -- on a Windows 8.1 workstation, and yes, it was fast. But this is Intel's latest and greatest RJ-45 adapter (with a great reputation) and I'm surprised that they're not making a more concerted effort to fix this problem for their enterprise customers.

I'd be happy to post any settings, screen shots, etc. that the community thinks might help diagnose this problem further and get it fixed. Thanks.

[JJoha1]

Thank you for joining the fun!

I wish I knew what was wrong with your setup but your machines shouldn't behave like the NICs mentioned in this thread since the X540 supports "real" RSS with MSI meaning the IndirectionTable should not be empty when a connection >= 1 Gbps is established.

Regarding SMB3 Multichannel I couldn't get it to work either: I've tried dedicated PCIe I350 cards with two and four Gbps ports that support RSS and MSI (and RSS is really working), installed on quad-core systems with Windows 8.1 x64 (latest Windows updates and Intel drivers). If I directly connect these systems to each other using two copper RJ-45 cables (to avoid the switch being the issue) I'm stuck at 113 MB/s at transfering large files over SMB meaning SMB3 Multichannel is not active.

All guides I've found online so far state that "SMB3 Multichannel is so simple, you just have to connect a capable server and client and it will automatically be active".

Since you come from an enterprise background could you maybe ask Intel to supply a SMB3 Multichannel check-list to follow to find the culprit that prevents it from working properly?

[JStud]

We have directly connected the X540-T2 in one of the workstations using two cables to two ports of the X540-T2 on the server and set up ram disks on each end to eliminate the switch and disk speed as potential bottlenecks. The workstations still exhibit the behavior in this thread (though the server reports RSS Capable as "True" and has populated indirection tables) and speed often drops below Gigabit speeds during these test transfers. These are all beefy workstations with recent i7s and at least 16 GB of RAM, and in every case the X540-T2 is seated in a PCIe3.0 slot (and reports as such from PowerShell). Again, I've seen SMB3 multichannel work on a Windows 8.1 machine against this same server and it was very fast.

One other data point: under the current drivers for the X540-T2, I see no drivers for Windows 10: just "Administrative Tools for Network Adapters":

https://downloadcenter.intel.com/search?keyword=intel%C2%AE+ethernet+converged++network+adapter+x540-t2 Drivers & Software

We have yet to open a support ticket as I was hoping this would have been resolved by now -- it's obvious Intel is aware of the issue -- but it's probably time.

[JJoha1]

The "only" trouble with Windows 10, as far as I know, consists of teaming and VLANs - both should not affect SMB Multichannel since the NICs operate individually here.

If you have the opportunity of experimentation (not productively used setups with recent backups) you can try the latest 20.5 driver package that supports 10 Gbps adapters and has NDIS65 (Windows 10) contents:

http://www.station-drivers.com/index.php?option=com_remository&Itemid=353&func=fileinfo&id=1961&lang=en http://www.station-drivers.com/index.php?option=com_remository&Itemid=353&func=fileinfo&id=1961&lang=en

Maybe these help in your case.

[JJoha1]

Coming back to the original RSS topic - it could be completely resolved:

- The I217 LM (and the other "entry-level" NICs) support RSS with a queue length of 2

- They do not support Message Signaled Interrupts (MSI-X) which is required to allow separate receive queues to be processed by separate CPU Cores

- To have RSS active with these NICs you have to have a single core CPU setup

- There haven't been any single core CPU releases for years so the advertised RSS function of these NICs is a little bit like giving a certain finger to customers.

[THatt2]

UPDATE

RSS is working in my case now on SMB3 share!

In my case it works now. I am working on a low power HCI with 3 x TP5 W2k16 Datacenter (Cluster with Storage Space Direct SAN + Hyper-V HA with 3 Nodes).

Today I noticed (at random) in taskman that all my nics are working on a file transfer together. I tried: Get-SmbClientNetworkInterface | ft *name*, rss*

Get-SmbClientNetworkInterface | ft *name*, rss*

RTL1 True

RTL2 True

vEthernet (i218) True

On the other nodes that wasn't.

RTL1 True

RTL2 True

i218 False

On the working node is hyper-v installed with i218 as sharing NIC!

After installing the same on my cluster node 2 the i218 also shows rss cabable = TRUE

It may be that this is also working in win10.