I211/I217-V Windows 10 LACP teaming fails

[FKurt]

Hello,

after the update to Windows 10 (x64, Build 10240) the creation of a teaming group (static or IEEE802.3ad) with a I211+I217-V NIC fails.

Drivers have been upgraded to the latest version available and multiple reinstallations with reboots din't help either. Whenever the group creation wizzard is used and a groupname (several tried), the adapters and LACP have been selected, a Windows pop-up appears to tell me group creation has failed.

However the Windows Device Manager shows a newly created "Intel Advanced Network Services Virtual Adapter", so some kind of configuration seems to get done.

Using Windows 7 SP1 x64 the exact same setup worked flawlessly for months, so Win10/the driver are the likely culprit.

Is anyone experiencing similar problems and/or is this a known bug? Feedback on this issue is greatly appreciated.

Thanks in advance!

Kind regards,

Famaku

[idata]

A few things:

- Intel isn't going to drop the price of their hardware adapters because of a software issue on a client OS. The feature set works 100% on Windows 7, so, by all means, use it on Windows 7 to your heart's content. This would be akin to saying HP should suddenly drop the price of a printer they have on the market because the scanning feature doesn't work any longer on Windows 10. It's not going to happen, nor should it happen.

- Great, you found somewhere buried in the documentation (that was last updated in March) showing "Windows 10" and "Teaming" on the same page. It's obviously out of date information and someone should get around to updating it. I suppose we agree there, I guess?

- If Microsoft is actively blocking this (hypothetically, stick with me here for a minute), is it not reasonable to assume (don't confuse this with being the right approach, mind you) that Microsoft wouldn't want to get in the middle of this and would prefer to just wait this out and hope that it eventually fades away? Again, if this is the case I think they *should* just out and say that, no doubt. But is it really that shocking to you this isn't high on their priority list to announce if they have no intention of this sort of functionality being supported within Windows 10 any longer (for whatever stupid reason that may be)?

- Have you actually taken a look at the Microsoft thread I linked earlier? If you haven't, take a look, you might learn something. Or not, whatever you want.

- There is this vibe you're putting off that I simply don't get. Seems to me like you're working under the assumption that Carl_Wilson is just somehow not giving you the "real deal" here or maybe just isn't working hard enough on this issue. If that's your position, do you think he's just a glutton for punishment and that he provides these periodic updates because it's good for his blood pressure or something? If he and/or Intel really didn't give two hoots about this issue, why would he be dragging it on? Why wouldn't Intel simply lock this thread and throw away the key. Seriously, why would he respond at all anymore and give any sort of updates about this? Just to screw with everyone? Doesn't make sense.

PS: I originally suggested you file a lawsuit because you seem pretty sure that Intel has some exposure here and I just want to make sure you cash in on your big settlement do you can go buy a bunch of Realtek cards and relish in their exceptional performance. Who knows, if you have 10 Realtek cards you might be able to agg enough together to get a full 1GbE worth of traffic through them!

[CARL_W_Intel]

The failure is related to setup/tear-down of the required miniport driver that allows teaming and multiple VLANs to function. This setup/teardown is 100% in the domain of the core OS. In most cases, this works fine (say ~90% of the time). So, as the Product Marketing guy, my first response would be 'well that's fine, lets enable it and then document how to manually work around the 10% of the time it fails'. That sounds reasonable and happens often while in the process of debugging a full fix is ongoing. Trust me, this was the first statement I made to our software development team.

So - what would keep me from pushing down that path? The one work around that is not acceptable. In the ~10% of the time the teardown fails, it requires a complete OS reinstall to correct. . We have explored all options - and this is the only one that puts it back into its original state. This, as you can imagine, is not acceptable. I can't image 'hey, that is a reasonable fix!' is a response I would expect to hear from anyone. Thus the reason to not enable it in the 21.1 release.

That's about as transparent as I can get.

I normally don't get involved in Community posts. I respond on this thread because this is an important feature for me personally, and an important feature for our customers. I'm the guy standing in the cubicles of the engineers who work on this code and getting answers from the management team on exactly what is going on. This team shares my passion and commitment to the feature.

And for the record - I'm already on high blood pressure medication (not related to my job, which i enjoy immensely).

[CUllr]

In the ~10% of the time the teardown fails, it requires a complete OS reinstall to correct.

That is a bit ugly, yes. Now that I think about it, way back in 10240 land, just after I installed Windows 10 the first time, I remember I had to reinstall because something had gone wrong. That may well have been with an early 20.x driver that had not been told it did not work.

That's about as transparent as I can get.

Pity. This would make a great blog post, if you could get (and allow, of course) one of the developers to go into the details. I think Windows driver development is best described as a black art, and any illumination would be highly welcome.

And for the record - I'm already on high blood pressure medication (not related to my job, which i enjoy immensely).

Be sure to keep up your health. There's always life after work.

Thanks for sharing what you could; we'll just have to find more patience somewhere.

[TED]

Hi Carl,

I sincerely appreciate your honesty and for being so transparent. It's hard to find a star such as yourself. I really wish Microsoft could be as honest and tell us what their intentions are.

But anyway with what you've said, now we are in a more realistic situation and I'm not in the pointing finger business, it is what it is.

So I guess, here are the alternatives:

a- Downgrade to Hateful 8

b- Install (and remain in) Win 10 build 10240

+

c- Keep sending daily reminders to Microsoft to quit screwing around with their loyal customers and enable the features that we IT pros need.

I can't afford to not run my business without VLANs enabled... I will probably downgrade to the much hated 8.1 and have the functionality I need back. Will also totally NOT recommend the upgrade yet to Win10 in my organization.

Until next update.

[idata]

Thank you Carl_Wilson for the candor. I for one can respect the position you are in and certainly appreciate the detail you were able to provide. It seems pretty clear to me that if Realtek is somehow able to make this work, it's on shaky ground and I for one respect the fact that Intel isn't willing to publicly release a driver package that may cause me to have to reinstall Windows in certain cases. Given that you've identified the issue is inherent to the OS stack, it actually speaks volumes about Intel's commitment to their customers that they DON'T release a driver package into the wild that could cause this (I mean, could you imagine the forum posts on THAT quagmire from people who unknowingly BLEW UP their machines as a result of installing the driver package? What a nightmare that would be). My hope is that folks like RocketTech will appreciate it as well (not holding my breath, but hey, you never know).

If I had one more question that I'm hoping you can answer for the moment with as much transparency as you are able: Should we consider this issue dead? In other words, is Intel genuinely continue to work on this with Microsoft, or have you guys pretty much written this off?

I certainly won't be happy if that's the case, but what would be worse is to hang on for weeks and months on the "hope" something will change when it's simply not going to happen.

If you're hopping mad about this, and plan on sticking on this issue until Microsoft does whatever they need to do, I'll hang on for the ride. But if there really is no change of this changing, I'd appreciate at least being given that heads up.

PS: To RocketTech's point, you probably should have someone update the documentation that Windows 10 doesn't support teaming and VLANs in more than just the download page, if that's going to be how this ends up.

[CUllr]

It seems pretty clear to me that if Realtek is somehow able to make this work, it's on shaky ground

Not necessarily; Realtek has separate drivers for teaming and VLAN, with Intel it's apparently one and the same. Just speculating here, but if the bug is related to having a single driver attached to two devices (as for teaming), then VLAN alone might actually work fine. Also, I keep pointing out (although none of my posts on that topic has yet come out of "moderation") that in the latest version of Realtek's driver and diagnostic tool, the teaming option is gone.

[CARL_W_Intel]

Does Intel consider this issue dead? Absolutely Not!. This is very much alive. To remove this feature from our driver package requires my express approval. If you were to ask 'who do I escalate this to at Intel' - that person would be me. These are my products and my responsibilities within Intel.

[JWeem1]

I think that we all would like to have the driver release at 90% success rate, as a "Beta" release and not an official supported release. Even if we have to download this version from a different location then the official locations, and is only available via this thread (almost everyone watching this thread would want it).

One thing that I am personally finding an issue with is the fact that you have a 90% success rate, and the 10% that fail have a work around of reinstalling the OS to fix it and this is a Microsoft issue? With the re installation of the OS is using Microsoft software and then installing the driver and it works, doesn't point to an issue with Microsoft software. I by no means am trying to state that Microsoft code doesn't have issues, it means that Intel just has not been able to figure out a way to achieve 100% success rate as of yet.

[VInga]

The driver is not on their site yet.

Here is the Realtek driver from their FTP. Put it up on my onedrive should you need it.

https://1drv.ms/u/s!AgL20NxXcHEIhplLgwd5QXhYdKTGHQ Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.

[idata]

VinnieI,

Can you confirm both VLAN *and* teaming is working with the Realtek driver package you were given?

Even with the VERY latest Win 10 update, v10.0.14905.1000 (Source: https://en.wikipedia.org/wiki/Windows_10_version_history Windows 10 version history - Wikipedia, the free encyclopedia)?

[VInga]

Indeed it all works as it should.

[CUllr]

The driver is not on their site yet.

Quite the opposite, it isn't there anymore. On the FTP site you posted the link to, there is now a package of the diagnostic tool dated 2016-08-08 (yours is 2015-08-20), and this new version is minus the teaming option. Or at least, it doesn't show up for the dual-8111 NIC I have tried it with. The actual teaming driver still gets installed, but cannot be configured afterwards.

Assuming Realtek had a reason to withdraw the feature, it appears plausible that that reason is the same, or close to, as Intel's. So before I'd want to trust that driver (which, by the way, is written to Windows 7 APIs, NDIS 6.2, not necessarily a bad thing), I would like to see that it does not kill the system over, say, a couple dozen cycles of install, configure, exchange half of a lot of GiBs, reboot, exchange the other half, uninstall, reboot again, da capo. Anyone have an idea how to automate that?

[idata]

w33mhz wrote:

I think that we all would like to have the driver release at 90% success rate, as a "Beta" release and not an official supported release. Even if we have to download this version from a different location then the official locations, and is only available via this thread (almost everyone watching this thread would want it).

One thing that I am personally finding an issue with is the fact that you have a 90% success rate, and the 10% that fail have a work around of reinstalling the OS to fix it and this is a Microsoft issue? With the re installation of the OS is using Microsoft software and then installing the driver and it works, doesn't point to an issue with Microsoft software. I by no means am trying to state that Microsoft code doesn't have issues, it means that Intel just has not been able to figure out a way to achieve 100% success rate as of yet.

Agree completely with the "Beta" release idea. I would definitely give it a try even if it meant re-installing OS. We then could test claims about the 90/10 rule.

[VInga]

Hey, you do what you need to do. Few pages back you see it works for me.

Be be worried or not, I'm just saying mine works for teaming and VLANs and has been since I posted it.

My multi pixel cameras and synologys have been chatting perfectly. Not a single blip and there have been reboots.

Id love nothing more than to swap back to my Intel card but I literral can't until teaming and VLANs are working.

[idata]

Carl_Wilson,

What is your comfort level with releasing COMPLETELY "non-supported" version of ANS that has the capability enabled in some non-standard channel (i.e., FTP download, OneDrive / Box / Dropbox share, etc) for users who are willing to accept the risk in order to have the feature set?

While I certainly understand on an "official release" where folks are downloading and installing at will, and in 99.99% of cases not being the slightest bit aware of this thread and the issues with this under Windows 10, you'd definitely want to make it as stable as possible without any sort of surprises.

But, for folks that understand exactly what they are doing and the risks of doing it... If was explicitly distributed as "as-is / your mileage may vary / expect zero official support"... Would that be so bad?

Joey

[CUllr]

I don't think that would be a very good idea for Intel.

Scenario 1: I try this unsupported beta driver, it hoses my system, and after reinstalling and trying a second time, this time successfully, I blog: "I tried this beta driver from Intel for my NIC. They said it could irretrievably break the OS. Yes, it can." What appears all over the 'net the next day (let's pretend I had that reach)? "Intel releases buggy driver, systems broken irretrievably." Bad press for Intel, even though they did nothing wrong.

Scenario 2: See above through "Yes, it can"; then someone asks me to send them the driver. I do, including all the dire warnings. Somehow, it escapes into the wild, and hundreds of people install it who did not get the dire warnings. The result is predictable, and Intel gets blamed. Bad press (and possibly lawsuits) for Intel, even though they did nothing wrong. (Remember: Nobody reads EULAs, even if they come in large, red letters.)

To be able to release the current state of the driver, even to a select group of testers, I think Intel would have to require formal NDAs. Since they already know what's wrong with it (albeit not how to fix it), and presumably have plenty of testing resources in-house, there is no incentive for them to go to all that trouble.

[idata]

chrullrich I see your point.

PS: I, personally, have no problem with signing an NDA if that gets me the software. I sort of get the feeling others might be cool with that as well (Intel's coolness with such an idea however...).

[TED]

I find it to be a very good idea. I suppose I could live with Beta Drivers. With a very well warning from intel to end users, this is doable imho. Although we all seem to understand the challenges Intel is facing here, I would not mind trying out these drivers. So Carl, would that something that could be arranged?

[idata]

Wow... I've engaged Microsoft through a Premier support request and I am absolutely, 100% satisfied, that it's Microsoft who is stonewalling this. No question about it.

It would probably be inappropriate for me to copy any paste directly what I was privy to, but suffice it to say if there was a "bug" in Windows 10 it's not that native LBFO has stopped working... The bug, in Microsoft's view, was that native LBFO was ever working in the first place, and Microsoft "fixed the bug" by *really* breaking it in subsequent releases / updates to Windows 10.

The door was left open for "third party" developers to have LBFO support via their utilities, such as Intel ANS, and that Microsoft would provide "developer support" to troubleshoot issues they run into... But the vibe I got reading between the lines from what I was seeing from support via copy / pastes of E-Mails from the Product Groups involved was that "if you want LBFO, use Server SKUs". It's NOT (and they actually used capital letters for emphasis there) supported in client SKUs - and if it ever worked natively in Windows 10 for a short time that was unintentional and absolutely not by design.

So, looks like we can pretty much kiss Windows 10 "native" LBFO goodbye, unless they have some massive change of heart. So the only hope, if Intel keeps on it, is that it may eventually work out with ANS. But probably not because Microsoft wants it to... More like because they feel like, as good development partners, they "have to".

Kind of like a kid being told he needs to eat his peas. He doesn't like it, he'll grit his teeth and do it, and it might take him 3 hours to finish dinner.

Pretty blown away by the response from Microsoft, to be honest. I'm challenging it and telling our contacts that, basically, the decision is crap. So we'll see if my little voice in this has any impact at all (not holding my breath).

But, yeah, it's Microsoft on this one. No doubt about it.

PS: Sorry, Carl, I just saw a little bit of the hell you guys are going through. Keep up the good fight!

[MVass]

As usual these b*s in Microsoft screw you as much as they can. And this is after you pay them a ton of money in license fees. So I guess - we have to abandon Windows 10 and move back to Server 2012R2 or wait for Server 2016.

What normal company in the world would do this to their customers.

Regards

Miroslav

[ALias]

Hi Carl,

Thanks for the feedback. I've been on the phone with Microsoft recently, and they are very aware of the problem. I didn't get the impression that this had any priority at all. As far as I understood, no timetable was set for a solution. If your team is depending on Microsoft, I have serious doubt about this getting fixed at all any time soon. What I understand from your explanation, is that Intel is not capable of finding a solution without using Microsoft assistance.

Armed with that knowledge, we've decided to move away from Microsoft as a desktop solution for our upcoming projects. There are several Linux alternatives these days that can substitute as a workable solution, including replacements for MS Office (great example of how Microsoft shoots itself in the foot on several levels). Only two machines would require Windows for compatibility reasons (Photoshop etc.), so we're not too saddened by this OS switch.

Either way, a while ago I noticed how Virtual Box (Oracle) developers ran into miniport problems as well while Microsoft was moving down Customer Destruction Street. I believe the problem for them was also introduced after build 10240. They tackled the problem somehow. Maybe you guys at Intel could reach out to Oracle/Virtual Box to see if there are similarities regarding this problem.

Good luck out there, and again thank you for the feedback. It's highly appreciated.

Andrew