Disabling EEE on 82566DM-2 Gbe controller.

[JMarx1]

I have a Sun Microsystems Ultra24 Workstation with the latest available BIOS (1.53) - It runs on Devuan ASCII - 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux with the system fully updated.

The Ultra24 has an on-board Intel 82566DM-2 Gigabit (rev 02) Ethernet Controller and the installed driver module is the latest available from Intel: 3.4.2.1-NAPI.

Because of a random freeze on shutdown problem I'm trying to solve, I need to disable the controller's tx-lpi timer.

Description of the problem:

Randomly, on shutdown the rig will freeze with all fans going at full blast with this output on the TTY1 screen:

Devuan GNU/Linux ascii devuan tty1 devuan login: [483.367459] EXT-fs (sdc1): re-mounted. Opts: (null) [485.772216] e1000e: eth0 NIC Link is Down [485.776885] kvm: exiting hardware virtualization [485.777756] sd 9:0:3:0: [sdf] Synchronizing SCSI cache [485.778154] sd 9:0:2:0: [sdf] Synchronizing SCSI cache [485.781519] e1000e: EEE TX LPI TIMER: 00000000 [485.785219] ACPI: Preparing to enter sleep state S5 [485.868007] reboot: Power down

From [485.868007] onwards, the only way out is a hard shutdown.

I have attempted to disable the EEE settings with the ethtool (both v. 4.8 and v. 4.19) utility but I have not been able to:

[root@devuan groucho]# ethtool --set-eee eth0 tx-lpi off Cannot get EEE settings: Operation not supported [root@devuan groucho]#

It's obvious from the screen output that the tx timer is active so the question at hand is: how to disable it?

Please advise.

Thanks in advance.

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for posting in Intel Ethernet Communities. We will look into the support of EEE for the 82566DM and driver, and update you soon. Have you contacted Sun (Oracle) about this issue to see if they have any knowledge of it? Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support

[JMarx1]

Good morning:

[quote]

We will look into the support of EEE for the 82566DM and driver, and update you soon.

[/quote]

Thank you.

[quote]

Have you contacted Sun (Oracle)

[/quote]

No.

My rig is from 2008 and obviously out of warranty.

In addition to that, access to anything related to Sun Microsystems hardware/software purchased before the takeover of the OEM by Oracle Corporation, including tech support for previously existing bugs/problems has been withdrawn and is subject to a very expensive (ie: absurdly expensive) service contract which is absolutely out of the question.

The latest available BIOS was as buggy as the previous version and to my knowledge a fix was never issued, leaving all owners of an Ultra24 out in the cold.

FYI

The shutdown problem I am experiencing is twofold.

On shutdown, the rig would do one of three things:

1. shut down properly

2. shut down properly and after about 5s. reboot start a reboot and freeze

3. freeze during the shutdown like shown in my OP.

The first part of the problem (#2) is documented in a few places on the web.

See:

[url]https://wiki.archlinux.org/index.php/Wake-on-LAN[/url]

and

[url]https://bbs.archlinux.org/viewtopic.php?id=173648[/url]

It seems to be related to buggy BIOSes and the working solution is to disable WoL.

I have disabled WoL (which is enabled by default by the driver but don't need) with the ethtool utility and a line in /etc/rc.local:

~$ cat /etc/rc.local #!/bin/sh -e # rc.local # This script is executed at the end of each multiuser runlevel. # Make sure that the script will "exit 0" on success or any other # value on error. # In order to enable or disable this script just change the execution # bits. # By default this script does nothing.   /sbin/ethtool -s eth0 wol d exit 0 devuan:~$    

Since I disabled WoL, (#2) does not happen anymore.

The second part of the problem (#3) seems to be related to the EEE Tx LPI Timer and the solution is most probably to disable it as I have not been able to find any source or script in my installation that could be triggering the entering into Sleep State 5, power management is 100% disabled and suspend state in BIOS is set to S1(POS).

Thank you for taking the time to write back.

Have a very Merry Xmas.

Cheers,

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Please let us know if the freeze during shutdown issue is corrected by disabling the controller. We will continue to look into the EEE support on this controller. Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support

[JMarx1]

Hello DanielD:

[quote]

... if the freeze during shutdown issue is corrected by disbling ...

[/quote]

I really cannot say.

Much to my chagrin, the BIOS (v. 1.56, latest available) in my Sun Microsystems Ultra 24 Workstation's motherboard does not have a working option for doing anything with the Gbe controller.

The Gbe option in the BIOS' screen is greyed out and unresponsive ie: inaccessible to the Admin.

And it is not because of the lack of administrative credentials as all passwords (both user and admin) to the BIOS are disabled.

I'm at a complete loss with respect to this rather strange instrumentation of the workstation's BIOS settings and have not been able to find any information as to how to change this particular feature.

Thank you for taking the time to write back.

Have a Happy New Year.

Cheers,

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for the reply. We will continue to investigate the issue and update you soon. Let us know if you have any other questions. Best regards, Daniel D Intel Customer Support

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for your patience. Please try removing the e1000e module prior to shut down. Does the system still freeze? Have you also tried using this command: ethtool --set-eee enp1s0 eee off "another option to turn off EEE" to turn off EEE? Please let us know if you have any questions. Best regards, Daniel D Intel Customer Support

[JMarx1]

Hello:

[quote]

Please try removing the e1000e module prior to shut down.

[/quote]

Please see my first post again.

This is a random occurrence (ie: does not happen regularly) and I have not been able to reproduce it, so I'd have to remove the module permanently and wait for it to happen.

From the screen output at the, it would seem that the problem is clearly related to the driver module trying to send the workstation to S5 ...

[485.781519] e1000e: EEE TX LPI TIMER: 00000000 [485.785219] ACPI: Preparing to enter sleep state S5

... so I'd think it is reasonable to expect that with the module inactive ie: not loaded, the shutdown process would be without problems.

Lke I mentioned in the same post, my use of ethtool to set EEE to 'off' results in this screen output ...

[root@devuan groucho]# ethtool --set-eee eth0 tx-lpi off Cannot get EEE settings: Operation not supported [root@devuan groucho]#

... so, at least in this specific case, it does not work.

Thanks for your input.

Cheers.

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for the reply. We will keep in mind that this issue cannot be reproduced due to the infrequency of occurrence. Has the "freezing during shutdown" issue occurred more than one time, and did it occur before and/or after solving the "restarting then freezing" issue by disabling WOL? Please let us know if a status is displayed for "ethtool --show-eee eth0". Let us know if anything changes or any questions come up. Best regards, Daniel D Intel Customer Support

[JMarx1]

Hello:

[quote]

... cannot be reproduced due to the infrequency of occurrence.

[/quote]

If it happened at every shutdown instance I'd just have more samples for analysis.

But it would not necessarily allow me to reproduce it.

[quote]

... "freezing during shutdown" issue occurred more than one time ...

[/quote]

Yes, many times.

[quote]

... before and/or after solving the "restarting then freezing" issue by disabling WOL?

[/quote]

Both before and after disabling WoL.

[quote]

... for "ethtool --show-eee eth0"

[/quote]

[root@devuan]# ethtool --show-eee eth0 Cannot get EEE settings: Operation not supported [root@devuan]#

Same as with "ethtool --show-eee".

[quote]

Let us know if anything changes.

[/quote]

I really don't expect anything to change unless there's an update.

But the question remains: how can I disable EEE in the e1000e driver?

Thanks a lot for your input.

Best regards,

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for the information. Please allow us to continue investigating this issue. We will update you as soon as possible. If any other questions come up please let us know. Best regards, Daniel D Intel Customer Support

[JMarx1]

Good morning:

[quote]

Please allow us to continue investigating this issue.

[/quote]

Yes, of course ...

But if I may say so, I am rather confused by the need for continuous investigating.

I mean, my post (in a specific Intel tech support forum) is related to Intel designed and manufactured hardware, working under a platform (Linux devuan 4.9.0-8-amd64) for which Intel wrote and published a driver supported as a (proprietary) loadable module and the question at hand is directly related to configuring the hardware through the mentioned module.

ie: how can I go about disabling the driver's EEE feature.

How is it that this needs investigation of any sort?

Are the driver features and configuration procedures not properly documented?

In any case, thank you for your efforts.

Best,

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for waiting while we look for more information to disable EEE. We can only provide best effort support for the 82566DM, as it is a discontinued controller and integrated by an OEM. Support resources are no longer readily available, and the OEM may make changes to facilitate board design. In general this type of product is best supported by the board manufacturer. We understand from your previous post that this is not an option. Please see the following for more information: https://www.intel.com/content/www/xa/en/support/articles/000005644/network-and-i-o/ethernet-products.html You may try the following to disable EEE "modprobe e1000e EEE=0" from the readme of the e1000e driver https://downloadmirror.intel.com/15817/eng/readme.txt We would also recommend checking with the support team for the e1000e drivers, as they would have further information on the driver features and module. Please see: https://sourceforge.net/projects/e1000/ Please inform us if the modprobe command works, or you receive a solution from the driver support team. Best regards, Daniel D Intel Customer Support

[JMarx1]

Good morning:

[quote]

... as it is a discontinued controller and integrated by an OEM.

[/quote]

I see ...

So, in all the years before it was discontinued, this issue never came up?

ie: disabling EEE on a 82566DM with this Intel driver was never an issue?

[quote]

... try the following to disable EEE "modprobe e1000e EEE=0" ...

... recommend checking with the support team for the e1000e drivers ...

[/quote]

As a matter of fact, that was exactly what I did before coming to this forum.

My post from 2018-12-19 has gone ignored since that date.

See here:

https://sourceforge.net/p/e1000/bugs/635/

I decided to post in this forum to see if I could get a straightforward answer in spite of my (more experienced) colleagues insisting that I would be wasting my time.

But I was adamant: after all, it is Intel hardware with Intel drivers developed for a specific platform (Linux).

Much to my chagrin, their being right in their appreciations cost me the wager they challenged me with if I was unable to get a solution here: a lunch for the three of them.

I will continue to look for an answer elsewhere and when I find it, I'll post it here so it will be of use to anyone with the same problem.

Please do not take this as an opinion of you or of your commitment to the task: you can only do so much with the limited data/information/training you have been given to work with.

Best,

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for the reply. We would recommend following up with the sourceforge ticket again, as it seems they have not replied to the other tickets recently. We will continue to check on our end. We do not have any documentation of issues involving the 82566DM and EEE, or issues similar to the issue you are experiencing. It is possible that issues encountered were resolved on Sun or Oracle's end. Best regards, Daniel D Intel Customer Support

[JMarx1]

Good morning:

[quote]

... have any documentation of issues involving the 82566DM and EEE ...

[/quote]

I see.

What I really cannot fathom is Intel Tech Support not having have any documentation/knowledge on how to disable EEE in the e1000e driver, a setting that (amazingly enough) is enabled by default in the driver module and inaccessible via the Intel sanctioned ethtool utility.

Being able disable EEE is/should be an option and as you surely understood, by itself has nothing to do with the issue I put forth: read my OP again and note that I asked for instructions to on how to disable EEE to troubleshoot a random freeze on shutdown problem I'm trying to solve and that I was fully aware that it's occurrence is most probably due to a buggy Sun Microsystems BIOS and not an Intel related issue.

I used to think that undocumented software/hardware belonged to the realm of lower tier IT companies or start-ups.

Not anymore.

JHM

[Daniel_D_Intel1]

Hello Jmarx1, Thank you for the reply. We typically turn off EEE using ethtool. Another method to do this is through modprobe as described in the readme for e1000e (https://downloadmirror.intel.com/15817/eng/readme.txt). It is not normal that EEE cannot be turned off with the previous methods, so it may be a change on the OEM end, and we are not aware of the modifications made by Sun. We strongly recommend to check with them on the root cause of your issue. Best regards, Daniel D Intel Customer Support

[JMarx11]

Hello:

It has taken me a while but I finally found the information I needed.

I think you may find it useful should anyone else need assistance with this.

As I have posted previously, for whatever reason, the Intel 82566DM-2 on board controller in my Sun Microsystems Ultra 24 workstation does not allow the ethtool (v4.19) utility to access or modify EEE settings.

Invariably, the output is Cannot get EEE settings: Operation not supported

Your suggestion of removing the e1000e module before shutdown does not solve the problem as the EEE TX LPI timer is still active even after the module has been removed.

A line to that effect showing up in the tty1 printout on shutdown would be evidence of that and it happens whether the e1000e module is removed via rmmod or modprobe.

What seems to be happening is that the module has indeed been removed but somehow the EEE TX LPI timer remains active.

Maybe waiting for an answer from another device? No idea.

The only way I have found to disable EEE on the 82566DM-2 on board controller is via a stanza added to the kernel command line :  igb.EEE=0

When disabled in this manner, the tty1 output does not contain any reference to either the status of the e1000e's EEE TX LPI timer.

eg.

With the e1000e module removed:

Devuan GNU/Linux ascii devuan tty1
devuan login: [483.367459] EXT-fs (sdc1): re-mounted. Opts: (null)
[485.772216] e1000e: eth0 NIC Link is Down
[485.776885] kvm: exiting hardware virtualization
[485.777756] sd 9:0:3:0: [sdf] Synchronizing SCSI cache
[485.778154] sd 9:0:2:0: [sdf] Synchronizing SCSI cache
[485.781519] e1000e: EEE TX LPI TIMER: 00000000
[485.785219] ACPI: Preparing to enter sleep state S5
[485.868007] reboot: Power down    <---- screen freezes at this point

With the igb.EEE=0 stanza added to the kernel command line:

Devuan GNU/Linux 3 devuan tty1
devuan login: [   864.785061] EXT4-fs (sda1): re-mounted. Opts: (null)
[   864.824466] kvm: exiting hardware virtualization
[   864.910856] sd 7:0:3:0: [sde] Syncronizing SCSI cache
[   864.911235] sd 7:0:2:0: [sdd] Syncronizing SCSI cache
[   864.911634] sd 5:0:0:0: [sda] Syncronizing SCSI cache
[   864.913092] sd 5:0:0:0: [sda] Stopping disk
[   865.013903] ACPI: Preparing to enter sleep state S5
[   865.014444] reboot: Power down

As you can see, there are no e1000e: eth0 NIC Link is Down or e1000e: EEE TX LPI TIMER: 00000000 entries, which would imply that the e1000e module has been uloaded by the system and EEE timer is not active.

Hopefully, it will be the end of my 'bad shutdown' problem.

I did not come up with this, not by a long shot, just lucky to find it.

Here's a post from 05/2012 with the solution:

https://thatbytes.co.uk/posts/fun-with-energy-efficient-ethernet-poweredge-720-i350-controller/

Best,

JHM

[JMarx11]

Hello:

... for whatever reason, the Intel 82566DM-2 on board controller in my Sun Microsystems Ultra 24 does not allow the ethtool (v4.19) utility to access or modify EEE settings ...

I have been informed that this has nothing to do with ethtool.

The reason being that ethtool is just that: a tool, a messenger if you will.

The ethtool request to query or modify EEE settings is passed to the kernel and (in this case) it is the 1000e NIC driver that will 1. either respond accordingly or 2. report that the instruction is not supported.

In this specific case, ethtool queries the current setting or attempts to modify it but fails in both cases.

Come to that, it's either the device which does not support getting and setting EEE parameters or that support in the e1000e driver is missing.

The fact that I have been able to disable EEE at boot with the addition of the igb.EEE=0 bit in the kernel command clearly points to the e1000e driver not having support for EEE queries/changes.

Best,

JHM