Wrong PTP (IEEE-1588v2) behavior with I218-V NIC

[JosephMatan]

Hi,

This question is relevant for those who are familiar with the Precision Time Protocol (PTP) or IEEE-1588v2.

I have a simple PTP setup:

[Master] <--- [2 meters cable] ---> [Slave]

There are no devices or switches between master and slave - only an ethernet cable.

The Master is I218-V NIC (which supports PTP and HW timestamping).

The Slave is a device that has been tested against other Masters such as Intel NIC I210 and Intel 82574L, and it worked well.

However, with the I218-V there is an issue that few seconds after starting with heavy network load the path delay is getting smaller.

With no traffic (only PTP traffic), the path delay is about 7-8 usec.

Under heavy traffic, the path delay (after about 10-15 seconds) start decreasing till it reaches about 0.8 usec and then it stabilized there (as long as I keep on the heavy traffic load).  

So the first thing I thought was that it's obviously something with EEE (Energy Efficient Ethernet) getting into action.

(if it was the flow-control in action, I would expect the delay to get bigger...) 

But just to be sure, I ran:

ethtool -A eth0 autoneg off rx off tx off

ethtool -a eth0 

        Pause parameters for eth0:
        Autonegotiate:  off         
        RX:             off        
        TX:             off   

ethtool --set-eee eth0 eee off

ethtool --show-eee eth0

EEE Settings for eth0:                                        
        EEE status: disabled                                  
        Tx LPI: 17 (us)                                      
        Supported EEE link modes:  100baseT/Full              
                                   1000baseT/Full            
        Advertised EEE link modes:  100baseT/Full            
                                    1000baseT/Full            
        Link partner advertised EEE link modes:  Not reported 

But there was no effect...

Any ideas what could cause this behavior ?

Thanks, 

Joseph

[Crisselle_Intel]

Hello Joseph,

Thank you for posting in Intel Ethernet Communities. 

We are sorry to hear that you are experiencing issue with your current setup. For us to proceed checking your request, kindly provide the following information.

1. Exact Operating System used.

2. Current kernel version

3. Model of the motherboard where the i218-V was embedded

4. Was the configuration known to be working fine before?

Looking forward to your reply.

We will follow up after 3 business days in case we don't hear from you.

Best regards,

Crisselle C.

Intel® Customer Support

[JosephMatan]

Hi Crisselle,

Thanks for replying my query.

1. I'm running on Ubuntu 18.04.1.
2. Linux Kernel version 5.4.0-74.
3. Motherboard: I think that is the Serial Number: "GERY63200554". Hope it's answer your question.
4. I don't think the configuration was known to be working fine before. However, I must say that this wrong behavior we are facing doesn't seems like a bug. It behaves like a feature. I guess it's something with power management of the NIC.

 Best Regards,

Joseph

[JosephMatan]

Hi Crisselle,

I would like to add additional information.

I'm having this issue also with I219-V. So I'll answer again your 4 questions:

1. I'm running on Ubuntu 18.04.1.
2. Linux Kernel version 4.15.0-29.
3. Motherboard: I think that is the Serial Number: "GEBE84400GJA". 
4. I don't think the configuration was known to be working fine before. However, I must say that this wrong behavior we are facing doesn't seems like a bug. It behaves like a feature. I guess it's something with power management of the NIC.

 Best Regards,

Joseph

[Mike_Intel]

Hello JosephMatan,

Thank you for the quick reply. However, what we need is the brand and model of the board for us to check. And also, you mentioned this "It behaves like a feature. I guess it's something with power management of the NIC".

Have you tried checking this with the manufacturer of the board if this is a feature by the board/BIOS since it is the one controlling the onboard network card?

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support Technician

[JosephMatan]

Hi Michael,

The brand is also Intel. Those NIC(s) (I218-V and I219-V) are embedded on my Intel NUC(s). The system description of each NUC is attached. I've additional NUC(s) with the same problem. I think these NUC(s) are newer, but I don't have yet their system description.

In general these are my NUC(s). Some with I218-V and some with I219-V:

Serial Number	Model
G6BE84300WCF	NUC8i7BEH
F44D306D7301	NUC7i7BNH
G6RY634004GU	NUC5i7RYH
G6BE84600BSG	NUC8i7BEH
G6BE84600BVQ	NUC8i7BEH
B8AEED7A13E7	NUC5i7RYH
F44D30613D45	NUC5i7RYH
G6BE92100GML	NUC8i7BEH
G6BE92100GLG	NUC8i7BEH

I've tried to verify it's not some OS power management configuration, so I've started with verifying that the /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
 is set to performance, and that /sys/class/net/enp0s25/power/control isn't set to auto.

I still suspect it's a power management feature, but not sure how to verify that all power management optimizations are disabled.

Best Regards,

Joseph 

[Crisselle_Intel]

Hello Joseph,

Thank you for the reply.

Since the Ethernet Controller is embedded on Intel NUC, please be informed that this request will be best handled by our NUC team. We will help you move this thread to the appropriate topic for further checking and they will get back to you as soon as possible.

Feel free to let us know if you have additional questions and clarifications. 

Best regards,

Crisselle C.

Intel® Customer Support

[DeividA_Intel]

Hello JosephMatan, 

Thank you for posting on the Intel® communities.   

In order to help you further, please confirm the following information:

1. Is the Intel® NUC Kit NUC8i7BEH the device that you are using?

2. Have you updated your drivers and firmware?

3. Are you using the NUC as a server or just for home/office tasks?

Regards,  

Deivid A.  

Intel Customer Support Technician  

[JosephMatan]

Hi Deivid,

I'm having several NUC(s), but each time I work only with a single NUC.

All of my NUC(s) suffers from the issue I was complaining about.

Regarding your questions:

1. Intel® NUC Kit NUC8i7BEH is one of the devices I'm using. It uses the I219-V as its NIC (networks interface card).
2. I've only updated the driver of the NIC (from 3.2.6-k to 3.8.7-NAPI) - but it didn't help. The firmware of the NIC is 0.4-4.
3. Not sure it answers your question, but this NUC is running Ubuntu Desktop 18.04.

Thanks for your support,

Joseph

[DeividA_Intel]

Hello JosephMatan, 

Thank you for the information provided.

I will proceed to check the issue internally and post back soon with more details. 

Best regards,  

Deivid A.  

Intel Customer Support Technician 

[JosephMatan]

Hi Deivid,

I would like to add additional information:

I took another PC (not a NUC), and replace it with the NUC.

The system information of this new pc is attached - I used Intel's ssu.sh script to dump this info.

But the interesting part is that I ran my test twice on that pc. Once when the PTP setup is connected through the I350 NIC, and in the second time when the setup is connected through the I219-V NIC.

Just a reminder, no switches or other devices are between.

With the I350 NIC, the path delay is stable (even under traffic).

With the I219-V NIC, the path delay is getting smaller under traffic. This is the same behavior I see in the NUC(s), which are using this same family of NIC(s).

So my guess is that it's something with the I219-V NIC (perhaps a feature, or a bug...).

Hope this additional information will be useful.

Best regards,

Joseph

[DeividA_Intel]

Hello JosephMatan, 

Based on your hardware and information please try the following:

- Bios recovery:

1. Steps: https://downloadmirror.intel.com/30430/eng/NUC-BIOS-Update-Readme.pdf

2. BIOS: https://downloadcenter.intel.com/download/30430/BIOS-Update-BECFL357-

Do not hesitate to let me know the outcome

Best regards,  

Deivid A.  

Intel Customer Support Technician 

[JosephMatan]

Hi Deivid,

1. Can I use this BIOS update for all of my NUC(s) ? (I'm using NUC8i7BEH / NUC7i7BNH / NUC5i7RYH models)
2. I'm don't believe the BIOS update will resolve the problem. Please read my last update - I used different PC (not a NUC), which has two different NIC(s) (Intel I219-V and Intel I350), and the problem reproduced only when I'm using the I219-V (which is the same NIC that we have on the NUC(s)).

Best Regards,

Joseph

[DeividA_Intel]

Hello JosephMatan, 

I would like to let you know that the BIOS version may not work for all your Nucs, I recommend you to try first with the Intel® NUC Kit NUC8i7BEH.

While you are trying the BIOS update/Recovery, I will continue the research with the information provided.

Best regards, 

Deivid A.  

Intel Customer Support Technician 

[DeividA_Intel]

Hello JosephMatan, 

Please be careful when updating the bios since not all NUCs are using the same version, they are different for each NUC model the one provided was for the NUC Intel® NUC Kit NUC8i7BEH. 

Updating the BIOS is a very important step for the process, please let me know if you were able to update the BIOS version.

Regards,  

Deivid A.  

Intel Customer Support Technician  

[JosephMatan]

Hi Deivid,

Thank you for your support!

Unfortunately, I'm yet not able to update the bIOS, since I'm not the only user of the NUC(s), and currently an optional fatal while updating the bIOS is a bit risky.

I would very appreciate if you still  keep research with the information that I provided, since this bad behavior don't occurs only on the NUC(s), but on other PC(s) that have this specific embedded I219-V NIC.

Thanks,

Joseph

[JosephMatan]

Hi Deivid,

If I may, perhaps upgrading the firmware of the NIC would help ? 

When I'm running ethtool info on the NIC, I see:

driver: e1000e
version: 3.2.6-k
firmware-version: 0.2-4

So, I tried to upgrade the driver version to 3.8.7, but it didn't help.

However, perhaps I can upgrade the firmware version ? (I have no idea how)

Thanks,

Joseph

[DeividA_Intel]

Hello JosephMatan, 

I do understand that a BIOS update may be risky, however, a BIOS recovery is the secure way to perform the update:

- Bios recovery:

1. Steps: https://downloadmirror.intel.com/30430/eng/NUC-BIOS-Update-Readme.pdf

2. BIOS: https://downloadcenter.intel.com/download/30430/BIOS-Update-BECFL357-

Also, updating the firmware of the Nic may help, you will find the driver for the Intel® Ethernet (LAN) Network Connection in the following link:

- https://downloadcenter.intel.com/download/30252/Intel-Ethernet-LAN-Network-Connection-Driver-for-Windows-10-for-Intel-NUC

If the BIOS recovery and the (LAN) Network updates did not help, you can get in contact with us directly to discuss the warranty options for your Intel® NUC Kit NUC8i7BEH. You can use the link that works better for you.

- US Canada: https://www.intel.com/content/www/us/en/support/contact-support.html#@11 

- Europe, Middle East, and Africa: https://www.intel.com/content/www/us/en/support/contact-support/emea-contact.html 

- Asia-Pacific: https://www.intel.com/content/www/us/en/support/contact-support/apac-contact.html 

- Latin America: https://www.intel.la/content/www/xl/es/support/contact-support/lar-contact.html 

Regards,  

Deivid A.  

Intel Customer Support Technician  

[JosephMatan]

Hi Deivid,

If the BIOS recovery and the (LAN) Network updates did not help, you can get in contact with us directly to discuss the warranty options for your Intel® NUC Kit NUC8i7BEH.

---

I'm sorry for being stubborn on that issue, but the warranty options for my NUC(s) is not my concern.

I'm having this issue on non-NUC devices that works well with other Intel NIC(s).

The problem is only with the NIC(s) family I218-V/I218-LM/I219-V/I219-LM.

Therefore, I think this issue should go back for the handling of the Intel Ethernet Team (@CrisselleF_C_Intel).

Again, sorry for being stubborn on this issue,

Joseph

[DeividA_Intel]

Hello JosephMatan, 

Thanks for the update.

I will proceed to review this situation further and check if there are more steps to try, as soon as I have any information I will let you know.

Regards,    

Deivid A. 

Intel Customer Support Technician 

[DeividA_Intel]

Hello JosephMatan, 

To be able to investigate further, I will appreciate it if you provide me the following information:

1. A recent report from the Intel® System Support Utility.

- https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-

2. Send videos/pictures/screenshots of the issue.

3. Steps to replicate the issue, this is to make sure that we are following your steps to have the same results.

4. Can you explain more about the issue and what is the results that you are waiting for?

5. Can you provide a list of the NUCs that are presenting this issue?

Regards,   

Deivid A. 

Intel Customer Support Technician