Problem with hardware timestamp on X550 (Intel Ethernet 10G 2P X550-t Adapter)

GustavoSalge · ‎11-04-2022

I have hardware that sends packets at a controlled time interval (208.33us in this case). I'm trying to measure this time between frames using a NIC (Intel Ethernet 10G 2P X550-t Adapter) that has the hardware timestamp feature. I performed the measurement through tcpdump, tshark and through a proprietary program developed based on Lipcap. The commands used in tcpdump and tshark are as follows:

1) tcpdump -i <interface-name> -j adapter_unsynced <file-name>

2) tshark -i <interface-name> --time-stamp-type adapter_unsynced -w <file-name>

I've been getting an absurd time (randomly) in the middle of the capture, representing a non-existent time interval. Apparently it is a NIC timestamp error, because when I capture using host timestamp this error does not appear. I'm currently using:

Ubuntu 18.04.06

ixgbe driver (5.1.0-k)

I've tried it on other versions of linux like Kali and got the same answers. The images below illustrate the times I have been capturing in the 3 softwares I mentioned.

Is there any configuration I should do? Is there any firmware update for the card?

Thaks in advance.

Mike_Intel · ‎11-06-2022

Hello GustavoSalge,

Thank you for posting in Intel Ethernet Communities.

For us to further check the issue, please provide the following details.

Are you using an onboard X550-t Adapter or is it a PCIe card?
If you are using a PCIe card, please send us photos of the card on both sides for us to validate the card
Please share the link of your latest driver.
Is it working before?

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-07-2022

1. Are you using an onboard X550-t Adapter or is it a PCIe card?

PCIe card

2. If you are using a PCIe card, please send us photos of the card on both sides for us to validate the card

3. Please share the link of your latest driver.

I've tried a few versions, but now I'm using the latest: 5.17.1

4. Is it working before?

No. Are my first tests.

Mike_Intel · ‎11-07-2022

Hello GustavoSalge,

Thank you for the quick reply. Upon checking the markings, I found out that your unit is an OEM unit from Dell. As much as we want to assist you, we have very limited support for OEM units. Since you already tried our drivers, please contact Dell for further assistance. They may have custom driver, software and firmware for your unit.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-08-2022

Thanks for the answer. Unfortunately I didn't imagine this when I bought the NIC. I have some doubts:

1. Isn't it the same NIC? If its the same hardware, is not possible to change firmware?

2. Does Dell customize (hardware) your NIC?

Mike_Intel · ‎11-08-2022

Hello GustavoSalge,

Thank you for understanding. If you have doubts in purchasing a network cards, one thing that you should look for is YottaMark* and BradyID* to easily identify if your network card is authentic or nor. Unfortunately your card don't have these stickers.

https://www.intel.com/content/www/us/en/support/articles/000007074/ethernet-products/gigabit-ethernet-adapters-up-to-2-5gbe.html

To answer your questions.

OEM units cannot use Intel's update file and vice versa because it can damage the card. That is why we always verify this if the customer's inquiry is about firmware. OEM manufacturer's should have their own firmware file and they may have different steps/ways to update the firmware.
OEM manufacturer can alter features and specifications of the network cards and those changes are unknown to us. They should have custom driver, firmware and software for OEM network cards.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-09-2022

I understood that OEM boards must have their own firmware and that manufacturers can alter features.

Really I was only able to update the firmware on the board (unfortunately it continued with the same behavior) through nvmupdate provided by Dell, even though it seems that both software (Dell/Intel) are produced by Intel itself as shown below.

Attempt to Update Firmware by Intel:

Attempt to Update Firmware by Dell:

However, understanding that it is the same hardware, is there any possibility for me to download the original Intel firmware as if it were an Intel retail network adapter? I assume the risks of any damage.

Mike_Intel · ‎11-09-2022

Hello GustavoSalge,

I understand your dilemma right now. I am sorry to inform you that whatever driver or firmware that is available in our download site which you already tried are the only drivers and firmware that we can provide.

However, I don't understand why you are having the same error when you tried updating the firmware using Dell's update. I think it is better to coordinate with Dell if they have different procedure/steps in updating the firmware of the unit.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-10-2022

I actually successfully updated the firmware using Dell's update, but the NIC continued to have the same issue. It's the reason why I want to try to download Intel firmware on the NIC.

I understand that naturally the NIC doesn't accept the intel firmware, only those supported by Dell. However, are you sure there is no way to force my NIC to accept intel firmware?

Mike_Intel · ‎11-10-2022

Hello GustavoSalge,

Thank you for the quick response and I understand your confusion with the issue. However, regarding your question, it is hard for us to answer that because we don't know how the network card was built. I know I have been saying the same thing, I am just being honest because we don't know how the card was built and how altered the unit is.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

Mike_Intel · ‎11-15-2022

Hello GustavoSalge,

I hope you're having a wonderful day. I am just sending a soft follow up hoping that you are now talking to Dell for further assistance regarding your inquiry.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

Mike_Intel · ‎11-20-2022

Hello GustavoSalge,

I hope this message finds you well. I am just sending another follow up hoping that you are now talking with Dell regarding the issue. Since we have not heard back from you, I need to close this inquiry.

If you need any additional information, please submit a new question as this thread will no longer be monitored.

Thank you and stay safe.

Best regards,

Michael L.

Intel® Customer Support

Mike_Intel · ‎11-21-2022

Hello GustavoSalge,

By the way, before we close the case. We just want to add that base on our firmware update file, https://downloadmirror.intel.com/727469/readme_3.60.txt

Limitations and Prerequisites

=============================

This package is intended to be used on Intel branded adapters. Please contact your OEM vendor for an appropriate package. In some cases this package may update an OEM device. This package only updates the NVM image for the device family listed on the package. Each Intel Ethernet product family has its own NVM Update Package.

Intel firmware is only intended to be used on Intel branded adapters. OEM adapters should be supported by OEM manufacturers.

If you need any additional information, please submit a new question as this thread will no longer be monitored.

Thank you and stay safe.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-22-2022

Hello Michael,

Sorry for the delay to reply. Unfortunately I haven't gotten support from Dell yet and I'm considering buying another card.

Can you help me to ensure that it is an intel-owned card? I need a NIC that provides hardware timestamp for all received packets and can synchronize via PTP (IEEE1588).

Does the model E810-XXVDA4 meet my needs?

Mike_Intel · ‎11-22-2022

Hello GustavoSalge,

Thank you for the update and I am sorry to hear that. To ensure that you are buying an authentic Intel Card, please refer to these links below:

https://www.intel.com/content/www/us/en/support/articles/000007074/ethernet-products/gigabit-ethernet-adapters-up-to-2-5gbe.html

https://www.intel.com/content/www/us/en/support/articles/000007060/ethernet-products.html

As for an Ethernet card that will work with your system. It is best also to check with Dell for compatible network cards that will work with your system to avoid incompatible issues.

If you have questions, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

GustavoSalge · ‎11-24-2022

Hello Michael,

After many tests I identified why errors occur in my timestamps, but I didn't understand the reason.

Every time my frame includes the 802.1Q tag (VLAN) the error appears. Note the error in the timestamp in the image below.

Without VLAN the error disappears. I know my board is OEM, but if you ever see any such error, could you please help me?

Mike_Intel · ‎11-24-2022

Hello GustavoSalge,

Let me try to check if we have this error in our database. Please give us 2 to 3 business days to provide an update on this.

Thank you and stay safe.

Best regards,

Michael L.

Intel® Customer Support

Reha · ‎11-24-2022

Hello,

As i spent some time on ptp issues , i can share my experiences on hw timestamping problems. We have got these experiences by using i350 NIC that can do hw timestamping as well.

You solely concentrated on NIC and NIC firmware , but the main cause of the problem may not be there. You can change the NIC and you may get the same problem.

Even the NIC supports hw timestamping you can not %100 guarantee that you will get an ordered output from tshark or dumpcap. If your linux machine is not fast enough you may get unordered packets as well . For example in our case even the NIC i350 did the hw timestamping correctly we had to use reordercap to order dumpcap . The main reason of that is the dumpcap software inserting packets coming from the NIC to the dumpcap file by using first in first out method, not by checking the timestamp.

But your case seems a bit different. There is something more strange in your case.

I assume that you are using linuxptp. If you didn't check before can you check who is the GrandMaster clock. If the GM clock is not the NIC itself there may be a problem.

Also you can consider using dumpcap filters to get rid of unwanted packets.

GustavoSalge · ‎11-25-2022

Hello,

Thanks for sharing your experience. I think my case is a little different because I control the source that is sending the packets and by analyzing the frame I know they are in order as you can see below. This protocol has a variable called sample count which is sent orderly as shown in the image and despite this the hardware timestamp is completely wrong.

Another detail is that in this example my NIC is not synchronized because I could not reach this condition using linux PTP and I think it is because of this timestamp error.

Mike_Intel · ‎11-28-2022

Hello GustavoSalge,

I hope this message finds you well.

Upon further checking, we do not provide support for 3rd party OEM adapters. 3rd party manufacturers already have customized firmware and drivers which is tailored for your adapter. They are in best position where you can seek assistance so you will not lose features or customizations of the Ethernet controller. We do apologize for any inconvenience caused by this issue.

If you still have clarifications, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support

Mike_Intel · ‎12-01-2022

Hello GustavoSalge,

I hope you're having a wonderful day. I am just sending a soft follow up hoping that you already tried contacting the OEM manufacturer for further assistance.

If you still have clarifications, please let us know. In case we do not hear from you, we will make a follow up after 3 workings days. Thank you.

Best regards,

Michael L.

Intel® Customer Support