Problems with the PTP Protocol on the Ethernet 10G 2P X520 Adapter (rev 01)

[tx-y-hiraka]

I would like to use linuxptp to synchronize PTP on an Ubuntu 22.04 environment.

Environment:
OS: Ubuntu 22.04.4 LTS
Intel NIC: X520
Driver: ixgbe-5.20.9 and ixgbe-5.20.10

Problem: When I set `twoStepFlag 0` and `time_stamping hardware` in /etc/linuxptp/ptp4l.conf to perform time synchronization in step1 mode of the PTP protocol, I encounter an error such as ioctl SIOCSHWTSTAMP failed: Numerical result out of range.

root@server01:~/intel# systemctl status ptp4l.service
● ptp4l.service - Precision Time Protocol (PTP) service for
Loaded: loaded (/lib/systemd/system/ptp4l.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2024-XX-22 XX:03:51 UTC; 1h 0min ago
Docs: man:ptp4l
Main PID: 14033 (ptp4l)
Tasks: 1 (limit: 629145)
Memory: 240.0K
CPU: 5ms
CGroup: /system.slice/ptp4l.service
mq14033 /usr/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf -i enp132s0f1

Jul XX XX:03:51 server01 systemd[1]: Started Precision Time Protocol (PTP) service for .
Jul XX XX:03:51 server01 ptp4l[14033]: [1566.808] selected /dev/ptp10 as PTP clock
Jul XX XX:03:51 server01 ptp4l[14033]: [1566.833] driver rejected most general HWTSTAMP filter
Jul XX XX:03:51 server01 ptp4l[14033]: [1566.833] ioctl SIOCSHWTSTAMP failed: Numerical result out of range
Jul XX XX:03:51 server01 ptp4l[14033]: [1566.852] port 1: INITIALIZING to FAULTY on FAULT_DETECTED (FT_UNSPECIFIED)
Jul XX XX:03:51 server01 ptp4l[14033]: [1566.852] port 0: INITIALIZING to LISTENING on INIT_COMPLETE

Additionally, I encounter an error when changing the hardware timestamp settings as shown below:

root@server01:~/intel# sudo hwstamp_ctl -i enp132s0f1 -t 1 -r 1
current settings:
tx_type 1
rx_filter 12
SIOCSHWTSTAMP failed: Numerical result out of range

The requested time stamping mode is not supported by the hardware.
These behaviors are very similar to the issue described in the following link:

https://x.gd/OR1YF

According to the patch report, the X520 uses the 82599 chip, so I believe there is a correlation: https://www.intel.com/content/www/us/en/products/sku/55353/intel-ethernet-server-adapter-x520da2/specifications.html

I manually applied similar changes to `/root/intel/ixgbe-5.20.10/src/ixgbe_ptp.c` as described in the patch and ran make install, but the error persisted.

Thank you for reading.

[Sachinks]

Hello tx-y-hiraka,

Thank you for posting in the Intel Community Forum. We sincerely apologize for the delay in our response. We have received your concern and would like to assure you that assisting you is our top priority.

Could you kindly let us know if the issue has been resolved, or if you require further assistance from us regarding this? Your prompt response will greatly help us in diagnosing and resolving the issue as quickly as possible.

Regards,

Sachin KS

Intel Customer Support Technician

[Poojitha]

Hello tx-y-hiraka,

Greetings for the day! 

We are following up to check if you were able to find the information we requested. Kindly confirm at your earliest convenience, so that we can continue assisting you in resolving this matter.

We appreciate your understanding!

Regards,

Poojitha

Intel Customer Support Technician

[tx-y-hiraka]

From my own findings, I’ve concluded that the X520 does not support one-step hardware synchronization in PTP. If possible, please let me know how I can check this in the X520 hardware specifications.

The full output of $ sudo ethtool -T enp129s0f1 is shown below. I believe that if there is no value for the third entry under 'Hardware Transmit Timestamp Modes:', it does not support one-step synchronization with hardware timestamps. The third value could be something like HWTSTAMP_TX_ONESTEP_SYNC.

sudo ethtool -T enp129s0f0
Time stamping parameters for enp129s0f1:
Capabilities:
        hardware-transmit
        software-transmit
        hardware-receive
        software-receive
        software-system-clock
        hardware-raw-clock
PTP Hardware Clock: 5
Hardware Transmit Timestamp Modes:
        off
        on
Hardware Receive Filter Modes:
        none
        all

The following is an excerpt from $ sudo lshw -class network."

*-network:0
description: Ethernet interface
product: Ethernet 10G 2P X520 Adapter
vendor: Intel Corporation
physical id: 0
bus info: pci@0000:84:00.0
logical name: enp132s0f0
version: 01
serial: a0:36:9f:dd:73:20
size: 10Gbit/s
capacity: 10Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical fibre 1000bt-fd 10000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=ixgbe driverversion=5.20.10 duplex=full firmware=0x8000095d, 19.5.12 latency=0 link=yes multicast=yes port=fibre speed=10Gbit/s
resources: irq:75 memory:c9d00000-c9dfffff ioport:9020(size=32) memory:c9e04000-c9e07fff memory:c9e80000-c9efffff memory:c9300000-c93fffff memory:c9400000-c94fffff

[Sachinks]

Hello tx-y-hiraka,

Thank you for the reply and thank you for the details. 

Kindly provide us the below details so that we can check this further.

1) Kindly let us know the system details in which this ethernet adapter is present ( motherboard model details).

2) Also kindly let us know if this adapter is built-in or was it manually added later to the system?

3) Also, we see that you mentioned the product as X520 Adapter. Kindly provide the exact model details of the adapter. ( ex : Ethernet Adapter X520-DA1 )

Regards,

Sachin KS

Intel Customer Support Technician

[tx-y-hiraka]

> 1) Kindly let us know the system details in which this ethernet adapter is present ( motherboard model details).

dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.

Handle 0x0200, DMI type 2, 8 bytes
Base Board Information
Manufacturer: Dell Inc.
Product Name: 072T6D
Version: A05

This is the motherboard of a DELL PowerEdge 730.

>  2) Also kindly let us know if this adapter is built-in or was it manually added later to the system?

It is a repurposed item from another organization, and there is no remaining information, but it is believed to be built-in.

> 3) Also, we see that you mentioned the product as X520 Adapter. Kindly provide the exact model details of the adapter. ( ex : Ethernet Adapter X520-DA1 )

According to the results from IPMI and lspci, it is displayed as "Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)," but it is likely the Intel X520 DP 10Gb DA/SFP+. Although the following is a Japanese document, it seems there are no other built-in Intel NICs from the X520 series that can be installed in the R730.
https://i.dell.com/sites/csdocuments/Business_smb_product_Documents/ja/jp/PER730configguide.pdf

[Sachinks · Accepted solution]

Hello tx-y-hiraka,

Thank you for the details.

We kindly request you to go through the below document in our RDC (Resource and documentation) page to know about PTP synchronization in ethernet adapaters.

Document ID : 635237

Link to the RDC page : https://www.intel.com/content/www/us/en/resources-documentation/developer.html#gs.5k1ay6

Please note that you need an Intel premier developer zone account with an active NDA to access this document.

How to Apply for an Intel® Resource and Documentation Center (RDC) and/or Intel® Developer Zone (Intel® DevZone) premier Account:

https://www.intel.com/content/www/us/en/support/articles/000058073/programs/resource-and-documentation-center.html

Kindly let us know if the above document is helpful.

Regards,

Sachin KS

Intel Customer Support Technician

[Akshaya1]

Hello tx-y-hiraka,

Greetings from Intel! 

This is a reminder mail regarding the issue you reported to us. 

We're eager to ensure a swift resolution and would appreciate any updates or additional information you can provide. 

 Regards,

Akshaya

Intel Customer Support Technician

[Sachinks]

Hello tx-y-hiraka,

Greetings for the day!

We would like to inform you that we are closing this request as no response has been received from our previous follow-ups. 

Please don't hesitate to ask any further questions in the future. Feel free to start a new conversation, as this thread will no longer be monitored.

Regards,

Sachin KS

Intel Customer Support Technician