Page Image
Run your own PTP server for fun and profit
Put a PTP server in your homelab to count nanoseconds
Modified:
Created:

As a long time lurker on the time-nuts mailing list and an aspiring timenut myself, I’ve always been enamored with time and timing related issues. Heck, I’m a proud owner of a Grace Hopper nanosecond . As it turns out, I’ve been working on SMPTE 2110 stuff lately, which needs a precise network time, and it was the perfect excuse to build a PTP server for my homelab.

Introduction

I should first preface this with the fact that, as usual, I’m standing on the back of wonderful engineers who came before me. A lot of what helped me get this project going can be found in James Clark’s PC PTP/NTP Guide . Also, it’s important to note that I’m setting this up on an Ubuntu 24.04 (x86) system. While the general steps should work pretty much everywhere, the specifics of your distribution might be different. Another great resource is Jeff Geerling’s Blog on the topic.

There are (at least) four time scales you’ll want to know about when working with this stuff. You can find much more detail about these at the leapsecond website , but here’s a concise breakdown:

  • Localtime - This is the time you’re probably used to. It’s the local time where you are. Of course, it’s different from others due to timezones. But it’s different from others in this list not only by the timezone offset, but by an offset due to leap seconds. This scale will get bumped by leap seconds.

  • UTC - You can think of this as just “standard” time. It’s a localtime w/ 0:00 offset. It too is impacted by leap seconds. This is the time scale used by NTP.

  • GPS - This the time as reported by GPS satellites. It is not impacted by leap seconds, so as leap seconds happen in local and UTC, GPS time drifts from those. Currently it’s 18 seconds ahead of UTC/local.

  • TAI - This is the time scale used by PTP (and more broadly the scientific community). It is not impacted by leap seconds, but is always 19 seconds of GPS time (for epoch reasons). Currently, it’s 37 seconds ahead of local/UTC .. at least until the next leap second.

Also, some definitions. Like everything technical, there’s a whole soup of acronyms:

  • NIC - Network Interface Card. This is the network card that connects to the Ethernet network.

  • NTP - Network Time Protocol. If you’re familiar with network time, this is the protocol you’re probably using. It has been around for a long time, is well understood, and is ultimately not very precise due to the fact that’s it’s generally a software based thing. Thus can only get down to the precision the software allows it to, say on the order of tens of milliseconds to tens of microseconds of uncertainty.

  • PTP - Precision Time Protocol (aka IEEE 1588). This is the protocol we’ll be using to sync down to the nanosecond scale. It takes advantage of specific hardware on the NIC to keep track of time and timestamp network packets. It’s not the protocol that makes PTP so precise, it’s this hardware support that makes it possible.

  • GPS - Global Positioning System. Make a u-turn when possible. We all know what GPS is. But it works by having atomic clocks (two actually) on each satellite. It needs incredibly precise timing, which means we get a free precision timing source right out of the air.

  • PPS - Pulse Per Second. This is a pulse generated by a GPS receiver (or other timing source) that provides “exactly” one pulse per second and the rising edge of that pulse is on the “exact” moment the second on the clock changes. Of course, I put exact in quotes here because nothing is ever exact, but we can get pretty darn close with a good PPS signal.

  • TOD - Time of Day. This is the rough time of day in the nearest second. This is what’s provided by a GPS normally if you aren’t using the PPS signal.

  • Epoch - A fancy word for a reference point in time. A time scale picks an arbitrary point in time to set as its zero point. You can look up the epochs for each scale I mentioned above, but it’s not really important (until you have to write code against it). Just know that each scale may pick a different epoch, so your counts may be different (beyond just those due to leap seconds and timezones).

  • TCXO - Temperature compensated crystal oscillator. This is a crystal oscillator that has some extra circuitry to null out frequency drift caused by temperature fluctuation. This generally means it’ll track a frequency with less error. Many of these also support being “disciplined”, which means you can give it a control signal to nudge it a bit faster or slower.

  • PHC - PTP Hardware Clock. This is the hardware clock kept by the NIC hardware. It uses the TAI scale (as PTP uses TAI) and is used to timestamp packets as well as timestamp the PPS input.

GPS & PPS & TOD, Oh My!

The PPS output of my GPS on the scope
Fig 1: Example PPS from GPS
PPS of two GPS modules
Fig 2: PPS from two GPS modules

At the top of our timing tree is a GPS receiver with a PPS and TOD output. The PPS signal provides an accurate reference point for the beginning of a second. Of course, you wouldn’t actually know what time it was with only a PPS signal, since it’s basically just a beat. So, along with the PPS, we also need a time of day signal. This is provided by the GPS as part of its normal GPS information output (via serial over USB). This output has a lot of uncertainty, but it’s plenty accurate enough to know what the time is down to a second (in actuality it appears to have something like 20ms or real jitter, but I didn’t take that close of a look).

Together these allow you to determine a precise time. How precise? The datasheet for my GPS module specs the PPS output to be 30-60ns accurate, meaning the edge of the pulse will line up with the actual edge of a seconds change with about 30-60ns of uncertainty. I measured the edge of the pulse on the scope and it appears the actual pulse has a rise and fall time of about 2.5ns. Impressive. Although, keep in mind the rise/fall time is only one potential inaccuracy (and not even the biggest). The uncertainty due to the GPS itself, is much higher (I’d guess at least an order of magnitude). There are GPS receivers that are optimized for time. As far as I know, they should be able to be dropped into this later w/o any changes (just an increase in accuracy). For today’s purposes, I chose a GPS board from Amazon that can be had for a few dollars. On thing that’s important to verify: the GPS PPS output swings from 0v to 3.3v. Mine does. I suspect that’s probably pretty standard, but I don’t know that for certain.

The GPS board is wired up to deliver the PPS via SMA connectors and coax because we want to keep that edge as crisp as possible (i.e. keep all the high frequency components from attenuating). The board itself is powered via USB, which is also how it delivers the time of day information.

To see how these inexpensive GPS receivers actually perform, I connected the PPS outputs of two separate modules to the GPS. I triggered off one PPS and the other shows up at some offset. I ran this for about half an hour w/ persistence to get an idea of what sort of uncertainty exists between two modules. Here’s what that looked like:

Seems like the datasheet’s 30-60ns figure was “ideal” and not exactly what I’m seeing real world. I don’t have actual statistics here, but it looks like maybe if we squint our eyes the 60ns figure makes sense for maybe 80% of the time. For now, I’ll be timing things up w/ this, but I’ve ordered a proper GPS disciplined oscillator (with a better, timing optimized, receiver and TCXO).

Network Interface Card w/ Skillz

Fig 3: Intel i226 w/ I/O pins that can be used for PPS In/Out
Fig 4: The TimeNIC

After the next step down our timing tree is the network interface card. And frankly, this is where a lot of the magic happens. Basically, we need three things from our network card:

  • A local oscillator that drives a timestamp counter in hardware.
  • A way to discipline that oscillator/counter with our PPS signal
  • A hardware timestamp engine to read the timestamp from the hardware counter and plop it into packets as they leave the card. Additionally, it should be able to attach that hardware timestamp to received packets

Generally, these features are advertised at IEEE 1588 (aka PTP) support. The Intel i226 implements all these things. And it’s pretty cheap and 2.5Gb as well. There are other cards like the Intel i210 that support this. For my purposes, I chose a slightly more expensive board called the TimeNIC , which uses the i226 controller but has some timing quality of life improvements. Specifically, it has much better temperature compensated local oscillator (TCXO) as well as SMA connectors for PPS input and output.

To Your Empty Server, Add Some GPSd

I used the Ubuntu packages for GPSd. They can be installed via sudo apt install gpsd gpsd-tools This installs GPSd version 3.25, which is important as that’s the first version to support the /run/chrony.clk.XXX.sock socket. If you use an older version, these directions won’t work as you’ll have to setup the link between GPSd and Chrony via SHM instead. This is a better way, since the socket uses a notification scheme, so it’s probably a good idea to upgrade GPSd if you don’t have at least this version.

A Touch of Persistence

When I’ve got specific hardware that I want to keep working as I might plug and unplug things, I find it’s useful to write udev rules to symlink devices to make sure they show up the same way all the time. Here’s the rule I added to make my GPS device(s) show up as /dev/gpsX. This allows me to later use the /dev/gpsX device in configuration instead of the typical /dev/ttyACMx or /dev/ttyUSBx or /dev/ttyXXX which could change on me due to initialization order.

/etc/udev/rules.d/99-gps.rules
# Once configured, probably best to lock the specific device to a specific GPS.  
# You can get the serial number via lsusb -v  
#SUBSYSTEM=="tty", ATTRS{idVendor}=="1546", ATTRS{idProduct}=="01a7", ATTRS{serial}=="000123456", SYMLINK+="gps0"  
  
# For now, we'll make it generic and just label all u-blox GPS devices as gps0, gps1, gps2, etc  
UBSYSTEM=="tty", ATTRS{idVendor}=="1546", ATTRS{idProduct}=="01a7", SYMLINK+="gps%n"

Reload the udev rules to make the /dev/gps0 (or more) device(s) to show up.

shell
sudo udevadm control --reload-rules
sudo udevadm trigger

Configure to Taste

Modify the /etc/default/gpsd file and add /dev/gps0 to the DEVICES variable and -n to the OPTIONS (to make gpsd poll the GPS all the time instead of only when a client is connected). For reference, here’s what my /etc/default/gpsd file looks like:

/etc/default/gpsd
# Devices gpsd should collect to at boot time.  
# They need to be read/writeable, either by user gpsd or the group dialout.  
DEVICES="/dev/gps0"  
  
# Other options you want to pass to gpsd  
GPSD_OPTIONS="-n"  
  
# Automatically hot add/remove USB GPS devices via gpsdctl  
USBAUTO="true"

Don’t Let AppArmor Sour It

AppArmor isn’t going to let gpsd push data to /run/chrony.clk.gps0.sock, so we need to modify the gpsd apparmor config file to add the following line:

/etc/apparmor.d/usr.sbin.gpsd
# default paths feeding GPS data into chrony  
 /{,var/}run/chrony.clk.*.sock rw,  # <--- ADD THIS LINE HERE
 /{,var/}run/chrony.tty{,S,USB,AMA}[0-9]*.sock rw,  
 /tmp/chrony.tty{,S,USB,AMA}[0-9]*.sock rw,

And Bake

Modify the gpsd systemd service to make sure it gets started after chrony. This can be done by running sudo systemctl edit gpsd.service and adding the following lines:

/etc/systemd/system/gps.service.d/override.conf
[Unit]
PartOf=chronyd.service

Restart gpsd and make sure it’s enabled at boot:

shell
sudo systemctl reload apparmor
sudo systemctl enable gpsd
sudo systemctl restart gpsd

Sprinkle In Some NTP

We’re going to use chrony, an NTP server, to read the GPS serial data to get time of day as well as other internet NTP servers. This will discipline the Linux kernel clock (a software construct inside the Linux kernel) called CLOCK_REALTIME. The CLOCK_REALTIME will later be used by the ts2phc application to get the current rough (to the nearest second) time of day when setting up the PHC (PTP Hardware Clock). I’ll probably gloss over some stuff here, since there are other better resources on setting up chrony.

I should, however, make note of my original intent to use the PHC for a Chrony timesource. Unfortunately, the i226 can’t handle timestamping the PPS input and PHC queries at the same time, which lead to a bug where the ts2phc application stops getting PPS samples. Since my priority is to get a stable PTP server, I decided to allow the NTP server to use more typical network and GPS serial time sources. Which, for NTP, is just fine. And remember the goal here is just to keep CLOCK_REALTIME accurate enough.

I installed chrony using sudo apt install chrony. Here’s the configuration I’m using:

/etc/chrony/chrony.conf
# chrony needs to know how many leap seconds there are to convert the PHC clock, which is TAI
# into NTP which is UTC
leapsectz right/UTC

# Not using the PHC directly as there seems to be a bug that ends up causing ts2phc to
# error out with "poll returns zero, no events" due to extts being paused while chrony
# reads the PHC
#refclock PHC /dev/ptp1 tai poll 0 delay 6e-8 precision 2e-8 prefer refid PPS
refclock SOCK /run/chrony.clk.gps0.sock offset 0 delay 0.1 poll 0 noselect refid GPS

pool ntp.ubuntu.com        iburst maxsources 4
pool 0.ubuntu.pool.ntp.org iburst maxsources 1
pool 1.ubuntu.pool.ntp.org iburst maxsources 1
pool 2.ubuntu.pool.ntp.org iburst maxsources 2

You should now be able to check that chrony can see all the time sources with the chronyc sources command. You should see something like:

text
MS Name/IP address         Stratum Poll Reach LastRx Last sample               
===============================================================================
#? GPS                           0   0    42     3    +61ms[  +61ms] +/-   51ms
^+ prod-ntp-3.ntp1.ps5.cano>     2  10   377   667   +108us[ +108us] +/-   84ms
^+ prod-ntp-4.ntp1.ps5.cano>     2  10   377   652  -1142us[-1142us] +/-   85ms
^+ prod-ntp-5.ntp4.ps5.cano>     2  10   367   318  -2105us[-2105us] +/-   83ms
^+ alphyn.canonical.com          2  10   377   23m  -1801us[-1898us] +/-   74ms
^+ 23.186.168.131                2  10   377   28m  -2714us[-2263us] +/-   52ms
^+ ntp.maxhost.io                2  10   371   741  -1036us[-1096us] +/-   50ms
^+ ntp4.glypnod.com              2  10   377   377   -595us[ -595us] +/-   62ms
^* pool-96-231-54-40.washdc>     1  10   377   683  +1366us[+1309us] +/-   51ms

Add The Key Ingredient: PTP

This is where I found it useful not to use the Ubuntu packages and use the latest and greatest version. You can get the source or binary builds form the Linux PTP Project . I built mine from source, which is easy as there are few (if any) dependencies. Their documentation was easy to follow. The thing I should note is that because of this, my binaries ended up in /usr/local/sbin instead of the more typical /usr/sbin that they’d be in had I installed a system package.

Whip the PHC Using PPS Timestamps

The PTP4L project provides a suite of applications besides just the PTP server. One is called ts2phc and its job is to grab the PPS timestamps from the network card and use them to discipline the PHC.

Below is the configuration I used. Some things to note: On the TimeNIC, the PPS input is on SDP1, thus the ts2phc.pin_index of 1. Also, the driver timestamps both rising and falling edge, but ts2phc knows how to deal with this. We set the polarity to both and the rough pulse width and ts2phc will only worry about rising edge timestamps and ignore the falling edge.

/etc/linuxptp/ts2phc.conf
[global]
leapfile /usr/share/zoneinfo/leap-seconds.list
step_threshold 0.1
clock_servo linreg
ts2phc.pulsewidth 100000000
ts2phc.tod_source generic
#logging_level 7
# Change this to the interface you're using for PTP (and have the PPS going to)
[enp2s0]
ts2phc.pin_index 1
ts2phc.extts_polarity both
ts2phc.perout_phase 0

Then there’s the small issue of starting the ts2phc on boot. We do this by adding a new systemd service:

/etc/systemd/system/ts2phc.service
[Unit]
Description=Synchronize PTP hardware clock from external timestamp
After=time-sync.target
After=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/sbin/ts2phc -s generic -f /etc/linuxptp/ts2phc.conf
[Install]
WantedBy=multi-user.target

From there, it’s the usual set of commands to enable and start it:

shell
sudo systemctl enable ts2phc
sudo systemctl restart ts2phc

You can then run sudo systemctl status ts2phc after a few seconds and you should see something like the following:

text
● ts2phc.service - Synchronize PTP hardware clock from external timestamp
     Loaded: loaded (/etc/systemd/system/ts2phc.service; enabled; preset: enabled)
     Active: active (running) since Sun 2025-06-01 01:00:13 PDT; 1 day 22h ago
   Main PID: 2189 (ts2phc)
Tasks: 1 (limit: 4586)
     Memory: 244.0K (peak: 1012.0K)
        CPU: 44.843s
     CGroup: /system.slice/ts2phc.service
             └─2189 /usr/local/sbin/ts2phc -s generic -f /etc/linuxptp/ts2phc.conf
Jun 02 23:25:27 voltaire ts2phc[2189]: [170452.636] /dev/ptp1 offset          3 s2 freq     -52
Jun 02 23:25:28 voltaire ts2phc[2189]: [170453.636] /dev/ptp1 offset          3 s2 freq     -49
Jun 02 23:25:29 voltaire ts2phc[2189]: [170454.636] /dev/ptp1 offset         26 s2 freq     -26
Jun 02 23:25:30 voltaire ts2phc[2189]: [170455.636] /dev/ptp1 offset          1 s2 freq     -38
Jun 02 23:25:31 voltaire ts2phc[2189]: [170456.636] /dev/ptp1 offset        -13 s2 freq     -51
Jun 02 23:25:32 voltaire ts2phc[2189]: [170457.636] /dev/ptp1 offset        -14 s2 freq     -65
Jun 02 23:25:33 voltaire ts2phc[2189]: [170458.636] /dev/ptp1 offset          0 s2 freq     -52
Jun 02 23:25:34 voltaire ts2phc[2189]: [170459.636] /dev/ptp1 offset          1 s2 freq     -51
Jun 02 23:25:35 voltaire ts2phc[2189]: [170460.636] /dev/ptp1 offset          0 s2 freq     -51
Jun 02 23:25:36 voltaire ts2phc[2189]: [170461.636] /dev/ptp1 offset          0 s2 freq     -51

This shows the PHC is tracking the PPS pretty well. The offset is the offset between the last PPS timestamp and the PHC clock (in nanoseconds). The freq is the frequency adjustment (in ppb) being applied to the PHC to keep it tracking the PPS.

And Serve Your PTP

And now the last thing to do is get the PTP server running. Here’s the configuration I use:

/etc/linuxptp/ptp4l.conf
[global]
masterOnly 1
clockClass 6
clockAccuracy 0x27

And just like the ts2phc application above, we’ll want to add a systemd service to start it. Note, the enp2s0 is the TimeNIC board in my machine. Yours might be different, so you’ll want to change it here if it is.

/etc/systemd/system/ptp4l.conf
[Unit]
Description=PTP Timeserver
After=time-sync.target
After=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf -i enp2s0 -H -4
[Install]
WantedBy=multi-user.target

And then enable and start ’er up:

shell
sudo systemctl enable ptp4l
sudo systemctl restart ptp4l

And finally, to check that everything is going well, if you run sudo systemctl status ptp4l after a few seconds, you should see something like:

text
● ptp4l.service - PTP Timeserver
     Loaded: loaded (/etc/systemd/system/ptp4l.service; enabled; preset: enabled)
     Active: active (running) since Sun 2025-06-01 11:22:28 PDT; 1 day 12h ago
   Main PID: 3553 (ptp4l)
      Tasks: 1 (limit: 4586)
     Memory: 292.0K (peak: 592.0K)
        CPU: 1min 10.275s
     CGroup: /system.slice/ptp4l.service
             └─3553 /usr/local/sbin/ptp4l -f /etc/linuxptp/ptp4l.conf -i enp2s0 -H -4
Jun 01 11:22:28 voltaire systemd[1]: Started ptp4l.service - PTP Timeserver.
Jun 01 11:22:28 voltaire ptp4l[3553]: [40673.865] selected /dev/ptp1 as PTP clock
Jun 01 11:22:28 voltaire ptp4l[3553]: [40673.868] port 1 (enp2s0): INITIALIZING to LISTENING on INIT_COMPLETE
Jun 01 11:22:28 voltaire ptp4l[3553]: [40673.868] port 0 (/var/run/ptp4l): INITIALIZING to LISTENING on INIT_COMPLETE
Jun 01 11:22:28 voltaire ptp4l[3553]: [40673.868] port 0 (/var/run/ptp4lro): INITIALIZING to LISTENING on INIT_COMPLETE
Jun 01 11:22:36 voltaire ptp4l[3553]: [40681.788] port 1 (enp2s0): LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
Jun 01 11:22:36 voltaire ptp4l[3553]: [40681.788] selected local clock 54494d.fffe.450016 as best master
Jun 01 11:22:36 voltaire ptp4l[3553]: [40681.788] port 1 (enp2s0): assuming the grand master role