Time Card mini adds Pi, GPS, and OCXO to your PC

For LTX 2023, I built this:

CM4 Timecard mini GPS locked

This build centers around the Time Card mini. Typically you'd install this PCI Express card inside another computer, but in my case, I just wanted to power the board in a semi-portable way, and so I plugged it into a CM4 IO Board.

The Time Card mini is a PCIe-based carrier board for the Raspberry Pi Compute Module 4, and by itself, it allows you to install a CM4 into a PC, and access the CM4's serial console via PCIe.

But the real power comes in 'sandwich' boards:

Time Card GPS sandwich boards with Raspberry Pi CM4

For the Raspberry Pi 4, people add on HAT—Hardware Attached on Top. But this configuration reverses that trend—the hardware addons are on bottom, so I'm calling it a Pi sandwich:

Time Card with U-blox GPS sandwich and CM4

Timebeat is selling a variety of GPS module sandwich boards, and the one I'm using in my build is the most premium, pairing a U-blox LEA-M8F with the Time Card mini.

(This blog post is a summary of my video on the Time Card mini on YouTube, embedded below:)

These modules don't run cheap—the LEA-M8F is a little over $250. But it's one of the best GPS modules on the market, and has another feature time nerds love: it can use an external OCXO (Oven-Controlled Crystal Oscillator). And Timebeat sent me a SiTime module capable of keeping a second to within 5 ppb!

SiTime OCXO on Time Card mini

An oven inside that little chip? Yes.

The OCXO isn't for baking cookies, though, it's a special highly-accurate oscillator. Much more accurate than the tiny little quartz crystals used in itty-bitty oscillators you'll find on practically every computing device, ever.

Quartz crystals are pretty accurate, but under normal conditions, they can drift as the temperature changes. OCXOs heat up the crystal inside to something like 100°C, and the stable high temperature makes it way more stable. In the case of the SiTime, accurate within 5 ppb of a true second.

Only CSACs (Chip-Scale Atomic Clocks) and exotic clocks like the NIST F2 cesium fountain are substantially better—though they certainly wouldn't look small compared to a Raspberry Pi CM4!

But why the need for accurate holdover if you already have GPS? GPS provides incredibly accurate time, and is the time base for most modern devices that need better-than-NTP accuracy.

Well, GPS isn't perfect. Like if you're inside, it's hard to get a good GPS signal.

And if you're in a noisy radio environment—or heck, maybe even over in Ukraine where Russia's been jamming GPS signals!—having accurate clock holdover is important.

But what for? NTP (Network Time Protocol) is plenty accurate if you just need to know time down to a few milliseconds.

But every day, more and more things rely on high-precision clocks:

  • VR uses accurate clocks for more accurate positioning data, down to the millimeter.
  • Submarines require precision clocks for anything from positioning to sonar.
  • Radio telescopes coordinate incoming data using highly-accurate clocks.
  • Data centers and banks timestamp all transactions for global and distributed sync.

Or—in my case—I wanted a timer to help manage my LTX 2023 Homelab Creator Livestream! To make sure we don't ramble on too long, I built this:

Timecard GPS locked with CM4 and Blinkstick Nano

This thing is kind of a frankenstein board, but I'm happy with it. I have the Time card mini plugged into a Compute Module 4 IO board using a 90° PCIe riser. The U-blox LEA-M8F is on the sandwich board with the SiTime OCXO connected, then a Compute Module 4 sits on top.

I have a USB Blinkstick to indicate how much time is left, splitting up an hour into 5 minute segments:

  • For the first three minutes, it is green.
  • For the fourth minute, it changes to amber.
  • For the fifth minute, it starts orange, then changes to red with 30 seconds left.
  • With 10 seconds remaining, it switches to flashing red.

And finally, there's an Adafruit miniPiTFT 1.3" display showing time and GPS status in the style of WOPR:

WOPR service Adafruit display HAT

My goal? To have the most accurate clock at LTX.

And to do that, I didn't even have to use half the features of the Time Card mini!

  • With the full height bracket, I could plug in a pulse-per-second input and output, or even another external 10 Mhz clock reference!
  • I could pair the Time Card mini with a NIC with PPS input (e.g. Nvidia Connect-X 6, Intel i225, etc.) for datacenter-scale PTP services.
  • I could even set up the CM4 itself for hardware PTP timestamping—the NIC supports it, though probably only for a few hundred systems.
  • Plugging it into a PC would expose the CM4 serial console and the U-blox directly to the host PC, making it easy to control without even going over the network (this is the first for a Pi-on-PCIe board, AFAICT).

For a deeper dive into my entire bringup experience—including testing the CM4 Pi OS image Timebeat maintains—check out this issue on my Pi PCIe site.

To be completely transparent, Timebeat sent these to me for testing. I probably couldn't justify buying one just for the fun of it, and technically, just using NTP or an external GPS module alone would be plenty accurate for the purposes of a 1-hour livestream! (Just... a lot less fun).

Even so, this solution is cheaper than most dedicated time appliances.

How it's made

For LTX 2023, I will be streaming from deep within the expo hall. Because I can't rely on GPS signals getting through, I'm powering the whole setup using a 12v 3000 mAh battery.

That's plugged into the CM4 IO Board, and the Time Card mini is plugged into the IO Board's PCIe slot (it doesn't require a Pi to be installed to provide power to the PCIe slot).

I rely on Timebeat's software to acquire the GPS signal and set the system clock (plus provide PTP out through the Pi's built-in Ethernet—should someone at LTX require it!). Then I wrote three small Python services to manage the Blinkstick and Adafruit display:

  • One service watches Timebeat to see if GPS time is locked in, and writes a file to the filesystem to indicate whether it is locked or not.
  • Another service prints text on the display: the time, and whether GPS time is locked in.
  • A third service manages the Blinkstick, changing colors at the proper times, dividing the hour up into 5 minute segments.

I don't know if I'll have a good time or a bad time, but I can guarantee I'll have the most accurate time at LTX.

Unless I find someone carrying around a stratum 0 clock in their backpack...

All the code's open source in my ltx2023 repo on GitHub, and I include detailed setup instructions. Thanks to my Patrons and Sponsors, I can continue open sourcing all my work!

Conclusion

LTX is just two days away. Make sure you sign up to get notified when our livestream begins!

Chris (from Crosstalk Solutions) and I will interview:

It's gonna be a blast, and we're hoping to raise some money for the ITDRC. Make sure to watch the livestream on Saturday, July 29!

Further reading

ltx
ltx2023
video
youtube
time card
gps
ocxo
raspberry pi
pcie

Comments

My immediate question is how much power does the OXCO use? I have also heard you can improve the accuracy of XO's in general if you model xtal aging, though if you also have GPS that can handle it.

I'm confused by the configuration options and the ala cart choices on their website. What would I need as a minimum to just have the fancy crystal oven to keep my time precise locally? I can sync upstream for NTP, relative time differences between my homelab and the internet doesn't really matter. For that reason the GPS module seems unnecessary?

Since you're dealing with highly-accurate time, it's worth remembering that the GPS time signal can't tell you time on its own, since it's the time the signal was broadcast. Its purpose is to help you resolve the positioning calculations by comparing with a high-accuracy clock at or near the receiver location. To use it for timekeeping you need the reverse: highly-accurate location information, which by definition you can't get from the GPS signal without accurate time.

Jeff (or anyone for that matter) - Do you know if the Timecard Mini is compatible with any of the CM4 carrier boards with power-over-Ethernet?

I’ve spent days trying to figure it out, but without buying a board + paying for shipping multiple times, I just don’t know. :(

That seems like the ultimate option to me!

FYI, One can use the bounceback test to check if the clock's are not synchronized relative to the bb time. (you may need to compile the latest iperf from master) https://sourceforge.net/projects/iperf2/

Below is an example:

root@raspberrypi:/usr/local/src/iperf2-code# iperf -c 192.168.1.33 -e -i 1 --trip-times --bounceback --bounceback-period 0
------------------------------------------------------------
Client connecting to 192.168.1.33, TCP port 5001 with pid 38489 (1/0 flows/load)
Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack)
TCP congestion control using cubic
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 85.0 KByte (default)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 1] local 192.168.1.32%eth0 port 42258 connected with 192.168.1.33 port 5001 (prefetch=16384) (bb w/quickack req/reply/hold=100/100/0) (trip-times) (sock=3) (icwnd/mss/irtt=14/1448/265) (ct=0.43 ms) on 2023-10-18 14:49:24.047 (PDT)
[ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS(avg)
[ 1] 0.00-1.00 sec 998 KBytes 8.18 Mbits/sec 10223=0.093/0.078/1.057/0.027 ms 0 14K/62 us 10729 rps
[ 1] 1.00-2.00 sec 1.06 MBytes 8.93 Mbits/sec 11166=0.086/0.077/0.225/0.003 ms 0 14K/61 us 11631 rps
[ 1] 2.00-3.00 sec 1.07 MBytes 8.94 Mbits/sec 11172=0.086/0.077/0.434/0.004 ms 0 14K/60 us 11633 rps
[ 1] 3.00-4.00 sec 1.06 MBytes 8.87 Mbits/sec 11092=0.087/0.079/0.376/0.005 ms 0 14K/62 us 11547 rps
[ 1] 4.00-5.00 sec 979 KBytes 8.02 Mbits/sec 10025=0.096/0.090/0.442/0.004 ms 0 14K/61 us 10402 rps
[ 1] 5.00-6.00 sec 960 KBytes 7.86 Mbits/sec 9831=0.098/0.090/0.413/0.008 ms 0 14K/61 us 10213 rps
[ 1] 6.00-7.00 sec 984 KBytes 8.06 Mbits/sec 10080=0.096/0.090/0.150/0.002 ms 0 14K/61 us 10461 rps
[ 1] 7.00-8.00 sec 983 KBytes 8.06 Mbits/sec 10070=0.096/0.090/0.168/0.002 ms 0 14K/61 us 10452 rps
[ 1] 8.00-9.00 sec 984 KBytes 8.06 Mbits/sec 10074=0.096/0.092/0.149/0.002 ms 0 14K/61 us 10455 rps
[ 1] 9.00-10.00 sec 982 KBytes 8.04 Mbits/sec 10056=0.096/0.087/0.446/0.004 ms 0 14K/64 us 10434 rps
[ 1] 0.00-10.01 sec 9.90 MBytes 8.29 Mbits/sec 103791=0.093/0.077/1.057/0.011 ms 0 14K/1729 us 10795 rps
[ 1] 0.00-10.01 sec OWD (ms) Cnt=103791 TX=0.770/-0.357/4.012/1.683 RX=-0.678/-3.709/0.789/1.686 Asymmetry=2.299/0.001/7.702/2.857
[ 1] 0.00-10.01 sec OWD-TX-PDF: bin(w=100us):cnt(103791)=1:14920,2:263,3:275,4:271,5:273,6:272,7:272,8:272,9:271,10:272,11:270,12:275,13:273,14:272,15:269,16:272,17:272,18:272,19:272,20:273,21:271,22:269,23:273,24:271,25:272,26:273,27:273,28:270,29:272,30:271,31:272,32:271,33:273,34:270,35:271,36:272,37:269,38:20475,39:625,40:2,41:1 (5.00/95.00/99.7%=1/100000/100000,Outliers=0,obl/obu=57994/0)
[ 1] 0.00-10.01 sec OWD-RX-PDF: bin(w=100us):cnt(103791)=1:12175,2:3139,3:3132,4:3136,5:51552,6:5,8:2 (5.00/95.00/99.7%=1/100000/100000,Outliers=0,obl/obu=30650/0)
[ 1] 0.00-10.01 sec BB8-PDF: bin(w=100us):cnt(103791)=1:100388,2:3136,3:258,4:4,5:4,11:1 (5.00/95.00/99.7%=1/1/2,Outliers=0,obl/obu=0/0)
[ 1] 0.00-10.01 sec Clock sync error count = 92028

Below is an example where the clock are sync'd - there is no Clock sync error count message

root@raspberrypi:/usr/local/src/iperf2-code# iperf -c 192.168.1.33 -e -i 1 --trip-times --bounceback --bounceback-period 0
------------------------------------------------------------
Client connecting to 192.168.1.33, TCP port 5001 with pid 38492 (1/0 flows/load)
Bounceback test (req/reply size = 100 Byte/ 100 Byte) (server hold req=0 usecs & tcp_quickack)
TCP congestion control using cubic
TOS set to 0x0 and nodelay (Nagle off)
TCP window size: 85.0 KByte (default)
Event based writes (pending queue watermark at 16384 bytes)
------------------------------------------------------------
[ 1] local 192.168.1.32%eth0 port 46112 connected with 192.168.1.33 port 5001 (prefetch=16384) (bb w/quickack req/reply/hold=100/100/0) (trip-times) (sock=3) (icwnd/mss/irtt=14/1448/291) (ct=0.46 ms) on 2023-10-18 14:51:27.555 (PDT)
[ ID] Interval Transfer Bandwidth BB cnt=avg/min/max/stdev Rtry Cwnd/RTT RPS(avg)
[ 1] 0.00-1.00 sec 1003 KBytes 8.22 Mbits/sec 10270=0.093/0.079/1.218/0.025 ms 0 14K/61 us 10775 rps
[ 1] 1.00-2.00 sec 1.06 MBytes 8.93 Mbits/sec 11166=0.086/0.078/0.322/0.005 ms 0 14K/60 us 11628 rps
[ 1] 2.00-3.00 sec 1.07 MBytes 8.94 Mbits/sec 11175=0.086/0.078/0.263/0.003 ms 0 14K/61 us 11635 rps
[ 1] 3.00-4.00 sec 1.06 MBytes 8.93 Mbits/sec 11167=0.086/0.078/0.313/0.004 ms 0 14K/60 us 11630 rps
[ 1] 4.00-5.00 sec 998 KBytes 8.18 Mbits/sec 10224=0.094/0.080/0.410/0.006 ms 0 14K/62 us 10614 rps
[ 1] 5.00-6.00 sec 958 KBytes 7.85 Mbits/sec 9811=0.098/0.088/0.432/0.009 ms 0 14K/61 us 10187 rps
[ 1] 6.00-7.00 sec 982 KBytes 8.05 Mbits/sec 10060=0.096/0.090/0.306/0.003 ms 0 14K/61 us 10437 rps
[ 1] 7.00-8.00 sec 980 KBytes 8.03 Mbits/sec 10035=0.096/0.090/0.927/0.015 ms 0 14K/61 us 10409 rps
[ 1] 8.00-9.00 sec 981 KBytes 8.04 Mbits/sec 10048=0.096/0.090/0.763/0.010 ms 0 14K/61 us 10424 rps
[ 1] 9.00-10.00 sec 982 KBytes 8.04 Mbits/sec 10054=0.096/0.092/0.287/0.003 ms 0 14K/61 us 10432 rps
[ 1] 0.00-10.01 sec 9.92 MBytes 8.31 Mbits/sec 104011=0.092/0.078/1.218/0.012 ms 0 14K/1009 us 10816 rps
[ 1] 0.00-10.01 sec OWD (ms) Cnt=104011 TX=0.040/0.030/0.680/0.005 RX=0.053/0.045/0.887/0.008 Asymmetry=0.013/0.000/0.847/0.006
[ 1] 0.00-10.01 sec OWD-TX-PDF: bin(w=100us):cnt(104011)=1:103967,2:31,3:9,4:2,6:1,7:1 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0)
[ 1] 0.00-10.01 sec OWD-RX-PDF: bin(w=100us):cnt(104011)=1:103769,2:226,3:13,4:1,8:1,9:1 (5.00/95.00/99.7%=1/1/1,Outliers=0,obl/obu=0/0)
[ 1] 0.00-10.01 sec BB8-PDF: bin(w=100us):cnt(104011)=1:100646,2:3146,3:198,4:10,5:4,6:1,7:1,8:2,10:2,13:1 (5.00/95.00/99.7%=1/1/2,Outliers=0,obl/obu=0/0)