Category Archives: Linux

Building My Own GPS Stratum-1 Time Server on a Raspberry Pi

Building My Own GPS Stratum-1 Time Server on a Raspberry Pi

Time silently hums away in the background, affecting everything we do today. Wake up, go to work, go to an appointment; all happen at specific times. Computers are no different, constantly logging time in logs, showing time in applications, and used for scheduling. In general, no one really cares about time because it's typically "accurate enough" for most tasks. Most computers just automatically have the correct time and are accurate down to a second or so, with most people perfectly content that it's technically approximately right. That being said, I'm neurotic and want nuclear-science-lab accuracy, knowing precisely what time it is.

While pool.ntp.org is a fantastic source of time, why settle when you can have sub-microsecond precision? No one is going to have a good time when you're technically having a bad time.

For this project, I wanted to see how precise time can be in my house, owning it end to end — no leaning on someone else's server out on the internet. In the industry, precision time is provided by what are called stratum-1 time servers and can sometimes be thousands of dollars. Given I'm cheap and don't have a money tree, I decided to build my own stratum-1 time server in a relatively affordable way: a Raspberry Pi that pulls time straight off the GPS satellites overhead and serves it to everything on my LAN, no internet required.

Along the way the project grew a second half, which is to visualize the satellite tracking and time sync. Once I had gpsd and chrony working, I wrote a small daemon that enables a live view of the calculated time, full gpsd + chrony stats, a satellite sky view, and an interactive 3D globe showing the GNSS constellation in space.

This build stands on the shoulders of the classic jacobdeane.com GPS time server guide, with two deliberate modernizations:

  • chrony instead of ntpd. chrony has native PPS support and a clean socket backend, so you skip the from-source NTP compile entirely.
  • Current Raspberry Pi OS paths. On Bookworm (and Trixie) the boot config lives at /boot/firmware/config.txt, not the old /boot/config.txt (which is now just a placeholder).

How it all fits together

Before wiring anything up, here's a quick overview :

   GPS/RTC HAT (u-blox + PPS)           Raspberry Pi 3 / 4 (host)
 ┌───────────────────────────┐    ┌──────────────────────────────────────────────┐
 │  receiver                 │    │                                              │
 │   • NMEA over UART  ──────┼──▶│  /dev/serial0 ─▶ gpsd ─┐                      │
 │   • PPS on GPIO18   ──────┼──▶│  /dev/pps0 ────────────┼─▶ chrony (stratum 1) │
 └───────────────────────────┘   │                        │        │  serves NTP │
                                 │   gpsd JSON :2947 ─┐    │        ▼   to LAN    │
                                 │   chronyc -c ──────┼────┴─▶ Rust daemon        │
                                 │   (optional) NAVCEN ───▶  :8080 (configurable) │
                                 │                    ├─▶ /data     (SSE telemetry)│
                                 │                    ├─▶ /satinfo  (opt. sat info)│
                                 │                    └─▶ /         (dashboard)   │
                                 └───────────────────────────────┬───────────────┘
                                                                │
                                                   browser ◀────┘  dashboard
  1. The timekeeping path — gpsd reads NMEA + PPS and feeds chrony; chrony disciplines the kernel clock and serves NTP to my network. This is what makes it a stratum-1 server. If you stop reading after Part 6, you've got a fully working time server.
  2. The telemetry path — a small daemon written in Rust that reads the same gpsd data plus chrony's stats and streams them to the dashboard over Server-Sent Events. It's strictly read-only and has zero effect on timekeeping accuracy.

What you'll need

Here's the parts list I used:

PartNotes
Raspberry Pi 3 or 4Both work identically here. The Pi 4 has dedicated Gigabit Ethernet; on the Pi 3, Ethernet shares the USB bus, so keep other USB devices off it. The Pi 5 needs different UART steps and isn't covered.
GPS HAT with PPSI used the Uputronics Raspberry Pi GPS/RTC Expansion Board, and it's what this guide is written against. PPS is mandatory for stratum-1; NMEA alone is only good to a few hundred milliseconds.
GPS antenna (active, timing-grade)Amphenol PCTEL 3978D-HR-DH-W — the exact antenna I used: a weatherproof (IP67) GPS L1 dome antenna with a built-in LNA + filter (~40 dB gain) and a TNC-female connector. A real step up from a puck. Needs a clear-ish view of the sky.
Antenna mounting bracketAmphenol PCTEL MMK1925 — the stainless-steel L-bracket for the WS397x-series dome (fits the 3978D). This and the antenna above are what I actually mounted.
Antenna coaxA run with the right ends for your antenna and HAT (see the note below). I didn't spring for timing-grade coax — I used a cheap SMA extension cable like this one and it's been perfectly fine.

All in all, the project cost ~$240 (inc. shipping, tax, etc). I didn't include the cost of a raspberry pi since I had one unused from a prior project, but assuming you bought a new Pi; you are ~$360.

Notes before we get started

Check your HAT's PPS pin. This guide assumes GPIO18 (the Uputronics default). Some HATs (e.g. Adafruit) use GPIO4. Adjust the gpiopin= value accordingly. Guess wrong and PPS will simply never show up.

Antenna connector & bias (if you're using the dome antenna above). The 3978D outputs TNC female, while most GPS HATs (Uputronics included) use SMA — so plan a coax run with the right ends (TNC-male at the antenna, SMA-male at the HAT) or grab an adapter. It's an active antenna (built-in LNA), so the HAT has to supply DC bias voltage on the coax; the Uputronics HATs do this, but confirm yours does. Being IP67 and bracket-mounted, it's meant to live outdoors on a mast with a clear sky view.

You don't have to mount it outdoors. The 3978D is a weatherproof, IP67, mast-mount dome. It's built to bolt onto a roof and shrug off the weather. I didn't put it on a roof. I bolted it to its MMK1925 bracket, set the whole thing up in my attic under asphalt shingles and 5/8" OSB, and ran it back to the HAT with a cheap SMA extension cable instead of proper timing-grade coax. On paper that's leaving signal on the table three different ways. In practice it's been tracking ~16-20 satellites and holding a sub-microsecond lock the entire time. So: buy the better antenna over fancy cable if you are on a limited budget and don't let "needs a clear view of the sky" scare you into a rooftop install (especially if you are HOA bound and limited to what you can mount outside). An attic on the sky-facing side of the house is likely going to do it for you.

Part 1 — The base OS

Flash Raspberry Pi OS Lite (64-bit) with Raspberry Pi Imager. In the Imager's advanced settings (the gear icon), enable SSH and set the hostname (I'll use timeserver for this guide) and locale before writing. Note: this Pi runs headless, which means you don't have a monitor connected to it, so you want to be able to reach it on first boot.

A word on Bookworm vs Trixie. As of mid-2026 there are active reports of a Trixie serial regression where the GPIO UART stops receiving GPS data on hardware that worked fine on Bookworm (a suspected kernel/PL011 driver issue). For a device whose entire job is reliable time, Bookworm might be the safer choice today; but so far things have worked fine for me. The config paths and overlays are identical — but if NMEA never appears despite correct wiring, jump to the Trixie diagnostic in Troubleshooting. There's also a Debian 12 vs 13 appendix at the bottom.

Boot, SSH to the Pi, and update first:

sudo apt update && sudo apt full-upgrade -y
sudo reboot

Part 2 — Freeing up the hardware UART

Here's the first "gotcha" the Pi throws at you. The Pi 3/4 wire the good UART (PL011) to the onboard Bluetooth by default. We need that UART for the GPS, and we also need to stop Linux from using the serial port as a login console.

2a. Disable the serial login shell (but keep the hardware port)

sudo raspi-config

Under Interface Options → Serial Port:

  • login shell accessible over serial? → No
  • serial port hardware enabled? → Yes

Choose Finish but don't reboot yet, we've got more config to add, and I'd rather reboot once.

2b. Disable Bluetooth to reclaim the UART

sudo vi /boot/firmware/config.txt

New to vi? Here's the reader's digest: press i to start typing (insert mode), make your edits, then hit Esc and type :wq followed by Enter to write and quit. If you fat-finger something and want to bail without saving, Esc then :q! throws your changes away. That's genuinely 90% of what you need to survive vi.

Add this at the end (we'll drop the PPS line in here too in Part 3, so it's a single reboot):

# Reclaim the PL011 UART for GPS by disabling onboard Bluetooth
enable_uart=1
dtoverlay=disable-bt

Older guides use dtoverlay=pi3-disable-bt; the current canonical name is just disable-btraspi-config may have already added an enable_uart=1 line — a duplicate is harmless, but you can keep just one if it bugs you.

Then stop the Bluetooth-UART service:

sudo systemctl disable hciuart

On a Lite image (and commonly on Trixie) this may report Unit hciuart.service does not exist — and that's totally fine. It ships with the pi-bluetooth package, which isn't always installed, and disable-bt already turns Bluetooth off at the device-tree level. The overlay is what actually matters here; this step is just belt-and-suspenders.

Part 3 — Enabling PPS

PPS (pulse-per-second) is the secret sauce. NMEA sentences tell you what second it is; the PPS pulse tells you exactly when that second begins, down to the nanosecond. No PPS, no real stratum-1.

3a. Load the PPS-over-GPIO overlay

Still in /boot/firmware/config.txt, add (adjust gpiopin for your HAT):

# Pulse-Per-Second input on GPIO18
dtoverlay=pps-gpio,gpiopin=18

Then load the kernel module at boot and reboot so both the UART and PPS changes take effect:

echo 'pps-gpio' | sudo tee -a /etc/modules
sudo reboot

3b. Verify raw GPS and PPS

First confirm the UART is on the GPIO header, then read raw NMEA:

ls -l /dev/serial0        # expect: /dev/serial0 -> ttyAMA0
sudo cat /dev/serial0

Always use /dev/serial0 — it's a symlink that always points to whichever UART is on the GPIO header, so you don't have to care whether it's ttyAMA0 or ttyS0 today. Ctrl-C to stop.

If you see garbled characters instead of clean $GNRMC/$GNGGA lines, don't panic, that's a baud-rate mismatch, not a fault. Bytes are flowing; the port's just reading them at the wrong speed. Set the rate explicitly and re-read until it's clean:

sudo stty -F /dev/serial0 9600 raw   && sudo cat /dev/serial0   # common u-blox default
sudo stty -F /dev/serial0 38400 raw  && sudo cat /dev/serial0
sudo stty -F /dev/serial0 115200 raw && sudo cat /dev/serial0   # mine with the Uputronics unit

Write down the rate that produces readable NMEA, you'll need it for gpsd in Part 4. It's a property of your GPS module, not the OS. Many u-blox modules use 9600, but others (including some Uputronics units) run at 115200. Mine was 115200.

Now confirm the pulse itself. This needs a satellite fix; most HATs blink a timepulse LED once they've locked:

sudo apt install -y pps-tools
sudo ppstest /dev/pps0

You want a new assert line every second, timestamped right on the whole second:

source 0 - assert 1699999590.000001416, sequence: 197334 - clear ...
source 0 - assert 1699999591.000000698, sequence: 197335 - clear ...

No output usually just means no fix yet, wait a couple mins or try moving the antenna to a different location. If you are running the antenna in a faraday cage, expect no results since it won't find a satellite.

Part 4 — gpsd

sudo apt install -y gpsd gpsd-clients
sudo vi /etc/default/gpsd

Point it at your HAT (using the baud rate you found in Part 3b):

START_DAEMON="true"
USBAUTO="false"
DEVICES="/dev/serial0 /dev/pps0"
GPSD_OPTIONS="-n -s 115200"

-n polls the GPS immediately at boot; -s 115200 pins the serial speed so gpsd doesn't autobaud its way into garbage. Enable and check:

sudo systemctl enable --now gpsd
sudo systemctl restart gpsd      # ensure it picks up the new config
gpsmon        # or: cgps -s

gpsmon shows the raw feed (fast-scrolling UBX packets are normal and healthy); cgps -s gives a calmer table. You want a 3D fix, satellites with good SNR, low DOP values, and a PPS line with a near-zero offset. That PPS line is the confirmation I was chasing.

Tip: gpsd takes exclusive control of the port. If you want to cat /dev/serial0 again for debugging, stop gpsd first: sudo systemctl stop gpsd gpsd.socket.

Part 5 — chrony with the socket backend

sudo apt install -y chrony

Here's the key design point, and the reason chrony is such an upgrade over the old ntpd dance: gpsd hands the precise PPS timestamp to chrony through a Unix socket (chrony's SOCK refclock). gpsd automatically writes to /run/chrony.<device>.sock once chrony has created it — neatly sidestepping the classic headache where chrony drops privileges and can no longer read /dev/pps0 directly.

sudo vi /etc/chrony/chrony.conf

Add this at the end:

# --- GPS + PPS reference clocks -------------------------------------------

# Coarse absolute time from gpsd (NMEA) via shared memory.
# 'noselect' = never sets the clock, only gives the PPS pulse a whole-second
# label. Tune 'offset' after calibration (Part 7d) if PPS won't lock.
refclock SHM 0 refid GPS precision 1e-1 offset 0.0 delay 0.2 noselect

# Precise PPS via gpsd's socket. gpsd writes here automatically.
# 'lock GPS' anchors each pulse to the GPS second above.
refclock SOCK /run/chrony.pps0.sock refid PPS lock GPS prefer

# Serve time to your LAN (adjust to your subnet) and keep serving during
# brief GPS dropouts.
allow 192.168.0.0/24
local stratum 1

# Optional internet fallback if the antenna fails. Remove for air-gapped use.
pool 2.debian.pool.ntp.org iburst

Restart both, in order — chrony first, so its socket exists by the time gpsd starts:

sudo systemctl restart chrony
sudo systemctl restart gpsd

Confirm the socket exists (it should be owned by root and actually be a socket — note the leading s in the permissions):

ls -l /run/chrony.*.sock      # expect /run/chrony.pps0.sock

Part 6 — Confirming you've actually got a stratum-1 server

Give it 5 minutes to settle, then watch the sources roll in:

watch -n 2 chronyc sources -v

You're looking for #* next to PPS. The asterisk means it's the selected source disciplining the clock. It starts at #? (not enough samples yet), climbs its reachability register (0 → 1 → 3 → 7 → 17 → 37 → 77 → 377), then finally promotes to #*. The internet servers drop to ^- (background sanity checks) once PPS takes over. Watching that register tick up is weirdly satisfying:

MS Name/IP address   Stratum Poll Reach LastRx Last sample
=========================================================================
#? GPS                     0    4  377     14   +42ms[  +42ms] +/- 200ms
#* PPS                     0    4  377     13   -242ns[ -324ns] +/- 1095ns
^- 2.debian.pool.ntp.org   2    6   77     45   -437us[ -436us] +/-  13ms

Then confirm the server itself:

chronyc tracking

Look for Stratum : 1Reference ID … (PPS), and a System time offset down in the sub-microsecond range. That, right there, is a working stratum-1 time server. I may have taken a screenshot.

Reading the numbers: System time / Last offset are your current accuracy (nanoseconds-to-microseconds once locked). RMS offset looks alarming at first (hundreds of µs) because it's a rolling average still dominated by the noisy warm-up; it falls steadily the longer it runs, so don't judge it too early. Root delay near zero and Reference ID = PPS confirm a direct hardware lock.

The falseticker trap. chrony will reject PPS if the GPS (NMEA) label is more than ±200 ms off the true second. If PPS stays stuck at #?, that's almost always the culprit, calibrate the offset (Part 7d).

At this point the time server itself is done. Below is the fun part of visualizing the setup. If you just wanted accurate time, congratulations, you can stop here.

Part 7 — The telemetry daemon

Now to create a live dashboard to visualize everything. The daemon does four things:

  1. Connects to gpsd's JSON socket on 127.0.0.1:2947 — the same protocol libgps speaks, so I get identical data without any C FFI.
  2. Polls chrony once a second with chronyc -c tracking (the -c flag hands back machine-readable CSV from chrony's command socket).
  3. Merges both, streams them to the browser over SSE at /data, and serves the dashboard.
  4. (Optional, off by default) fetches and caches the NAVCEN GPS almanac and serves it at /satinfo for satellite-health enrichment (more in Part 8).

Get the code, the daemon and dashboard live on GitHub: JackStromberg/gps-timeserver-daemon. Clone it directly onto the Pi:

sudo apt install -y git
git clone https://github.com/JackStromberg/gps-timeserver-daemon.git
cd gps-timeserver-daemon
ls -al

You should end up with:

gps-timeserver-daemon/
├── Cargo.toml
├── src/main.rs
└── static/index.html      # already set to live mode (USE_SIMULATION = false)

7a. Install Rust and build

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # press Enter for defaults
source "$HOME/.cargo/env"
cargo --version                                                  # should print a version
sudo apt install -y build-essential                              # C linker (Lite images lack it)
cd gps-timeserver-daemon
cargo build --release

cargo: command not found right after installing? The installer can't modify an already-running shell's PATH. Run source "$HOME/.cargo/env" (or just open a new terminal). If ls ~/.cargo/bin/cargo doesn't exist, the install didn't finish — re-run the curl … | sh line.

error: linker 'cc' not found? Install build-essential (above) and rebuild.

Fair warning: the first build on a Pi is slow. It compiles tokio, axum, reqwest, and the whole rustls TLS stack from scratch (several minutes), and it may look completely frozen at the final link step. That's normal. Subsequent builds are incremental and quick.

7b. Run it

./target/release/gps-timeserver-daemon

It prints its listen address and stays running (it's serving, so it won't hand you back a prompt). Open http://<pi-ip>:8080 from a browser on your LAN and you should see the dashboard light up.

The port is configurable — see Part 8. If 8080 is already taken (a Docker service, say), you'll get AddrInUse; set a different PORT rather than editing code.

7c. The /data contract

The daemon emits one JSON object per second in this shape (the dashboard reads exactly these field names):

{
  "timestamp": "2026-07-07T12:34:56Z",
  "time":   { "gps_time": "...", "system_time": "...", "offset_us": 0.42,
              "time_since_last_fix": 0.3, "leap_seconds": 18 },
  "gps":    { "mode": 3, "satellites_visible": 12, "satellites_used": 9,
              "hdop": 0.8, "vdop": 1.1, "pdop": 1.3, "gdop": 1.5, "tdop": 0.7,
              "ept": 0.005, "latitude": 51.5, "longitude": -0.12, "altitude": 35.2,
              "epx": 3.1, "epy": 4.2, "epv": 6.0, "track": 0.0, "speed": 0.0,
              "climb": 0.0, "eps": 0.5 },
  "chrony": { "reference_id": "PPS", "stratum": 1, "system_time": 0.0000004,
              "last_offset": 0.0000004, "rms_offset": 0.0000012,
              "frequency_ppm": -2.3, "residual_freq_ppm": 0.001, "skew_ppm": 0.02,
              "root_delay": 0.00001, "root_dispersion": 0.00002,
              "update_interval": 16.0, "leap_status": "Normal" },
  "device": { "path": "/dev/serial0", "driver": "u-blox", "subtype": "...",
              "bps": 115200, "parity": "N", "native": true, "cycle": 1.0, "mincycle": 1.0 },
  "version":{ "release": "3.25", "proto_major": 3, "proto_minor": 15 },
  "satellites": [
    { "prn": 5, "system": "GPS", "elevation": 70, "azimuth": 251, "snr": 42, "used": true }
  ]
}

Telemetry::to_json() in src/main.rs is the single source of truth for this contract — if you want to add a field, that's the one place to touch.

7d. Calibrating the GPS offset (optional)

If PPS refuses to lock, let it run ~15 minutes, then read the measured GPS offset:

chronyc sourcestats

Take the GPS source's Offset (say +512ms), convert to seconds, and set it as offset on the SHM 0 line in chrony.conf (e.g. offset 0.512). Restart chrony. This nudges the NMEA label back inside the ±200 ms window so PPS gets accepted — the fix for that falseticker trap I mentioned above.

Part 8 — Configuration via environment variables

I wanted to be able to change behavior without recompiling, so the daemon reads a handful of environment variables at startup. Set them when running manually, or via Environment= lines in the systemd unit (Part 9).

VariableDefaultPurpose
PORT8080Port to bind (binds 0.0.0.0:PORT).
LISTEN_ADDR(unset)Full host:port; overrides PORT. Use 127.0.0.1:8088 to serve only locally, 0.0.0.0:8088 for the LAN.
SATINFO_ENABLED(off)1/true/yes/on enables the optional NAVCEN satellite-info fetcher. Off means the daemon makes zero outbound network calls — safe for air-gapped setups.
SATINFO_INTERVAL_HOURS168 (weekly)How often to refresh the NAVCEN data. The constellation changes only a few times a year, so weekly (or 720 ≈ monthly) is plenty.
SATINFO_URLNAVCEN YUMA almanacOverride the source if needed.

The optional satellite-info fetcher

When SATINFO_ENABLED=1, a background task fetches the NAVCEN GPS almanac (US Coast Guard, public-domain US-government data), caches it in memory, and serves it at /satinfo. The dashboard uses it to add live health status to each GPS satellite's hover tooltip. On failure it retries in an hour; on success it waits the full interval. Left disabled (the default), /satinfo returns {"enabled": false} and the dashboard silently falls back to its built-in reference table — so nothing breaks either way.

Quick checks once you've enabled it:

journalctl -u timeserver-dashboard -n 20 --no-pager   # look for: satinfo: fetched N satellites
curl -s localhost:8080/satinfo | head -c 300          # expect "enabled":true and a satellites object

On data sources & licensing (not legal advice): NAVCEN is public-domain US-government data — no account, redistribution-safe. The dashboard's built-in reference table is assembled from public GPS records. If you instead point live mode at CelesTrak, respect their access etiquette (cache; don't poll more than roughly every 2 hours) and cite them. There's a fuller Licensing appendix at the bottom.

Part 9 — Running it at boot

I want this thing to just come back on its own after a power blip, so let's give it a systemd unit:

sudo vi /etc/systemd/system/timeserver-dashboard.service
[Unit]
Description=GPS Timeserver Dashboard
After=network-online.target gpsd.service chrony.service
Wants=network-online.target

[Service]
# Adjust User and paths to where you put the project.
User=pi
WorkingDirectory=/home/pi/gps-timeserver-daemon
ExecStart=/home/pi/gps-timeserver-daemon/target/release/gps-timeserver-daemon
Restart=on-failure
RestartSec=3

# Optional configuration (see Part 8):
Environment=PORT=8088
# Environment=SATINFO_ENABLED=1
# Environment=SATINFO_INTERVAL_HOURS=168

[Install]
WantedBy=multi-user.target

WorkingDirectory matters more than it looks. The daemon serves static/index.html relative to where it runs, and systemd does not inherit your shell's current directory. Point this at the wrong place and you'll get a 404 for the dashboard.

Enable and verify:

sudo systemctl daemon-reload
sudo systemctl enable --now timeserver-dashboard
systemctl status timeserver-dashboard          # expect: active (running)
journalctl -u timeserver-dashboard -n 5 --no-pager   # confirm the listen address

Whenever you edit the unit file (say, to change an Environment= line), run sudo systemctl daemon-reload and then sudo systemctl restart timeserver-dashboard. Skipping the reload is the classic "my change didn't take" gotcha. systemctl show timeserver-dashboard -p Environment prints what systemd is actually passing, which is handy when you're sure you fixed it but it disagrees.

One nice quality-of-life note: editing only static/index.html (HTML/CSS/JS) needs no rebuild and no restart — the daemon serves it from disk at request time. Just hard-refresh the browser (Ctrl-Shift-R). Only changes to src/main.rs or Cargo.toml require cargo build --release.

Part 10 — Serving time to the rest of the network

Everything so far only disciplines the Pi's own clock. To let other machines sync from it, chrony has to answer NTP requests — and it refuses by default, which is a good safety default but not what we want here.

10a. Match the allow line to your LAN

hostname -I               # your Pi's IP, e.g. 192.168.1.50

If the Pi is 192.168.1.50, your subnet is likely 192.168.1.0/24. Edit chrony.conf so the allow line matches — a whole subnet (allow 192.168.1.0/24), a single host (allow 192.168.1.100), or allow all only if you deliberately mean to serve the public — then restart:

sudo systemctl restart chrony

10b. Confirm it's serving

sudo ss -ulnp 'sport = :123'    # something should be bound to UDP 123
chronyc clients                 # machines that have requested time (empty until clients connect)

If you run a firewall on the same Raspberry Pi (ufwnftables, or Docker-managed iptables), open NTP: sudo ufw allow 123/udp. A plain Lite install has no firewall and needs nothing.

10c. Point your clients at the Pi

Your Pi is stratum 1, so anything syncing to it becomes stratum 2 — perfect for a LAN. Here's how I pointed a few common clients at it:

  • Router: set its NTP server to the Pi's IP — this pushes time locally to every device automatically, using it as an ntp source.
  • Linux: add server <pi-ip> iburst to /etc/chrony/chrony.conf, restart chrony.
  • Windows: w32tm /config /manualpeerlist:"<pi-ip>" /syncfromflags:manual /update then w32tm /resync.
  • macOS: System Settings → General → Date & Time, or sudo sntp -sS <pi-ip>.

Verify from a client with chronyc -h <pi-ip> trackingsntp <pi-ip>, or ntpdate -q <pi-ip>. Back on the Pi, chronyc clients will list everyone pulling time from you.

Part 11 — Actually using the dashboard

Open http://<pi-ip>:<port> (default 8080, or 8088 in my examples above). Everything updates once per second from the live feed.

Time header. A large clock with millisecond precision, plus GPS time, system time, and the chrony offset. The UTC / Local toggle under the clock switches every time on the page between UTC and your browser's local zone, and it remembers your choice across reloads.

Orbital globe (the hero panel, and my favorite). A holographic Earth with each GNSS satellite placed in true 3D from its azimuth/elevation, color-coded by constellation, with beams running from the satellites in use down to a marker at your location.

  • Drag to rotate (mouse or touch): horizontal spins around the poles, vertical tilts.
  • Home button (top-right) or double-click the globe to glide back to the default view.
  • Hover a satellite for its details: PRN, constellation, elevation, azimuth, SNR, and whether it's in the solution — plus, for GPS, catalog facts (block, manufacturer, clock type, launch year, name, SVN) and, if /satinfo is enabled, live health.
  • Hover a legend entry (GPS/GLONASS/Galileo/BeiDou/SBAS) for a one-line explanation of that constellation.

Status panels. GPS Status, Chrony Status, Position & Navigation, and System Info. Hover any metric label (they've got a dotted underline) for an explanation of what it means and what a good value looks like — HDOP, TDOP, EPX/EPY/EPV, the chrony offsets, and so on. I added these mostly so I'd stop having to look them up myself.

Sky view + satellite table. Polar az/el plot, plus a detailed per-satellite table.

Satellite-info modes (dashboard side)

Near the top of the <script> in static/index.html:

  • SATINFO_MODE — 'daemon' (default; asks this server's /satinfo, no CORS, falls back to the built-in table if the server has fetching disabled), 'offline' (built-in table only, never fetches), 'celestrak' (fetch CelesTrak directly from the browser; may hit CORS), or 'off'.
  • The built-in table always supplies block/manufacturer/clock/name/launch; /satinfo adds live health on top.

Previewing with simulated data

static/index.html ships with const USE_SIMULATION = false; (live). To preview the UI without any hardware — say you're building this on your laptop before the HAT arrives — open a copy with it set to true. It generates fake data (a fixed location, invented satellites, a locked stratum-1 state) so every visual works offline.

Heads up: the globe and all hover tooltips are mouse-based, so the hover interactions won't appear on touchscreens (dragging the globe still works via touch).

Part 12 — Optional: running the dashboard in Docker

You can containerize the dashboard daemon — but keep gpsd + chrony on the host. The kernel clock is a single host-wide resource; putting chrony in a container means granting it SYS_TIME and steering the host clock across the container boundary — extra layers wrapped around your most timing-sensitive component. The daemon, on the other hand, is a read-only consumer and containerizes cleanly with zero precision impact.

Here's an example multi-stage Dockerfile:

# Build
FROM rust:1-bookworm AS build
WORKDIR /app
COPY Cargo.toml Cargo.lock* ./
COPY src ./src
RUN cargo build --release

# Runtime
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
      chrony ca-certificates && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=build /app/target/release/gps-timeserver-daemon /usr/local/bin/
COPY static ./static
CMD ["gps-timeserver-daemon"]

Note that chrony is installed here only for its chronyc client (the daemon shells out to it) — the container does not run a chrony daemon. Run it sharing the host network and chrony's command socket:

docker build -t timeserver-dashboard .

docker run -d --name timeserver-dashboard \
  --network host \
  -e PORT=8088 \
  -e SATINFO_ENABLED=1 \
  -v /run/chrony:/run/chrony:ro \
  --restart unless-stopped \
  timeserver-dashboard
  • --network host lets the daemon reach host gpsd at 127.0.0.1:2947 and keeps PORT/LISTEN_ADDR working normally.
  • Mounting /run/chrony gives the container's chronyc access to chrony's command socket.
  • You do not need --device=/dev/serial0 or /dev/pps0 here — the daemon never touches the hardware directly; it reads gpsd and chronyc. (Those devices only matter if you were containerizing gpsd/chrony too, which, again, I'd steer you away from.)

Troubleshooting

Here's a running list of things I've stumbled across:

SymptomLikely cause / fix
cat /dev/serial0 shows nothingUART not freed. Confirm dtoverlay=disable-bt is in /boot/firmware/config.txt (not /boot/), hciuart disabled, rebooted. Check ls -l /dev/serial0 points to ttyAMA0.
cat /dev/serial0 shows garbled textBaud-rate mismatch (not a fault). Set the rate with stty -F /dev/serial0 <rate> raw and re-read; use the working rate in gpsd's -s.
Silent serial on Trixie only, same HW worked on BookwormSuspected Trixie UART/PL011 regression. Diagnose: pinctrl poll 14-15 &, then trigger GPS data — if GPIO15 (RX) never changes level, no bytes are reaching the pin (a kernel issue, not your config). Try linux-modules-extra-raspi, pin an older kernel, or fall back to Bookworm.
ppstest prints nothingNo fix yet, or wrong PPS pin. Give it sky view; verify gpiopin= (4 vs 18).
chrony PPS stuck at #?GPS/NMEA offset > 200 ms → falseticker. Calibrate the SHM 0 offset (Part 7d); make sure /dev/pps0 is in gpsd DEVICES and /run/chrony.pps0.sock exists.
Dashboard "Waiting for data..."Daemon can't reach gpsd. Check systemctl status timeserver-dashboard and that gpsd is up on :2947.
Dashboard returns 404WorkingDirectory in the unit doesn't point at the project root, so static/index.html isn't found. Fix the path, daemon-reload, restart.
Daemon exits with AddrInUseThe port's taken (e.g. Docker on 8080). Set Environment=PORT=8088 (or another free port); sudo ss -ltnp 'sport = :8080' shows the culprit.
/satinfo shows enabled:falseSATINFO_ENABLED not set/true, or daemon-reload skipped after editing the unit. systemctl show … -p Environment to verify.
/satinfo enabled:true but count:0NAVCEN fetch/parse failed (check the error field and the journal). Network blip → it retries within an hour; persistent → check outbound HTTPS.
cargo: command not foundPATH not loaded: source "$HOME/.cargo/env". If ~/.cargo/bin/cargo is missing, re-run the rustup installer.
cargo build → linker 'cc' not foundsudo apt install -y build-essential, then rebuild.
Clients can't syncThe allow subnet in chrony.conf has to match your LAN; open UDP/123 if you run a firewall.
Satellites look frozen on the globeExpected! GPS is MEO (~2 orbits/day) and moves only gradually in az/el; the receiver also updates angles slowly. Check back in 20–30 min and the pattern will have shifted.

What you end up with

So where does all this leave you? Two things:

  • stratum-1 NTP server disciplined by GPS + PPS, accurate to the microsecond or better, serving your whole LAN with no internet dependency. Every device on my network now sets its clock from a little box in my attic pulling time straight out of the sky, and I find that quietly delightful.
  • live dashboard showing the calculated time, full gpsd + chrony stats with hover explanations, an interactive 3D constellation globe with per-satellite detail, and a sky view — all fed by the same hardware over SSE, with a clean JSON contract you can extend however you like.

If you want to keep tinkering, there are really just two files worth revisiting: src/main.rs (where Telemetry::to_json() defines the data contract) and static/index.html (METRIC_HELPGPS_SAT_DB, and the render functions). Everything I'd want to customize lives in one of those two places.

If you made it this far, congrats on your new clock! Your clock is now more accurate than it has any reason to be!


Appendix A — Debian 12 (Bookworm) vs Debian 13 (Trixie)

Both work; most steps are identical. I verified everything on Debian 12 & 13 exactly as written above.

TopicDebian 12 / BookwormDebian 13 / Trixie
Boot config path/boot/firmware/config.txtSame
disable-bt / pps-gpio overlaysSameSame
Serial device/dev/serial0 → ttyAMA0 (always prefer serial0)Same
systemctl disable hciuartUsually succeedsOften Unit … does not exist — harmless
GPIO UART receiving GPSReliableRegression risk (mid-2026): no RX bytes despite correct config. Diagnose with pinctrl poll 14-15; may need linux-modules-extra-raspi, an older kernel, or Bookworm
gpsd / chronyStock; config as writtenNewer; same config syntaxchronyc -c tracking CSV unchanged
RecommendationPreferred for a time serverWorks, but verify the serial link early

Remember: serial baud rate is a property of your GPS module, not the OS — it's handled identically on both.

Appendix B — Environment variables (quick reference)

Daemon (systemd Environment= or shell):

VariableDefaultNotes
PORT8080Bind port.
LISTEN_ADDR(unset)Full host:port; overrides PORT.
SATINFO_ENABLEDoff1/true/yes/on to enable NAVCEN fetch. Off = no outbound calls.
SATINFO_INTERVAL_HOURS168Refresh cadence (weekly).
SATINFO_URLNAVCEN YUMASource override.

Dashboard (static/index.html constants):

ConstantDefaultNotes
USE_SIMULATIONfalsetrue = fake data for previewing.
SATINFO_MODE'daemon'daemon / offline / celestrak / off.

Appendix C — Licensing & data sources

Not legal advice. For personal, non-commercial use you're clear on all of the below.

  • GPS orbital/almanac data is US-government, public-domain. NAVCEN (US Coast Guard) serves it with no account and is redistribution-safe — the recommended live source here.
  • CelesTrak (an alternative live source) redistributes the same government data and adds convenient PRN/block labeling. Respect their access etiquette: cache results and don't poll more than roughly once every two hours; cite CelesTrak; note that it isn't an official source. Commercial redistribution is a "contact them first" situation.
  • The dashboard's built-in reference table is factual data (which PRN, block, launch year, manufacturer) assembled from public records; facts aren't copyrightable.
  • For anything commercial or internet-facing at scale using, review each source's current terms directly.

Headless Raspberry Pi Image based on Debian Trixie (Like Raspbian Lite)

I'm in the process on repurposing a few Raspberry Pis and noticed Debian 13 has officially been released (at time of writing), but Raspberry Pi OS is still on Debian 12.

If you're looking to create a custom Raspberry Pi OS image tailored for headless deployment, this guide walks you through using the official pi-gen tool to build your own image — based on Debian Trixie (Debian 13).

While I've tried to write this document in the context of a how-to, it's more of a reflection on what I did vs something formal.

⚠️ Disclaimer: Since this image is not built or maintained by the Raspberry Pi Foundation, you may encounter compatibility issues, hardware quirks, or missing Raspberry Pi-specific features. Official support for this setup is limited, and it's best suited for experimentation or advanced use cases. I don't work for the Raspberry Pi project, so follow at your own risk.

That all being said, here's what following this page would yield:

  • Uses the same tool the Raspberry Pi Foundation uses to generate official images
  • Builds a minimal system similar to Raspberry Pi OS Lite
  • Builds an Arm64 image (not 32-bit)
  • Has SSH enabled
  • Supports customization of locale, timezone, and user credentials

✅ Why Build Your Own Image?

You may want a custom image when:

  • You need headless deployment out of the box (no keyboard/mouse/monitor)
  • You want SSH access enabled by default
  • You want to preconfigure locales or packages for embedded or production use

🔧 Prerequisites

To follow this guide, it assumes usage of a few linux utilities / command. It assumes you are using and have available:

  • A macOS (Apple Silicon) or Linux machine
  • git, docker, and optionally qemu-user-static (only needed for x86)
  • ~10–20GB of free disk space

🧰 Step 1: Clone pi-gen

git clone --branch arm64 https://github.com/RPi-Distro/pi-gen.git
cd pi-gen

From the docs at time of this article:

Note: 32 bit images should be built from the master branch. 64 bit images should be built from the arm64 branch.

Tip: They have different branches based on debian OS version you want to target.


🧾 Step 2: Create the config File

You can define a file to customize that tells the image how to be customized out of box. Since we are a headless setup, we don't want to have to connect a monitor, keyboard, mouse, etc; so this file will define how the operating system should be customized on first startup.

Inside the pi-gen directory, create a file named config:

touch config

Then edit it with your preferred editor (nano, vim, VSCode, etc.):

Example config File

# Debian release to build (note, you should target the applicable branch when changing releases)
RELEASE=trixie

# Name of the image
IMG_NAME="raspios-trixie-arm64"

# Build 64-bit image
ARCH=arm64

# Enable SSH by default
ENABLE_SSH=1

# Set default username and password
FIRST_USER_NAME="pi"
FIRST_USER_PASS="raspberry"

# Don't compress the image – output raw .img file
DEPLOY_COMPRESSION=none

# Set timezone to UTC+0
TIMEZONE_DEFAULT="Etc/UTC"

# Don't use QEMU (set to 1 if not on Arm based architecture)
USE_QEMU=0


🌍 Locales, Keyboard, and Timezone

To change the system locale or timezone, you can add these optional variables:

# Locale
LOCALE_DEFAULT="en_US.UTF-8"

# Keyboard layout
KEYBOARD_KEYMAP="us"
KEYBOARD_LAYOUT="English (US)"

# Timezone (examples)
TIMEZONE_DEFAULT="Etc/UTC"       # UTC+0
TIMEZONE_DEFAULT="America/New_York"
TIMEZONE_DEFAULT="Europe/London"
TIMEZONE_DEFAULT="Asia/Tokyo"

You can find all available timezones on a Linux system via:

ls /usr/share/zoneinfo

🏗️ Step 3: Build the Image

If you don't want the desktop environment (headless setup), create the following skip files to eliminate creation of the full desktop images:

touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES

To start the build, run the following:

./build-docker.sh

This will:

  • Pull the necessary Docker images
  • Run through the build stages (stage0 to stage2 (assuming you created the skips))
  • Output a raw .img file(s) in the deploy/ folder

Example output without skip

copying log from container pigen_work to deploy/
total 35235816
drwxr-xr-x  10 jack  staff   320B Sep  6 11:58 .
drwxr-xr-x  23 jack  staff   736B Sep  6 11:57 ..
-rw-r--r--   1 jack  staff   8.3G Sep  6 11:57 2025-09-06-raspios-trixie-arm64-full.img
-rw-r--r--   1 jack  staff   245K Sep  6 11:57 2025-09-06-raspios-trixie-arm64-full.info
-rw-r--r--   1 jack  staff   2.6G Sep  6 11:56 2025-09-06-raspios-trixie-arm64-lite.img
-rw-r--r--   1 jack  staff    76K Sep  6 11:56 2025-09-06-raspios-trixie-arm64-lite.info
-rw-r--r--   1 jack  staff   5.8G Sep  6 11:56 2025-09-06-raspios-trixie-arm64.img
-rw-r--r--   1 jack  staff   213K Sep  6 11:56 2025-09-06-raspios-trixie-arm64.info
-rw-r--r--   1 jack  staff   1.0M Sep  6 11:58 build-docker.log
-rw-r--r--   1 jack  staff    10K Sep  6 11:57 build.log

Example output with the skip files

copying results from deploy/
copying log from container pigen_work to deploy/
total 5525296
drwxr-xr-x   6 jack  staff   192B Sep  6 12:03 .
drwxr-xr-x  23 jack  staff   736B Sep  6 12:03 ..
-rw-r--r--   1 jack  staff   2.6G Sep  6 12:03 2025-09-06-raspios-trixie-arm64-lite.img
-rw-r--r--   1 jack  staff    76K Sep  6 12:03 2025-09-06-raspios-trixie-arm64-lite.info
-rw-r--r--   1 jack  staff   308K Sep  6 12:03 build-docker.log
-rw-r--r--   1 jack  staff   5.8K Sep  6 12:03 build.log

💾 Step 4: Flash the Image to SD Card

After the build completes, write the image to an SD card. You can do this via command line or via the official Raspberry Pi Imager.

WARNING: make absolutely sure if you use the commands below you do not target the wrong disk. These commands can result in data loss. You do not hold me liable for data loss if using this approach!

On macOS or Linux:

  1. Identify the SD card:
diskutil list       # macOS
lsblk               # Linux
  1. Unmount the disk (macOS):
diskutil unmountDisk /dev/disk2
  1. Write the image:
sudo dd if=deploy/2025-09-06-raspios-trixie-arm64-lite.img of=/dev/rdisk5 bs=4m conv=fsync

⚠️ Replace /dev/rdisk5 with the correct device for your SD card.


🔌 Step 5: Boot the Pi

  • Insert the SD card into your Pi
  • Connect an Ethernet cable
  • Power on the Pi

You should:

  • See the Pi boot
  • Get a login prompt (no GUI)
  • Be able to SSH in using the IP address assigned via DHCP

🧩 Optional: Add SSH Key or Preseed Files

You can also preconfigure:

  • Your SSH public key
  • A custom hostname
  • Static IP
  • Additional packages

Let me know if you want a follow-up post for those.


✅ Recap

FeatureConfigured?
Debian Version✅ Trixie (Debian 13)
SSH Enabled✅ Yes
GUI/Desktop❌ No
Output Format✅ Raw .img
Build Host✅ Apple Silicon (no QEMU)
Locale / Timezone✅ UTC (editable)

🧭 Conclusion

By using pi-gen, you're following the official image-building process used by the Raspberry Pi Foundation — but with full control over your system. This process can also be used for embedded projects, production environments, or anyone who needs a tightly configured headless system.

How to update Z-Wave JS Docker Container

This document is written to help those that are using Z-Wave JS and Home Assistant as Docker containers. This tutorial goes hand-in-hand with this: How to update Home Assistant Docker Container | Jack Stromberg

Validate your current version

First, validate what version of Z-Wave JS you are running. To do this, navigate to the Z-Wave JS webpage and hover over the i icon to validate what versions of the software you are running. The Z-Wave JS webpage can typically be accessed at http://yourip:8091.

Get the current name of your container and version

sudo docker ps

In running this command, note the NAME of your container as well as the IMAGE.

Stop and delete the container

Replace the name of the container in the command below with the value you had.

sudo docker stop zwave-js
sudo docker rm zwave-js

Update packages

Some versions of HA require newer versions of Python, Docker, etc. I may consider updating to latest package versions first.

sudo apt-get update
sudo apt-get upgrade

Pull the latest container from Docker Hub

Replace the value below with your IMAGE value you documented in the previous steps.

sudo docker pull zwavejs/zwavejs2mqtt:latest

Deploy the container

Make sure your replace the name and value of the image with the values in the previous step. In addition, ensure you specify the correct path to where you existing configuration files exist to have the container load your existing configurations.

sudo docker run -d --restart=always  -p 8091:8091 -p 3000:3000 --device=/dev/ttyACM0 --name="zwave-js" -e "TZ=America/Chicago" -v /home/docker/zwave-js:/usr/src/app/store zwavejs/zwavejs2mqtt:latest

Validate your version number

After a few minutes, navigate back to the Z-Wave JS page. Upon load, you should now be on the latest versions.

Notes:

You can find the latest, stable, and development builds out on docker hub here: https://hub.docker.com/r/zwavejs/zwavejs2mqtt

How to add buster-backports to a Raspberry Pi

What are backports?

Debian has a really good write up here on what backports are. Copying directly from their introduction paragraph:

You are running Debian stable, because you prefer the Debian stable tree. It runs great, there is just one problem: the software is a little bit outdated compared to other distributions. This is where backports come in.

Backports are packages taken from the next Debian release (called "testing"), adjusted and recompiled for usage on Debian stable. Because the package is also present in the next Debian release, you can easily upgrade your stable+backports system once the next Debian release comes out. (In a few cases, usually for security updates, backports are also created from the Debian unstable distribution.)

Backports cannot be tested as extensively as Debian stable, and backports are provided on an as-is basis, with risk of incompatibilities with other components in Debian stable. Use with care!

It is therefore recommended to only select single backported packages that fit your needs, and not use all available backports.

Once I enable backports will all packages use them?

No! Any new packages and updates to existing stable packages will prefer the stable releases. The only time you will leverage a new backport package is if you explicitly specify to pull from them.

How do I enable backports?

First you need to add the new backport source to your sources.list file. Edit the file in vi:

sudo vi /etc/apt/sources.list

Arrow down to the last row, press o to create a new line and then enter the following:

deb http://deb.debian.org/debian buster-backports main

Press escape and then type :wq to save the changes and exit via.

Next, we need to specify a keyserver to verify the authenticity of these packages. Note we use Ubuntu's key servers to validate the packages. Interestingly, Debian has a keyring to validate the packages, however the keyring doesn't contain the backports for buster on the raspberry pi at time of writing this. Ubuntu's servers will work fine to validate the authenticity of these packages and you will ultimately pull the packages from Debian rather than Ubuntu.

sudo bash
gpg --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC
gpg --keyserver keyserver.ubuntu.com --recv-keys 648ACFD622F3D138

gpg --export 04EE7237B7D453EC | sudo apt-key add -
gpg --export 648ACFD622F3D138 | sudo apt-key add -
exit

How do I obtain a package from backport?

You can leverage one of the follow formats to specify the backport package:

apt install <package>/buster-backports
apt-get install <package>/buster-backports

or

apt install -t buster-backports <package>
apt-get install -t buster-backports <package>

or

aptitude install <package>/buster-backports

How to update Home Assistant Docker Container

Continuing from my previous guide on how to setup Home Assistant + Docker + Z-Wave + Raspberry Pi, this tutorial will show you how to update Home Assistant to the latest version. Updating Home Assistant to the latest version is critical to ensure you have the latest bug fixes, integrations, and security patches.

Note: during the update your devices will continue to work fine, but please note any automations or access to the application will not be available, so it's recommended to do this during a time that you know no automations will be running.

Validate your current version

Navigate to the Developer Tools section of Home Assistant. Here you can validate the latest version you currently have deployed.

Get the current name of your container and version

sudo docker ps

In running this command, note the NAME of your container as well as the IMAGE.

Stop and delete the container

Replace the name of the container in the command below with the value you had.

sudo docker stop home-assistant
sudo docker rm home-assistant

Update packages

Some versions of HA require newer versions of Python, Docker, etc. I may consider updating to latest package versions first.

sudo apt-get update
sudo apt-get upgrade

Pull the latest container from Docker Hub

Replace the value below with your IMAGE value you documented in the previous steps.

sudo docker pull ghcr.io/home-assistant/raspberrypi4-homeassi                                                                                                             stant:stable

Deploy the container

Make sure your replace the name and value of the image with the values in the previous step. In addition, ensure you specify the correct path to where you existing configuration files exist to have the container load your existing configurations.

sudo docker run -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" --device=/dev/ttyACM0 -v /home/docker/home-assistant:/config --net=host ghcr.io/home-assistant/raspberrypi4-homeassistant:stable

Note: If you are now using the Z-Wave JS docker container, you will not want to attach the Z-Wave stick to the Home-Assistant container. In this case, run the following command:

sudo docker run -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" -v /home/docker/home-assistant:/config --net=host ghcr.io/home-assistant/raspberrypi4-homeassistant:stable

Validate your version number

After a few minutes, navigate back to the Developers Tools page. Upon load, you should now be on the latest version of Home Assistant.

Notes:

You can find the latest, stable, and development builds out on docker hub here: https://hub.docker.com/u/homeassistant

For example, for raspberrypi4 builds, here you can validate the versions of all the different containers offered: https://hub.docker.com/r/homeassistant/raspberrypi4-homeassistant/tags

In newer versions of the docker container --init should not be specified in the docker run command. Specifying --init will result with the following error: "s6-overlay-suexec: fatal: can only run as pid 1". This was mentioned as a breaking change in: 2022-06-01 update: 2022.6: Gaining new insights! - Home Assistant (home-assistant.io)

Home Assistant + Docker + Z-Wave + Raspberry Pi

Notice: Home Assistant has released a new integration called Z-Wave JS. You should be using that integration vs the older Z-Wave integration that this article covers. I will be updating this guide soon.

A few years back I had a SmartThings Hub and for the most part it worked great. It was simple to setup, can be accessed anywhere, and for the most part automatically updated itself. Unfortunately, with the acquisition of it by Samsung, it seems to have turned into bloatware with poor responsiveness, the mobile application's UI is horrific, and they have a less than desirable security/privacy policy.

Luckily, the open source community has thrown together Home Assistant, an open source home automation project backed by hundreds/thousands of individuals. Over the years, they have now brought native support for mobile devices, at time of writing this there are 1500+ integrations for dang near any device, and the software puts you in control of who has access to and where your data is accessible.

The one trade-off though is while Home Assistant works well and is very extensible, the documentation and usability of the application can be overwhelming to understand for someone new to home automation, unfamiliar with Linux/Open Source technologies, or new to debugging/command line interfaces.

In this case, I've tried to document a crash course in getting Home Assistant up and running as quickly as possible for those that want to get started with Z-Wave devices and Home Assistant.

Hardware

You can leverage pretty much any hardware with Home Assistant, but here are the two items I used in my venture. Home Assistant has a full list of recommendations for what hardware to use for Home Assistant (https://www.home-assistant.io/getting-started/#suggested-hardware) as well as what Z-Wave controllers are supported (https://www.home-assistant.io/docs/z-wave/controllers/).

Update your Raspberry Pi

First things first, update your Raspberry Pi with the latest updates. Open up Terminal or SSH to your Raspberry Pi and execute the following command:

sudo apt-get update && sudo apt-get upgrade

Prepare your Z-Wave USB Stick

Plug in your Z-Wave USB stick. Once plugged in, we need to find the device path so that we can reference it for Home Assistant. Execute the lsusb command to find your device ID. In this case, you can see my device ID begins with 0658.

root@raspberrypi:/dev# lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Next, let's find what the device path is for the USB stick. You can do this by executing the following command: dmesg | egrep '0658|acm' Please note, if you purchased a difference device, 0658 may be a different number. In this case, you can see my device is presented on ttyACM0.

root@raspberrypi:/dev# dmesg | egrep '0658|acm'
[    1.405327] usb 1-1.2: New USB device found, idVendor=0658, idProduct=0200, bcdDevice= 0.00
[    3.468875] cdc_acm 1-1.2:1.0: ttyACM0: USB ACM device
[    3.471348] usbcore: registered new interface driver cdc_acm
[    3.471359] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters

Install Docker

Home Assistant doesn't require Docker, but by leveraging Docker you can easily copy/backup your configuration and simply redeploy the container if something goes wrong. As updates are made, you can simply remove your container and redeploy. To install Docker, execute the following command:

curl -sSL https://get.docker.com | sh

Deploy Home Assistant Docker Container

Once Docker is installed, you can deploy the container from Docker Hub. Docker Hub is a public repository that has tons of different prebuilt containers to deploy. Here you can find the official homeassistant containers: https://hub.docker.com/u/homeassistant

To deploy the container, execute the following line, replacing the following variables with your desired configuration:

  • --name="the name of your container"
  • -e "TM=YourTimezone"
  • --device=/dev/ttyACM0
    • This allows the container to leverage the Z-Wave USB device. Make sure you specify the path to your device found in the previous step
  • -v /home/docker/home-assistant:/config
    • This is the path that the home assistant configuration files should be stored to. You can specify a fileshare or other path to place your configuration files.
  • --net=host homeassistant/raspberrypi4-homeassistant:stable
    • The first half of this is the container you wish to deploy and the second half is the version. You can find all of Home Assistant's official containers here: https://hub.docker.com/u/homeassistant
sudo docker run -d --restart=always --name="home-assistant" -e "TZ=America/Chicago" --device=/dev/ttyACM0 -v /home/docker/home-assistant:/config --net=host homeassistant/raspberrypi4-homeassistant:stable

Note: In newer versions of the docker container --init should not be specified in the docker run command. Specifying --init will result with the following error: "s6-overlay-suexec: fatal: can only run as pid 1". This was mentioned as a breaking change in: 2022-06-01 update: 2022.6: Gaining new insights! - Home Assistant (home-assistant.io)

Setup Home Assistant

Give the container a few minutes to deploy and configure itself for the first time. After a few minutes, try opening your web browser and navigating to the IP address assigned to your machine, using port number 8123: http://192.168.1.2:8123/

When the page loads, it should first ask for your Name, Username, and Password. This is the username and password you will use to login to Home Assistant.

Next, specify the location of where your Home Assistant deployment is located. Oddly enough, you cannot type in a location, but you can place the pin near your location by dragging the map around and clicking once to set the pin.

Once you click Next, Home Assistant may have already found a few devices connected to your network. You can add them now or skip and add them later.

Tell Home Assistant to use your Z-Wave USB Stick

Although we granted access to the container to use the Z-Wave USB Stick, you need to tell Home Assistant how to leverage the device. To do so, you will need to open up Terminal or SSH to your machine and edit the configuration.yaml file to point to the device. Before we get into modifying the configuration.yaml file, first execute the following command to generate a Z-Wave Security Key. This key may be required by Z-Wave security devices (Door Locks, Keypads, etc), as an extra layer of security. More information on this can be found here: https://www.home-assistant.io/docs/z-wave/adding#network-key

Execute the following command via Terminal or SSH:

cat /dev/urandom | tr -dc '0-9A-F' | fold -w 32 | head -n 1 | sed -e 's/\(..\)/0x\1, /g' -e 's/, $//'

Once you execute the command, it should give you a string of characters that look something like:

"0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10"

Next, we need to edit the configuration.yaml file, which can be found in the path specified when the Docker container was deployed (using the -v parameter). For the purpose of this article, /home/docker/home-assistant/configuration.yaml is where the file is located. Using your favorite text editor, add the following lines of code:

zwave:
  usb_path: /dev/ttyACM0
  network_key: "0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10"
configuration.yaml file with Z-Wave configuration

Once saved, go back to Home Assistant and click the Gear icon and then select Server Controls

Select the Restart button to restart Home Assistant. Any time you make a change to the configuration.yaml file, you will need to restart Home Assistant to pickup the configuration changes.

Click OK to Restart

Upon restart, navigate back to the Gear icon and you should see a new entry in the Config portal for Z-Wave. If you do not see the "Z-Wave" section, scroll down to the troubleshooting step at the end of this article.

Add a Z-Wave device

Once you see that your Z-Wave network has started, adding a device is a piece of cake. First click the Add Node button. When you click the button, nothing will happen, but go ahead and put your device in inclusion mode. Once the device is in inclusion mode, Home Assistant should automatically add the device.

At this point, if you navigate back to Configuration (Gear icon) and select Devices

You should see your newly added Z-Wave device!

At this point, you can select the Device to give it a friendly name or start to work on building your own home automation actions.

Hope this helped! If you have any comments or suggestions on how to improve this guide, please drop it below.

Troubleshooting Missing Z-Wave Configuration

The first time I ran through this, I noticed I was missing the Z-Wave configuration tile after making changes to the configuration.yaml file. It turned out I specified the wrong device path in the configuration file. To verify, you can check the logs from your Docker container by executing the following command in your Terminal or via SSH. (Replace home-assistant with the name of your container if you specified something else)

sudo docker logs home-assistant

In my case, I had the following error:

2020-02-16 21:08:01 INFO (MainThread) [homeassistant.components.scene] Setting up scene.homeassistant
2020-02-16 21:08:02 INFO (MainThread) [homeassistant.components.zwave] Z-Wave USB path is /dev/ttyACM01
2020-02-16 21:08:02 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Z-Wave (import from configuration.yaml) for zwave
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__
    raise ZWaveException(u"Can't find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: "Zwave Generic Exception : Can't find device /dev/ttyACM01 : ['NoneType: None\\n']"

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 215, in async_setup
    hass, self
  File "/usr/src/homeassistant/homeassistant/components/zwave/__init__.py", line 369, in async_setup_entry
    config_path=config.get(CONF_CONFIG_PATH),
  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 81, in __init__
    raise ZWaveException(u"Error when retrieving device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))
openzwave.object.ZWaveException: 'Zwave Generic Exception : Error when retrieving device /dev/ttyACM01 : [\'Traceback (most recent call last):\\n\', \'  File "/usr/local/lib/python3.7/site-packages/openzwave/option.py", line 78, in __init__\\n    raise ZWaveException(u"Can\\\'t find device %s : %s" % (device, traceback.format_exception(*sys.exc_info())))\\n\', \'openzwave.object.ZWaveException: "Zwave Generic Exception : Can\\\'t find device /dev/ttyACM01 : [\\\'NoneType: None\\\\\\\\n\\\']"\\n\']'

Here you can see I accidentally specified /dev/ttyACM01 vs /dev/ttyACM0. Simply updating the configuration.yaml file with the correct device path solved the issue.

Setting up an email server on a RaspberryPI (Postfix+Dovecot+MariaDB+Roundcube)

Edit in 2025: This guide was originally published in 2020 and works with Dovecot 2.3. Dovecot 2.4 introduces many new changes that prevent this guide from working as-is. While the premise is the same, the configuration file for Dovecot will need to be updated.

There's a few things in this journey that you should be aware of when running your own mailserver before we begin.

  1. Invest in a static public IP
  2. Don't open mail relay
  3. Leverage proper DNS records to help mitigate your email from being marked spam
  4. Leverage something to scrub your email
  5. Don't open mail relay
  6. Don't open mail relay
  7. Verify your domain or IP address hasn't been placed on a blacklist from a previous owner

In my career in doing IT, handling email is one of the most tedious tasks to setup/maintain due to so many moving pieces; many of which may be out of your control. Dealing with spam, blacklisting, having emails non-deliverable for several reasons, handling dns records, certificates, etc.... it's sometimes worth paying a few extra bucks to have someone else host your email and have peace of mind the message will be delivered. That being said, if you have the extra time on your hands and like the challenge of solving problems, here's a quick way to get started.

Preamble

This guide took me several hours to compile through trial and error. If you have any thoughts, notice any errors/typos, or have ideas on how to further secure/optimize, please leave feedback below to further improve this guide. Thank you and good luck on the deployment of your mail server!

Assumptions

  • You have previously followed my guide on building a LEMP stack
  • You are running Ubuntu or Debian as per the above guide (you can still follow this guide, you may have to slightly change which commands you use for your distribution -- configuration should remain the same though)

DNS

Let's first start at getting your DNS records configured properly. This guide will talk about configuring MX, SPF, and PTR records. We won't be covering Domain Keys in this article, maybe in a separate article if someone donates to my paypal on the right side of the website 😉

MX Record

Via your nameservers, add a new mx record for your domain name. Here's a list of tutorials for some of the major domain registrars:

SPF Record

Contrary to many websites that say you need to create a "SPF" record type, the SPF record type was never ratified by RFC standards. In this case, the proper way to create a SPF record is via a TXT record with the SPF value (as per RFC 7208).

You can leverage my SPF generator to create a new TXT record in the root of your domain.

PTR Record

To help decrease the odds of your emails being labeled as spam, I'd recommend creating a PTR record that will resolve your IP address to a DNS name (we call this a reverse lookup). For example, if my mail server's domain name was mail.mydomain.com and it resolved to 123.123.123.123, I would create a PTR record for 123.123.123.123 that points to mail.mydomain.com.

In many cases, you will need to either work with your ISP (Internet Service Provider) or domain registrar if you own your own IP block to make changes to the record for your IP address block.

When you are ready, you can leverage the nslookup command on Windows to validate the name from the IP address.

nslookup 123.123.123.123

Or on linux you can leverage the host command to verify the reverse lookup as well:

host 123.123.123.123

Get the OS ready

Download the latest packages and actually perform any updates.

sudo sh -c 'apt update && apt upgrade'

Prepare MariaDB for virtual users/aliases

One of the primary reasons we need to configure a database is it is what will contain the information about all of our users and their corresponding email addresses (aliases). To do so, we need to create 3 new tables inside of a new database.

Login to the database

sudo mariadb -u root -p

Create the database, database user, and tables

Create a new database for our users (in this case, I'm calling the database mailserver). Note: This command must be run in the context of mariadb, this is not a bash command.

create database mailserver;

Create a new user called mailuser, grant them access to the entire database, require the user to only create connections from 127.0.0.1 (localhost), and specify a password for the user.

GRANT SELECT ON mailserver.* TO 'mailuser'@'127.0.0.1' IDENTIFIED BY 'mysupersecretpassword';

Execute the following command to apply the changes

FLUSH PRIVILEGES;

Create a table for each of the domain names we will leverage for our email addresses.

CREATE TABLE `mailserver`.`virtual_domains` (
  `id` int(11) NOT NULL auto_increment,
  `name` varchar(50) NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Create a table that will hold each of the users that will need mailboxes.

CREATE TABLE `mailserver`.`virtual_users` (
  `id` int(11) NOT NULL auto_increment,
  `domain_id` int(11) NOT NULL,
  `password` varchar(106) NOT NULL,
  `email` varchar(100) NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `email` (`email`),
  FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Create a table that will hold aliases (additional email addresses) for a particular user.

CREATE TABLE `mailserver`.`virtual_aliases` (
  `id` int(11) NOT NULL auto_increment,
  `domain_id` int(11) NOT NULL,
  `source` varchar(100) NOT NULL,
  `destination` varchar(100) NOT NULL,
  PRIMARY KEY (`id`),
  FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Insert a new user into the database

First, we need to add our first domain name into the domains table

INSERT INTO `mailserver`.`virtual_domains`
  (`name`)
VALUES
  ('mydomain.com');

Second, we need to create the user. Replace mysupersecretpassword with your password.

INSERT INTO `mailserver`.`virtual_users`
  (`domain_id`, `password` , `email`)
VALUES
  ('1', ENCRYPT('mysupersecretpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), '[email protected]');

Third, we can optionally specify an alias (secondary email address) for the user.

INSERT INTO `mailserver`.`virtual_aliases`
  (`domain_id`, `source`, `destination`)
VALUES
  ('1', '[email protected]', '[email protected]');

Type exit once you are done to leave the context of MariaDB.

Install Packages for Postfix and Dovecot

Postfix is what we call a Mail Transport Agent (MTA) and is responsible for actually sending/receive the messages from the internet. Later, we will talk about Dovecot which will be our MDA (Mail Delivery Agent) (what actually interacts with the mailbox).

The following command will install postfix, dovecot, and pull the packages to interact with MySQL. Although these are labeled MySQL, they should interact fine with MariaDB.

sudo apt-get install postfix postfix-mysql dovecot-core dovecot-imapd dovecot-lmtpd dovecot-mysql

During the installation of Postfix, you will be prompted to configure the connection type to the mail server. In this case, select Internet Site for the mail configuration.

On the second installation prompt, it will ask for the domain name used in receiving email. In this prompt, specify one of the domain names you will be using for your users. For example, if your email addresses are going to be [email protected] you would specify mydomain.com for this prompt. Don't worry if you have multiple email addresses, we will cover that later on.

Configure Postfix to leverage MariaDB

First, let's create a backup of the Postfix configuration, so we have a baseline to refer back to.

sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.bak

Copy the following configuration and replace the domain name example.com with yours. Credit to linode for sharing their configuration as it not only defines integration into a database, but also hardens the Postfix deployment.

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# Debian specific:  Specifying a file name will cause the first
# line of that file to be used as the name.  The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/letsencrypt/live/mydomain.com/fullchain.pem
smtpd_tls_key_file=/etc/letsencrypt/live/mydomain.com/privkey.pem
smtpd_use_tls=yes
smtpd_tls_auth_only = yes
smtp_tls_security_level = may
smtpd_tls_security_level = may
smtpd_sasl_security_options = noanonymous, noplaintext
smtpd_sasl_tls_security_options = noanonymous

# Authentication
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

# Restrictions
smtpd_helo_restrictions =
        permit_mynetworks,
        permit_sasl_authenticated,
        reject_invalid_helo_hostname,
        reject_non_fqdn_helo_hostname
smtpd_recipient_restrictions =
        permit_mynetworks,
        permit_sasl_authenticated,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain,
        reject_unlisted_recipient,
        reject_unauth_destination
smtpd_sender_restrictions =
        permit_mynetworks,
        permit_sasl_authenticated,
        reject_non_fqdn_sender,
        reject_unknown_sender_domain
smtpd_relay_restrictions =
        permit_mynetworks,
        permit_sasl_authenticated,
        defer_unauth_destination

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

myhostname = example.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
mydomain = mydomain.com
myorigin = $mydomain
mydestination = localhost
relayhost =
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all

# Handing off local delivery to Dovecot's LMTP, and telling it where to store mail
virtual_transport = lmtp:unix:private/dovecot-lmtp

# Virtual domains, users, and aliases
virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-users.cf
virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-mailbox-aliases.cf,
        mysql:/etc/postfix/mysql-virtual-mailbox-users.cf

# Even more Restrictions and MTA params
disable_vrfy_command = yes
strict_rfc821_envelopes = yes
#smtpd_etrn_restrictions = reject
#smtpd_reject_unlisted_sender = yes
#smtpd_reject_unlisted_recipient = yes
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtp_always_send_ehlo = yes
#smtpd_hard_error_limit = 1
smtpd_timeout = 30s
smtp_helo_timeout = 15s
smtp_rcpt_timeout = 15s
smtpd_recipient_limit = 40
minimal_backoff_time = 180s
maximal_backoff_time = 3h

# Reply Rejection Codes
invalid_hostname_reject_code = 550
non_fqdn_reject_code = 550
unknown_address_reject_code = 550
unknown_client_reject_code = 550
unknown_hostname_reject_code = 550
unverified_recipient_reject_code = 550
unverified_sender_reject_code = 550

Next, we need to create the mappings of domain names, users, and aliases. In the same directory as the main.cf (/etc/postfix) we need to first create a file that will tell postfix how to lookup what domain names exist. You can open the documents with your favorite text editor; I use vi since it's universally installed.

sudo vi /etc/postfix/mysql-virtual-mailbox-domains.cf

Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.

user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name='%s'

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Next, we will create another file that is used to lookup each user's mailbox.

sudo vi /etc/postfix/mysql-virtual-mailbox-users.cf

Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.

user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT email FROM virtual_users WHERE email='%s'

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Last, we will create another file that is used to map an alias to a user's mailbox.

sudo vi /etc/postfix/mysql-virtual-mailbox-aliases.cf

Press i to get vi into insert mode and paste the following, replacing the password with the mailuser we specified earlier in this tutorial.

user = mailuser
password = mysupersecretpassword
hosts = 127.0.0.1
dbname = mailserver
query = SELECT destination FROM virtual_aliases WHERE source='%s'

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Restart the Postfix service for the changes to take effect

sudo service postfix restart

Next, to enable port 587 and 465 to connect securely with email clients, we need to modify /etc/postfix/master.cf. First, let's create a backup of the master.cf file.

sudo cp /etc/postfix/master.cf /etc/postfix/master.cf.bak

Next, we need to modify the master.cf file. Modify the document (mostly uncomment many of the lines) to look similar to the code below.

sudo vi /etc/postfix/master.cf
submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_tls_auth_only=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING
smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_sasl_type=dovecot
  -o smtpd_sasl_path=private/auth
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Note: If you'd like to allow open relay locally, you can prefix permit_mynetworks to the smtpd_relay_restrictions list as well.

Restart the Postfix service for the changes to take effect

sudo service postfix restart

Configure Dovecot

Now that we have our MTA configured, we now need to configure our MDA. You can think of Postfix as a shipping center and Dovecot as the courier, who interfaces directly with your mailbox. Roundcube will be our MUA (mail user agent) that interfaces with Dovecot to display your mail. The goal for this section is to ensure Dovecot requires SSL.

First, we'll create backups of each of the Dovecot configuration files

sudo cp /etc/dovecot/dovecot.conf /etc/dovecot/dovecot.conf.orig
sudo cp /etc/dovecot/conf.d/10-mail.conf /etc/dovecot/conf.d/10-mail.conf.orig
sudo cp /etc/dovecot/conf.d/10-auth.conf /etc/dovecot/conf.d/10-auth.conf.orig
sudo cp /etc/dovecot/dovecot-sql.conf.ext /etc/dovecot/dovecot-sql.conf.ext.orig
sudo cp /etc/dovecot/conf.d/10-master.conf /etc/dovecot/conf.d/10-master.conf.orig
sudo cp /etc/dovecot/conf.d/10-ssl.conf /etc/dovecot/conf.d/10-ssl.conf.orig

Execute the following command to enable support for imap and lmtp (pop3 can be added, but ensure you install the dovecot-pop3d package).

sudo sed -i '/^\!include_try \/usr\/share\/dovecot\/protocols.d\/\*.protocol/a protocols=imap lmtp' /etc/dovecot/dovecot.conf

Next, we need to edit /etc/dovecot/conf.d/10-mail.conf to define where mailboxes are stored. Execute the following commands:

sudo sed -i 's/mail_location = mbox.*/mail_location = maildir:\/var\/mail\/vhosts\/%d\/%n\//g' /etc/dovecot/conf.d/10-mail.conf
sudo sed -i 's/^#mail_privileged_group = mail/mail_privileged_group = mail/g' /etc/dovecot/conf.d/10-mail.conf

Next, we need to make directories for each of your domain names. Execute the following command for each of your domain names.

sudo mkdir -p /var/mail/vhosts/example.com

Now we need to create a user and group called vmail, assigned with an id of 5000, and set the directory with the owner of vmail

sudo groupadd -g 5000 vmail
sudo useradd -g vmail -u 5000 vmail -d /var/mail
sudo chown -R vmail:vmail /var/mail

Next we need to edit the user authentication file (/etc/dovecot/conf.d/10-auth.conf) to tell Dovecat to leverage MariaDB for our users. Execute the following commands:

sudo sed -i 's/^#disable_plaintext_auth = yes/disable_plaintext_auth = yes/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^#auth_mechanisms = plain login/auth_mechanisms = plain login/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^!include auth-system.conf.ext/#!include auth-system.conf.ext/g' /etc/dovecot/conf.d/10-auth.conf
sudo sed -i 's/^#!include auth-sql.conf.ext/!include auth-sql.conf.ext/g' /etc/dovecot/conf.d/10-auth.conf

Once we have the authentication file configured, we need to update the sql driver (/etc/dovecot/conf.d/auth-sql.conf.ext) to point to our mailboxes. You will need to uncomment the passdb section and uncomment the userdb driver that is static.

sudo vi /etc/dovecot/conf.d/auth-sql.conf.ext

Press i to get vi into insert mode and paste the following configuration

# Authentication for SQL users. Included from 10-auth.conf.
#
# <doc/wiki/AuthDatabase.SQL.txt>

passdb {
  driver = sql

  # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext
  args = /etc/dovecot/dovecot-sql.conf.ext
}

# "prefetch" user database means that the passdb already provided the
# needed information and there's no need to do a separate userdb lookup.
# <doc/wiki/UserDatabase.Prefetch.txt>
#userdb {
#  driver = prefetch
#}

#userdb {
#  driver = sql
#  args = /etc/dovecot/dovecot-sql.conf.ext
#}

# If you don't have any user-specific settings, you can avoid the user_query
# by using userdb static instead of userdb sql, for example:
# <doc/wiki/UserDatabase.Static.txt>
userdb {
  driver = static
  args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n
}

Press : and then type wq and press enter to write the changes to the file and quit in vi.

The final Dovecot file we need to modify will set our database settings (/etc/dovecot/dovecot-sql.conf.ext). Execute the following commands to uncomment the correct settings. Note: be sure to replace the password with the database password we configured earlier.

sudo sed -i 's/^#driver = /driver = mysql/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i 's/^#connect =/connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mysupersecretpassword/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i 's/^#default_pass_scheme = MD5/default_pass_scheme = SHA512-CRYPT/g' /etc/dovecot/dovecot-sql.conf.ext
sudo sed -i '/^#password_query = \\/i password_query = SELECT email as user, password FROM virtual_users WHERE email=\x27%u\x27;' /etc/dovecot/dovecot-sql.conf.ext

After making the changes to the dovecot-sql.conf.ext file, next we need to change the owner and the group of the dovecot folder to the vmail user:

sudo chown -R vmail:dovecot /etc/dovecot
sudo chmod -R o-rwx /etc/dovecot 

Next, we need to disable the unencrypted versions of IMAP and SMTP.

sudo vi /etc/dovecot/conf.d/10-master.conf

We need to edit the /etc/dovecot/conf.d/10-master.conf file and set ports to 0 to disable non-encrypted imap/pop3. Find service imap-login { and make it look like the following.

service imap-login {
  inet_listener imap {
    port = 0
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }

  # Number of connections to handle before starting a new process. Typically
  # the only useful values are 0 (unlimited) or 1. 1 is more secure, but 0
  # is faster. <doc/wiki/LoginProcess.txt>
  #service_count = 1

  # Number of processes to always keep waiting for more connections.
  #process_min_avail = 0

  # If you set service_count=0, you probably need to grow this.
  #vsz_limit = $default_vsz_limit
}

service pop3-login {
  inet_listener pop3 {
    port = 0
  }
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}

In the same file, find service lmtp { and replace the whole block down to the third } with the following:

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }

  # Create inet listener only if you can't use the above UNIX socket
  #inet_listener lmtp {
    # Avoid making LMTP visible for the entire internet
    #address =
    #port =
  #}
}

In the same file, find service auth { and replace the whole block down to the third } with the following:

service auth {
  # auth_socket_path points to this userdb socket by default. It's typically
  # used by dovecot-lda, doveadm, possibly imap process, etc. Users that have
  # full permissions to this socket are able to get a list of all usernames and
  # get the results of everyone's userdb lookups.
  #
  # The default 0666 mode allows anyone to connect to the socket, but the
  # userdb lookups will succeed only if the userdb returns an "uid" field that
  # matches the caller process's UID. Also if caller's uid or gid matches the
  # socket's uid or gid the lookup succeeds. Anything else causes a failure.
  #
  # To give the caller full permissions to lookup all users, set the mode to
  # something else than 0666 and Dovecot lets the kernel enforce the
  # permissions (e.g. 0777 allows everyone full permissions).
  unix_listener auth-userdb {
    mode = 0600
    user = vmail
    #group =
  }

  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0600
    user = postfix
    group = postfix
  }

  # Auth process is run as this user.
  user = dovecot
}

In the same file, find service auth-worker { and replace the whole block down to the } with the following:

service auth-worker {
  # Auth worker process is run as root by default, so that it can access
  # /etc/shadow. If this isn't necessary, the user should be changed to
  # $default_internal_user.
  user = vmail
}

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Last, we need to tell dovecot where our SSL certificate is for encryption. We will modify the /etc/dovecot/conf.d/10-ssl.conf file. Make sure to update the directory with the correct path for your certificates.

Execute the following commands, replacing

sudo sed -i 's/^ssl = yes/ssl = required/g' /etc/dovecot/conf.d/10-ssl.conf
sudo sed -i 's/^ssl_cert = .*/ssl_cert = <\/etc\/letsencrypt\/live\/mydomain.com\/fullchain.pem/g' /etc/dovecot/conf.d/10-ssl.conf
sudo sed -i 's/^ssl_key = .*/ssl_key = <\/etc\/letsencrypt\/live\/mydomain.com\/privkey.pem/g' /etc/dovecot/conf.d/10-ssl.conf

Last, restart devocot to enable all of our changes.

sudo service dovecot restart

Configure Roundcube

Install dependencies for Roundcube

Roundcube requires several PHP PEAR modules. To install the bare minimum featureset, execute the following command:

sudo apt-get install php7.3-mbstring php-pear php-net-idna2 php-net-smtp  php-mail-mime

Create a database for Roundcube

First, we need to create a new database and user for Roundcube. We can do this by logging into MariaDB and executing the create and grant commands.

sudo mariadb -u myusername -p
CREATE DATABASE roundcubemail CHARACTER SET utf8 COLLATE utf8_general_ci;
GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost IDENTIFIED BY 'myreallyreallysecretpassword';
FLUSH PRIVILEGES;
exit

Request SSL Certificates for Roundcube

We will want to ensure all traffic to and from the client is encrypted in transit when trying to access Roundcube. To do this, I leverage Let's Encrypt, which will allow you to request a free SSL certificate. If you have your own SSL certificate, go ahead and copy it to a location on the server so we can reference it later.

sudo apt-get install certbot
sudo certbot certonly --authenticator standalone -d webmail.mydomain.com --pre-hook "service nginx stop" --post-hook "service nginx start"

Create a directory for Roundcube

We will need to create a directory that will hold Roundcube's files to serve to the web. Let's create a new directory to serve these files and limit permissions to www-data.

sudo mkdir /var/www/webmail.mydomain.com
sudo chown -R www-data:www-data /var/www/webmail.mydomain.com

Copy Roundcube Files to the web directory

We will need to grab the latest copy of Roundcube's code to run the website. Note: please ensure you substitute the correct version for Roundcube when executing the commands below as the version listed in the guide will likely be out of date as time goes on:

cd /tmp
wget https://github.com/roundcube/roundcubemail/releases/download/1.4.1/roundcubemail-1.4.1.tar.gz
tar -xf roundcubemail-1.4.1.tar.gz
mv roundcubemail-1.4.1 /var/www/webmail.mydomain.com

Populate the SQL Database

You will need to execute the following SQL command to populate your Roundcube database with the tables needed to run Roundcube. To do so, execute the following commands.

sudo mariadb roundcubemail < /var/www/webmail.mydomain.com/SQL/mysql.initial.sql

Install Roundcube dependencies

Roundcube doesn't ship with several javascript dependencies. To ensure the Roundcube pages load properly, you will need to execute the following command to pull down the javascript dependencies.

sudo php /var/www/webmail.mydomain.com/bin/install-jsdeps.sh

Configure NGINX

Let's configure NGINX to point to our web directory for the website. When doing so, it is very important you protect your installation by preventing access to some sensitive files from the web.

First, create a virtual-host file within the nginx sites-available folder:

sudo vi /etc/nginx/sites-available/webmail.mydomain.com

Press i to get vi into insert mode and paste the following. Note: Please replace the values with the path to your SSL Certificate we generated earlier.

##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# https://www.nginx.com/resources/wiki/start/
# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
# https://wiki.debian.org/Nginx/DirectoryStructure
#
# In most cases, administrators will remove this file from sites-enabled/ and
# leave it as reference inside of sites-available where it will continue to be
# updated by the nginx packaging team.
#
# This file will automatically load configuration files provided by other
# applications, such as Drupal or WordPress. These applications will be made
# available underneath a path with that package name, such as /drupal8.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##

# Default server configuration
#

server {
        listen 443 ssl;
        listen [::]:443 ssl;
        server_name webmail.mydomain.com;

ssl_certificate /etc/letsencrypt/live/webmail.mydomain.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/webmail.mydomain.com/privkey.pem;
  ssl_session_timeout 1d;
  ssl_session_cache shared:SSL:50m;
  ssl_session_tickets off;
  ssl_protocols TLSv1.1 TLSv1.2;
  ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK';
  ssl_prefer_server_ciphers on;
  ssl_stapling on;
  ssl_stapling_verify on;
  ssl_trusted_certificate /etc/letsencrypt/live/webmail.mydomain.com/chain.pem;


        # SSL configuration
        #
        # listen 443 ssl default_server;
        # listen [::]:443 ssl default_server;
        #
        # Note: You should disable gzip for SSL traffic.
        # See: https://bugs.debian.org/773332
        #
        # Read up on ssl_ciphers to ensure a secure configuration.
        # See: https://bugs.debian.org/765782
        #
        # Self signed certs generated by the ssl-cert package
        # Don't use them in a production server!
        #
        # include snippets/snakeoil.conf;

        root /var/www/webmail.mydomain.com;

        # Add index.php to the list if you are using PHP
        index index.php index.html index.htm;

       # Revoke access to sensitive files and directories
       location ~ ^/(README|INSTALL|LICENSE|CHANGELOG|UPGRADING)$ {
                deny all;
       }
       location ~ ^/(config|temp|bin|SQL|logs)/ {
                deny all;
       }

        # pass PHP scripts to FastCGI server
        #
        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
        #
        #       # With php-fpm (or other unix sockets):
                fastcgi_pass unix:/run/php/php7.3-fpm.sock;
        #       # With php-cgi (or other tcp sockets):
        #       fastcgi_pass 127.0.0.1:9000;
        }

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        location ~ /\. {
                deny all;
                access_log off;
                log_not_found off;
       }
}

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Last, we need to create a link of the virtual host file to /etc/nginx/sites-enabled. You will need to execute the following commands to create the link as well as restart nginx to apply the changes.

sudo ln -s /etc/nginx/sites-available/webmail.mydomain.com /etc/nginx/sites-enabled/webmail.mydomain.com
sudo service nginx restart

Run the Roundcube installer

At this point, if you navigate to https://webmail.mydomain.com/installer, you should see the Roundcube Webmail Installer page. You should see a series of items show OK, NOT AVAILABLE, or NOT OK. You will need to remediate any items that show NOT OK for Roundcube to successfully run.

In this installer, I primarily focused on Step 1 (Checking the environment) and Step 2 (Checking the database). Once both show OK (don't worry about if email is successful or fails (likely it is failing still), move the installer directory to your home drive to secure the environment (IT IS VERY DANGEROUS TO LEAVE THIS PAGE!!! DON'T SKIP THIS STEP).

sudo mv /var/www/webmail.mydomain.com/installer ~

Update Roundcube configuration

I couldn't get Roundcube to actually work during the installation with this setup until I manually specified a few items via the Roundcube configuration file. Within the /var/www/webmail.mydomain.com/config/config.inc.php file, ensure you have the following code snippets to allow Roundcube to properly authenticate to your mailserver.

sudo vi /var/www/webmail.mydomain.com/config/config.inc.php

Ensure you have the following code snippets (typically there is a section under // IMAP that has the config we can start with). To do so, press i to get vi into insert mode and paste the following.

$config['default_port'] = 993;
$config['default_host'] = 'imaps://localhost';
$config['mail_domain'] = '%d';
$config['imap_conn_options'] = array(
 'ssl'         => array(
     'verify_peer'       => true,
     'verify_peer_name' => false,
  ),
);

// SMTP
$config['smtp_server'] = 'ssl://localhost';
$config['smtp_port'] = 465;
$config['smtp_auth_type'] = 'LOGIN';
// Required if you're running PHP 5.6 or later
$config['smtp_conn_options'] = array(
    'ssl' => array(
        'verify_peer'      => true,
        'verify_peer_name' => false,
    ),
);

Press : and then type wq and press enter to write the changes to the file and quit in vi.

Verify

At this point, you should be able to login to https://webmail.mydomain.com and send/receive email!

As with all technology, ensure you keep up-to-date with all the latest security patches to keep your environment stable and secure.

If you made it to this point, were able to successfully send/receive mail via Roundcube, pat yourself on the back and grab a fine beverage!

Troubleshooting

Here are some useful commands to help troubleshoot your deployment.

sudo postqueue -p can be used to check if any pending emails are queued.

sudo postmap -q mydomain.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf can be used to validate what domain names are accepted. You should receive the value of 1 if it exists.

sudo postmap -q [email protected] mysql:/etc/postfix/mysql-virtual-mailbox-users.cf will validate if a user account exists with the specified email address. You should receive the value of the email address of the user if it exists.

sudo postmap -q [email protected] mysql:/etc/postfix/mysql-virtual-mailbox-aliases.cf can be used to validate the alias of an email address. You should receive the email address of the user account if it does map back to another user.

tail -f /var/log/mail.log can be useful watching how emails are handled by postfix/dovecot to troubleshoot how messages are being handled

Roundcube installation instructions (documentation): https://github.com/roundcube/roundcubemail/wiki/Installation

How to install NodeJS on a Raspberry Pi

Installing NodeJS on a Raspberry Pi can be a bit tricky.  Over the years, the ARM based processor has gone through several versions (ARMv6, ARMv7, and ARMv8), in which there are different flavors of NodeJS to each of these architectures.

Depending on the version you have, you will need to manually install NodeJS vs grabbing the packages via a traditional apt-get install nodejs.

Step 1: Validate what version of the ARM chipset you have

First let's find out what ARM version you have for your Raspberry Pi.  To do that, execute the following command:

uname -m

You should receive something like: armv61

Step 2: Find the latest package to download from nodeJS's website

Navigate to https://nodejs.org/en/download/ and scroll down to the latest Linux Binaries for ARM that match your instance.  Right click and copy the address to the instance that matches your processor's architecture.  For example, if you saw armv61, you'd copy the download for ARMv6

Step 3: Download and install nodeJS

Within your SSH/console session on the Raspberry Pi, change to your local home directory and execute the following command (substituting in the URL you copied in the previous step in what's outlined in red below).  For example:

cd ~
wget https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-armv6l.tar.xz

Next, extract the tarball (substituting in the name of the tarball you downloaded in the previous step) and change the directory to the extracted files

tar -xvf node-v8.11.3-linux-armv6l.tar.xz
cd node-v8.11.3-linux-armv6l

Next, remove a few files that aren't used and copy the files to /usr/local

rm CHANGELOG.md LICENSE README.md
cp -R * /usr/local/

Step 4: Validate the installation

You can validate that you have successfully installed NodeJS by running the following commands to return the version numbers for NodeJS and npm

node -v
npm -v

That's it!  Have fun!

 

How to build a LEMP stack

Growing up it was always common to spin up a "LAMP" box to host a website.  The typical setup was:
Linux
Apache
MySQL
PHP

Over the past few years, this model has slightly changed due to new open source technologies bringing new ideas to solve performance and licensing issues at massive scale.  In this tutorial, we are going to look at setting up a LEMP box on Debian Stretch (9.1).
Linux
nginx [engine x]
MariaDB
PHP

Please note, MariaDB could easily be swapped out with MySQL in this tutorial, however many have opted to jump over to MariaDB as an open source alternative (actually designed by the original developers of MySQL) over fear Oracle may close source MySQL.

Installing Linux

This tutorial assumes you already have either a copy of Ubuntu 14+ or Debian 7+.  This probably works on earlier versions as well, but I haven't tested them.  On a side note, I typically don't install Linux builds with an interactive desktop environment, so grab yourself a copy of Putty and ssh in or open up Terminal if you have interactive access to the Desktop Environment.  Before continuing, go ahead and update apt-get repos and upgrade any packages currently installed:

apt-get update && apt-get upgrade

Installing nginx

Grab a copy of nginx

apt-get install nginx

Installing MariaDB

Grab a copy of MariaDB

apt-get install mariadb-server

Installing PHP

In this case, I want to roll with PHP7.  You can specify php5 or php7 depending on your application, but PHP7 has some great performance enhancements, so for new apps, I'd leverage it.  The biggest thing here is to make sure you use the FastCGI Process Manager package.  If you specify just php or php7, package manager will pull down apache2 as a dependency.  That is not what we want in our LEMP stack.

apt-get install php7.3-fpm

Once installed, fire up your favorite text editor (it's ok if it's vi :)) and edit the default site for nginx

vi /etc/nginx/sites-enabled/default

Search for the comment # Add index.php to the list if you are using PHP and add index.php to the line below it.  For example:

index index.html index.htm index.php index.nginx-debian.html;

Next, find the comment # pass PHP scripts to FastCGI server and change the block of code to the following to tell nginx to process .PHP files with FastCGI-PHP:

# pass PHP scripts to FastCGI server
#
location ~ \.php$ {
include snippets/fastcgi-php.conf;
#
# # With php-fpm (or other unix sockets):
fastcgi_pass unix:/var/run/php/php7.3-fpm.sock;
# # With php-cgi (or other tcp sockets):
# fastcgi_pass 127.0.0.1:9000;
}

Save the file.  If using vi, you can do that by executing :wq

Next, reload the nginx service to pickup the new changes to our configuration:

service nginx reload

Test

At this point, we can create a php file to validate things are working well. Go ahead and create a new file /var/www/html/info.php and add the following line:

<?php
phpinfo();

If you see a page listing the PHP version and the corresponding environment configuration, congratulations, you have finished setting up your new LEMP stack! 🙂

[Tutorial] Adding firewall rules via system-config-firewall-tui on CentOS 6

Here is a quick tutorial on how to add an ingress firewall rule on your CentOS 6 machine.  In this example, we will be forwarding port 443 for HTTPS.

  1. Open up terminal if you are on the GUI version of CentOS 6
    CentOS6 - Terminal
  2. Execute the following command
    1. system-config-firewall-tui
      Terminal - system-config-firewall-tui
  3. Use your arrow keys to select Customize and hit enter
    system-config-firewall-tui - Customize Rules
  4. Use your arrow keys to select which service you would like to allow.  Hit the spacebar to enable or disable the rule and then select Close once you have enabled/disabled the rules you wish.
    1. In this case, I arrowed down to HTTPS and hit the spacebar.
      system-config-firewall-tui - Select Rules
  5. Select OK
    system-config-firewall-tui - Apply Rules
  6. Select Yes
    system-config-firewall-tui - Apply Rules - Confirmation