hardware/mac-studio/services/nginx.md

Table of Contents

Nginx — Central Reverse Proxy (Mac Studio)

Host: Mac Studio — 192.168.1.189 Install: Homebrew — /opt/homebrew/etc/nginx Binary: /opt/homebrew/opt/nginx/bin/nginx Config root: /opt/homebrew/etc/nginx/nginx.conf + /opt/homebrew/etc/nginx/sites-enabled/*.conf

As of 2026-09-20, this Mac Studio is the central nginx server for all *.dph.am (and some qwestly.co) traffic. The router forwards ports 22, 80, and 443 to this machine's LAN IP (192.168.1.189), and wildcard DNS (*.dph.am) points at the home's public IP. Any subdomain that doesn't have its own sites-enabled conf falls through to a default block serving ~/Work.

This file is the current source of truth for the nginx server setup itself. ../overview.md and nginx-auth-service.md have been updated (2026-09-20) to point here instead of describing nginx as living on the old Mac Mini dev-server (192.168.1.185).

Related docs:

  • nginx-auth-service.md — the auth_request login wall and per-app backend list.
  • nginx-pwa-injection.md — how self-hosted apps get PWA manifests/icons and injected UX scripts via sub_filter, served from ~/Work/pwa/<app>/.

Why nginx moved here

The nginx config (sites-enabled/, ~40 site .conf files) was copied over from the old Mac (dev-server, z.dph.am / 192.168.1.185) to make this Mac Studio the new central server. Certbot and its certs (/etc/letsencrypt) had to be migrated too — see below.

Certbot / SSL

  • Installed via Homebrew: brew install certbot → binary at /opt/homebrew/bin/certbot
  • Cert store: /etc/letsencrypt (standard path, 0700 root, not Homebrew-prefixed)
  • How certs got here: the old Mac (z.dph.am, SSH port 22, user dominick) held the real certs. Copied over via tarball (plain rsync can't read root-owned dirs remotely without a TTY for the remote sudo password):
    # on old Mac (interactive, so sudo can prompt):
ssh -p 22 dominick@z.dph.am
sudo tar -czf /tmp/letsencrypt-backup.tgz -C /etc letsencrypt
sudo chown dominick /tmp/letsencrypt-backup.tgz

# on this Mac:
scp -P 22 dominick@z.dph.am:/tmp/letsencrypt-backup.tgz /tmp/
sudo tar -xzf /tmp/letsencrypt-backup.tgz -C /etc
  • Each site's .conf references /etc/letsencrypt/live/<domain>/fullchain.pem + privkey.pem, plus the shared options-ssl-nginx.conf and ssl-dhparams.pem — all came over in that same tarball.

Running nginx as root (critical gotcha)

Homebrew's default nginx install runs as a user-level LaunchAgent (~/Library/LaunchAgents/homebrew.mxcl.nginx.plist), started as your login user. That's fundamentally incompatible with this setup:

  • Can't read /etc/letsencrypt/live/* — Certbot locks it 0700 root.
  • Can't reliably bind ports <1024 (80/443) long-term.

Fix: run nginx as a real root LaunchDaemon.

brew services stop nginx                 # kill the user-level agent
sudo pkill -f "nginx: master process"
sudo pkill -f "nginx: worker process"
sudo brew services start nginx           # registers /Library/LaunchDaemons/sh.brew.nginx.plist

Gotcha: Bootstrap failed: 5: Input/output error

Hit this twice. Root causes, in order encountered:

  1. Port already held by the stale user-level master — bootout/pkill it first (see above).
  2. Plist ownership — Homebrew writes /Library/LaunchDaemons/sh.brew.nginx.plist as root:admin. launchd wants root:wheel for system daemons. Fix:
    sudo launchctl bootout system/sh.brew.nginx   # clear stale/half-registered state
sudo chown root:wheel /Library/LaunchDaemons/sh.brew.nginx.plist
sudo chmod 644 /Library/LaunchDaemons/sh.brew.nginx.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/sh.brew.nginx.plist

Verify:

ps aux | grep "[n]ginx"                              # master should show USER root
sudo lsof -nP -iTCP:80 -iTCP:443 -sTCP:LISTEN         # both ports LISTEN
sudo launchctl print system/sh.brew.nginx | grep state # "state = running"

Gotcha: worker runs as nobody → 403s on ~/Work

With no explicit user directive, an nginx master started as root defaults its workers to nobody. nobody can't traverse into /Users/dominick/... (home dir isn't world-readable), so anything serving out of ~/Work (the default/fallback site) 403s.

Fix — add to the top of nginx.conf:

user dominick staff;

Then force a full daemon restart (not just nginx -s reload) so workers respawn under the new user:

sudo nginx -t
sudo launchctl kickstart -k system/sh.brew.nginx

sites-enabled load order matters

nginx.conf includes sites-enabled/*.conf in alphabetical/glob order. Digits sort before _ in ASCII, so 0.dph.am.conf loads before _localhost.conf. Since neither had default_server set explicitly, nginx picked whichever loaded first as the implicit default for port 80 — which silently changed behavior (unmatched hostnames started hitting 0.dph.am's return 404 instead of falling through to the ~/Work catch-all) once 0.dph.am.conf actually got loaded (see next section).

Fix: _localhost.conf explicitly declares itself default:

server {
    listen       80 default_server;
    server_name  localhost;
    ...
}

There's a separate, deliberate default_server for port 443 in sites-enabled/default.conf — catches unmatched HTTPS hosts and returns 444 (connection closed, no response). That one's fine as-is.

The "why was 443 refused at all" saga

Short version: there was a stale nginx master process running since 2026-09-15 (well before the certs/configs were copied over), started under the old user-level LaunchAgent. Every nginx -s reload after that failed silently with emerg: cannot load certificate ... Permission denied (because non-root can't read /etc/letsencrypt), so the ancient in-memory config — from before most sites-enabled files existed — kept serving. It had no listen 443 block at all, hence "connection refused" on HTTPS despite the router correctly forwarding 443. Killing that stale process and moving to the root LaunchDaemon (above) was the actual fix, not just fixing the referenced cert paths.

Lesson: if nginx -t passes but behavior doesn't match the config on disk, check ps for how long the running master has actually been up — a reload can fail silently and leave a very old process serving forever.

Gotcha: proxy_pass to a LAN IP fails with No route to host (macOS Local Network Privacy)

Symptom: 502s on any site proxying to a real LAN address, error log shows connect() to 192.168.1.x:PORT failed (65: No route to host), while curl to the exact same address from a shell on the same box works fine. tcpdump shows zero packets for nginx's attempt.

Cause: macOS 15+ (confirmed on 26.6.2) gates outbound connections to RFC1918 addresses per-process, in the kernel. Denial returns EHOSTUNREACH and emits no packets. Grants attach to the responsible process — a shell-run nginx inherits the terminal's grant and works, but the launchd-spawned daemon is its own responsible process with no GUI session, so it's denied by default. Denials are not logged anywhere.

Fix: System Settings → Privacy & Security → Local Network → enable nginx. Applies immediately, no restart needed.

How to confirm it's this (not the config, not a firewall): run the same nginx binary from an interactive shell against a minimal conf that proxies to the LAN target — if the shell-run copy gets a 200 while the daemon 502s, it's the permission gate and nothing else.

mkdir -p /tmp/lnx/logs && cat > /tmp/lnx/nginx.conf <<'EOF'
worker_processes 1; daemon off;
error_log /tmp/lnx/logs/error.log info; pid /tmp/lnx/nginx.pid;
events { worker_connections 64; }
http { access_log off; server { listen 127.0.0.1:8099;
  location / { proxy_pass http://192.168.1.153:30141; proxy_connect_timeout 3s; } } }
EOF
/opt/homebrew/opt/nginx/bin/nginx -p /tmp/lnx -c /tmp/lnx/nginx.conf &
curl -so /dev/null -w "shell nginx: %{http_code}\n" --max-time 5 http://127.0.0.1:8099/
curl -sko /dev/null -w "daemon    : %{http_code}\n" --max-time 5 -H "Host: pi-win.dph.am" https://127.0.0.1/

A brew upgrade of nginx changes its ad-hoc signature and may drop the grant — if LAN proxying breaks right after an upgrade, check this pane first. Machine-wide context (applies to every third-party daemon on this box, not just nginx): hardware/mac-studio/local-network-privacy.md.

Quick reference

# check status
ps aux | grep "[n]ginx"
sudo launchctl print system/sh.brew.nginx | grep -i state
sudo lsof -nP -iTCP:80 -iTCP:443 -sTCP:LISTEN

# reload config (safe, no downtime)
sudo nginx -t && sudo nginx -s reload

# full restart (needed after nginx.conf top-level changes, e.g. `user` directive)
sudo nginx -t && sudo launchctl kickstart -k system/sh.brew.nginx

# add a new certbot-managed site
sudo certbot --nginx -d newsite.dph.am    # or certonly + manual nginx conf if DNS-01/cloudflare

Open items / not yet done

  • Confirm OpenClaw's trusted-proxy IP allowlist — done 2026-09-20, took two passes: first bumped trustedProxies to 192.168.1.189 (nginx's new LAN IP), but OpenClaw's trusted_proxy_local_interface_source guard rejects trusting an IP that's one of the app's own local interfaces (nginx + OpenClaw are now same-host). Real fix: route max.dph.am.conf's proxy_pass through 127.0.0.1 (true loopback) and set trustedProxies: ["127.0.0.1", "::1"] + trustedProxy.allowLoopback: true + trustedProxy.allowUsers: ["dominick"] in ~/.openclaw/openclaw.json. Full writeup in nginx-auth-service.md. Confirmed working — max.dph.am 200, authenticated session over remote=127.0.0.1.
  • Confirm Certbot renewal (certbot renew) works end-to-end now that nginx runs as root via LaunchDaemon — the renewal hook that reloads nginx may reference the old user-level service.
  • Homebrew flagged that upgrading/reinstalling nginx will need manual sudo rm on some paths it took root:admin ownership of (/opt/homebrew/Cellar/nginx/..., /opt/homebrew/opt/nginx, /opt/homebrew/var/homebrew/linked/nginx) — expect friction on the next brew upgrade nginx.