orchard

  1# CLAUDE.md
  2
  3Guidance for Claude Code (claude.ai/code) and for anyone else working in this
  4repository. Start with `README.md`, this file is the working detail behind it.
  5
  6**Anything visual is in `DESIGN.md`, and that file is the authority.** The
  7palette, the type, the grid, the home page skeleton and its word budget, and the
  8rules that are easy to get wrong all live there. Read it before touching a
  9template or a stylesheet, and update it in the same commit when a design
 10decision changes. There is no shared stylesheet here on purpose, so five sites
 11looking alike is something a person keeps true by following that file.
 12
 13## What this is
 14
 15One repo for every site Isaac Bythewood runs, plus the shared Cloudflare Tunnel
 16and Caddy that front them. Eleven sites, all Go, all served from a desktop behind a
 17tunnel rather than a rented server.
 18
 19| Directory | What it serves |
 20|---|---|
 21| `sites/isaacbythewood.com/` | Portfolio. Animation heavy, framework free, no third party Go dependency |
 22| `sites/blog.bythewood.me/` | Markdown blog. A PDF and a social card per post, and an Atom feed |
 23| `sites/analytics.bythewood.me/` | Self hosted analytics. SQLite, GeoIP, Typst PDF reports |
 24| `sites/status.bythewood.me/` | Self hosted uptime monitoring. SQLite, Lighthouse audits, crawler |
 25| `sites/logging.bythewood.me/` | Self hosted log aggregation. Every other site ships its slog records here. SQLite, retention and rollups, Typst PDF reports |
 26| `sites/repos.bythewood.me/` | Self hosted git remote. Push to it over HTTPS with a token, and it mirrors the GitHub account as a backup. Everything git is a subprocess |
 27| `sites/dash.bythewood.me/` | Dashboard. Markets off Yahoo, Hacker News and Lobsters, the weather, and whether the other sites are answering. One poller, server sent events out, no database |
 28| `sites/auth.bythewood.me/` | The front door. One account, a six digit code pushed over ntfy, and an opaque session every other site checks against it |
 29| `sites/search.bythewood.me/` | Answers a question against the web, checking every sentence it writes against the passage it cites |
 30| `sites/chat.bythewood.me/` | A conversation with a local model, with tools, attachments, history in SQLite and an incognito mode that writes nothing. Beside it an offline Wikipedia, served by kiwix off a 12.5GB ZIM |
 31| `sites/llm.bythewood.me/` | The model gateway. One set of weights on one card behind an API key, with every prompt and completion logged unless the caller marks the call incognito |
 32| `edge/` | The shared `cloudflared` tunnel, the Caddy that reverse proxies to each site, and the ntfy every alert is published to |
 33
 34## The one structural rule
 35
 36**Every site is its own Go module and owns its own copy of `web/`.**
 37
 38There is no module at the repo root. `go.work` exists so repo wide `make`
 39targets and an editor can see all eleven at once, and nothing depends on it. Each
 40site builds standalone:
 41
 42```sh
 43cd sites/blog.bythewood.me && GOWORK=off go build ./...
 44```
 45
 46and its Docker build context is that directory alone, so a site is a folder you
 47can copy into its own repository.
 48
 49`web/` is the small HTTP layer every site needs: the Vite manifest reader,
 50request logging, panic recovery, security headers, the static and edge cache
 51policies, graceful shutdown, `shipper.go`, the tee handler that copies every log
 52record to logging.bythewood.me, and `session.go`, which asks auth.bythewood.me
 53whether the cookie on a request is a live session.
 54
 55**A fix in `web/` has to be made eleven times.** Do not add a shared parent module
 56to avoid it, and keep `shipper.go` and `session.go` byte identical across the
 57eleven, since both are wire formats as much as files.
 58
 59## Commands
 60
 61From the repo root:
 62
 63```sh
 64make install                         once per machine: tunnel, secrets, containers, alerts
 65make up                              edge, then every site; idempotent, and the repair command
 66make deploy SITE=blog.bythewood.me   rebuild one site and replace it
 67make doctor                          tunnel, network, containers, and the data volumes
 68make wiki                            download the offline wikipedia into its volume, 12.5GB
 69make down                            stop everything
 70
 71make run SITE=blog.bythewood.me      vite watch + go run
 72make build SITE=blog.bythewood.me
 73make check                           gofmt, then vet and build every site
 74make test                            every site's tests, including its own web/ copy
 75```
 76
 77There is no default `SITE`, so a bare `make deploy` cannot replace the wrong
 78one. `up` does not pass `--build`, so it starts what is missing and reuses
 79existing images, which makes it both safe on a healthy system and the repair
 80command on a broken one, and `deploy` is the only thing that rebuilds. Each site
 81has `run`, `build` and `clean` in its own `Makefile`, but no `serve` target,
 82since `go run .` without Vite leaves no manifest and that is fatal.
 83
 84## Conventions that will bite you
 85
 86**Port 8000, everywhere.** Every container listens on 8000 internally, in dev
 87and in prod, and no host ports are published. `cloudflared` terminates the
 88tunnel and hands plain HTTP to Caddy, which reverse proxies to each app by
 89container name on the `orchard-edge` network. The one other port in the repo is
 90`orchard-logging:9001`, a plain TCP socket Caddy writes its access log to,
 91reachable on the bridge and nowhere else.
 92
 93**Everything is named `orchard-<first label>`.** `orchard-caddy` and
 94`orchard-cloudflared` for the edge, then `orchard-blog`, `orchard-analytics`,
 95`orchard-status`, `orchard-isaacbythewood`, `orchard-logging`, `orchard-repos`
 96and `orchard-dash`, plus an `orchard-<label>-data` volume for each of the four
 97sites with SQLite. One prefix, so `docker ps --filter name=orchard` is the whole
 98system and the Makefile derives a container name from a site directory without a
 99lookup table.
100
101**Six things reference a container by name, and all six bake it in.**
102`edge/caddy/Caddyfile` reverse-proxies to each site and writes its access log to
103`tcp/orchard-logging:9001`, `sites/isaacbythewood.com/site.go` fetches
104`http://orchard-blog:8000/latest.json` for the latest-posts panel,
105`web/shipper.go` in every site posts to `http://orchard-logging:8000/ingest`,
106`alerts.go` in status and logging posts to `http://orchard-ntfy:8000`, and
107`sites/dash.bythewood.me/systems.go` reads `http://orchard-logging:8000/aggregate`
108and probes every other site at `http://orchard-<label>:8000/healthz`, and
109`web/session.go` in every site asks `http://orchard-auth:8000/verify`. None reads
110the name at runtime, so renaming one means rebuilding Caddy, the portfolio and
111every site, not just editing a compose file. No SQLite database refers to a
112container name.
113
114**The seventh is the one exception.** `sites/chat.bythewood.me/tools/wikipedia.go`
115reaches `http://orchard-wiki:8000` through `WIKI_URL`, which compose sets and
116which falls back to that name, so it is the only one of these a dev run can
117point somewhere else without a rebuild.
118
119**Alerts leave through ntfy in the edge, and reading them is authenticated.**
120status publishes to the `status` topic, logging to `logging` and auth to `auth`,
121all to `http://orchard-ntfy:8000` on the bridge with a write-only token from
122each site's `.env`, and reading is over the tunnel at `ntfy.bythewood.me` with a
123read-only account. There are three accounts, not two: `orchard-auth` writes the
124auth topic and nothing else, because the token in status' and logging' `.env`
125files can publish to their topics and sharing it would let a copy of either mint
126its own login codes. Two fences hold, ntfy runs `auth-default-access: deny-all`
127and decides who may publish, and Caddy refuses every publish route on the public
128hostname, so the write token on its own gets an outsider nowhere. ntfy has no
129source-based ACL, which is why that second fence has to live at the edge.
130`make ntfy` creates the accounts and `make ntfy-token` mints the token.
131
132**The Caddy publish fence has to stay a denylist.** Blocking `POST` is not
133enough, because ntfy publishes over GET too: `/<topic>/publish`, `/send` and
134`/trigger` all publish with no body, and `POST /` publishes with the topic in a
135JSON body, all confirmed against the running container. A rule blocking
136`POST /<topic>` leaks three ways.
137
138**Cloudflare bounds what you can push to repos and nothing here can raise it.**
139It rejects any proxied request body over 100MB on Free and Pro, and a tunnel
140hostname has to stay proxied because `<uuid>.cfargotunnel.com` is not publicly
141routable and a DNS-only record would resolve to nothing. `http.postBuffer` does
142not help either, git's own documentation says raising it only disables chunked
143encoding for servers that cannot handle it. Everyday pushes send new objects
144only and are kilobytes, so this bites once per repository, on the first push.
145Two ways past it, in order of least effort:
146
1471. **Seed over the Docker bridge.** From a container on `orchard-edge`, push to
148   `http://orchard-repos:8000/<name>.git`. Cloudflare is not in the path.
1492. **Push in slices,** which works from anywhere:
150   `git log --oneline --reverse main | awk 'NR % 500 == 0' | cut -d' ' -f1 | while read sha; do git push origin +$sha:refs/heads/main; done`
151   then a final `git push origin main`.
152
153The repository page meters each repo against the limit and names which route to
154use once one is over it. `orchard` itself packs to about 95MB, so it is the one
155to watch.
156
157**Secrets are a `.env` beside each site's compose file.** Compose reads it
158because that directory is the project directory, so nothing is exported in a
159shell and nothing is forwarded through the Makefile. `.env` is gitignored by
160bare name and this repo is public, so check it with `git check-ignore -v`
161instead of assuming. Every site needing one commits a `.env.example`, and
162`make env` turns each example into a `.env` with a generated value for every
163empty `*_PASSWORD`, printing them once. It skips a site that already has one,
164because rewriting `repos`' password signs every open session out and rewriting
165either of the other two loses the ntfy token with it.
166
167**Editing anything in `edge/` needs `make edge`.** The Caddyfile and ntfy's
168`server.yml` are baked into images and `make up` does not pass `--build`, so an
169edited config that was never rebuilt is a silent no-op. cloudflared is worse
170again, its config lives in a volume so compose sees no change and will not
171restart it, and the tunnel serves the old ingress while a newly added hostname
172404s from the Cloudflare edge. `make edge` restarts it explicitly, but a restart alone
173still serves the old ingress, because the config it reads is the copy in the
174volume and nothing has replaced it.
175
176Adding a hostname is five changes: a Caddy site block, a `cloudflared` ingress
177rule, the name in `HOSTNAMES` in `edge/setup-tunnel.sh`, a proxied CNAME to
178`<tunnel-id>.cfargotunnel.com`, and then `make tunnel` to reseed the
179volume before `make edge`. Skip the reseed and the container is healthy, Caddy
180is right, and the hostname still 404s from the Cloudflare edge. The DNS route
181calls in that script fail with an authentication error unless `cert.pem` covers
182that zone, which does not matter when the CNAME already exists.
183
184**Containers run as UID 65532, and base images are pinned by digest.** The four
185Alpine sites create a real user at that UID and the two scratch ones use the
186bare number, since there is no `/etc/passwd` to name one in. A `/data` volume
187created root-owned stays root-owned, so a new one has to be chown'd once.
188
189**The binary is its own health check.** `-healthcheck` does a loopback GET
190against `/healthz` and exits 0 or 1. A `FROM scratch` image has no shell for
191`HEALTHCHECK` to call, so compose and the Dockerfiles both use the flag.
192
193**Logging is `log/slog`, JSON to stdout, UTC.** `web.SetupLogging()` runs in
194every main before anything else logs. UTC is forced through `ReplaceAttr`
195because it is not slog's default, and local time in a container silently differs
196from the host's.
197
198**Log records are teed, never replaced.** `web.ShipLogs(source, web.HTTPSink())`
199runs straight after `SetupLogging`, past the healthcheck branch so a
200`HEALTHCHECK` invocation does not start a queue it will never flush. It wraps
201whatever handler is installed and copies each record onto a bounded channel a
202goroutine flushes to `orchard-logging`. Nothing on that path blocks a caller, so
203a full queue, a failed POST and a 429 all drop the record, and stdout stays the
204source of truth, which means the worst a broken logging site can do is lose
205lines from a dashboard. The shipper writes its own state changes to stderr and
206never calls `slog`, which would enqueue a record about failing to ship, and
207`logging.bythewood.me` passes a local sink, because posting to itself would be
208an ingest request that logs a request that becomes an ingest request.
209
210**Caddy ships, cloudflared and ntfy do not.** Caddy can't carry a Go handler, so
211it writes its access log to a socket with its own `net` writer and
212`logging.bythewood.me` listens on 9001 for it, turning each line into the record
213`web.Logged` would have written for the same request. `soft_start` is required
214there, or Caddy refuses to boot whenever the logging site is down, which `make
215up` guarantees since the edge starts first. A second `log console` block keeps
216the same events on stderr so `docker logs orchard-caddy` is unchanged, and only
217the access log ships, never Caddy's runtime log, since that's where a failing
218`net` writer reports itself and shipping it over the failing connection would
219loop. cloudflared and ntfy log to stdout and can't write to a socket, and
220pointing either at a file takes its stdout away, so neither ships.
221
222**A site block with no host matcher gets one logger, not a list.** The `:80`
223catch-all in the Caddyfile becomes the server's `default_logger_name`, which is
224a single string, so naming two loggers there silently keeps one and drops the
225other. It carries the console logger alone for that reason.
226
227**`Shipper.Close()` has to stay bounded.** Draining a 4096 deep
228queue with a synchronous sink call every 500 records runs far past Docker's ten
229second stop grace when the far end is wedged, and one hung container would then
230get every other site SIGKILLed on the next `make deploy`, skipping their
231`db.Close()`. Close waits on a timer instead, and every compose file sets
232`stop_grace_period: 30s`.
233
234**Deploys need `sudo`, and `sudo` then eats the password.** The Docker socket is
235`root:root` mode 660 and the `docker` group does not help. But sudoers here sets
236`env_reset`, so `ANALYTICS_PASSWORD=... sudo docker compose up` starts compose
237with the variable stripped and the `${VAR:?}` guard aborts, complaining about
238the shell you just set it in. The Makefile forwards each one as a sudo-level
239assignment (`sudo VAR="$VAR" docker compose ...`), which survives. Every docker
240command here, `edge/setup-tunnel.sh` included, goes through a `SUDO` variable so
241a host that needs no sudo can turn it off.
242
243**Frontends are bun and Vite 8.** Output goes to `sites/<name>/build/dist/` with
244content hashed filenames, and the Go binary reads
245`build/dist/.vite/manifest.json` to resolve them. A missing manifest is fatal,
246since serving a page whose script tag points at a file that was never built is
247worse than refusing to start. Vite 8 bundles with Rolldown, which imports
248`styleText` from `node:util`, and Node 18 does not export it, so a build under
249an old Node dies with a `SyntaxError`. It needs bun 1.4 or a modern Node, and
250the pinned `oven/bun:1-alpine` in every frontend stage is already 1.4.
251
252**`build/` holds every generated file, and only generated files.** Vite output,
253the blog's PDFs and cards, analytics' topojson. It is gitignored and `make
254clean` deletes it. A dev build reads it off disk, and `make build` passes
255`-tags embed`, swapping `assets_disk.go` for `assets_embed.go` so `//go:embed`
256compiles the directory into the binary. blog and isaacbythewood.com come out as
257a single self-contained file, while analytics, status and logging still need
258typst on disk, status also bun and chromium, and repos git, because those are
259programs and not assets. The tag exists so a fresh clone still builds, since
260`//go:embed` fails at compile time on a missing directory, and it lives under
261`build/` because a directive cannot reference a path above its own package.
262
263**Typst runs at build time, not on the request path.** Post PDFs, the resume and
264every social card are compiled during `docker build` and served as files, which
265is how the blog and the portfolio end at `FROM scratch`. Analytics, status and
266logging keep Typst in the runtime image because their reports come from live
267data over an arbitrary date range, with no finite set to precompile.
268
269**Fonts for Typst must be TrueType.** Geist comes from `bun add geist`, Vercel's
270package and not `@fontsource/geist`, which ships woff2 only, and Typst reaches
271it through `--font-path`. A missing face does not error, it falls back to a
272serif, which is how `blog_post.typ` asked for Inter and rendered in DejaVu.
273
274## dash.bythewood.me
275
276The seventh site, built 2026-08-30. Markets, Hacker News, Lobsters, the weather
277and a health strip for the other six, all on one page that updates itself. It is
278public and has no login, which is the constraint everything below follows from.
279
280**Yahoo's `v7/finance/quote` is gone.** It answers 401 Unauthorized to anything
281that has not carried a cookie and a crumb through their handshake, and it is the
282endpoint every tutorial still points at. `v7/finance/spark` and
283`v8/finance/chart` both still answer with no session at all, and spark takes the
284whole symbol list in one request and returns the same meta block plus the
285intraday closes, so the entire markets panel costs one call per poll. It needs a
286browser-like User-Agent or the response is a block page, and Yahoo's edge sends
287`cache-control: max-age=10`, so polling faster than that returns bytes you
288already have.
289
290**Hacker News comes from Algolia, not from the official API.** The Firebase API
291hands back 500 bare story ids and charges one request per story to resolve each
292one. `hn.algolia.com/api/v1/search?tags=front_page` returns all thirty with
293titles, scores and comment counts in a single response.
294
295**Futures replace the cash indexes outside the session**, driven by a New York
296clock in `market.go` rather than by a holiday calendar. There is no calendar on
297purpose, and the case it would catch is caught instead by the age of the S&P's
298own quote: a cash index that has not printed in half an hour during what the
299clock calls regular hours means the clock is wrong.
300
301**The health strip asks the bridge first and the public hostname only as a
302fallback.** Cloudflare will serve a cached 200 for `/healthz` long after the
303origin behind it has stopped answering, and two of these sites do exactly that
304right now, so a public probe is not evidence a site is up. A row built from the
305fallback says `cached` rather than `up`.
306
307**What logging hands over is counts and nothing else.** `/aggregate` returns a
308record, error and 5xx count per source plus the watchdog's up flag, never a
309message, a path, a status code or an address, because dash publishes it to the
310internet. Caddy refuses the path on the public hostname the same way it refuses
311`/ingest`, and a test on each side asserts the field list rather than trusting
312the handler to stay honest.
313
314**Everything is fetched once and pushed to every browser.** One poller per
315source writes into a store and the store broadcasts the whole state as JSON over
316`/events`, so ten open tabs still cost one request upstream. The markets poll
317drops from 30 seconds to 5 minutes when nobody is connected, which keeps the
318page warm for the first visitor without polling Yahoo all night for no one.
319
320**`web/server.go` here sets `WriteTimeout: 0`,** the third version of that file
321in this repo. Go's write bound covers the whole response, so any value at all is
322a ceiling on how long a stream may stay open. An idle stream is held up by a
323comment frame every 25 seconds instead, which is inside Cloudflare's 100 second
324idle drop.
325
326**Caddy's `encode` takes a matcher in this site's block** rather than the
327blanket one in `(site)`, because a compressed `text/event-stream` buffers.
328
329**Every card on the strip is drawn against one New York trading day.** It runs
3309:30 to 16:00 and rolls at the open rather than at midnight, weekends included,
331so gold and crude and bitcoin reset in the morning with the exchanges instead of
332whenever Yahoo thinks their day starts. Bars before the open are dropped and
333whatever prints after the close stretches the window, and the eight cards share
334one window so the right edge of one means the same hour as the right edge of the
335next.
336
337**A card whose market shut gets its dead stretch marked.** The VIX stops at the
338bell and the seven cards beside it keep trading, so it ends its line about 60%
339across an evening card, and an empty right hand third on one of eight reads as a
340broken card rather than a closed market. It takes a faint dashed rule at the last
341print and the amber cursor is dropped, since on every other card that cursor means
342here is now and it would be saying the opposite thing in the same place. The space
343right of the rule is left alone, because the dotted baseline already runs the width
344of the card and a filled block there shouted over everything else on the page. A
345card only counts as shut once it is more than half an hour behind the latest bar
346anywhere on the strip, because the futures normally trail bitcoin by about ten
347minutes without either having stopped.
348
349**The previous close comes off the bars for anything that trades around the
350clock.** Yahoo dates bitcoin's day by UTC and a future's by its contract, so
351their own `chartPreviousClose` measures from a different moment than the S&P's
352and the eight cards disagree about what day it is. The four cash indexes and the
353VIX keep Yahoo's number, since theirs is already the 4pm one and it is what every
354other site quotes.
355
356**The strip goes out as two requests, at two ranges.** The cash indexes only need
357the day they are in, and everything else needs enough history to find 4pm
358yesterday, which over a weekend is three days back. `1d` and `5d`, and it is
359still two batches because the split lands under the ten symbol limit either way.
360
361**The earnings panel is the top hundred of the S&P 500, and that takes two
362sources.** Nasdaq's screener answers every US listing sorted by market cap in one
363call, and `sp500.go` holds the index membership, because a cap floor on its own
364fills the panel with ASML, TSM and ARM. The hundredth name is worth about $120B
365right now, so the old $10B floor was twelve times too loose and the panel was
366showing companies nobody has heard of.
367
368**Nasdaq drops the time of day once a date is in the past.** An upcoming row says
369`time-pre-market` or `time-after-hours` and the same row a week later says
370`time-not-supplied`, so which of the two sessions around a report carried it is
371not knowable from the calendar. The reaction takes the one that moved, which is
372right for both cases and does not matter when neither moved. The presence of an
373actual EPS is also what sorts a row into reported or upcoming, rather than
374comparing its date to today, so a company that reported before the bell this
375morning reads correctly.
376
377**Nothing free publishes guidance.** A beat the market sold is the closest this
378gets, so a row where the result and the move disagree by more than 1.5% is
379tagged `SOLD THE BEAT` or `BOUGHT THE MISS` and the panel says nothing else about
380the outlook.
381
382**`range=1d` for BTC-USD is the UTC day and not the last 24 hours.** It rolls at
3838pm New York, so a poll at 8:05pm came back with six bars and drew a straight
384line across the card. That is what `carrySparks` was written for, and the wider
385range fixes it at the source, but it stays as a guard against Yahoo genuinely
386having a moment. It keeps the previous shape when a poll returns fewer than five
387points and the last one had more, within one symbol and one trading day so a card
388never shows yesterday's chart or the other instrument's. Only the shape is held
389back, the price and the percent are always fresh.
390
391## Signing in
392
393**One account, and it lives on auth.bythewood.me.** analytics, status, logging
394and repos each carried a near identical `auth.go` with its own password until
3952026-08-31. They now have none: signing in is a username and a six digit code
396pushed to a phone over the `auth` ntfy topic, and the four sites ask
397`orchard-auth:8000/verify` whether the cookie on a request is live.
398
399**The session cookie is opaque and checked on every request.** It is 32 random
400bytes on `.bythewood.me`, stored here only as a SHA-256, and a site cannot
401validate it alone. That is what revocation needs: a signed cookie stays valid
402until it expires whatever the issuer says, so signing a device out would mean
403rotating a key and ending every other session with it. Nothing caches the
404answer, because a cache is a window in which a revoked session still works.
405
406**With orchard-auth down, all four dashboards are unreachable.** That is
407inherent, and it is why the break-glass is ten Argon2id recovery codes rather
408than a password, and why `make auth-init` prints the first set. ntfy is behind
409the same tunnel as the sites it gates, so a bad tunnel takes the code path with
410it.
411
412**What bounds abuse is a per account ceiling, not a per IP one.** The username is
413in this public repository, so anyone can post it to `/login`. One code is
414outstanding at a time and a repeat request publishes nothing, and at most five
415notifications go out per hour whatever address they are asked from. A per IP
416limit is bypassed by sending one request from each of a thousand proxies. Codes
417publish at ntfy priority `low` so a flood is silent, and only a session that
418actually opened is `high`.
419
420**A code is bound to the browser that asked for it** by a second short lived
421cookie, so a code visible on a lock screen cannot be typed into somebody else's
422session.
423
424**Cloudflare's location headers are only trustworthy because of the tunnel.**
425`CF-IPCountry` and the visitor location transform feed the session list and the
426notification that says where a login came from. Nothing publishes a host port,
427so nothing reaches an app without passing cloudflared and Caddy. The day one is
428directly reachable a client can set its own country.
429
430**auth cannot change other containers, and must not learn how.** It does not hold
431the ntfy account password, which ntfy stores hashed and will not hand back, and
432it does not hold a Cloudflare token that could write WAF rules. Both would make
433an internet facing container into the thing that owns the machine. `make
434ntfy-passwd` changes the ntfy password.
435
436**repos keeps its own push tokens.** git cannot answer a code prompt, so the
437Argon2id credentials on the wire stay in repos' database and are unaffected by
438any of this. Only its browser UI moved.
439
440## Rules learned the hard way
441
442**An hourly rollup cannot answer an unaligned window.** `logging`'s
443`rollups.hour` is an hour-floored timestamp, so `hour >= start` against a
444`now`-relative start drops the bucket that contains the start, whole, and every
445tile then lost up to an hour of data while the raw-backed panels beside them did
446not. Floor the start of the window and the rollup sum equals the raw count. A
447test using an hour-aligned base with a `base-1 .. base+1` window cannot catch
448this, so give the test an unaligned base.
449
450**Percentiles in SQLite want `CUME_DIST`, not `PERCENT_RANK`.** `PERCENT_RANK`
451assigns exactly 1.0 to the largest row of every partition, so `pr <= 0.95` never
452selects the slowest sample and a path with few samples reports a value well
453below the percentile its column claims. `CUME_DIST` reaches 1.0 at the maximum,
454so `MIN(CASE WHEN cd >= 0.95 ...)` is the nearest-rank percentile it says it is,
455in one query instead of one per percentile.
456
457**A template listed with no file behind it ships and then crash-loops.**
458`web.NewRenderer` resolves its page list at boot, not at build, so deleting
459`login.html` while leaving it listed compiled, passed every test, built an image
460and then failed at startup with `pattern matches no files`. Every site now keeps
461`layoutTemplates` and `pageTemplates` as package vars and a test parses them.
462
463**A container health check is not traffic.** Every one is the binary probing
464itself over loopback with no `CF-Ray`, and in `logging` they outnumbered real
465requests and were counted by the latency percentiles and the busiest-paths
466ranking, dragging both away from anything a visitor sees. They are demoted to a
467rollup counter rather than dropped, so the proof that each site answered its
468probe survives without a raw row.
469
470**Never put `$(MAKE)` on a recipe line that also does something.** GNU make runs
471any recipe line containing that string even under `-n`, so a one-line shell
472conditional ending in `$(MAKE) doctor` is executed in full by a dry run, side
473effects and all, which is enough to replace a running container. Keep `$(MAKE)`
474on a line of its own.
475
476**Do not let a poller that probes this process start before the listener.**
477dash's health strip asks itself over loopback, and starting the pollers before
478`web.Serve` has bound the socket makes the dashboard report the site it is
479running on as down or unknown until the next round, which is every deploy. Split
480the synchronous first fetch from the loops and gate the loops on a health check.
481
482**A middleware that wraps the ResponseWriter has to implement `Unwrap`.** Both
483wrappers in `web/middleware.go` do now. Without it `http.ResponseController`
484cannot reach the real writer, so a server-sent events handler behind `Logged`
485cannot flush and dash's `/events` answered 500 for every request. Assert on
486`http.Flusher` and you get the wrapper, not the connection.
487
488**Never animate a layout property.** Animating `height`, `width`, `top` or
489`left` relays out the page every frame, and the browser scores each frame as a
490layout shift even when the animation is intentional and covers the whole
491viewport, which is enough to fail CLS with no image involved. Use `transform`
492(`scaleY`, `translateX`) with `transform-origin`, which is composited, looks
493identical and scores zero. Verify with `document.getAnimations()` and a pixel
494diff instead of by eye.
495
496**`og:image` must be a raster format.** Facebook, X, LinkedIn, Slack, iMessage
497and Discord all refuse `image/svg+xml`.
498
499**Do not put `s-maxage` in a cache policy here.** Per RFC 9111 it carries
500`proxy-revalidate` semantics, so Cloudflare treats it as never serve stale
501without asking first and it disables both `stale-while-revalidate` and
502`stale-if-error`. The origin is a desktop at the end of a tunnel, so
503`stale-if-error` is what keeps the edge serving the last good copy instead of a
504530 when the house goes dark. Use plain `max-age` with both stale directives,
505and keep Cloudflare's Always Online off, since it makes `stale-if-error`
506ignored.
507
508**Error responses get `no-store` explicitly.** Once a Cache Rule marks a zone
509eligible, Cloudflare stamps its own browser TTL on a header-less response and
510will hold a 404 at the edge, so publishing a post at a URL somebody already
511missed would serve them the 404 for hours.
512
513**Hardcode identity, never hardcode credentials.** Base URLs and site names are
514constants in `site.go`, and the only environment variables that exist are real
515secrets and paths set by the Dockerfiles. A `BASE_URL` that defaults to empty is
516worse than no variable at all, since it ships a site whose own tracking silently
517does nothing.
518
519## Tests
520
521`make test` runs every site's suite plus its `web/` copy. There
522are no linter configs, and `make check` is gofmt, vet and build. The portfolio
523has no Go tests of its own, being templates and handlers over static data, and
524is covered by its `web/` package plus browser checks.
525
526`web/shipper_test.go` is one of the eleven identical copies and covers the parts
527easy to get quietly wrong: that a record reaches both the original handler and
528the queue, that `WithAttrs` and `WithGroup` still tee, that a full queue drops
529instead of blocking, and that logging after `Close` does not panic.
530`logging.bythewood.me` also renders every page against a real seeded database,
531because a template referencing a field that does not exist fails at execute time
532rather than at parse time.
533
534## Writing
535
536Comments explain why, and only when the why is not visible from the code around
537them. Three lines is the ceiling and most should be one or two. Delete anything
538that restates the line, narrates what the code used to be, or quotes a number
539from a single run. Nothing written here points at anything outside this
540repository, since a reader only ever has the repo. Commit subjects are short,
541capitalised, imperative and carry no trailing period, like `Add atom feed` or
542`Fix rollup window off by an hour`, and they carry no body. Prose has no em
543dashes and no semicolons, use a comma, a full stop, or and.