Skip to content

perf(build): write the /cite/ redirects from the compiler - #154

Merged
maehr merged 2 commits into
textrefs:stagingfrom
maehr:perf/cite-redirects-compiler
Sep 7, 2026
Merged

maehr merged 2 commits into
textrefs:stagingfrom
maehr:perf/cite-redirects-compiler

Conversation

@maehr

@maehr maehr commented Sep 7, 2026 •

Copy link
Copy Markdown
Member

What this changes

/cite/ minted 172,794 pages — two thirds of the built site. Astro rendered each one through the full route pipeline to produce four tags and no content. Pagefind then opened every one of them and found no data-pagefind-body.

Those pages are output, not routes. scripts/compile.ts now writes them after astro build, next to the /dump/ resources it already writes there. The redirect stays a static file, so curl, crawlers, and every non-browser client keep working, and GitHub Pages still needs no redirect layer.

Measured on the full registry

before after
npm run build 1m46s 1m12s
astro build 259,815 pages in 1m42s 87,021 pages in 54s
Pagefind 22.5s 8.4s
dist/cite/ 77.1 MB 63.9 MB
files in dist/ 346,596 346,596

The move changed nothing (first commit)

Every one of the 172,794 pages came out byte-identical to the page Astro rendered, minus the module script Astro injected into each one. A page that leaves before it paints has no use for the prefetch runtime.

The check ran over all 172,794 files: hash the built page, put the one script tag back, compare against a manifest taken from the build before this change. 172,794 checked, 0 missing, 0 different, 0 extra.

Then the page got smaller (second commit)

The page is ours now, so it carries only what earns its place — 446 bytes to 370:

<!doctype html><html lang="en"><meta charset="utf-8"><title>Redirecting</title><meta http-equiv="refresh" content="0;url=/id/ref/{uuid}/"><link rel="canonical" href="/id/ref/{uuid}/"><p>Redirecting to <a href="/id/ref/{uuid}/">the record</a>.
  • The refresh performs the redirect. The canonical link states the target for a client that ignores the refresh.
  • <title> and lang stay: WCAG 2.2 SC 2.4.2 and SC 3.1.1, which the site holds everywhere else.
  • <html>, <head> and <body> are implied and left out. The target is spelled twice, not three times — as the link text it cost 44 bytes a page and repeated the address bar.

On-disk size does not move, because 172,794 files each hold a 4 KB block either way. The Pages artifact does.

Also in this PR

  • iriToLocal moves to standard/iri.ts, beside the four IRI builders it inverts. The compiler cannot import src/lib/registry.ts, because that module imports the compiler. src/lib/registry.ts re-exports it, so no call site moves.
  • src/integrations/cite-dev.ts serves /cite/ in astro dev. It calls the same citeRedirectHtml the compiler writes with, so the dev server and the published site cannot disagree.
  • npm run check:links now runs npm run build instead of astro build, so dist/ still holds /cite/ when lychee reads it. The linkcheck CI job reads the sources, not dist/, so it needs no change.
  • npm run build:fast never runs the writer, so scripts/compile.test.ts covers the page body, its size, the :// alias filter, and the ADR-0003 noindex rule.

Verification

  • npm run verify — format, astro check, 184 tests, full build
  • npm run validate:data — 86,492 records valid, 15 frozen identifiers unchanged
  • all 172,794 pages are 370 bytes, the path set is unchanged, and a sample of 2,000 states one /id/ target in the refresh, the canonical link and the anchor
  • a browser follows the smaller page through to the record page
  • astro dev — bare alias, qualified alias, unknown alias that still 404s, malformed escape

Note for the reviewer

Scoping the Pagefind index was the other half of the original plan. This PR already took Pagefind from 22.5s to 8.4s, so the remaining prize is about 8s of a 72s build. Deferred to #155, which records the measurement and the catch.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Shr415rgnDYoMDzfnjv9c2

`/cite/` minted 172,794 pages, two thirds of the built site. Astro rendered
each one through the full route pipeline to produce four tags. Pagefind then
opened every one of them and found no `data-pagefind-body`.

The pages are output, not routes. `scripts/compile.ts` now writes them after
`astro build`, next to the `/dump/` resources it already writes. The redirect
stays a static file, so `curl` and crawlers keep working, and GitHub Pages
needs no redirect layer.

Measured on the full registry:

- `npm run build`: 1m46s to 1m17s
- `astro build`: 259,815 pages in 1m42s to 87,021 pages in 58s
- Pagefind: 22.5s to 8.4s
- `dist/` holds the same 346,596 files

Every one of the 172,794 pages is byte-identical to the rendered page, minus
the module script Astro injected into each one. A page that leaves before it
paints has no use for the prefetch runtime.

Also:

- `iriToLocal` moves to `standard/iri.ts`, beside the four IRI builders. The
  compiler cannot import `src/lib/registry.ts`, which imports the compiler.
- `src/integrations/cite-dev.ts` serves `/cite/` in `astro dev`, from the same
  `citeRedirectHtml` the compiler writes with.
- `npm run check:links` runs `npm run build`, so `dist/` still holds `/cite/`
  when lychee reads it.
- `npm run build:fast` never runs the writer, so `scripts/compile.test.ts`
  covers the page body, the alias filter, and the `noindex` rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Shr415rgnDYoMDzfnjv9c2

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new dev middleware can throw on malformed percent-encoded URLs (decodeURIComponent), which can crash astro dev instead of returning a normal 404.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Moves generation of /cite/ redirect pages out of Astro’s routing/render pipeline and into the post-build compiler step (scripts/compile.ts), significantly reducing the number of pages Astro renders and Pagefind scans while keeping /cite/ working as static HTML output. It also keeps astro dev behavior aligned with production output via a dev-only integration.

Changes:

  • Generate /cite/{alias}/index.html redirect pages during tsx scripts/compile.ts (after astro build) via writeCiteRedirects() and shared citeRedirectHtml().
  • Remove the Astro route for /cite/ and add an astro dev middleware integration to serve redirects during development.
  • Move iriToLocal() to standard/iri.ts and re-export it from src/lib/registry.ts to avoid import cycles.
File summaries
File Description
standard/iri.ts Adds iriToLocal() alongside the IRI builders so both compiler and runtime can use it without cycles.
src/pages/cite/[...alias].astro Removes the /cite/ route now that redirects are emitted as build output instead of rendered routes.
src/lib/registry.ts Re-exports iriToLocal() from standard/iri.ts to preserve call sites without duplicating logic.
src/integrations/cite-dev.ts Adds dev-server middleware to serve /cite/ redirects in astro dev using the same HTML generator as production.
scripts/compile.ts Adds citeRedirectHtml() and writeCiteRedirects() and runs the writer in the CLI entry path post-build.
scripts/compile.test.ts Adds unit tests covering the redirect HTML and redirect writer behavior (including noindex rules and :// filtering).
package.json Updates check:links to run npm run build so /cite/ output exists when lychee scans dist/.
astro.config.mjs Registers the new citeDev() integration to make /cite/ work during astro dev.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/integrations/cite-dev.ts Outdated
The page is ours now, not Astro's output, so it carries only what earns its
place. The refresh performs the redirect. The canonical link states the target
for a client that ignores the refresh. The title and the `lang` attribute hold
the WCAG 2.2 AA baseline (SC 2.4.2 and SC 3.1.1). The paragraph gives a person
a link to follow.

`<html>`, `<head>` and `<body>` are implied and left out. The target is spelled
twice, not three times: as the link text it cost 44 bytes a page and told the
reader what the address bar already says.

446 bytes to 370 bytes, so `dist/cite/` falls from 77.1 MB to 63.9 MB. On-disk
size does not move, because 172,794 files each hold a 4 KB block either way;
the Pages artifact does.

Also guards `decodeURIComponent` in the dev middleware, as the review asked. A
malformed escape now takes the same path as an unknown alias. `astro dev`
answers 500 to a malformed escape on every route, `/` and `/reg/` included, so
this keeps `/cite/` consistent with the rest of the site rather than throwing
from our own code.

Verified: a browser follows the smaller page to the record, all 172,794 pages
are 370 bytes and state one `/id/` target in all three places, and the path set
is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Shr415rgnDYoMDzfnjv9c2
@maehr
maehr merged commit 8c47ad4 into textrefs:staging Sep 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants