Skip to content
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f42b58c
docs: add a BetterMan verification skill and feature map
cursoragent Oct 12, 2026
57a6a90
fix: make the verification skill launch and drive a live app
cursoragent Oct 12, 2026
ea6d9ff
Merge branch 'main' into cursor/verify-betterman-skill-701a
amanthanvi Oct 15, 2026
4f90782
fix: harden the verification skill against false ready and leftover s…
cursoragent Oct 15, 2026
9ffef36
fix: forward Playwright filters from verify:drive
cursoragent Oct 15, 2026
2bfdba4
fix: keep verification seed on a local anonymous Convex deployment
cursoragent Oct 15, 2026
66077da
fix: keep Playwright specs off the ESM verify runtime
cursoragent Oct 15, 2026
d8091a8
fix: retry the distro select until the URL updates
cursoragent Oct 15, 2026
2477ebb
fix: require a positive info pageCount before verify is ready
cursoragent Oct 15, 2026
9a6d3f1
fix: match verify reuse to the recorded listener
cursoragent Oct 15, 2026
06b3dc4
fix: keep verify evidence, state, and seed on one checkout
cursoragent Oct 15, 2026
3bce7a4
fix: give the manual smoke workflow a valid verify id
cursoragent Oct 15, 2026
d8934dd
fix: refuse to seed a Next server this checkout does not own
cursoragent Oct 15, 2026
18221d0
fix: snapshot generated files only after verify owns Next
cursoragent Oct 15, 2026
0a25625
fix: keep verify state files private and type the path helper
cursoragent Oct 15, 2026
9b5776c
fix: match verify repo paths on a separator boundary
cursoragent Oct 15, 2026
1f6c0f9
fix: keep convex env set off the parent dataset stage
cursoragent Oct 15, 2026
b4f8cdc
fix: match verify listeners by repo path prefix
cursoragent Oct 15, 2026
176da33
fix: keep the skill typecheck project off Node's spec loader
cursoragent Oct 15, 2026
548e2ff
fix: treat ipv6 loopback as a verify listener target
cursoragent Oct 15, 2026
347bf14
fix: refuse an ipv6 loopback verify launch
cursoragent Oct 15, 2026
f72bdb8
fix: keep a live session snapshot on a refused launch
cursoragent Oct 15, 2026
fc022b0
fix: drop a stale verify snapshot when Next is not running
cursoragent Oct 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 204 additions & 0 deletions .agents/skills/verify-betterman/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
---
name: verify-betterman
description: "Drive the BetterMan Next.js UI on a local dev server with Playwright. Use to launch the app, run the read-only doctor check, exercise search and man pages like a user, and capture screenshots and traces."
disable-model-invocation: true
---

# Verify BetterMan

BetterMan is a Next.js web app. Readers search Unix man pages, open a page, and use the palette, find bar, bookmarks, and reading preferences. There is no account system. Personal state lives in `localStorage` and cookies. Page data is read from a local Convex deployment.

The feature map is [`references/features/README.md`](references/features/README.md). Open the one file that matches the change. Walk the README top to bottom for a broad sweep, and finish with `journeys.md`.

A maintain pass (`maintain-verification-skill`) edits only this directory: `SKILL.md`, `references/features/`, `scripts/`, and `playwright.config.mjs`. Product code stays untouched. Outcomes are `clean`, `changed` (one PR), or `blocked`.

## Isolate

This skill owns one app on `http://127.0.0.1:3000`. Playwright gives each test a fresh browser context, so bookmarks, theme, and recent history stay inside that context. The Convex dataset is shared by every context on this machine.

When doctor already reports `READY` and the process on port 3000 matches the pid and start time recorded for this checkout, launch reuses that server. Cleanup stops only the tmux sessions recorded in this checkout's `run.json`. A server this skill did not start is left running.

State and session names are per checkout. The id is `VERIFY_ID` when that is set, otherwise the first 8 hex characters of the sha256 of the repo root. The state directory is `/tmp/betterman-verify-<id>/` and the tmux sessions are `bmv-<id>-convex` and `bmv-<id>-next`. Two checkouts do not share them.

Launch refuses a base URL other than port 3000 on loopback. It also refuses to start a second Next server when port 3000 is already taken by someone else. Seed runs only when `/api/v1/info` is a 404 or `pageCount` is 0, and only when `.env.local` points Convex at loopback.

## Launch

From the repo root, with Node 26 on `PATH`:

```bash
export PATH="${NVM_DIR:-$HOME/.nvm}/versions/node/v26.5.1/bin:$HOME/.local/bin:$PATH"
pnpm install --frozen-lockfile
bash scripts/install-playwright-ci.sh
pnpm verify:launch
```

`pnpm verify:launch` is `node .agents/skills/verify-betterman/scripts/launch.mjs`. It does the following:

1. If doctor is `READY`, print `REUSED` and exit 0. `READY` includes an IPv4 listen socket and a hydration probe. An IPv6-only server is not reused.
2. If port 3000 is open, doctor reports `empty-dataset`, and Convex is already listening on the loopback URL in `.env.local`, seed that dataset and check doctor again. On `READY`, print `SEEDED` and leave the existing Next process running. If it is still not `READY`, exit 2 without restarting it.
3. If port 3000 is open for any other reason and this skill did not start it, exit 2 and leave that process alone.
4. Run `pnpm install --frozen-lockfile` when `node_modules` is missing.
5. Run `CONVEX_AGENT_MODE=anonymous pnpm convex:check` when `.env.local` is missing.
6. Install Playwright Chromium (`pnpm -C nextjs exec playwright install chromium`). Set `VERIFY_SKIP_BROWSER_INSTALL=1` to skip. If Chromium's system libraries are missing, the command above `bash scripts/install-playwright-ci.sh` installs them.
7. Start tmux session `bmv-<id>-convex` with `npx convex dev --tail-logs disable` when ports from `.env.local` (`VITE_CONVEX_URL`, usually 3210, and `VITE_CONVEX_SITE_URL`, usually 3211) are closed. The session name is written to `run.json` immediately, before the ready wait. The shell unsets `CONVEX_DEPLOY_KEY`, `CONVEX_DEPLOYMENT`, `CONVEX_URL`, every `CONVEX_DEPLOY*` variable, and the other remote Convex variables, then exports only `CONVEX_AGENT_MODE=anonymous`, an `anonymous:` deployment name when `.env.local` has one, and the loopback URLs from that file. Ready means the log line `Convex functions ready`.
8. Start tmux session `bmv-<id>-next` with `pnpm -C nextjs dev --hostname 127.0.0.1 --port 3000`. That session is recorded in `run.json` as soon as it starts. After `.env.local` is sourced, the shell unsets inherited Convex variables again and exports only the anonymous deployment name and loopback URLs. `--hostname 127.0.0.1` is required. A `next dev` bound only to IPv6 `::` can still answer `http://127.0.0.1:3000` for documents, while Playwright's HMR websocket fails with `ERR_INVALID_HTTP_RESPONSE` and clicks never reach React. Doctor treats that listener as `NOT_READY` `ipv6-only`. It also opens Chromium, waits until `Cycle theme` has React props, and clicks it. `READY` requires `localStorage['bm-theme']` to change. Launch will not reuse an IPv6-only server. Stop that process yourself, then run launch again.
9. If doctor reports `empty-dataset` for a Next process this skill started, seed only when `.env.local` names an `anonymous:` deployment or has no deployment name, both Convex URLs are loopback, and no deploy key is configured. The seed process inherits a scrubbed environment. It receives the anonymous deployment name and loopback URLs, and it does not receive `CONVEX_DEPLOY_KEY` or a cloud `CONVEX_URL`. Launch then stops Next, waits until `127.0.0.1:3000` is closed, starts Next again, waits until that address is bound by a process whose cwd or command line is this repo, records that pid and start time, and waits until doctor reports hydration. A fixed sleep is not the ready check. Reuse also requires the live listener pid and start time to match `run.json`.

The seeded fixture publishes `tar(1)`, `gzip(1)`, alias `gunzip(1)`, `printf(1)`, `printf(3)`, `openssl(1ssl)`, `bash(1)`, and an EVP page for Debian, Ubuntu, and Fedora. Ubuntu `tar(1)` contains the sentence `Ubuntu variant: this page is intentionally different for E2E testing.` The license package list is empty.

Logs and state:

- `/tmp/betterman-verify-<id>/` is mode `0700` and must be owned by the current user. Launch refuses a pre-created directory owned by someone else. Set `VERIFY_ID` to choose the id, or `VERIFY_STATE_DIR` to choose the directory.
- `run.json` lists sessions this launch started, including a Convex session that failed before Next opened port 3000, plus the Next listener pid and start time
- `bmv-<id>-convex.log` and `bmv-<id>-next.log`
- `evidence/` is created at launch and kept by cleanup. Override the directory with `VERIFY_EVIDENCE_DIR`. Playwright specs read that variable. They do not import the ESM runtime, because the test runner loads them as CommonJS.

`pnpm next:dev` (`convex dev --start "pnpm -C nextjs dev"`) is the README command. The helper splits Convex and Next so the dataset can be seeded before Next caches an empty render. Use the helper.

## Doctor

Run this first whenever a drive looks wrong. It sends GET requests, checks that port 3000 is listening on IPv4, and opens a throwaway Chromium window to confirm the page hydrated. It does not seed data or start servers.

```bash
pnpm verify:doctor
```

`READY` (exit 0) means all of the following:

- `GET /` is 200 and the HTML contains `Search man pages`
- `GET /api/v1/search?q=tar&limit=5` is 200 and includes `tar` in section `1`
- `/proc/net/tcp` shows an IPv4 listen socket for port 3000, and a TCP connect to `127.0.0.1` succeeds. IPv6 `::` alone is `ipv6-only`
- Chromium sees React props on `Cycle theme`, and clicking it writes `bm-theme`

`GET /api/v1/info` is printed every time. A cached `pageCount` of 0 while search already returns `tar(1)` is a stale Next.js cache. Doctor can still report `READY` and prints `info cache is empty while search returns tar(1)`.

Anything else prints `NOT_READY` and exits 2. Codes:

| Code | Meaning |
| --- | --- |
| `unreachable` | Nothing answered on the base URL |
| `home-unhealthy` | The home page is up without the search box, or the status is not 200, and this is not an empty dataset |
| `empty-dataset` | Search has no `tar(1)` and info is 404 or `pageCount` is 0, or the home error says the dataset release is missing. Launch may seed. It does not restart a Next process it did not start |
| `info-unreadable` | Home rendered, but info did not return the expected JSON and search did not prove `tar(1)` |
| `search-missing-tar` | Info reports pages, and `tar(1)` is absent. Launch does not overwrite that dataset |
| `ipv6-only` | Port 3000 is listening on IPv6 `::` and not on IPv4. Launch does not reuse it |
| `ipv4-unreachable` | Documents answered, but TCP to `127.0.0.1` on that port failed |
| `not-hydrated` | The search API is fine, but Cycle theme did not write `bm-theme` or the page threw |
| `hydration-unproven` | Playwright Chromium is missing, so hydration could not be checked |

```bash
node .agents/skills/verify-betterman/scripts/doctor.mjs --help
node .agents/skills/verify-betterman/scripts/doctor.mjs --json
node .agents/skills/verify-betterman/scripts/doctor.mjs --base-url http://127.0.0.1:3000
```

Doctor prints who is listening on port 3000. It uses `ss -ltnp` when `ss` exists, and otherwise reads `/proc/net/tcp`. A screenshot taken while doctor is `NOT_READY` is not evidence.

## Drive

Prefer the Playwright runner in this skill. It uses the same accessible names and `data-bm-*` hooks as `nextjs/e2e/` and `DESIGN.md`.

Fast path (home, search `tar`, open `tar(1)`):

```bash
pnpm verify:smoke
```

Feature sweep (every file under `references/features/` except the production-only service worker registration):

```bash
pnpm verify:drive
```

Both commands run doctor first. If doctor exits 2, the browser does not open.

One feature, after doctor is `READY`:

```bash
pnpm verify:drive -- -g "find in page"
```

Run that from the repo root. The wrapper passes an absolute `--config` and sets `NODE_PATH` to `nextjs/node_modules`. A `pageerror` event fails the test.

Stable handles:

- Roles and names from `DESIGN.md`: banner `Site header`, contentinfo `Site footer`, searchbox `Search man pages`, combobox `Command palette input`, textbox `Find in page`, button `Cycle theme`, `Search`, `Find in page`, `Reading preferences`, `Open contents`, `Load more results`, link names like `tar(1)`.
- Hooks: `[data-bm-app-header]`, `[data-bm-app-footer]`, `[data-bm-sidebar]`, `[data-bm-findbar]`, `[data-bm-home-search]`, `[data-bm-page-search]`, `mark[data-bm-find]`, `mark[data-bm-opt]`, `body[data-bm-font-size]`.
- Single-key shortcuts (`?`, `/`, `d`, `b`, `t`, `h`, `p`, `m`) are ignored while focus is in a text field or `<select>`. Blur the active element, or click the page, before sending them. Wait until `Cycle theme` is visible so the shell listener is attached.
- Palette modes: plain text searches, `>` actions, `#` headings, `@ubuntu tar` switches distro inside the query. `Escape` closes the palette and the find bar.
- The header control named `Search` opens the palette. The home search box submits to `/search`.

The full e2e contract remains `pnpm next:e2e`. Use it when the change touches a behavior the sweep does not list. `verify:smoke` is the short loop.

## Evidence

Proof artifacts go to `/tmp/betterman-verify-<id>/evidence/` (override with `VERIFY_EVIDENCE_DIR`). Cleanup never deletes that directory. Launch prints the path.

- `*.png` screenshots taken by the specs at named paths (`smoke-home.png`, `smoke-search.png`, `smoke-tar.png`, and the drive shots such as `man-tar.png`)
- `test-results/` Playwright traces (`trace.zip` per test, trace is `on`)
- `playwright-report/` HTML report

A proof exercises the user path: type in the search box, click a result, press the shortcut, change the select. Reading `/api/v1/info` is the doctor check, not the proof that search works.

Capture the action and the end state. Examples: the search box filled, then the `tar(1)` heading; the bookmark button pressed, then the Bookmarks region on `/#bookmarks` after a reload or navigation; `localStorage['bm-theme']` after cycling theme and again after reload.

Side effects worth reading:

- URL, heading, and `aria-current`
- `localStorage` keys `bm-theme`, `bm-bookmarks`, `bm-recent`
- `document.documentElement.dataset.theme`
- `body` dataset `bmFontSize` (`data-bm-font-size`)
- clipboard text after Copy code block or Copy link
- `GET /api/v1/search?q=` with no query returns 400, while the empty search page shows try links

The E2E seed is the local dataset behind the same Convex reads production uses. Do not stub those reads. `pnpm next:dev` does not register `/sw.js` (`NODE_ENV` is development). Service-worker caching is `INCONCLUSIVE` on the dev server. Observe it with `pnpm -C nextjs build && pnpm -C nextjs start` only when that production path is what you are proving. The offline banner still updates from the browser `offline` event in dev.

## Cleanup

```bash
pnpm verify:cleanup
```

That runs `node .agents/skills/verify-betterman/scripts/cleanup.mjs`. It `tmux kill-session`s only `bmv-<id>-convex` and `bmv-<id>-next` when `run.json` lists them, including a session recorded before its ready check failed. It does not kill processes by name. It waits until the recorded Next pid is gone, then restores generated files. It leaves the evidence directory in place and prints that path. After every failed launch or drive, run cleanup before starting again so a half-started session does not keep the ports.

`next dev` rewrites `nextjs/next-env.d.ts` and creates `nextjs/AGENTS.md` and `nextjs/CLAUDE.md`. The generated `AGENTS.md` says to commit that file. Do not. Launch snapshots those paths, and cleanup restores them after the sessions stop. A verification run should leave `git status` clean.

Confirm the evidence directory still exists after cleanup before citing it.

## Helpers

| Command | Script |
| --- | --- |
| `pnpm verify:launch` | `scripts/launch.mjs` |
| `pnpm verify:doctor` | `scripts/doctor.mjs` |
| `pnpm verify:smoke` | `scripts/smoke.mjs` (doctor, then `smoke.spec.ts`) |
| `pnpm verify:drive` | `scripts/smoke.mjs --drive` (doctor, then `drive.spec.ts`) |
| `pnpm verify:cleanup` | `scripts/cleanup.mjs` |

```bash
node .agents/skills/verify-betterman/scripts/launch.mjs --help
node .agents/skills/verify-betterman/scripts/doctor.mjs --help
node .agents/skills/verify-betterman/scripts/smoke.mjs --help
node .agents/skills/verify-betterman/scripts/cleanup.mjs --help
```

`scripts/smoke.mjs --drive` is the long sweep. `scripts/evidence.ts` writes screenshots into `VERIFY_EVIDENCE_DIR`.

## How to report

Put a Proof section on the pull request:

```markdown
### Proof
- Commands: the exact commands, including `pnpm verify:doctor` and the drive you ran
- Evidence: screenshot and trace paths that still exist after cleanup
- Results: pass or fail per command, plus the observable end state (URL, heading, storage, attribute)
- INCONCLUSIVE: any step that could not run, and the concrete reason (port busy, dataset refused, browser missing, dev server has no service worker)
```

A missing check is `INCONCLUSIVE`. Do not describe it as passed.

## Feature map

[`references/features/README.md`](references/features/README.md) is the sweep order. Each feature file has the same four headings: Sub-features, How to get to it (user POV), Driving it with Playwright, Gotchas.
33 changes: 33 additions & 0 deletions .agents/skills/verify-betterman/playwright.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import { createRequire } from 'node:module'
import path from 'node:path'
import { fileURLToPath } from 'node:url'

import { evidenceDir } from './scripts/runtime.mjs'

const here = path.dirname(fileURLToPath(import.meta.url))
const require = createRequire(path.join(here, '../../../nextjs/package.json'))
const { defineConfig, devices } = require('@playwright/test')

export default defineConfig({
testDir: './scripts',
testMatch: '*.spec.ts',
timeout: 60_000,
expect: { timeout: 15_000 },
retries: 0,
fullyParallel: false,
workers: 1,
reporter: [['list'], ['html', { open: 'never', outputFolder: path.join(evidenceDir, 'playwright-report') }]],
outputDir: path.join(evidenceDir, 'test-results'),
use: {
baseURL: process.env.E2E_BASE_URL ?? 'http://127.0.0.1:3000',
trace: 'on',
screenshot: 'only-on-failure',
video: 'off',
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
],
})
68 changes: 68 additions & 0 deletions .agents/skills/verify-betterman/references/features/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# BetterMan feature map

Behavior-level inventory of the BetterMan web app. Agents use it to decide what to drive. The order below is the regression sweep.

## Baseline preconditions

- Doctor reports `READY` on `http://127.0.0.1:3000` before the first drive, and again after a failed drive.
- Local dataset is the E2E seed unless doctor found an existing `tar(1)`. Seeded pages: `tar(1)`, `gzip(1)`, `gunzip(1)` alias, `printf(1)`, `printf(3)`, `openssl(1ssl)`, `bash(1)`. Distros with data: Debian (default), Ubuntu, Fedora.
- The seed's license package list is empty.
- `pnpm next:dev` does not register the service worker. Say `INCONCLUSIVE` for offline cache hits on that server.
- Fresh Playwright context. Bookmarks, theme, and recent history start empty in that context.
- Prefer roles, accessible names, and `data-bm-*` hooks. Wait until the heading, mark, URL, or attribute you care about is visible.
- Blur text fields and `<select>` before single-key shortcuts. Wait until the `Cycle theme` button is visible first.
- Desktop viewport unless the file says 390x844.

## Proof and skip reporting

Drive the user path listed in the feature file: the success path, and the empty, error, and hostile cases the change can affect. Record the action and the end state. When a path cannot be reached, name it and the prerequisite, and cover the closest path that still runs.

## Full sweep

Walk this list top to bottom, then `journeys.md`.

## Chrome

- [site-chrome](site-chrome.md): Header, footer, skip link, wordmark, offline banner.

## Find a page

- [home](home.md): Home search, browse sections, empty recent and bookmarks.
- [search](search.md): Search results, try links, typos, section filter, load more.
- [section-browse](section-browse.md): Section index, letter jump, in-section search, pagination.
- [command-palette](command-palette.md): Palette search, `>` actions, `#` headings, `@distro`.
- [keyboard-shortcuts](keyboard-shortcuts.md): `?` dialog and the global chords.

## Read a page

- [man-page](man-page.md): Page body, options table, see also, related, alias, ambiguous name, copy.
- [find-in-page](find-in-page.md): Find bar, match count, next and previous.
- [table-of-contents](table-of-contents.md): Desktop sidebar and mobile contents drawer.
- [reading-preferences](reading-preferences.md): Font, measure, line height, code theme, default distro.
- [distro-variants](distro-variants.md): Per-page distro select and `?distro=`.

## Local reader state

- [theme](theme.md): System, light, and dark cycle.
- [bookmarks](bookmarks.md): Bookmark toggle and the home list.
- [recent](recent.md): Recent pages and `/history`.

## Other pages

- [licenses](licenses.md): Dataset attribution and package filter.
- [not-found-and-errors](not-found-and-errors.md): Unknown routes, missing names, render error.
- [offline-and-install](offline-and-install.md): Offline banner, web manifest, service worker.
- [discovery-and-share](discovery-and-share.md): robots, sitemaps, JSON-LD, share metadata.

## Journeys

- [journeys](journeys.md): Search, open, bookmark, and come back.

## Entry contract

Every feature file uses the same four H2s:

1. `Sub-features`
2. `How to get to it (user POV)`
3. `Driving it with Playwright`
4. `Gotchas`
Loading
Loading