Repository navigation
ci(e2e): run the dashboard e2e suite on pull requests and main - #206
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The suite only ran on a laptop, so a broken signup or onboarding could merge unnoticed. CI starts next dev on the runner with TEST_DATABASE_URL and replays the agent steps recorded in the committed .e2e/cache, so a run costs one model judgment instead of every step. Recordings store the start path, and the workspace slug changed every run, so nothing replayed outside the machine that recorded it. Agent steps now take the slug as unique(), which the runner swaps in at replay. A cold next dev also outlasted the 5s assertion default on the first visit to each route. The workflow keeps its own concurrency group: sharing test.yml's would let a queued e2e run cancel a pending Test run, which gates deploys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BLzw17iv19ctMndm8WmVGz
stewartjarod
force-pushed
the
ci/e2e-workflow
branch
from
October 28, 2026 16:46
8a3867b to
e495535
Compare
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #205. Adds
.github/workflows/e2e.yml. It runs the suite on PRs into main and on pushes to main, skipping website, docs, plans, and Markdown-only changes.A run replays the agent steps recorded in
apps/e2e/.e2e/cache, which is now committed. In a fresh worktree withCI=1and no.env.local, all 29 passed in about 110 seconds. Cost was $0.003, one model call for the singleagent.assert. Unreplayed, the same run cost $0.11.Getting replays to work outside my machine took two fixes:
unique(slug), which the runner swaps in at replay.next devcompile outlasted the 5s assertion default on the first visit to each route. It's 30s now.The job uses
TEST_DATABASE_URL, the Neon branchtest-webuses. e2e only inserts uniquely named rows, so it can't delete a fixturetest-webreads. It keeps its own concurrency group. Sharingtest.yml's would let a queued e2e run cancel a pending Test run, and Test gates the deploy.AI_GATEWAY_API_KEYis set as a repo secret. Uploaded artifacts are the report, JUnit, and traces. The app log stays out because e2e writes it unredacted.If a UI change breaks a recording, CI still passes on the live agent but pays for it every run. The fix is
pnpm test:e2elocally, then commit the changed cache files.docs/agents/ci-and-deploy.mdhas the note.This PR targets
test/e2e-dashboard, so the workflow won't run here. Once #205 merges and this retargets to main, push once or close and reopen to get the first run.🤖 Generated with Claude Code
https://claude.ai/code/session_01BLzw17iv19ctMndm8WmVGz