Skip to content

Commit e495535

Browse files
stewartjarodclaude
stewartjarod
and
claude
committed
ci(e2e): run the dashboard e2e suite on pull requests and main
The suite only ran on a laptop, so a broken signup or onboarding could merge unnoticed. CI starts next dev on the runner with TEST_DATABASE_URL and replays the agent steps recorded in the committed .e2e/cache, so a run costs one model judgment instead of every step. Recordings store the start path, and the workspace slug changed every run, so nothing replayed outside the machine that recorded it. Agent steps now take the slug as unique(), which the runner swaps in at replay. A cold next dev also outlasted the 5s assertion default on the first visit to each route. The workflow keeps its own concurrency group: sharing test.yml's would let a queued e2e run cancel a pending Test run, which gates deploys. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BLzw17iv19ctMndm8WmVGz
1 parent 6ee4314 commit e495535

11 files changed

Lines changed: 512 additions & 20 deletions

‎.github/workflows/e2e.yml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
name: E2E
2+
3+
on:
4+
push:
5+
branches: [main]
6+
paths-ignore:
7+
- "apps/website/**"
8+
- "docs/**"
9+
- "plans/**"
10+
- "**/*.md"
11+
pull_request:
12+
branches: [main]
13+
paths-ignore:
14+
- "apps/website/**"
15+
- "docs/**"
16+
- "plans/**"
17+
- "**/*.md"
18+
workflow_dispatch:
19+
20+
permissions:
21+
contents: read
22+
23+
# Not test.yml's test-db-<ref> group: GitHub keeps one pending run per group,
24+
# so a queued e2e run would cancel a pending Test run, and deploy-api.yml
25+
# deploys only when Test completes. e2e only inserts uniquely named rows
26+
# (a fresh owner and org per run), so it can't delete a fixture test-web reads.
27+
concurrency:
28+
group: e2e-${{ github.ref }}
29+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
30+
31+
jobs:
32+
e2e:
33+
# Forks get no secrets; the setup test needs the database and the model key.
34+
if: github.repository == 'wraps-team/wraps'
35+
runs-on: ubuntu-latest
36+
timeout-minutes: 30
37+
38+
steps:
39+
- name: Checkout code
40+
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v4
41+
42+
- name: Setup pnpm
43+
uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
44+
45+
- name: Setup Node.js 24.x
46+
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
47+
with:
48+
node-version: 24.x
49+
cache: 'pnpm'
50+
51+
- name: Install dependencies
52+
run: pnpm install --frozen-lockfile
53+
54+
- name: Build web workspace dependencies
55+
run: pnpm --filter "@wraps/web^..." build
56+
57+
- name: Install Chromium
58+
run: pnpm --filter @wraps/e2e exec e2e-web install chromium --with-deps
59+
60+
# Replays the committed recordings in apps/e2e/.e2e/cache (read-only in
61+
# CI). A step whose recording no longer matches the UI falls back to the
62+
# live agent, which is what the model key is for.
63+
- name: Run e2e
64+
run: pnpm test:e2e --reporter list,junit
65+
env:
66+
DATABASE_URL: ${{ secrets.TEST_DATABASE_URL }}
67+
BETTER_AUTH_SECRET: test-secret-key-for-testing-only
68+
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
69+
70+
# Not .e2e/logs: the app log is written unredacted, and artifacts on a
71+
# public repo are downloadable by anyone signed in.
72+
- name: Upload report
73+
if: ${{ !cancelled() }}
74+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
75+
with:
76+
name: e2e-report
77+
path: |
78+
apps/e2e/.e2e/report.json
79+
apps/e2e/.e2e/junit.xml
80+
apps/e2e/.e2e/results
81+
if-no-files-found: warn
82+
retention-days: 7

‎apps/e2e/.e2e/cache/27fe7ce721153a7c918f7db69bc71b370740ea5d15a53cf2b30d3f7ce888cac4.json‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
{
2+
"schemaVersion": "trace-1",
3+
"createdAt": "2026-10-26T22:23:49.470Z",
4+
"payload": {
5+
"actions": [
6+
{
7+
"name": "tap",
8+
"summary": "tap button \"Add contact\"",
9+
"target": {
10+
"role": "button",
11+
"name": "Add contact"
12+
}
13+
},
14+
{
15+
"name": "type",
16+
"summary": "type \"ada@e2e.wraps.test\" into textbox \"Email address\"",
17+
"target": {
18+
"role": "textbox",
19+
"name": "Email address",
20+
"placeholder": "contact@example.com"
21+
},
22+
"value": "ada@e2e.wraps.test"
23+
},
24+
{
25+
"name": "type",
26+
"summary": "type \"Ada\" into textbox \"First name\"",
27+
"target": {
28+
"role": "textbox",
29+
"name": "First name",
30+
"placeholder": "John"
31+
},
32+
"value": "Ada"
33+
},
34+
{
35+
"name": "tap",
36+
"summary": "tap button \"Add Contact\"",
37+
"target": {
38+
"role": "button",
39+
"name": "Add Contact"
40+
}
41+
}
42+
],
43+
"executor": {
44+
"name": "e2e-default-agent",
45+
"version": "2"
46+
},
47+
"recordedFor": {
48+
"testId": "tests/contacts.e2e.ts::the%20owner%20adds%20a%20contact%20by%20hand",
49+
"targetId": "chromium",
50+
"instructionDigest": "d3c053047e2ff27bc94b503e153507dc99fa0f21807a3dc6a70f39e6f259d3e6",
51+
"paramsDigest": "5679c80a86c6abe433c50bd869592ed746923f8d80af15173107576b023b6c4f",
52+
"callIndex": 0,
53+
"agent": "default"
54+
},
55+
"recordedBy": {
56+
"provider": "gateway",
57+
"model": "anthropic/claude-sonnet-5.5"
58+
},
59+
"summary": "In e2e-mvptoww7 contacts, added a contact with email ada@e2e.wraps.test and first name Ada. A \"Contact created\" toast appeared. The contacts list now shows ada@e2e.wraps.test as Active, \"Showing 1-1 of 1 contacts\".",
60+
"startPath": "/{{param:/workspace}}/contacts",
61+
"endPath": "/{{param:/workspace}}/contacts",
62+
"endAnchors": [
63+
{
64+
"role": "heading",
65+
"name": "Contact Growth"
66+
},
67+
{
68+
"text": "New contacts added over time"
69+
},
70+
{
71+
"role": "button",
72+
"name": "Refresh contact growth"
73+
},
74+
{
75+
"text": "+1"
76+
},
77+
{
78+
"text": "+100%"
79+
},
80+
{
81+
"role": "button",
82+
"name": "What these figures cover"
83+
},
84+
{
85+
"text": "Whole organization, not the filtered list below"
86+
},
87+
{
88+
"text": "Engagement"
89+
}
90+
],
91+
"goneAnchors": [
92+
{
93+
"role": "heading",
94+
"name": "No contacts yet"
95+
},
96+
{
97+
"text": "Add contacts manually, import a CSV, or use the Platform SDK to manage contacts programmatically."
98+
},
99+
{
100+
"text": "create-contact.ts"
101+
},
102+
{
103+
"text": "import { createClient } from '@wraps.dev/platform'; const client = createClient({ organizationId: 'org_...', apiKey: process.env.WRAPS_API_KEY, }); const { data } = await client.POST('/v1/contacts/', { body: { email: 'user@example.com', firstName: 'Jane', lastName: 'Doe', }, });"
104+
},
105+
{
106+
"role": "link",
107+
"name": "Read the docs"
108+
},
109+
{
110+
"role": "button",
111+
"name": "Import CSV"
112+
},
113+
{
114+
"role": "button",
115+
"name": "Add contact"
116+
}
117+
],
118+
"endWaitMs": 14472,
119+
"keyedBy": {
120+
"cacheSchema": "trace-1",
121+
"policyVersion": "conservative/6",
122+
"project": "6270d378e32b1181044da35433df046bd2ea7bbc8b700e3a614fe441981c5456",
123+
"platform": "web",
124+
"engineName": "web",
125+
"engineSpiVersion": 1,
126+
"appIdentity": "f9f49ed79c2662c7f6e0522da33af26cd0588919fce8b674530ba7c83e5d217c",
127+
"agentContextDigest": "d436824420520e934fb52613db21890f6152750d6c044528fbf9cb9ed5b0ab6d"
128+
}
129+
}
130+
}

‎apps/e2e/.e2e/cache/97c4e238d5d6e41ef99d0d7f2a48f73a5b0a76573c6e7efdc0caf6df11e42863.json‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
{
2+
"schemaVersion": "trace-1",
3+
"createdAt": "2026-10-26T22:23:38.093Z",
4+
"payload": {
5+
"actions": [
6+
{
7+
"name": "tap",
8+
"summary": "tap button \"Create API Key\" (1 of 2)",
9+
"target": {
10+
"role": "button",
11+
"name": "Create API Key",
12+
"position": {
13+
"index": 0,
14+
"of": 2
15+
}
16+
}
17+
},
18+
{
19+
"name": "type",
20+
"summary": "type \"CI deploys\" into textbox \"Name\" in \"Create API Key\"",
21+
"target": {
22+
"role": "textbox",
23+
"name": "Name",
24+
"placeholder": "Production API Key",
25+
"within": "Create API Key"
26+
},
27+
"value": "CI deploys"
28+
},
29+
{
30+
"name": "tap",
31+
"summary": "tap button \"Create API Key\"",
32+
"target": {
33+
"role": "button",
34+
"name": "Create API Key"
35+
}
36+
}
37+
],
38+
"executor": {
39+
"name": "e2e-default-agent",
40+
"version": "2"
41+
},
42+
"recordedFor": {
43+
"testId": "tests/api-keys.e2e.ts::the%20owner%20creates%20an%20API%20key",
44+
"targetId": "chromium",
45+
"instructionDigest": "9327f197ffea6ccc67a6e73c2df3106fa154431622ca38770e0f14f444f6f3cf",
46+
"paramsDigest": "cd767e925690ff4596425f8639c9b2c6994503d8e49e70972502a0c2ec2bd1a7",
47+
"callIndex": 0,
48+
"agent": "default"
49+
},
50+
"recordedBy": {
51+
"provider": "gateway",
52+
"model": "anthropic/claude-sonnet-5.5"
53+
},
54+
"summary": "Created API key \"CI deploys\" in e2e-mvptoww7; \"API key created\" toast and \"API Key Created\" dialog (showing the key once) appeared.",
55+
"startPath": "/{{param:/workspace}}/settings/api-keys",
56+
"endPath": "/{{param:/workspace}}/settings/api-keys",
57+
"endAnchors": [
58+
{
59+
"role": "alert",
60+
"name": "Important This is the only time you'll see this API key. Make sure to copy it and store it securely."
61+
},
62+
{
63+
"text": "API key created"
64+
},
65+
{
66+
"role": "heading",
67+
"name": "API Key Created"
68+
},
69+
{
70+
"text": "Copy your API key now. You won't be able to see it again!"
71+
},
72+
{
73+
"text": "Important"
74+
},
75+
{
76+
"text": "This is the only time you'll see this API key. Make sure to copy it and store it securely."
77+
},
78+
{
79+
"text": "Your API Key"
80+
},
81+
{
82+
"role": "button",
83+
"name": "Toggle visibility"
84+
}
85+
],
86+
"goneAnchors": [
87+
{
88+
"role": "status",
89+
"name": "Connect your AWS account to start sending emails. Get Started Dismiss banner"
90+
},
91+
{
92+
"role": "image",
93+
"name": "Wraps Logo"
94+
},
95+
{
96+
"text": "EW"
97+
},
98+
{
99+
"text": "E2E Workspace"
100+
},
101+
{
102+
"text": "owner"
103+
},
104+
{
105+
"text": "Overview"
106+
},
107+
{
108+
"text": "Audience"
109+
},
110+
{
111+
"text": "Email"
112+
}
113+
],
114+
"endWaitMs": 12560,
115+
"keyedBy": {
116+
"cacheSchema": "trace-1",
117+
"policyVersion": "conservative/6",
118+
"project": "6270d378e32b1181044da35433df046bd2ea7bbc8b700e3a614fe441981c5456",
119+
"platform": "web",
120+
"engineName": "web",
121+
"engineSpiVersion": 1,
122+
"appIdentity": "f9f49ed79c2662c7f6e0522da33af26cd0588919fce8b674530ba7c83e5d217c",
123+
"agentContextDigest": "d436824420520e934fb52613db21890f6152750d6c044528fbf9cb9ed5b0ab6d"
124+
}
125+
}
126+
}

0 commit comments

Comments
 (0)