Repository navigation
347 lines (334 loc) · 19.5 KB
/
Copy pathintegration-drizzle.yml
File metadata and controls
347 lines (334 loc) · 19.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
name: Integration — Drizzle (EQL v3)
# Real ZeroKMS ciphertext against a real Postgres, on BOTH database variants.
#
# The Drizzle adapter talks straight to the database, so it does not need
# PostgREST — but it does need to work on managed Postgres, where the `postgres`
# role is not a superuser, the EQL install takes its self-skipping path, and the
# ORE domains cannot hold data. The Supabase compose file brings up PostgREST
# too; the Drizzle suites simply ignore it.
#
# Separate from `tests.yml` on purpose: these suites need CipherStash credentials
# and a database, and they THROW rather than skip when unconfigured. Keeping them
# out of the unit job is what lets `pnpm test` stay runnable with neither.
on:
push:
branches: [main]
paths:
- 'languages/typescript/packages/stack/src/eql/v3/**'
- 'languages/typescript/packages/stack-drizzle/**'
# Source layers the adapter's encoding/round-trip rests on: a break here
# (not just under src/eql/v3) can produce wrong rows, so trigger the live
# suite that would catch it.
- 'languages/typescript/packages/stack/src/encryption/**'
- 'languages/typescript/packages/stack/src/schema/**'
- 'languages/typescript/packages/stack/integration/**'
# The WASM family suite (integration/wasm/**) exercises this entry:
- 'languages/typescript/packages/stack/src/wasm-inline.ts'
# The DynamoDB adapter, and the entry/type modules the suites import
# directly. `integration/shared/v2-decrypt-compat` and its
# `integration/wasm/` twin (both selected below) are the repo's only live
# EQL v2 read coverage, and they exercise the DynamoDB legacy path, so a
# change here must run them. Pinned by
# scripts/__tests__/integration-workflow-paths.test.mjs.
- 'languages/typescript/packages/stack/src/dynamodb/**'
- 'languages/typescript/packages/stack/src/index.ts'
- 'languages/typescript/packages/stack/src/types.ts'
# Those same v2 suites mint their fixtures by importing
# `@cipherstash/protect-ffi` directly. A native-module bump is the change
# most able to break v2 payload deserialization and it touches NO source
# directory, so without these two entries the only suites that would catch
# it never start. They are the files a bump actually edits: exact pins
# (`protect-ffi`, `@cipherstash/eql`) live in the package manifest,
# `catalog:` ones in the workspace catalog.
#
# `pnpm-lock.yaml` is deliberately NOT listed. It changes on roughly every
# dependency bump in the monorepo — far more often than either file here —
# and these are credentialed, database-backed jobs. Nothing is lost: a
# protect-ffi version change cannot reach the lockfile without editing
# one of the two manifests below first, and an auth one edits its own
# manifest, under the auth paths below.
- 'languages/typescript/packages/stack/package.json'
- 'pnpm-workspace.yaml'
- 'languages/typescript/packages/test-kit/**'
- 'languages/typescript/packages/cli/src/installer/**'
- 'local/docker-compose.postgres.yml'
- 'local/docker-compose.supabase.yml'
- 'local/supabase-init.sql'
- '.github/workflows/integration-drizzle.yml'
- '.github/actions/integration-setup/**'
- '.github/actions/integration-db/**'
- '.github/actions/build-ffi-binding/**'
- '.github/actions/build-auth-binding/**'
# `@cipherstash/auth` is a workspace package, and the SDK loads its napi
# module (and, for `wasm-inline`, its wasm) on import, so a change to what
# it ships arrives here rather than in `pnpm-workspace.yaml`. The paths of
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
# Absorbing protect-ffi put it in-tree, so a crate change can now
# break them in a PR that touches no TypeScript at all.
#
# The manifests are here for the same reason as the sources: a
# dependency bump in Cargo.lock changes the compiled encryption core
# without touching a .rs file, and package.json / mise.toml carry the
# build scripts and the toolchain pin. Matching the filter in
# tests-rust.yml, which already covers all four.
- 'languages/typescript/packages/protect-ffi/crates/**'
- 'languages/typescript/packages/protect-ffi/src/**'
- 'languages/typescript/packages/protect-ffi/Cargo.toml'
- 'languages/typescript/packages/protect-ffi/Cargo.lock'
- 'languages/typescript/packages/protect-ffi/package.json'
- 'languages/typescript/packages/protect-ffi/mise.toml'
# The non-cargo half of `build:wasm`, and the reason this job passes
# `wasm: 'true'`: `tsc -p tsconfig.wasm-errors.json`, then a
# `postbuild:wasm` hook running `scripts/inline-wasm.mjs`, which is what
# emits `dist/wasm/protect_ffi_inline.js` — the bundle the
# `integration/wasm/**` suites selected below actually load. Both files
# are in the WASM cache key of `.github/actions/build-ffi-binding`, so an
# edit to either misses that cache and REBUILDS the bundle. Without these
# two entries the workflow does not trigger at all, so neither the rebuild
# nor the only suite that loads it ever runs.
# Pinned by scripts/__tests__/wasm-build-inputs-paths-filter.test.mjs.
- 'languages/typescript/packages/protect-ffi/tsconfig.wasm-errors.json'
- 'languages/typescript/packages/protect-ffi/scripts/inline-wasm.mjs'
# Out of that package, and compiled into the same two artifacts: the
# cdylib crate carries
# `eql-bindings = { path = "../../../../../../packages/eql/crates/eql-bindings" }`, so the
# EQL wire types are linked into `index.node` and into
# `protect_ffi_inline.js`. Both cache keys in
# `.github/actions/build-ffi-binding` hash them, so an edit here misses
# that cache and rebuilds the bundle the `integration/wasm/**` suites
# load. Same argument as the two entries above, same guard.
- 'packages/eql/crates/**'
- 'packages/eql/Cargo.toml'
pull_request:
branches: ['**']
# Repeated verbatim: GitHub Actions does not support YAML anchors/aliases.
paths:
- 'languages/typescript/packages/stack/src/eql/v3/**'
- 'languages/typescript/packages/stack-drizzle/**'
# Source layers the adapter's encoding/round-trip rests on: a break here
# (not just under src/eql/v3) can produce wrong rows, so trigger the live
# suite that would catch it.
- 'languages/typescript/packages/stack/src/encryption/**'
- 'languages/typescript/packages/stack/src/schema/**'
- 'languages/typescript/packages/stack/integration/**'
# The WASM family suite (integration/wasm/**) exercises this entry:
- 'languages/typescript/packages/stack/src/wasm-inline.ts'
# The DynamoDB adapter, and the entry/type modules the suites import
# directly. `integration/shared/v2-decrypt-compat` and its
# `integration/wasm/` twin (both selected below) are the repo's only live
# EQL v2 read coverage, and they exercise the DynamoDB legacy path, so a
# change here must run them. Pinned by
# scripts/__tests__/integration-workflow-paths.test.mjs.
- 'languages/typescript/packages/stack/src/dynamodb/**'
- 'languages/typescript/packages/stack/src/index.ts'
- 'languages/typescript/packages/stack/src/types.ts'
# Those same v2 suites mint their fixtures by importing
# `@cipherstash/protect-ffi` directly. A native-module bump is the change
# most able to break v2 payload deserialization and it touches NO source
# directory, so without these two entries the only suites that would catch
# it never start. They are the files a bump actually edits: exact pins
# (`protect-ffi`, `@cipherstash/eql`) live in the package manifest,
# `catalog:` ones in the workspace catalog.
#
# `pnpm-lock.yaml` is deliberately NOT listed. It changes on roughly every
# dependency bump in the monorepo — far more often than either file here —
# and these are credentialed, database-backed jobs. Nothing is lost: a
# protect-ffi version change cannot reach the lockfile without editing
# one of the two manifests below first, and an auth one edits its own
# manifest, under the auth paths below.
- 'languages/typescript/packages/stack/package.json'
- 'pnpm-workspace.yaml'
- 'languages/typescript/packages/test-kit/**'
- 'languages/typescript/packages/cli/src/installer/**'
- 'local/docker-compose.postgres.yml'
- 'local/docker-compose.supabase.yml'
- 'local/supabase-init.sql'
- '.github/workflows/integration-drizzle.yml'
- '.github/actions/integration-setup/**'
- '.github/actions/integration-db/**'
- '.github/actions/build-ffi-binding/**'
- '.github/actions/build-auth-binding/**'
# `@cipherstash/auth` is a workspace package, and the SDK loads its napi
# module (and, for `wasm-inline`, its wasm) on import, so a change to what
# it ships arrives here rather than in `pnpm-workspace.yaml`. The paths of
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
# Absorbing protect-ffi put it in-tree, so a crate change can now
# break them in a PR that touches no TypeScript at all.
#
# The manifests are here for the same reason as the sources: a
# dependency bump in Cargo.lock changes the compiled encryption core
# without touching a .rs file, and package.json / mise.toml carry the
# build scripts and the toolchain pin. Matching the filter in
# tests-rust.yml, which already covers all four.
- 'languages/typescript/packages/protect-ffi/crates/**'
- 'languages/typescript/packages/protect-ffi/src/**'
- 'languages/typescript/packages/protect-ffi/Cargo.toml'
- 'languages/typescript/packages/protect-ffi/Cargo.lock'
- 'languages/typescript/packages/protect-ffi/package.json'
- 'languages/typescript/packages/protect-ffi/mise.toml'
# The non-cargo half of `build:wasm`, and the reason this job passes
# `wasm: 'true'`: `tsc -p tsconfig.wasm-errors.json`, then a
# `postbuild:wasm` hook running `scripts/inline-wasm.mjs`, which is what
# emits `dist/wasm/protect_ffi_inline.js` — the bundle the
# `integration/wasm/**` suites selected below actually load. Both files
# are in the WASM cache key of `.github/actions/build-ffi-binding`, so an
# edit to either misses that cache and REBUILDS the bundle. Without these
# two entries the workflow does not trigger at all, so neither the rebuild
# nor the only suite that loads it ever runs.
# Pinned by scripts/__tests__/wasm-build-inputs-paths-filter.test.mjs.
- 'languages/typescript/packages/protect-ffi/tsconfig.wasm-errors.json'
- 'languages/typescript/packages/protect-ffi/scripts/inline-wasm.mjs'
# Out of that package, and compiled into the same two artifacts: the
# cdylib crate carries
# `eql-bindings = { path = "../../../../../../packages/eql/crates/eql-bindings" }`, so the
# EQL wire types are linked into `index.node` and into
# `protect_ffi_inline.js`. Both cache keys in
# `.github/actions/build-ffi-binding` hash them, so an edit here misses
# that cache and rebuilds the bundle the `integration/wasm/**` suites
# load. Same argument as the two entries above, same guard.
- 'packages/eql/crates/**'
- 'packages/eql/Cargo.toml'
jobs:
integration:
name: Drizzle v3 integration (db=${{ matrix.db }})
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 35
# No concurrency group: `integration-db` gives each job its own compose
# project and ephemeral host ports, so live-DB jobs no longer contend and do
# not need serialising. See that action for why the old
# `integration-live-db-<db>` group had to go (it cancelled a third
# contender rather than queueing it).
#
# Fork PRs have no secrets. Skip cleanly rather than fail on something the
# contributor cannot fix — `tests.yml` still gives them a green signal.
# Every OTHER event runs, which is why this gates on "not a fork PR" rather
# than listing the event names allowed through: the listing form skipped
# the job on `workflow_dispatch` in integration-protect-ffi.yml, making a
# declared manual trigger do nothing. Enforced by
# scripts/__tests__/workflow-dispatch-job-conditions.test.mjs.
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
strategy:
fail-fast: false
# Drizzle talks straight to Postgres, so it runs against BOTH databases.
# The Supabase variant is not a formality: its `postgres` role is not a
# superuser, so the EQL install takes its self-skipping path and the ORE
# domains become unusable. A suite that passes on a superuser database can
# still fail there.
# (The supabase compose file starts PostgREST anyway. Drizzle does not use
# it, but `integration-db` supplies the URL regardless, which lets the
# harness assert the `anon` path on the database it is actually running
# against.)
matrix:
db: [postgres, supabase]
env:
CS_WORKSPACE_CRN: ${{ vars.CS_WORKSPACE_CRN }}
CS_CLIENT_ID: ${{ vars.CS_CLIENT_ID }}
CS_CLIENT_KEY: ${{ secrets.CS_CLIENT_KEY }}
CS_CLIENT_ACCESS_KEY: ${{ secrets.CS_CLIENT_ACCESS_KEY }}
# The identity suites federate a freshly-minted Clerk M2M JWT into a CTS
# token. Only they read these; the other suites ignore them. If one is
# unset the relevant suite fails loudly (throw, never skip) rather than
# taking the whole job down, so they are NOT in the require-cs-secrets
# preflight. `_B` is a SECOND machine (a distinct `sub`) in the same Clerk
# instance, used only by the cross-identity test.
CLERK_MACHINE_TOKEN: ${{ secrets.CLERK_MACHINE_TOKEN }}
CLERK_MACHINE_TOKEN_B: ${{ secrets.CLERK_MACHINE_TOKEN_B }}
# EXPLICIT, never inferred. The variant decides whether EQL is installed
# with `--supabase` (and therefore whether the role grants are applied).
# Inferring it from `PGRST_URL` reported `postgres` for this job's Supabase
# cell and silently skipped the grants.
CS_IT_DB_VARIANT: ${{ matrix.db }}
# Scoped by directory, never by named file, so a renamed suite cannot
# silently drop from CI. `integration/shared/` holds the adapter-agnostic
# suites (harness, bloom, ope-term, the crypto/SQL matrices);
# `integration/identity/` holds the Clerk-federated lock-context suites
# (they need CLERK_MACHINE_TOKEN + a workspace with the Clerk issuer
# registered); the Supabase adapter suites are not run here — they have
# their own job.
# The Drizzle adapter suites (incl. the Clerk-federated lock-context one)
# now live in @cipherstash/stack-drizzle (run below via its own
# test:integration). This glob scopes what still lives in @cipherstash/stack:
# the adapter-agnostic `shared/` core suites, the model-path
# `identity/matrix-identity` suite, and the `wasm/` family suite (the
# `@cipherstash/stack/wasm-inline` adapter over the shared v3 matrix).
CS_IT_SUITE: >-
integration/shared/**/*.integration.test.ts,
integration/identity/**/*.integration.test.ts,
integration/wasm/**/*.integration.test.ts
steps:
- uses: actions/checkout@v6
- uses: ./.github/actions/integration-setup
# Fast pre-flight: fail in seconds if a secret was rotated or cleared,
# before the binding build and the docker pull. The in-test
# `requireIntegrationEnv` is the correctness guarantee; this is the cheap
# one, so nothing expensive may be ordered ahead of it.
- name: Require CipherStash secrets
uses: ./.github/actions/require-cs-secrets
with:
workspace-crn: ${{ vars.CS_WORKSPACE_CRN }}
client-id: ${{ vars.CS_CLIENT_ID }}
client-key: ${{ secrets.CS_CLIENT_KEY }}
client-access-key: ${{ secrets.CS_CLIENT_ACCESS_KEY }}
# `wasm: true` because CS_IT_SUITE below includes `integration/wasm/**`
# — the `@cipherstash/stack/wasm-inline` adapter over the shared v3
# matrix — which loads the real WASM, unlike the mocked unit tests.
- name: Build the protect-ffi binding
uses: ./.github/actions/build-ffi-binding
with:
wasm: 'true'
- name: Build the @cipherstash/auth binding
uses: ./.github/actions/build-auth-binding
with:
wasm: 'true'
# No pre-`up` cleanup step any more: the project name is unique per job, so
# a container leaked by a hard-killed prior run cannot hold this job's
# name or its (ephemeral) port. Blanket-pruning would now be actively
# unsafe — without the concurrency group, another job's stack may be live
# on this runner.
- name: Start ${{ matrix.db }}
id: db
uses: ./.github/actions/integration-db
with:
db: ${{ matrix.db }}
# `globalSetup` installs EQL v3 by shelling out to the real
# `stash eql install --eql-version 3`, so an installer regression fails
# here rather than hiding behind a test-only SQL apply.
#
# Step env, not a `.env` file: `dotenv/config` does not override an
# already-set `process.env`, so these win and no secret is written to disk.
# Through turbo, not a bare `pnpm --filter`: `test:integration` declares
# `dependsOn: ["^build", "build"]`, and the only build in this job is the
# setup action's `--filter stash`, which reaches these packages purely by
# coincidence of stash's own dependency graph. `--env-mode=loose` because
# turbo defaults to strict and would otherwise withhold the step env below
# from the test process (#787 review follow-up).
- name: Drizzle v3 integration suites
run: pnpm exec turbo run test:integration --filter @cipherstash/stack-drizzle --env-mode=loose
env:
DATABASE_URL: ${{ steps.db.outputs.database-url }}
PGRST_URL: ${{ steps.db.outputs.pgrest-url }}
# A second vitest invocation (stack's shared/ + identity suites live in a
# different package now). Its globalSetup calls the same EQL v3 install, but
# `isInstalled` short-circuits against the DB the first invocation already
# provisioned — a fast no-op check, not a second schema apply.
- name: Shared core + identity + wasm integration suites
run: pnpm exec turbo run test:integration --filter @cipherstash/stack --env-mode=loose
env:
DATABASE_URL: ${{ steps.db.outputs.database-url }}
PGRST_URL: ${{ steps.db.outputs.pgrest-url }}
# Guarded on the project being set: if the stack never came up, there is
# nothing to tear down and an unguarded `-p ""` would fail the job with a
# confusing error that masks the real one.
- name: Stop ${{ matrix.db }}
if: always() && env.CS_COMPOSE_PROJECT != ''
run: docker compose -p "$CS_COMPOSE_PROJECT" -f "$CS_COMPOSE_FILE" down -v