Skip to content

Commit abf9de5

Browse files
authored
Merge pull request #1210 from cipherstash/ci/dependabot-eql-ignore-suite-crates
ci(dependabot): keep the EQL entry out of the root workspace's manifests
2 parents 8f610a5 + 4153d0a commit abf9de5

4 files changed

Lines changed: 414 additions & 11 deletions

File tree

‎.github/dependabot.yml‎

Lines changed: 67 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,16 @@ updates:
6666
# found" on a log page nobody visits, and the symptom is simply that no PR
6767
# ever arrives. The e2e test asserts this path holds a Cargo.toml.
6868
directory: /languages/typescript/packages/protect-ffi
69+
# Lockfile only, like the EQL entry below and for the same reason: this
70+
# workspace depends by path on eql-bindings, a member of the EQL
71+
# workspace. A requirement this entry raised could be written into
72+
# packages/eql/crates/eql-bindings/Cargo.toml without the EQL Cargo.lock,
73+
# the #1205 shape one edge over. An update that needs a manifest change,
74+
# in this workspace or that one, is dropped and made by hand. This applies
75+
# to Dependabot security updates too; osv-scanner still reports the
76+
# advisory. Held by
77+
# scripts/__tests__/dependabot-cargo-cross-workspace.test.mjs.
78+
versioning-strategy: lockfile-only
6979
# Monthly, where npm and actions are weekly. The deviation is about
7080
# validation cost, not risk appetite: a Cargo.lock bump is checked by
7181
# tests-rust.yml — one Blacksmith job, 45-minute timeout, cargo test +
@@ -138,23 +148,39 @@ updates:
138148
- version-update:semver-major
139149

140150
# ── Cargo (packages/eql — the second Rust workspace) ───────────
141-
# The EQL subtree brought a second Cargo workspace: five members
142-
# (crates/eql-{domains,codegen,tests-macros,bindings} plus tests/sqlx) and
143-
# its own Cargo.lock. It needs its own entry because Dependabot's cargo
144-
# `directory:` is a single workspace root, not a glob — the protect-ffi entry
145-
# above cannot reach it. The lockfile-to-ecosystem check in
146-
# e2e/tests/supply-chain.e2e.test.ts asserts coverage per ECOSYSTEM, so it
147-
# was already satisfied by that entry and would NOT have caught this gap;
148-
# the symptom would simply have been that no PR ever arrived.
151+
# The EQL subtree brought a second Cargo workspace: six members
152+
# (crates/eql-{domains,codegen,tests-macros,bindings}, tests/sqlx and
153+
# tests/encryption) and its own Cargo.lock. It needs its own entry because
154+
# Dependabot's cargo `directory:` is a single workspace root, not a glob —
155+
# the protect-ffi entry above cannot reach it. The lockfile-to-ecosystem
156+
# check in e2e/tests/supply-chain.e2e.test.ts asserts coverage per
157+
# ECOSYSTEM, so it was already satisfied by that entry and would NOT have
158+
# caught this gap; the symptom would simply have been that no PR ever
159+
# arrived.
149160
- package-ecosystem: cargo
150161
# The workspace root, where Cargo.toml and Cargo.lock live. Not `/`, and
151162
# not a crate directory — same trap as the protect-ffi entry documents.
152163
directory: /packages/eql
164+
# Lockfile only: this entry must never edit a manifest. The workspace
165+
# depends by path on root-workspace crates (see the ignores below), so a
166+
# requirement it would raise can live in the ROOT Cargo.toml, outside this
167+
# workspace; editing it there without the root Cargo.lock broke every root
168+
# `--locked` build (#1205, #1207). An update that needs a manifest change is
169+
# dropped rather than proposed, and is made by hand.
170+
#
171+
# That includes Dependabot SECURITY updates: `versioning-strategy` governs
172+
# them as well as version updates, so a security fix that needs a
173+
# requirement raised gets no PR from this entry. osv-scanner still reports
174+
# the advisory (it reaches this lockfile with `--recursive ./`), and the fix
175+
# is made by hand. Held, with the other two cargo entries, by
176+
# scripts/__tests__/dependabot-cargo-cross-workspace.test.mjs.
177+
versioning-strategy: lockfile-only
153178
# Monthly, matching protect-ffi and for the same reason: validating a
154179
# Cargo.lock bump here means the sharded SQLx matrix in test-eql.yml, which
155180
# compiles the full dependency tree and runs against live Postgres with
156-
# CipherStash credentials. Security fixes are unaffected — those are driven
157-
# by alerts, not by `schedule`.
181+
# CipherStash credentials. The schedule does not delay security fixes —
182+
# those are driven by alerts, not by `schedule` — though `lockfile-only`
183+
# above still drops one that needs a manifest edit.
158184
#
159185
# No `day:`, for the reason recorded above: it applies to `interval: weekly`
160186
# and would be configuration that does nothing.
@@ -195,6 +221,26 @@ updates:
195221
# a registry version for a crate we release here would fight the lockstep
196222
# version hook (scripts/sync-lockstep-versions.mjs).
197223
- dependency-name: "eql-bindings"
224+
# Inherited from the ROOT workspace. tests/encryption depends by path on
225+
# packages/stack-kms and packages/stack-encrypt, and eql-bindings on
226+
# stack-encrypt behind its optional `stack-encrypt` feature. Those crates
227+
# declare these `{ workspace = true }`, so their requirements come from
228+
# the root Cargo.toml even when built here. The root entry below ignores
229+
# them because they move by hand with cipherstash-suite: the root
230+
# manifest pins recipher and cllw-ore with `=`, and so does the Go guest.
231+
# Without these lines this entry bumped them in this lockfile and, to
232+
# satisfy the exact pins, rewrote the root Cargo.toml without touching
233+
# the root Cargo.lock or the guest's pin, breaking every root `--locked`
234+
# build and the guest (#1205, #1207). vitaminc* also keeps eql-bindings
235+
# on the line stack-encrypt and the guest build against.
236+
# versioning-strategy above already stops the manifest edit; these keep
237+
# the lockfile from moving these crates apart from the root's.
238+
# scripts/__tests__/dependabot-eql-inherited-ignores.test.mjs fails when
239+
# the root entry ignores a crate this entry does not.
240+
- dependency-name: "recipher"
241+
- dependency-name: "cllw-ore"
242+
- dependency-name: "zerokms-protocol"
243+
- dependency-name: "vitaminc*"
198244
# Major bumps are reviewed and applied manually, not by Dependabot.
199245
- dependency-name: "*"
200246
update-types:
@@ -213,6 +259,17 @@ updates:
213259
- /packages/stack-encrypt/fuzz
214260
- /languages/golang/encrypt/guest
215261
- /languages/golang/auth/guest
262+
# Lockfile only, like the other two cargo entries. Every one of these
263+
# workspaces shares crates by path with another: the fuzz crates and both
264+
# guests depend on root-workspace crates, the encrypt guest also on
265+
# eql-bindings, and the EQL workspace on stack-encrypt and stack-kms. Those
266+
# root crates take `{ workspace = true }` requirements from the root
267+
# Cargo.toml, so a manifest edit made for any one of these lockfiles can
268+
# leave the others stale under `--locked`. An update that needs a manifest
269+
# change, security updates included, is dropped and made by hand; osv-scanner
270+
# still reports the advisory. Held by
271+
# scripts/__tests__/dependabot-cargo-cross-workspace.test.mjs.
272+
versioning-strategy: lockfile-only
216273
# Monthly, matching the other two cargo entries.
217274
schedule:
218275
interval: monthly

‎packages/eql/crates/eql-codegen/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ thiserror = "2"
1313
quote = "1"
1414
proc-macro2 = "1"
1515
syn = { version = "2", features = ["full"] }
16-
prettyplease = "=0.2.37"
16+
prettyplease = "0.2"
1717

1818
[[bin]]
1919
name = "eql-codegen"
Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
import { existsSync, readFileSync, statSync } from 'node:fs'
2+
import { join, relative, resolve, sep } from 'node:path'
3+
import yaml from 'js-yaml'
4+
import { beforeAll, describe, expect, it } from 'vitest'
5+
import { REPO_ROOT } from './lib/repo-root.mjs'
6+
7+
/**
8+
* Every cargo Dependabot entry whose workspaces share crates with another
9+
* workspace by path must be `versioning-strategy: lockfile-only`.
10+
*
11+
* Each cargo `directory:` is one Cargo workspace with its own Cargo.lock, and
12+
* a Dependabot PR updates exactly that one lockfile. A path dependency crosses
13+
* that line in both directions:
14+
* - OUTWARD: a workspace that depends by path on another workspace's crate
15+
* resolves that crate's requirements, including any `{ workspace = true }`
16+
* ones from the other workspace's root Cargo.toml. To satisfy a bump it may
17+
* edit those manifests, outside its own workspace, and leave the other
18+
* workspace's Cargo.lock behind. That is #1205: the EQL entry rewrote the
19+
* root Cargo.toml, and every root `--locked` build failed (#1207).
20+
* - INWARD: a workspace whose crate another workspace depends on by path can
21+
* raise a requirement in its own manifest and update its own lockfile, and
22+
* the depending workspace's Cargo.lock is then stale under `--locked`.
23+
*
24+
* `lockfile-only` stops both: the entry never edits a manifest. An update that
25+
* needs one is dropped and made by hand, across every affected lockfile.
26+
*
27+
* The edges are read from the tracked manifests, so a new path dependency is
28+
* covered the day it lands.
29+
*/
30+
31+
const DEPENDABOT = join(REPO_ROOT, '.github/dependabot.yml')
32+
33+
/** Repo-relative, `/`-separated, no leading slash: the form Cargo uses. */
34+
function repoRel(abs) {
35+
return relative(REPO_ROOT, abs).split(sep).join('/')
36+
}
37+
38+
function readManifest(dir) {
39+
return readFileSync(join(REPO_ROOT, dir, 'Cargo.toml'), 'utf8')
40+
}
41+
42+
/**
43+
* The crate directories of the workspace rooted at `dir`: its `members`, plus
44+
* the root itself when it is also a package (a detached single-crate
45+
* workspace, such as a fuzz crate or a Go guest).
46+
*/
47+
function workspaceCrates(dir, manifest) {
48+
const crates = []
49+
if (/^\[package\]/m.test(manifest)) crates.push(dir)
50+
const block = manifest.match(/^members\s*=\s*\[([\s\S]*?)\]/m)
51+
if (block) {
52+
for (const [, member] of block[1].matchAll(/"([^"]+)"/g)) {
53+
expect(
54+
member,
55+
`${dir}/Cargo.toml: glob members are not expanded here`,
56+
).not.toMatch(/[*?[]/)
57+
crates.push(repoRel(resolve(REPO_ROOT, dir, member)))
58+
}
59+
}
60+
return crates
61+
}
62+
63+
/**
64+
* The crates a manifest depends on by path, repo-relative. A `path =` that
65+
* names a file (`[lib]`, `[[bin]]`) is not a dependency and is skipped.
66+
*/
67+
function pathDependencies(crateDir, manifest) {
68+
const deps = []
69+
for (const [, path] of manifest.matchAll(/\bpath\s*=\s*"([^"]+)"/g)) {
70+
const abs = resolve(REPO_ROOT, crateDir, path)
71+
if (!existsSync(abs) || !statSync(abs).isDirectory()) continue
72+
if (!existsSync(join(abs, 'Cargo.toml'))) continue
73+
deps.push(repoRel(abs))
74+
}
75+
return deps
76+
}
77+
78+
/**
79+
* Every cross-workspace path dependency, as
80+
* `{ from, to, fromDir, toDir }`: crate `from` in the workspace at `fromDir`
81+
* depends on crate `to` in the workspace at `toDir`. `toDir` is null when no
82+
* Dependabot cargo directory covers `to`.
83+
*/
84+
function crossWorkspaceEdges(workspaces) {
85+
const owner = new Map()
86+
for (const [dir, crates] of workspaces) {
87+
for (const crate of crates) owner.set(crate, dir)
88+
}
89+
const edges = []
90+
for (const [dir, crates] of workspaces) {
91+
for (const crate of crates) {
92+
for (const to of pathDependencies(crate, readManifest(crate))) {
93+
const toDir = owner.get(to) ?? null
94+
const seen = edges.some((e) => e.from === crate && e.to === to)
95+
if (toDir !== dir && !seen) {
96+
edges.push({ from: crate, to, fromDir: dir, toDir })
97+
}
98+
}
99+
}
100+
}
101+
return edges
102+
}
103+
104+
/** The directories of a Dependabot entry, without the leading `/`. */
105+
function entryDirs(entry) {
106+
const dirs = entry.directories ?? [entry.directory]
107+
return dirs.map((d) => d.replace(/^\/+/, '').replace(/\/+$/, ''))
108+
}
109+
110+
describe('cargo Dependabot entries that share crates across workspaces never edit a manifest', () => {
111+
let entries
112+
let edges
113+
114+
beforeAll(() => {
115+
const config = yaml.load(readFileSync(DEPENDABOT, 'utf8'))
116+
entries = config.updates.filter((u) => u['package-ecosystem'] === 'cargo')
117+
const workspaces = new Map()
118+
for (const entry of entries) {
119+
for (const dir of entryDirs(entry)) {
120+
workspaces.set(dir, workspaceCrates(dir, readManifest(dir)))
121+
}
122+
}
123+
edges = crossWorkspaceEdges(workspaces)
124+
})
125+
126+
it('finds cross-workspace path dependencies', () => {
127+
// If these all go away the rule below has nothing to hold, and this file
128+
// can be retired rather than left vacuously green.
129+
expect(edges).not.toHaveLength(0)
130+
})
131+
132+
it('covers every crate a path dependency reaches', () => {
133+
const uncovered = edges.filter((e) => e.toDir === null)
134+
expect(
135+
uncovered.map((e) => `${e.from} -> ${e.to}`),
136+
'path dependencies on crates no cargo Dependabot directory covers',
137+
).toEqual([])
138+
})
139+
140+
it('makes every entry on either end of one lockfile-only', () => {
141+
const involved = new Set(edges.flatMap((e) => [e.fromDir, e.toDir]))
142+
const offenders = entries
143+
.filter((entry) => entryDirs(entry).some((d) => involved.has(d)))
144+
.filter((entry) => entry['versioning-strategy'] !== 'lockfile-only')
145+
.map((entry) => {
146+
const dirs = new Set(entryDirs(entry))
147+
const why = edges
148+
.filter((e) => dirs.has(e.fromDir) || dirs.has(e.toDir))
149+
.map((e) => `${e.from} -> ${e.to}`)
150+
return `${(entry.directories ?? [entry.directory]).join(', ')}: ${why.join('; ')}`
151+
})
152+
expect(
153+
offenders,
154+
'these cargo entries share crates by path with another workspace, so a manifest ' +
155+
"edit can leave the other workspace's Cargo.lock behind. Set " +
156+
'`versioning-strategy: lockfile-only` on each.',
157+
).toEqual([])
158+
})
159+
})

0 commit comments

Comments
 (0)