ci: run every suite on Dependabot PRs and deploy their review apps - #3079
Open
mfal wants to merge 3 commits into
Open
ci: run every suite on Dependabot PRs and deploy their review apps#3079mfal wants to merge 3 commits into
mfal wants to merge 3 commits into
Conversation
Contributor
Coverage Report for ./packages/components/
File CoverageNo changed files found. |
Contributor
🚀 Preview DeploymentPreview environments are ready:
Images:
|
mfal
marked this pull request as ready for review
September 2, 2026 09:24
Two gaps on a Dependabot PR, both rooted in the same fact: `secrets.*` in a
Dependabot-triggered run resolves against the repository's *Dependabot* secret
store, not the Actions one.
**Review apps.** The images were always built and pushed; only the deploy job
was skipped, and `cleanup-previews.yml` failed on every Dependabot merge
("Missing required environment variables", after the merge, where nobody
looks). Both halves now run in a `workflow_run` from the default branch, which
has the secrets — the same mechanism `dependabot-auto-merge.yml` uses. Neither
new workflow ever checks out the PR's code: the PR reaches them only as the two
images its build job pushed. The actor conditions are the exact complement of
the guards in the two ordinary preview workflows, so a push a person makes onto
a `dependabot/*` branch is still deployed there and not a second time here. The
cleanup workflow skips its *steps* rather than its job for that actor: a
job-level `if` concludes the run `skipped`, and only a run that completes is a
dependable trigger.
**Tests.** The full cross-version suite (both harnesses, every published
version) was the one suite a Dependabot PR never ran — it was label-only, while
visual has run unconditionally for a while. A dependency bump is exactly the
change that moves rendered structure without touching a component, so it now
runs there too, and `dependabot-auto-merge.yml` waits for both suites. Neither
is a required status check, so this workflow is the only thing between a red run
and a merge.
The gate reads the verdicts back per workflow instead of trusting the payload,
ordered by run `id` with `cancelled`/`skipped` dropped. One commit carries four
to five runs of the same suite: Dependabot attaches its own labels right after
opening the PR, each `labeled` event starts a run, and the concurrency group
cancels the rest — with `created_at` equal to the second, and the surviving run
not necessarily the highest id.
Also corrects the rationale several of these workflows carried. They claimed a
Dependabot-triggered run gets a read-only `GITHUB_TOKEN`; the run log says
`PullRequests: write`, and the ghcr push works. Only the secrets are withheld.
CONTRIBUTE.md now records both halves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
mfal
force-pushed
the
claude/dependabot-mrs-review-apps-tests-17e0bd
branch
from
September 2, 2026 11:53
367777e to
bd0a141
Compare
One conflict, in `.github/workflows/cleanup-previews.yml`: the actions group bump (#3069) re-pinned `actions/checkout` and `pnpm/action-setup` on the very lines where this branch adds the `CLEAN_UP_HERE` step guards. Kept both intents — the guards, on the new pins. The two workflows this branch adds could not conflict, because main has never seen them, so they still carried the pre-#3069 shas. Carried the new pins over by hand: leaving them behind would have made this PR reintroduce the exact versions #3069 just replaced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two gaps on a Dependabot PR, both rooted in the same fact:
secrets.*in a Dependabot-triggered run resolves against the repository's Dependabot secret store, not the Actions one.Review apps
The images were always built and pushed — only the deploy job was skipped.
cleanup-previews.ymlhad it worse: it ran without a guard and failed on every Dependabot merge withMissing required environment variables, after the merge, where nobody looks.deploy-previews-dependabot.yml(new) —workflow_runon Build & Deploy Preview Apps, runs from the default branch where the secrets are ordinary, and posts the preview URLs.cleanup-previews-dependabot.yml(new) — the same pattern behind Cleanup Preview Apps.dependabot/*branch is still deployed there — not a second time here.cleanup-previews.ymlskips its steps, not its job, for that actor: a job-levelifconcludes the whole runskipped, and only a run that actually completes is a dependable trigger.Tests
The full cross-version suite — both harnesses against every published version — was the one suite a Dependabot PR never ran (
skippingon all four open ones). It was label-only, while visual has run unconditionally for a while. A dependency bump is exactly the change that moves rendered structure without touching a component.test-cross-version-label.ymlnow also runs onopened/reopened/synchronizefor Dependabot PRs.dependabot-auto-merge.ymlwaits for both suites. Neither is a required status check in the ruleset, so this workflow is the only thing between a red run and a merge.The gate reads each suite's verdict back for the PR head instead of trusting the payload, ordered by run
idwithcancelled/skippeddropped. That ordering is not pedantry: one commit carries four to five runs of the same suite, because Dependabot attaches its own labels right after opening the PR and everylabeledevent starts one that the concurrency group then cancels. Theircreated_atare equal to the second, and the surviving run is not necessarily the highest id.Corrected rationale
Several of these workflows claimed a Dependabot-triggered run gets a read-only
GITHUB_TOKEN. The run log saysPullRequests: write, and the ghcr push works — only the secrets are withheld. Comments fixed in four workflows (no behaviour change there), and CONTRIBUTE.md now records both halves of what such a run does and does not get.Verifying this
Like every
workflow_runworkflow, the two new ones are inert until they land onmain— they cannot be exercised from this PR. After merge,@dependabot recreateon one of the open bump PRs produces a push → build → deploy chain to watch.Cost
The cross-version suite is unsharded (~25 min, 45 min timeout) and now runs on every Dependabot PR, plus the cancelled label-storm runs. At four PRs a week with the rebase cascade, that is the most noticeable part of this change.