Repository navigation
simplex tests: a file is overdue counted from its forward, however long that took #2310
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # fragment CI on Linux: `cargo xtask check` (every first-party JavaScript | |
| # file through `node --check` on the pinned Node, host tests and clippy on | |
| # host and wasm, warnings denied), and `cargo xtask e2e`: the cell under | |
| # `wrangler dev` (workerd), with fakes only at the vendor boundaries | |
| # (docs/cloudflare-v1.md, phase 2). The celld e2e on macOS is at tag | |
| # celld-final. | |
| # | |
| # The e2e runs in four shards (crates/e2e/src/lanes/mod.rs, SHARDS: every | |
| # section in exactly one, a host test checks), each on a runner of its own | |
| # with its own build and node; the `e2e` job is green when every shard is. | |
| # | |
| # Caches are restored on every run and saved only from master's pushes (a | |
| # PR reads master's, and so does production's). `check`'s Rust cache is | |
| # Swatinem/rust-cache's, keyed by the job, rustc, and every Cargo.toml and | |
| # Cargo.lock, and saved at the job's end with only the dependencies' | |
| # builds: a checkout's fresh mtimes rebuild the workspace's own crates | |
| # anyway. It is given the target directories by name (`cache-targets: | |
| # false`), not `target` whole: its cleaning would leave target/tools an | |
| # empty tree, which xtask would take for the pinned Node. The e2e's builds, | |
| # our own crates' with them, are actions/cache's, one entry per master | |
| # commit, with the tree they were built from, so cargo rebuilds only what | |
| # a change touched (.github/actions/e2e-build, saved by the first shard | |
| # or, when a push's shards skip, by `e2e-build`; `prune` keeps the newest | |
| # alone); | |
| # their tools are rust-cache's, and the stub image's layers BuildKit's, | |
| # in the same Actions cache. The pinned Node and the JavaScript caches are | |
| # actions/cache's, each key naming what it holds below, saved right after | |
| # the step that fills it. | |
| # | |
| # A green push deploys (`deploy`, below): master's to the preview, | |
| # production's to production. A push of a tree a green run already | |
| # tested (master's, for production; for master, its merge queue's or its | |
| # pull request's, when master had not moved since) skips the shards | |
| # (`proven`, below), and so does a merge group into master whose tree a | |
| # green run already tested. | |
| name: ci | |
| on: | |
| push: | |
| branches: [master, production] | |
| pull_request: | |
| merge_group: | |
| types: [checks_requested] | |
| workflow_dispatch: | |
| # a pull request's newer run cancels its older one; a push's run is never | |
| # cancelled, as it may be deploying: a newer push's waits for it (and | |
| # replaces one already waiting) | |
| concurrency: | |
| group: ci-${{ github.ref }} | |
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | |
| jobs: | |
| # A push to master or production, or a merge group into master: whether | |
| # a run of this workflow (by its id, not its name) already proved what it | |
| # holds. A job can only need | |
| # jobs of its own run, so without this a promotion tested again, for 20 | |
| # minutes, what master's run had (Paul, 2026-10-20), and a merge what its | |
| # pull request's run had (Paul, 2026-10-23). What a run proves is a tree: | |
| # its success is its check's and every shard's, on the tree each checked | |
| # out, which `e2e` records (an artifact named `tested-tree-<tree>`). So a | |
| # commit is proven by a run that succeeded and is one of these: | |
| # | |
| # - on a push to master of a commit whose tree is this one's; for | |
| # production, master's runs of this very SHA too, recorded or not (as | |
| # before trees were). Master's runs test and then deploy the preview, | |
| # or deploy a tree proven so: such a run's success is its deploy's too. | |
| # - on master only, the run of the pull request this merge commit merges, | |
| # whose head is this commit's second parent, that recorded this tree. A | |
| # pull request's run tests GitHub's merge of its head into master as | |
| # master stood then: if master has not moved since, the merge commit's | |
| # tree is that one; if it has, they differ, and the tests run. The head | |
| # binds the run to this merge: another branch's run (a fork's included) | |
| # proves nothing here, whatever tree it names, and this one's workflow | |
| # is the one this merge brings to master. A squash or a rebase merge | |
| # has no such parent. | |
| # - on master only, a merge_group run from this repository's master | |
| # queue (`gh-readonly-queue/master/…`) that recorded this tree, whose | |
| # head really has this tree too. The queue tests the combined changes | |
| # against the base they will land on; moving master rebuilds the | |
| # group. Neither another base's queue nor a fork can prove it. | |
| # | |
| # A merge group into master is held to master's rules: its head is the | |
| # merge commit master will move to (the group's last pull request's head | |
| # its second parent), so it is proven exactly when that push would be. | |
| # Same tree, same files: the proving run ran this very workflow and these | |
| # very tests. | |
| # | |
| # Proven, the e2e's shards skip, `e2e` is green on the proof, and `deploy` | |
| # runs once `check` is green; otherwise (a hotfix pushed straight to | |
| # production, master moved since the pull request's run, a run red or | |
| # still running) they run as on any push. Should this job fail, no shard | |
| # runs, `e2e` is red, and nothing merges or deploys. | |
| # | |
| # `check` runs on every push, proven or not: besides the tree's own | |
| # checks it holds the tree's schema lock to the one at what the push | |
| # moved from (`github.event.before`; xtask/src/schema_lock.rs), which no | |
| # tree can prove ahead of the push. A tree proven before a migration | |
| # step shipped, and pushed again after it (a revert), drops that step, | |
| # and every object that took it refuses code without it (docs/MODEL.md, | |
| # Schemas): its `check` fails, and nothing deploys. On production too: | |
| # its last deploy can hold a step its new commit lost (a hotfix's, lost | |
| # in a merge back), which no run of master's checked against. `check` | |
| # takes about two minutes, beside `proven`; the shards are the 15 to 20. | |
| # A merge group's `check` holds the lock to the group's exact base. | |
| proven: | |
| if: >- | |
| ${{ (github.event_name == 'push' && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/production')) | |
| || (github.event_name == 'merge_group' && github.event.merge_group.base_ref == 'refs/heads/master') }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 5 | |
| permissions: | |
| actions: read # this workflow's runs and their artifacts, through the API | |
| contents: read # the commits' trees and parents | |
| outputs: | |
| proven: ${{ steps.ask.outputs.proven }} | |
| steps: | |
| # the runs the API's filters find, each fetched and held to the rules | |
| # above again; every one found is printed, with its state | |
| - id: ask | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| # the branch this lands on: a push's own, a merge group's base | |
| ONTO: ${{ github.event_name == 'merge_group' && github.event.merge_group.base_ref || github.ref }} | |
| run: | | |
| set -euo pipefail | |
| case "$ONTO" in | |
| refs/heads/master | refs/heads/production) ;; | |
| *) echo "::error::$GITHUB_EVENT_NAME lands on $ONTO, which no rule here proves"; exit 1 ;; | |
| esac | |
| repo=repos/$GITHUB_REPOSITORY | |
| workflow=$(gh api "$repo/actions/runs/$GITHUB_RUN_ID" --jq .workflow_id) | |
| commit=$(gh api "$repo/git/commits/$GITHUB_SHA") | |
| tree=$(jq -r .tree.sha <<< "$commit") | |
| merged=$(jq -r 'if (.parents | length) == 2 then .parents[1].sha else "" end' <<< "$commit") | |
| echo "$GITHUB_SHA's tree is $tree${merged:+; it merges $merged}" | |
| ids=$(gh api -X GET "$repo/actions/artifacts" -f name="tested-tree-$tree" -f per_page=100 --jq '.artifacts[].workflow_run.id') | |
| if [ "$ONTO" = refs/heads/production ]; then | |
| ids+=$'\n'$(gh api -X GET "$repo/actions/workflows/$workflow/runs" -f head_sha="$GITHUB_SHA" -f branch=master -f event=push -f per_page=100 --jq '.workflow_runs[].id') | |
| fi | |
| for id in $(sort -u <<< "$ids"); do | |
| run=$(gh api "$repo/actions/runs/$id") | |
| jq -r '"\(.html_url), attempt \(.run_attempt): \(.event) of \(.head_sha) (\(.head_repository.full_name) \(.head_branch)), \(.status) \(.conclusion // "")"' <<< "$run" | |
| jq -e --argjson w "$workflow" '.workflow_id == $w and .status == "completed" and .conclusion == "success"' <<< "$run" > /dev/null || continue | |
| head=$(jq -r .head_sha <<< "$run") | |
| if jq -e --arg r "$GITHUB_REPOSITORY" '.event == "push" and .head_branch == "master" and .head_repository.full_name == $r' <<< "$run" > /dev/null; then | |
| [ "$head" = "$GITHUB_SHA" ] || [ "$(gh api "$repo/git/commits/$head" --jq .tree.sha)" = "$tree" ] || continue | |
| why="master's run of $head proved this tree" | |
| elif [ "$ONTO" = refs/heads/master ] && [ -n "$merged" ] && jq -e --arg h "$merged" '.event == "pull_request" and .head_sha == $h' <<< "$run" > /dev/null; then | |
| why="the run of the pull request this merges, at its head $head, proved this tree" | |
| elif [ "$ONTO" = refs/heads/master ] && jq -e --arg r "$GITHUB_REPOSITORY" '.event == "merge_group" and .head_repository.full_name == $r and (.head_branch | startswith("gh-readonly-queue/master/"))' <<< "$run" > /dev/null; then | |
| [ "$(gh api "$repo/git/commits/$head" --jq .tree.sha)" = "$tree" ] || continue | |
| why="master's merge queue run of $head proved this tree" | |
| else | |
| continue | |
| fi | |
| echo "$why: $GITHUB_SHA's e2e need not run again" | |
| echo proven=true >> "$GITHUB_OUTPUT" | |
| exit 0 | |
| done | |
| echo "no run of this workflow proved $GITHUB_SHA's tree: testing it first" | |
| echo proven=false >> "$GITHUB_OUTPUT" | |
| # on every run, a proven push's too (`proven`, above, says why) | |
| check: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: dtolnay/rust-toolchain@stable | |
| with: | |
| targets: wasm32-unknown-unknown | |
| components: clippy | |
| # the debug, test and clippy builds of the two workspaces | |
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | |
| with: | |
| workspaces: | | |
| . -> target | |
| cell -> target | |
| cache-targets: false | |
| cache-directories: | | |
| target/debug | |
| cell/target | |
| cache-bin: false | |
| save-if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }} | |
| # the pinned Node `node --check` runs on, unpacked: the e2e's key, | |
| # which its first shard saves from master | |
| - uses: actions/cache/restore@v4 | |
| with: | |
| path: target/tools | |
| key: node-${{ runner.os }}-${{ hashFiles('crates/devstack/src/node_release.rs') }} | |
| # the schema lock master holds, which this tree's keeps step for | |
| # step (xtask/src/schema_lock.rs): a pull request's base, or the | |
| # master a push moved on from, or a merge group's exact base (not | |
| # master, which can move while the group is being tested) | |
| - run: git fetch --no-tags --depth=1 origin "$BASE" && echo "FRAGMENT_SCHEMA_BASE=FETCH_HEAD" >> "$GITHUB_ENV" | |
| env: | |
| BASE: ${{ github.event_name == 'push' && github.event.before || github.event.merge_group.base_sha || github.base_ref || 'master' }} | |
| - run: cargo xtask check | |
| # Also exercise image-input resolution on PRs/queue trees, without | |
| # secrets or registry writes. The real image job is master-only. | |
| - run: cargo xtask image preview --dry-run --output "$RUNNER_TEMP/preview-image-plan.json" | |
| e2e-shard: | |
| name: e2e shard ${{ matrix.shard }}/4 | |
| # on every run but a push or merge group whose tree a run proved | |
| # (`proven`, which skips on every other run: `!cancelled()` keeps its | |
| # skip from skipping this), and never after `proven` fails | |
| needs: proven | |
| if: ${{ !cancelled() && (needs.proven.result == 'skipped' || (needs.proven.result == 'success' && needs.proven.outputs.proven == 'false')) }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 20 | |
| env: | |
| # xtask/src/build.rs, WORKER_BUILD_VERSION | |
| WORKER_BUILD_VERSION: 0.8.5 | |
| strategy: | |
| fail-fast: false | |
| matrix: | |
| shard: [1, 2, 3, 4] | |
| steps: | |
| - uses: actions/checkout@v4 | |
| # the caches, the stub image and the build (.github/actions/e2e-build); | |
| # the first shard saves them on master's pushes | |
| - id: build | |
| uses: ./.github/actions/e2e-build | |
| with: | |
| save: ${{ matrix.shard == 1 && github.event_name == 'push' && github.ref == 'refs/heads/master' }} | |
| worker-build-version: ${{ env.WORKER_BUILD_VERSION }} | |
| # one shard's sections on a fresh node; the run's scratch (each node | |
| # boot's log, the lanes' screenshots) is kept when a check fails, and | |
| # uploaded below | |
| - run: cargo xtask e2e --no-build --shard ${{ matrix.shard }}/4 | |
| env: | |
| BUILDX_BUILDER: ${{ steps.build.outputs.builder }} | |
| CHROME_BIN: /usr/bin/google-chrome | |
| # after the run, from any shard that shot a preview card: miniflare | |
| # fetches its Chrome for the first one during the run (the first | |
| # shard to save wins; the others find the key taken) | |
| - if: always() && github.event_name == 'push' && github.ref == 'refs/heads/master' && steps.build.outputs.js-cache-hit != 'true' && hashFiles('target/cache/.wrangler/chrome/**/.miniflare-install-complete') != '' | |
| uses: actions/cache/save@v4 | |
| with: | |
| path: target/cache | |
| key: ${{ steps.build.outputs.js-cache-key }} | |
| - if: failure() | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: e2e-scratch-${{ matrix.shard }} | |
| path: target/e2e/ | |
| retention-days: 7 | |
| # The check named `e2e`, which pull requests and merge groups require: | |
| # green when every shard is (a re-run's, by each shard's latest attempt), | |
| # or when `proven` found a run that proved this tree, so none ran. It | |
| # runs whatever came before it, unless the run was cancelled: a required | |
| # check that skips reads as green, so a `proven` that failed, or shards | |
| # that skipped or failed, make it red, never skipped. | |
| e2e: | |
| needs: [proven, e2e-shard] | |
| if: ${{ !cancelled() }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 5 | |
| permissions: | |
| contents: read # the tested commit's tree, through the API | |
| steps: | |
| - env: | |
| PROVEN: ${{ needs.proven.result == 'success' && needs.proven.outputs.proven == 'true' }} | |
| PROVEN_RESULT: ${{ needs.proven.result }} | |
| SHARDS: ${{ needs.e2e-shard.result }} | |
| run: | | |
| if [ "$PROVEN" = true ]; then echo "a run that proved this tree ran its e2e (\`proven\`'s log names it): none ran here"; exit 0; fi | |
| test "$SHARDS" = success || { echo "the shards did not all pass (shards: $SHARDS; proven: $PROVEN_RESULT): their logs, or proven's, say why"; exit 1; } | |
| # The tree every shard tested, for `proven`: GITHUB_SHA's (a pull | |
| # request's merge commit, a merge group's head, a push's head), | |
| # which each checked out, | |
| # recorded once they are all green as an artifact named by it (the | |
| # commit and the tree in it). `proven` counts it only for a run that | |
| # succeeded, `check` and every shard of its latest attempt included. | |
| # A re-run's replaces its own. Recording is a saving, not a test: a | |
| # step of it that fails warns and leaves the run as green as its | |
| # shards, unrecorded (a push of its tree is then tested again). | |
| # Only shards that ran here record: a proven run's proof is another's. | |
| - id: tree | |
| if: needs.e2e-shard.result == 'success' | |
| continue-on-error: true | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| set -euo pipefail | |
| tree=$(gh api "repos/$GITHUB_REPOSITORY/git/commits/$GITHUB_SHA" --jq .tree.sha) | |
| [[ $tree =~ ^[0-9a-f]{40}([0-9a-f]{24})?$ ]] || { echo "::warning::the API named no tree for $GITHUB_SHA: '$tree'"; exit 1; } | |
| echo "this run tested $GITHUB_SHA, whose tree is $tree" | |
| echo "$GITHUB_SHA $tree" > "$RUNNER_TEMP/tested-tree" | |
| echo "tree=$tree" >> "$GITHUB_OUTPUT" | |
| - if: steps.tree.outcome == 'success' | |
| continue-on-error: true | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: tested-tree-${{ steps.tree.outputs.tree }} | |
| path: ${{ runner.temp }}/tested-tree | |
| if-no-files-found: error | |
| overwrite: true | |
| retention-days: 30 | |
| # Master follows one image, keyed by its inputs rather than its commit. | |
| # A registry tag preserves the digest across cache eviction and re-runs; | |
| # a changed cell or shell never builds a new image. Prepare it before | |
| # deploy can start. This job never changes the running Worker. The merge | |
| # queue builds and prepares a group's image ahead (preview-image.yml), | |
| # which this job copies to its input tag; it builds only when neither | |
| # is there. | |
| preview-image: | |
| if: github.event_name == 'push' && github.ref == 'refs/heads/master' | |
| runs-on: ubuntu-latest | |
| # Build/push plus prepare.rs's 40-minute preparation limit. | |
| timeout-minutes: 65 | |
| permissions: | |
| actions: read # the merge queue's image build of this commit, through the API | |
| contents: read | |
| environment: preview | |
| # Share the manual Image workflow's publication lock: two cold builds | |
| # must not race to replace the same input tag with different digests. | |
| concurrency: | |
| group: image-preview | |
| cancel-in-progress: false | |
| outputs: | |
| # A whole reference contains the secret account ID, so GitHub would | |
| # suppress it as a secret output. Deploy uses its config's account. | |
| digest: ${{ steps.ready.outputs.digest }} | |
| key: ${{ steps.ready.outputs.key }} | |
| env: | |
| CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} | |
| CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} | |
| steps: | |
| - uses: actions/checkout@v4 | |
| with: | |
| persist-credentials: false | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | |
| with: | |
| shared-key: preview-image | |
| cache-targets: false | |
| cache-directories: target/debug | |
| cache-bin: false | |
| - uses: actions/cache/restore@v4 | |
| with: | |
| path: target/tools | |
| key: node-${{ runner.os }}-${{ hashFiles('crates/devstack/src/node_release.rs') }} | |
| - run: command -v skopeo > /dev/null || { sudo apt-get update -q && sudo apt-get install -y -q skopeo; } | |
| # A merge group's head is the commit master moves to, so the queue's | |
| # image run for this very commit, when one is still going, is | |
| # building these inputs: wait for it rather than build them beside | |
| # it. It ends within its own 65 minutes; past 30, or if it failed, | |
| # this job resolves and builds as it would without it. | |
| - name: wait for the merge queue's image build of this commit | |
| timeout-minutes: 30 | |
| continue-on-error: true | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| set -euo pipefail | |
| run=$(gh api -X GET "repos/$GITHUB_REPOSITORY/actions/workflows/preview-image.yml/runs" -f head_sha="$GITHUB_SHA" -f event=merge_group -f per_page=1 --jq '.workflow_runs[0] // empty | "\(.id) \(.html_url)"') | |
| [ -n "$run" ] || { echo "the merge queue built no image for $GITHUB_SHA"; exit 0; } | |
| id=${run%% *} | |
| echo "the merge queue's image run for $GITHUB_SHA: ${run#* }" | |
| until [ "$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$id" --jq .status)" = completed ]; do sleep 15; done | |
| echo "it ended: $(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$id" --jq .conclusion)" | |
| - id: probe | |
| run: | | |
| cargo xtask image preview --output "$RUNNER_TEMP/preview-image.json" | |
| echo "build=$(jq -r .build "$RUNNER_TEMP/preview-image.json")" >> "$GITHUB_OUTPUT" | |
| - id: buildx | |
| if: steps.probe.outputs.build == 'true' | |
| uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1 | |
| # The CLI stage now copies only its inputs (build.rs's host test). | |
| # This cache saves its unchanged stage, the bridge's, and the large | |
| # base. One master-only scope (~3 GB), not one entry per commit. | |
| # Registry input tags, not this evictable cache, preserve identity. | |
| - if: steps.probe.outputs.build == 'true' | |
| uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0 | |
| env: | |
| DOCKER_BUILD_SUMMARY: false | |
| DOCKER_BUILD_RECORD_UPLOAD: false | |
| with: | |
| context: . | |
| file: images/hermes/Dockerfile | |
| platforms: linux/amd64 | |
| provenance: false | |
| cache-from: type=gha,scope=preview-hermes | |
| cache-to: type=gha,scope=preview-hermes,mode=max,ignore-error=true,timeout=5m | |
| - id: ready | |
| env: | |
| BUILDX_BUILDER: ${{ steps.buildx.outputs.name }} | |
| run: | | |
| cargo xtask image preview --build --output "$RUNNER_TEMP/preview-image.json" --summary "$GITHUB_STEP_SUMMARY" | |
| echo "digest=$(jq -er .digest "$RUNNER_TEMP/preview-image.json")" >> "$GITHUB_OUTPUT" | |
| echo "key=$(jq -er .key "$RUNNER_TEMP/preview-image.json")" >> "$GITHUB_OUTPUT" | |
| # Continuous deployment, on a green push: master's to the preview | |
| # (`--branch preview`), production's to production, after `check` and | |
| # the e2e, or after `check` alone when a run proved its tree (`proven`), | |
| # and never after a red test or a failed `proven`. Only Paul updates | |
| # `production` (a protected branch): his push, or the pull request he | |
| # merges into it, is the approval. Each | |
| # deploys from a GitHub environment of its name, `preview` or | |
| # `production`, which the repo's settings let | |
| # only its branch use, and which holds the secrets DEPLOY_REPO_TOKEN (below), | |
| # CLOUDFLARE_API_TOKEN (wrangler's, and the DNS token), the preview's | |
| # PREVIEW_TEST_SECRET, and, optional, CLOUDFLARE_ACCOUNT_ID (below). | |
| # Its config is its environment's file in the private repo | |
| # finitecomputer/fragment-deploy, `preview.jsonc` or `production.jsonc` | |
| # at the root of its main as it stands when the job runs: names and | |
| # paths, no secret's value, kept out of this public repo and its logs, | |
| # and readable by the team, as a secret is not. Every computer image is | |
| # pinned by digest (`--pinned-only`): this job builds none; preview's | |
| # separate image job resolves its input-keyed pin before it can start | |
| # (xtask/src/deploy.rs, `pinned_only`, says why). | |
| deploy: | |
| needs: [proven, check, e2e, preview-image, cli-version, cli-release] | |
| if: >- | |
| ${{ !cancelled() && github.event_name == 'push' | |
| && (github.ref == 'refs/heads/master' || github.ref == 'refs/heads/production') | |
| && needs.check.result == 'success' | |
| && (github.ref == 'refs/heads/production' || needs.preview-image.result == 'success') | |
| && ((needs.proven.result == 'success' && needs.proven.outputs.proven == 'true') | |
| || needs.e2e.result == 'success') }} | |
| runs-on: ubuntu-latest | |
| # the deploy waits up to 20 minutes for its images to be ready on | |
| # Cloudflare's network before it builds (xtask/src/prepare.rs, | |
| # DEPLOY_LIMIT), and, after wrangler's deploy, up to 10 to finish one | |
| # whose Worker is live and 7 for every object class to answer it, and | |
| # 7 more for the release before when it rolls back (rollout.rs) | |
| timeout-minutes: 60 | |
| permissions: | |
| contents: read | |
| environment: ${{ github.ref_name == 'production' && 'production' || 'preview' }} | |
| # one deploy at a time to each, never cancelled mid-way | |
| concurrency: | |
| group: deploy-${{ github.ref_name }} | |
| cancel-in-progress: false | |
| env: | |
| # xtask/src/build.rs, WORKER_BUILD_VERSION | |
| WORKER_BUILD_VERSION: 0.8.5 | |
| # the environment's config, in fragment-deploy's checkout (below) | |
| CONFIG_FILE: ${{ github.ref_name == 'production' && 'production' || 'preview' }}.jsonc | |
| steps: | |
| # the deploy reads its commit, and needs no git credentials | |
| - uses: actions/checkout@v4 | |
| with: | |
| persist-credentials: false | |
| # the config, from fragment-deploy's main as it stands now (a change | |
| # there deploys at the environment's next deploy), read with | |
| # DEPLOY_REPO_TOKEN, a fine-grained token with Contents · Read on that | |
| # repo alone, which its checkout does not keep. It is checked out | |
| # after this repo, whose checkout cleans the tree, into | |
| # deploy-config/, which .gitignore names: the tree stays clean, so the | |
| # deploy's id is its commit (xtask/src/deploy.rs, `git_head`). The job | |
| # prints the config's commit, so a deploy's config is traceable, and | |
| # never its contents. | |
| - env: | |
| DEPLOY_REPO_TOKEN: ${{ secrets.DEPLOY_REPO_TOKEN }} | |
| run: test -n "$DEPLOY_REPO_TOKEN" || { echo "::error::the environment of $GITHUB_REF_NAME's deploy has no DEPLOY_REPO_TOKEN (a fine-grained token with Contents · Read on finitecomputer/fragment-deploy), so it has no config"; exit 1; } | |
| - uses: actions/checkout@v4 | |
| with: | |
| repository: finitecomputer/fragment-deploy | |
| ref: main | |
| token: ${{ secrets.DEPLOY_REPO_TOKEN }} | |
| path: deploy-config | |
| persist-credentials: false | |
| - run: | | |
| at=$(git -C deploy-config rev-parse --short HEAD) | |
| test -s "deploy-config/$CONFIG_FILE" || { echo "::error::finitecomputer/fragment-deploy has no $CONFIG_FILE at its root on main ($at)"; exit 1; } | |
| echo "$GITHUB_REF_NAME deploys with finitecomputer/fragment-deploy's $CONFIG_FILE at $at" | |
| - id: rust | |
| uses: dtolnay/rust-toolchain@stable | |
| with: | |
| targets: wasm32-unknown-unknown | |
| # the e2e's caches, restored and never saved here: a deploy builds what | |
| # its build does (xtask in target/debug, the cell's wasm32 builds, with | |
| # worker-build and its tools). Every input of each but save-if is the | |
| # e2e build's own (.github/actions/e2e-build), as the paths are part | |
| # of a cache's version. Its builds are this commit's when its first | |
| # shard saved them (a master push the e2e ran on; a proven push's | |
| # `e2e-build` saves them beside this job, and so mostly after its | |
| # restore), and their mtimes are made as there: the cell's | |
| # crates are then all fresh, and the Wasm this ships is made from the | |
| # compile the e2e ran. On production they are master's newest (a | |
| # branch reads the default branch's caches), as its shards save none, | |
| # and a proven push runs none; what changed since then is rebuilt. | |
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 | |
| with: | |
| shared-key: e2e-tools-worker-build-${{ env.WORKER_BUILD_VERSION }} | |
| workspaces: | | |
| . -> target | |
| cell -> target | |
| cache-targets: false | |
| cache-directories: ~/.cache/worker-build | |
| save-if: false | |
| - uses: actions/cache/restore@v4 | |
| with: | |
| path: | | |
| target/built-tree | |
| target/debug | |
| target/release | |
| cell/target | |
| key: e2e-builds-${{ runner.os }}-${{ steps.rust.outputs.cachekey }}-${{ github.sha }} | |
| restore-keys: e2e-builds-${{ runner.os }}-${{ steps.rust.outputs.cachekey }}- | |
| # as e2e-shard's | |
| - run: | | |
| rustc --edition 2021 -O -o "$RUNNER_TEMP/stamp" xtask/src/stamp.rs | |
| "$RUNNER_TEMP/stamp" restore target/built-tree "$RUNNER_TEMP/stamp-state" | |
| - uses: actions/cache/restore@v4 | |
| with: | |
| path: target/tools | |
| key: node-${{ runner.os }}-${{ hashFiles('crates/devstack/src/node_release.rs') }} | |
| - run: worker-build --version | grep -qF "$WORKER_BUILD_VERSION" || cargo install worker-build --version "$WORKER_BUILD_VERSION" --locked | |
| # the two files the config reads by path, at the paths a CI config | |
| # names (deploy/example.jsonc): the token, its DNS token too, and the | |
| # preview's test secret, each 0600. printf is the shell's own, so no | |
| # value is ever a process's argument, and nothing here echoes one. | |
| # CLOUDFLARE_ACCOUNT_ID is named only to be masked: the runner masks | |
| # every secret a job names, wherever it appears in the job's logs | |
| # (wrangler's and xtask's registry paths and URLs); unset, it is empty | |
| # and masks nothing. | |
| - env: | |
| CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} | |
| PREVIEW_TEST_SECRET: ${{ secrets.PREVIEW_TEST_SECRET }} | |
| CLOUDFLARE_ACCOUNT_ID_MASKED: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} | |
| run: | | |
| umask 077 | |
| test -n "$CLOUDFLARE_API_TOKEN" || { echo "the environment of $GITHUB_REF_NAME's deploy has no CLOUDFLARE_API_TOKEN"; exit 1; } | |
| mkdir -p ~/.config/fragment/secrets | |
| printf '%s\n' "$CLOUDFLARE_API_TOKEN" > ~/.config/fragment/secrets/cloudflare-api-token | |
| if [ -n "$PREVIEW_TEST_SECRET" ]; then printf '%s\n' "$PREVIEW_TEST_SECRET" > ~/.config/fragment/secrets/preview-test-secret; fi | |
| # The private config remains the source for deployment settings. | |
| # Preview's effective image is resolved by this checkout's inputs, | |
| # and applied in memory; this summary is its auditable image trail. | |
| - if: github.ref_name == 'master' | |
| env: | |
| DIGEST: ${{ needs.preview-image.outputs.digest }} | |
| INPUT_KEY: ${{ needs.preview-image.outputs.key }} | |
| run: | | |
| test -n "$DIGEST" && test -n "$INPUT_KEY" || { echo "::error::preview image resolution is missing; no deploy"; exit 1; } | |
| printf '### Resolved preview image\n\nInputs: `%s`\n\nDigest: `%s`\n\nDefault `hermes` and compatibility name `hermes-canary` both use this digest.\n' "$INPUT_KEY" "$DIGEST" >> "$GITHUB_STEP_SUMMARY" | |
| # green only when every object class answers this commit's release | |
| # (`git rev-parse --short=12 HEAD`) at /healthz/objects; a deploy that | |
| # cannot be finished or verified rolls back, and fails (rollout.rs) | |
| - run: | | |
| args=() | |
| if [ "$GITHUB_REF_NAME" = master ]; then args=(--branch preview --preview-image-digest "$PREVIEW_IMAGE_DIGEST"); fi | |
| cargo xtask deploy --config "deploy-config/$CONFIG_FILE" --pinned-only "${args[@]}" | |
| env: | |
| CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} | |
| PREVIEW_IMAGE_DIGEST: ${{ needs.preview-image.outputs.digest }} | |
| # Empty after a failed lookup/build: deploy still proceeds, with | |
| # no update header. Only a verified published release is named. | |
| FRAGMENT_RELEASED_CLI_VERSION: ${{ needs.cli-release.result == 'success' && needs.cli-release.outputs.version || (needs.cli-version.result == 'success' && needs.cli-version.outputs.version) || '' }} | |
| # Release new CLI inputs after green tests and before production deploys. | |
| # Unchanged inputs reuse their published release. A failed lookup/build | |
| # never blocks deploy, which then advertises no available CLI version. | |
| cli-version: | |
| needs: [proven, check, e2e] | |
| if: >- | |
| ${{ !cancelled() && github.event_name == 'push' && github.ref == 'refs/heads/production' | |
| && needs.check.result == 'success' | |
| && ((needs.proven.result == 'success' && needs.proven.outputs.proven == 'true') | |
| || needs.e2e.result == 'success') }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 5 | |
| permissions: | |
| contents: read | |
| outputs: | |
| tag: ${{ steps.next.outputs.tag }} | |
| version: ${{ steps.next.outputs.version }} | |
| publish: ${{ steps.next.outputs.publish }} | |
| steps: | |
| - uses: actions/checkout@v4 | |
| with: | |
| persist-credentials: false | |
| - uses: dtolnay/rust-toolchain@stable | |
| - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 | |
| with: | |
| save-if: false | |
| - id: next | |
| env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| cargo xtask cli-release plan >> "$GITHUB_OUTPUT" | |
| cli-release: | |
| needs: cli-version | |
| if: ${{ !cancelled() && needs.cli-version.result == 'success' && needs.cli-version.outputs.publish == 'true' }} | |
| permissions: | |
| contents: write # the tag and the release | |
| uses: ./.github/workflows/release.yml | |
| with: | |
| tag: ${{ needs.cli-version.outputs.tag }} | |
| sha: ${{ github.sha }} | |
| # A master push whose shards skip (`proven`) still builds the e2e and | |
| # saves it, as the first shard does on any other master push: pull | |
| # requests and merge groups restore master's newest build, the stub | |
| # image's layers and the tools, and without this they built from the | |
| # last master push that ran its shards (on 2026-10-30, about 160 s more | |
| # for the stub's bridge and 100 s more of Rust in every shard). Nothing | |
| # waits for it: `deploy` restores whatever is newest, as it did. | |
| e2e-build: | |
| needs: proven | |
| if: ${{ !cancelled() && github.event_name == 'push' && github.ref == 'refs/heads/master' && needs.proven.result == 'success' && needs.proven.outputs.proven == 'true' }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 20 | |
| env: | |
| # xtask/src/build.rs, WORKER_BUILD_VERSION | |
| WORKER_BUILD_VERSION: 0.8.5 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: ./.github/actions/e2e-build | |
| with: | |
| save: true | |
| worker-build-version: ${{ env.WORKER_BUILD_VERSION }} | |
| # The e2e's builds are kept one entry per master commit (`e2e-shard`'s | |
| # first shard, or `e2e-build`), each about 0.5 GB of the repository's | |
| # 10 GB of caches, and a run restores the newest: after a master push's | |
| # shards or its `e2e-build`, every older entry is deleted. A job of its own, as deleting one needs `actions: write`, | |
| # which no job that runs the repo's code is given. | |
| prune: | |
| needs: [e2e-shard, e2e-build] | |
| if: ${{ !cancelled() && github.event_name == 'push' && github.ref == 'refs/heads/master' }} | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 5 | |
| permissions: | |
| actions: write | |
| steps: | |
| - env: | |
| GH_TOKEN: ${{ github.token }} | |
| run: | | |
| set -euo pipefail | |
| entries=$(gh api -X GET "repos/$GITHUB_REPOSITORY/actions/caches" -f key=e2e-builds- -f ref=refs/heads/master \ | |
| -f sort=created_at -f direction=desc -f per_page=100 --jq '.actions_caches[] | "\(.id) \(.key) \(.size_in_bytes / 1e6 | floor) MB"') | |
| echo "$entries" | head -n 1 | sed 's/^/kept: /' | |
| echo "$entries" | tail -n +2 | while read -r id rest; do | |
| [ -n "$id" ] || continue | |
| echo "deleted: $id $rest" | |
| gh api -X DELETE "repos/$GITHUB_REPOSITORY/actions/caches/$id" | |
| done |