Repository navigation

images #1515

Workflow file for this run

	# The computer images' Rust (images/: the bridge and hermes-boot,
	# docs/bridge.md): its own workspace, so its own jobs.
	#
	# - `check`: host tests and clippy, warnings denied, on every change to
	# images/, what it compiles in from outside it, or this workflow
	# (`changes`, below).
	# - `docker (real Hermes)`: the Docker lane (images/bridge/tests/docker.rs),
	# both images built for linux/amd64 and run in Docker against the fake
	# API and a scripted model, real Hermes included (`cargo test -p
	# fragment-bridge --test docker -- --ignored`): two jobs side by side,
	# the stub image's and the Hermes image's, and the check that needs both. It runs only when a change touches what it
	# builds or runs (`changes` below: images/hermes, images/bridge,
	# images/stub, the images' manifest and lock, the Hermes image's CLI
	# stage's inputs, or this workflow), in merge groups and on master's
	# pushes, on every run by hand, and whenever `changes` itself did not
	# succeed. Never on a pull request's own runs: the merge queue runs it on
	# the very tree that merges (as ci.yml's e2e), and a pull request's run
	# of it only held the queue back by its 20 minutes (Paul, 2026-11-02).
	# Skipped there, the required check passes; a run by hand runs it on a
	# branch's head before it queues.
	#
	# The workflow runs on every pull request, merge group and master push,
	# with no path filter of its own: master's ruleset requires `images check`
	# and `docker (real Hermes)`, and a check a workflow-level `paths` filter
	# keeps from starting never reports, which blocks a pull request that
	# touches none of them for good. Each job decides instead (`changes`): one
	# its inputs did not change skips, which a required check counts as a
	# pass, and one `changes` could not decide for runs.
	#
	# Their Rust cache (the registry and images/target's dependencies) is
	# Swatinem/rust-cache's, one key for both jobs, saved only from master's
	# pushes by `check` (a PR reads master's), as ci.yml's.
	name: images

	on:
	push:
	branches: [master]
	pull_request:
	# `changes` compares a merge group's exact base and head, as it does a
	# pull request's files and a push's since its `before`.
	merge_group:
	types: [checks_requested]
	workflow_dispatch:

	concurrency:
	group: images-${{ github.ref }}
	cancel-in-progress: true

	jobs:
	check:
	# ci.yml also has `check`; required checks need unambiguous names.
	name: images check
	needs: changes
	# as the Docker lane's: skips when images/ and this workflow did not
	# change, and runs whenever `changes` did not succeed
	if: ${{ !cancelled() && (github.event_name == 'workflow_dispatch' \|\| needs.changes.result != 'success' \|\| needs.changes.outputs.check == 'true') }}
	runs-on: ubuntu-latest
	defaults:
	run:
	working-directory: images
	steps:
	- uses: actions/checkout@v4
	- uses: dtolnay/rust-toolchain@stable
	with:
	components: clippy
	- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
	with:
	shared-key: images
	workspaces: images -> target
	cache-bin: false
	save-if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
	- run: cargo clippy --workspace --all-targets --locked -- -D warnings
	- run: cargo test --workspace --locked

	# Whether `check` and the Docker lane run: a change to what each checks,
	# builds or runs. Its own job, since a job's `if` reads only another
	# job's outputs. The changed
	# files are the pull request's (from the API, every page), or the push's
	# since its `before`, or the merge group's since its exact base (by git,
	# hence the checkout); a run by hand runs the lane whatever it says.
	changes:
	runs-on: ubuntu-latest
	timeout-minutes: 5
	outputs:
	check: ${{ steps.paths.outputs.check }}
	docker: ${{ steps.paths.outputs.docker }}
	steps:
	- uses: actions/checkout@v4
	- id: paths
	uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
	with:
	base: ${{ github.event.merge_group.base_sha \|\| github.ref }}
	ref: ${{ github.event.merge_group.head_sha \|\| github.ref }}
	filters: \|
	check:
	- images/**
	- .github/workflows/images.yml
	# what images/ compiles in from outside it, which a host test
	# (xtask/src/image/preview.rs) holds this list to
	- cli/SKILL.md
	- templates/skills/**
	- crates/e2e/fixtures/chat-preview.pdf
	docker:
	- images/hermes/**
	- images/bridge/**
	- images/stub/**
	- images/Cargo.toml
	- images/Cargo.lock
	- .github/workflows/images.yml
	# the Hermes image's other inputs, its CLI stage's (#715):
	# xtask/src/image/preview.rs `INPUTS`, which a host test
	# holds this list and the paths above to
	- .dockerignore
	- Cargo.toml
	- Cargo.lock
	- cli/**
	- crates/core/**
	- crates/nip98/**
	- crates/proto/**
	- crates/templates/**
	- templates/blank/**
	- templates/calories/**
	- templates/doc/**
	- templates/inbox/**
	- templates/notes/**
	- templates/todo/**
	- crates/devstack/Cargo.toml
	- crates/e2e/Cargo.toml
	- crates/fakes/Cargo.toml
	- xtask/Cargo.toml

	# The Docker lane. The images are built first by the runner's own Docker
	# (its BuildKit, straight into its image store, under the tags the tests
	# run), then the tests run on them with FRAGMENT_DOCKER_SKIP_BUILD, in
	# parallel (one per CPU). Measured on ubuntu-latest (4 CPUs, 16 GB, a
	# 145 GB disk with about 85 GB free): the Hermes image builds in about
	# 7 minutes (its two Rust stages, the CLI's and the bridge's, about
	# 5.5 of them), the tests take about 3. Its disk, measured on ubuntu-24.04
	# (PR #890): from the 31 GB used once the disk is freed, a peak of 44.7
	# GiB used, about 14 GiB of the lane's own.
	#
	# No layer cache, by measurement (PR #252): with BuildKit's layers in the
	# Actions cache (mode=max), an image whose every input was unchanged built
	# in 2 minutes, but that is a re-run's case. The CLI's stage, the build's
	# longest, reads the root workspace, which 108 of master's 147 commits
	# from 2026-10-01 to 10-08 changed: of the 33 pairs of master's
	# image-touching commits in that time, the CLI's stage was unchanged
	# across 6. What the cache would still hit (the Hermes base, its own
	# layer since 2026-10-09) is off the build's critical path, which is the
	# Rust. It cost 3.3 GB of the repository's
	# 10 GB of caches, and about 1.5 GB more at each write, against the
	# caches ci.yml reads on every run.
	#
	# The tests' container logs are kept on failure.
	# Whether master's merge queue already ran the Docker lane on this push's
	# very tree, as ci.yml's `proven` asks for the e2e: its `docker (real
	# Hermes)` recorded the tree (`docker-lane-tree-<tree>`, below) once both
	# halves passed. Then master's push runs it no more: the queue ran it on
	# the tree that landed (about 25 minutes of a runner a merge, 2026-11-02).
	# Only a merge group of master's counts, its run this workflow's and
	# green, its head's tree this push's; anything else, or a question the
	# API does not answer, runs the lane.
	docker-proven:
	if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/master' }}
	runs-on: ubuntu-latest
	timeout-minutes: 5
	permissions:
	actions: read # this workflow's runs and their artifacts, through the API
	contents: read # the commits' trees
	outputs:
	proven: ${{ steps.ask.outputs.proven }}
	steps:
	- id: ask
	env:
	GH_TOKEN: ${{ github.token }}
	run: \|
	set -euo pipefail
	repo=repos/$GITHUB_REPOSITORY
	workflow=$(gh api "$repo/actions/runs/$GITHUB_RUN_ID" --jq .workflow_id)
	tree=$(gh api "$repo/git/commits/$GITHUB_SHA" --jq .tree.sha)
	echo "$GITHUB_SHA's tree is $tree"
	for id in $(gh api -X GET "$repo/actions/artifacts" -f name="docker-lane-tree-$tree" -f per_page=100 --jq '.artifacts[].workflow_run.id' \| sort -u); do
	run=$(gh api "$repo/actions/runs/$id")
	jq -r '"\(.html_url), attempt \(.run_attempt): \(.event) of \(.head_sha) (\(.head_branch)), \(.status) \(.conclusion // "")"' <<< "$run"
	jq -e --argjson w "$workflow" --arg r "$GITHUB_REPOSITORY" '.workflow_id == $w and .status == "completed" and .conclusion == "success" and .event == "merge_group" and .head_repository.full_name == $r and (.head_branch \| startswith("gh-readonly-queue/master/"))' <<< "$run" > /dev/null \|\| continue
	[ "$(gh api "$repo/git/commits/$(jq -r .head_sha <<< "$run")" --jq .tree.sha)" = "$tree" ] \|\| continue
	echo "master's merge queue ran the Docker lane on this tree: it need not run again"
	echo proven=true >> "$GITHUB_OUTPUT"
	exit 0
	done
	echo "no merge group's run proved this tree: the Docker lane runs"
	echo proven=false >> "$GITHUB_OUTPUT"

	# The Docker lane is two jobs, side by side, each building only the image
	# its tests run: the stub image's one test (`the_stub_image`), and the
	# Hermes image's, every other. Built one after the other on one runner,
	# the stub's 4.6 minutes were the Hermes image's to wait out (the two
	# share no layer: their stages' cache mounts and copies differ), on the
	# merge queue's longest job (25 to 26.5 minutes, 2026-11-02). `docker
	# (real Hermes)`, the required check, is the job that needs both.
	docker-stub:
	name: docker lane (stub)
	needs: [changes, docker-proven]
	# as `docker-hermes`'s, which says why
	if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.docker-proven.outputs.proven != 'true' && (github.event_name == 'workflow_dispatch' \|\| needs.changes.result != 'success' \|\| needs.changes.outputs.docker == 'true') }}
	# the Docker `docker-hermes` runs on, which says why
	runs-on: ubuntu-24.04
	timeout-minutes: 40
	defaults:
	run:
	working-directory: images
	steps:
	- uses: actions/checkout@v4
	- uses: ./.github/actions/free-disk
	- uses: dtolnay/rust-toolchain@stable
	- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
	with:
	shared-key: images
	workspaces: images -> target
	cache-bin: false
	save-if: false
	- run: cargo test -p fragment-bridge --test docker --locked --no-run
	- name: the runner
	run: nproc; free -g; df -h /; docker version --format '{{.Server.Version}}'
	- name: build the stub image
	working-directory: .
	run: docker build --platform linux/amd64 -f images/stub/Dockerfile -t fragment-stub:test .
	- name: the Docker lane, the stub image's
	run: cargo test -p fragment-bridge --test docker --locked -- --ignored --exact the_stub_image --show-output
	env:
	FRAGMENT_DOCKER_SKIP_BUILD: "1"
	RUST_BACKTRACE: "1"
	- if: failure()
	uses: actions/upload-artifact@v4
	with:
	name: docker-lane-stub-logs
	path: images/target/tmp/*/.log
	retention-days: 7

	docker-hermes:
	name: docker lane (Hermes)
	needs: [changes, docker-proven]
	# fails closed: a `changes` that did not succeed (it failed, or ran out
	# its time, as in run 41100441445) runs the lane rather than skip it,
	# which a required check would count as a pass; a pull request's own
	# run skips it, as its merge group runs it
	if: ${{ !cancelled() && github.event_name != 'pull_request' && needs.docker-proven.outputs.proven != 'true' && (github.event_name == 'workflow_dispatch' \|\| needs.changes.result != 'success' \|\| needs.changes.outputs.docker == 'true') }}
	# ubuntu-24.04 (Docker 28), not ubuntu-latest: since 2026-10-25,
	# ubuntu-latest is 26.04 with Docker 29, whose containerd image store
	# names `containerd` in every container's root mount, so Hermes takes
	# even the Containers-shaped run (no /.dockerenv) for a container, and
	# `an_agents_home_is_the_same_on_either_runtime` proves nothing (it says
	# so). Containers on Cloudflare show no such mount.
	runs-on: ubuntu-24.04
	timeout-minutes: 40
	defaults:
	run:
	working-directory: images
	steps:
	- uses: actions/checkout@v4
	# its disk freed first (the action says why: PR #890), and sampled
	- uses: ./.github/actions/free-disk
	- uses: dtolnay/rust-toolchain@stable
	# the check job's cache, read only
	- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
	with:
	shared-key: images
	workspaces: images -> target
	cache-bin: false
	save-if: false
	- run: cargo test -p fragment-bridge --test docker --locked --no-run
	- name: the runner
	run: nproc; free -g; df -h /; docker version --format '{{.Server.Version}}'
	# as the tests' own `build` does (docker.rs), from the repo's root (it
	# carries the fragment CLI, of the root workspace)
	- name: build the Hermes image
	working-directory: .
	run: docker build --platform linux/amd64 -f images/hermes/Dockerfile -t fragment-hermes:test .
	# the tests build only the SimpleX lab's image, from its own
	# Dockerfile: the build's cache goes before them
	- name: disk, built
	run: docker system df; docker builder prune -af > /dev/null; df -h /
	- name: the Docker lane, the Hermes image's
	run: cargo test -p fragment-bridge --test docker --locked -- --ignored --skip the_stub_image --show-output
	env:
	FRAGMENT_DOCKER_SKIP_BUILD: "1"
	RUST_BACKTRACE: "1"
	- if: always()
	name: disk, after
	run: \|
	docker system df; df -h /; free -h
	echo "peak: $(awk '$1 > d { d = $1 } END { print d }' "$RUNNER_TEMP/disk-used") MiB of the disk used, $(awk '$2 > m { m = $2 } END { print m }' "$RUNNER_TEMP/disk-used") MiB of the memory"
	# each container's log (Container's drop) and the tests' own
	- if: failure()
	uses: actions/upload-artifact@v4
	with:
	name: docker-lane-hermes-logs
	path: images/target/tmp/*/.log
	retention-days: 7

	# The required check: green when both halves of the lane are, skipped when
	# the lane is not to run (a pull request's own run, or nothing it builds
	# changed), and red otherwise. It runs through a cancelled run (`always()`,
	# never `!cancelled()`): a required check that skips counts as passed, so
	# one cancelled with a half still running would let the merge through
	# (Codex review of #981). Its check runs whatever came before it, and a
	# half cancelled, skipped or failed is red.
	docker:
	name: docker (real Hermes)
	needs: [changes, docker-proven, docker-stub, docker-hermes]
	if: ${{ always() && github.event_name != 'pull_request' && needs.docker-proven.outputs.proven != 'true' && (github.event_name == 'workflow_dispatch' \|\| needs.changes.result != 'success' \|\| needs.changes.outputs.docker == 'true') }}
	runs-on: ubuntu-latest
	timeout-minutes: 5
	steps:
	- if: always()
	env:
	STUB: ${{ needs.docker-stub.result }}
	HERMES: ${{ needs.docker-hermes.result }}
	run: \|
	echo "the stub image's half: $STUB; the Hermes image's: $HERMES"
	test "$STUB" = success && test "$HERMES" = success
	# The tree both halves tested, for `docker-proven`: recorded as an
	# artifact named by it, which counts only for a merge group's green
	# run. Recording is a saving, not a test: a step of it that fails
	# warns, and master's push of the tree runs the lane again.
	- id: tree
	continue-on-error: true
	env:
	GH_TOKEN: ${{ github.token }}
	run: \|
	set -euo pipefail
	tree=$(gh api "repos/$GITHUB_REPOSITORY/git/commits/$GITHUB_SHA" --jq .tree.sha)
	[[ $tree =~ ^[0-9a-f]{40}([0-9a-f]{24})?$ ]] \|\| { echo "::warning::the API named no tree for $GITHUB_SHA: '$tree'"; exit 1; }
	echo "$GITHUB_SHA $tree" > "$RUNNER_TEMP/docker-lane-tree"
	echo "tree=$tree" >> "$GITHUB_OUTPUT"
	- if: steps.tree.outcome == 'success'
	continue-on-error: true
	uses: actions/upload-artifact@v4
	with:
	name: docker-lane-tree-${{ steps.tree.outputs.tree }}
	path: ${{ runner.temp }}/docker-lane-tree
	if-no-files-found: error
	overwrite: true
	retention-days: 30

Navigation Menu

Repository navigation

images #1515

Workflow file

images #1515

Uh oh!

Workflow file for this run