feat(exec): enroll SUITE_LOADER — the hosted api.qa/vitest@1 runner goes live #23

Workflow file for this run

.github/workflows/api-qa-example.yml at 247c5be

	name: api.qa gate (example)

	# Gate a pipeline on a PINNED reusable suite, the way you would gate on Newman.
	#
	# WHAT MAKES THIS A GATE (not a rubber stamp):
	# - `autonomous-qa suite ... --expect-digest <pin>` refuses to run unless
	# the suite text hashes to the ratified pin, then exits NON-ZERO if ANY probe
	# (or, with --iteration-data, ANY iteration) fails or the target is
	# unreachable/refused. The job step inherits that exit code, so a regression
	# turns the check RED. Exit 0 only when everything passed.
	# - The JUnit + JSON reports are uploaded as artifacts on every run (pass or
	# fail) so the failure detail survives even when the gate goes red.
	#
	# Copy this into your OWN repo, then set:
	# - API_QA_TARGET the deployed URL to verify (repo/environment variable)
	# - API_QA_SUITE_DIGEST the suite pin (mint once: `npx autonomous-qa spec-digest <suite>`)
	# ...and in YOUR repo, invoke the published CLI instead of the local build:
	# npx --yes autonomous-qa suite <suite.json> --env prod ...
	#
	# ── Why this file is CONFIGURED-ONLY, and why it builds the CLI (2026-08-06, #4)
	#
	# It had been red on `main` for 11 consecutive runs, for three stacked reasons.
	# All three are fixed here; recording them so they are not reintroduced:
	#
	# 1. exit 127, `autonomous-qa: not found`. THIS repo is itself the package
	# `autonomous-qa`, so inside this checkout `npx --yes autonomous-qa`
	# resolves to the LOCAL package.json and its bin `dist/cli/index.js` —
	# which is gitignored and absent in a fresh CI checkout. npx never
	# consulted the registry, so the step died before probing anything.
	# => here we `npm ci && npm run build` and run the LOCAL build on purpose:
	# the verifier gates using the verifier it just built (dogfood). A repo
	# that COPIES this file has no such collision and uses npx (see above).
	#
	# 2. `vars.API_QA_TARGET` / `vars.API_QA_SUITE_DIGEST` are not set in this
	# repo — this file is documentation here, not this repo's own gate. With an
	# empty target the suite fell back to its baked-in `prod` baseUrl
	# (`https://apis.directory`), a THIRD-PARTY surface knowingly sitting at the
	# AX floor. api.qa's `main` was therefore permanently red for someone
	# else's non-conformance. The deploy-path convention already established in
	# `deploy.yml` applies: when unconfigured, SKIP cleanly and say so.
	# NOTE the anti-Goodhart line — the fix is to stop grading an unconfigured
	# target, NOT to repoint the suite at something that happens to pass.
	#
	# 3. `dorny/test-reporter` defaults to `fail-on-empty: true`, so once (1)
	# guaranteed no `reports/api-qa.junit.xml` was ever written, the publish
	# step reported `No test report files were found` and failed on top of the
	# real failure. A step that cannot pass is not a gate, it is noise that
	# teaches everyone to ignore CI. It is now guarded on the file existing.

	on:
	push:
	branches: [main]
	pull_request:
	workflow_dispatch:

	permissions:
	contents: read

	jobs:
	verify:
	name: Verify agent-first contract
	runs-on: ubuntu-latest
	env:
	API_QA_TARGET: ${{ vars.API_QA_TARGET }}
	API_QA_SUITE_DIGEST: ${{ vars.API_QA_SUITE_DIGEST }}
	steps:
	- name: Checkout
	uses: actions/checkout@v4

	- name: Set up Node.js
	uses: actions/setup-node@v4
	with:
	node-version: '20'
	cache: npm

	# This repo IS `autonomous-qa`; gate with the build from this commit.
	- name: Install and build the verifier
	if: env.API_QA_TARGET != ''
	run: \|
	npm ci
	npm run build

	- name: Run the pinned api.qa suite (fails the check on regression)
	if: env.API_QA_TARGET != ''
	run: \|
	mkdir -p reports
	node dist/cli/index.js suite examples/golden-scenario.suite.json \
	--env prod \
	--target "${API_QA_TARGET}" \
	--expect-digest "${API_QA_SUITE_DIGEST}" \
	--reporter cli \
	--reporter junit --reporter-junit-out reports/api-qa.junit.xml \
	--reporter json --reporter-json-out reports/api-qa.json

	- name: Upload api.qa reports
	if: always()
	uses: actions/upload-artifact@v4
	with:
	name: api-qa-reports
	path: reports/
	if-no-files-found: ignore

	# Publishing is REPORTING — it must never be the thing that decides the
	# colour of this check. The suite step above is the gate and already
	# carries the verdict in its exit code. Three separate ways this step used
	# to (or would) redden a run independently of that verdict, all closed:
	# - `fail-on-empty` defaults to TRUE: the file is absent whenever the
	# gate is unconfigured, or when the suite step died before writing it.
	# That produced `No test report files were found` on every run.
	# - `fail-on-error` defaults to TRUE: the action RE-JUDGES the report and
	# fails the step whenever a testcase failed — double-counting a verdict
	# the suite step has already delivered. Measured on run 31096622566.
	# - a missing file is now also gated on by `hashFiles`, so the action is
	# never handed a path that does not exist.
	- name: Publish JUnit test report
	if: always() && hashFiles('reports/api-qa.junit.xml') != ''
	uses: dorny/test-reporter@v1
	with:
	name: api.qa results
	path: reports/api-qa.junit.xml
	reporter: java-junit
	fail-on-empty: false
	fail-on-error: false

	- name: Note when unconfigured
	if: env.API_QA_TARGET == ''
	run: \|
	echo "API_QA_TARGET is not set in this repo — the example gate is documentation here, so it SKIPPED."
	echo "To make it gate for real, set the repo variables:"
	echo " gh variable set API_QA_TARGET --repo <owner>/<repo> --body '<deployed url>'"
	echo " gh variable set API_QA_SUITE_DIGEST --repo <owner>/<repo> --body \"\$(npx autonomous-qa spec-digest <suite.json>)\""

Navigation Menu

feat(exec): enroll SUITE_LOADER — the hosted api.qa/vitest@1 runner goes live #23

Workflow file

feat(exec): enroll SUITE_LOADER — the hosted api.qa/vitest@1 runner goes live #23

Uh oh!

Workflow file for this run