Skip to content

[feature] Emit runner-info.xml with build SHA + JAR sha256 alongside JUnit output#51

Draft
joewiz wants to merge 1 commit into
eXist-db:developfrom
joewiz:feature/runner-info-emission
Draft

[feature] Emit runner-info.xml with build SHA + JAR sha256 alongside JUnit output#51
joewiz wants to merge 1 commit into
eXist-db:developfrom
joewiz:feature/runner-info-emission

Conversation

@joewiz
Copy link
Copy Markdown
Member

@joewiz joewiz commented May 10, 2026

Summary

Captures the runner JAR's git build SHA, sha256, and the embedded exist-core version into a sibling runner-info.xml written to the output directory root. The metadata is consumed by eXist-db/exist's compare-results.xslt to surface a comparison-warning element when two runs' runner builds differ.

Parent issue: eXist-db/exist#6326 (mitigation A).
Companion eXist PR: https://github.com/eXist-db/exist/pull/

Why

The same eXist source can produce dramatically different XQTS results when run against two different runner JAR builds. Concrete data point from a 2026-05-09 investigation, identical eXist SHA both runs:

May 9 runner JAR May 10 runner JAR (applyVersionHint cap-at-3.1 patch)
Pass rate 90.9% (23,250/25,574) 92.5% (24,081/26,014) -- +831 tests
Total wallclock 5,362s (89 min) 215s (3m 35s) -- 25x faster

Without metadata in the output, CI's compare-results transform attributes those deltas to the PR's source change. It's wrong. This patch is the first half of making such drift directly attributable.

A second drift mode surfaced during PR #6331's perf-6322 investigation: locally-built runner JARs shade exist-core, so two runs of the same runner-source git SHA can still differ if assembled from different exist-core snapshots. runner-info.xml captures both signals (runner JAR's git-sha + sha256, and embedded exist-core's version).

What changed

  • New RunnerInfo helper reads the runner JAR's META-INF/MANIFEST.MF for the fields already populated by Compile / packageBin / packageOptions (Git-Commit, Build-Tag, Build-Timestamp, ...), computes the JAR's sha256 via the existing Checksum utility, and reads the embedded exist-core's META-INF/maven/org.exist-db/exist-core/pom.properties for version + groupId + artifactId. All fields gracefully degrade to unknown=\"true\" when not available (e.g., when the runner is launched via sbt run from a class directory rather than an assembled JAR, or when a manifest field happens to be missing).
  • JUnitResultsSerializerActor captures the run's start timestamp at actor construction and the xqts-version + test count from incoming TestSetResults, then writes <output-dir>/runner-info.xml during FinalizeSerialization. Failures are logged but never propagated -- emitting metadata must not affect the run's exit status.

Sample output

<runner-info xmlns=\"http://exist-db.org/exist-xqts-runner/runner-info\">
    <runner-jar>
        <git-sha>1a5c00f3...</git-sha>
        <git-sha-abbrev>1a5c00f</git-sha-abbrev>
        <build-tag>v2.0.0-SNAPSHOT</build-tag>
        <build-version>N/A</build-version>
        <built-timestamp>20260510114215</built-timestamp>
        <built-by>ci</built-by>
        <build-jdk>21.0.5+11-LTS</build-jdk>
        <sha256>9c2d8e7f...</sha256>
        <jar-path>/.../exist-xqts-runner_2.13-2.0.0-SNAPSHOT.jar</jar-path>
    </runner-jar>
    <embedded-exist-core>
        <version>7.0.0-SNAPSHOT</version>
        <group-id>org.exist-db</group-id>
        <artifact-id>exist-core</artifact-id>
        <pom-properties-source>META-INF/maven/org.exist-db/exist-core/pom.properties</pom-properties-source>
    </embedded-exist-core>
    <run-info>
        <started>2026-05-10T11:32:15Z</started>
        <completed>2026-05-10T11:35:50Z</completed>
        <xqts-version>3.1</xqts-version>
        <test-count>26014</test-count>
    </run-info>
</runner-info>

Test plan

  • `sbt compile` clean (warnings unchanged from baseline -- all in pre-existing files).
  • Companion XSLT side hand-tested against synthetic `runner-info.xml` files in 5 scenarios (SHAs differ, SHAs match, missing on one side, malformed, embedded-exist-core-only drift) -- see eXist-side PR.
  • Smoke test: end-to-end XQTS run on next CI execution should produce `runner-info.xml` at the artifact root and round-trip through `actions/upload-artifact` automatically.

Risk and rollback

The change is purely additive. Reverting removes `runner-info.xml` emission; the runner runs normally otherwise. The companion XSLT side degrades to a no-op when `runner-info.xml` is absent, so the two PRs can land in either order.

@line-o
Copy link
Copy Markdown
Member

line-o commented May 20, 2026

This PR seems to depend on another PR?

@joewiz joewiz marked this pull request as draft May 20, 2026 13:10
@joewiz
Copy link
Copy Markdown
Member Author

joewiz commented May 20, 2026

[This response was co-authored with Claude Code. -Joe]

@line-o — yes, you're right: this branch is built on top of `feature/qt4-xquery-update` and effectively depends on the QT4 bundle (#49). Since #49 is being moved to draft pending the post-7.0 XQ 4 / XQUF / XQFT integration push (eXist 7 is narrowing to XQ 3.1, code freeze for the beta is later today), I'm moving this one to draft as well.

The runner-info.xml emission itself is suite-agnostic and could in principle be rebased onto `develop` for the 3.1 release, but that's a non-trivial rebase I'd rather not attempt today. Will revisit once #49 lands or split this out fresh against `develop` if there's appetite to include it in 7.0.

…JUnit output

Captures the runner JAR's git build SHA, sha256, and the embedded
exist-core version into a sibling `runner-info.xml` written to the
output directory root. Consumed by `eXist-db/exist`'s
`compare-results.xslt` to surface a `comparison-warning` element when
two runs' runner builds differ -- making runner-JAR drift directly
attributable in CI XQTS comparison reports.

Why: the same eXist source can produce dramatically different XQTS
results when run against two different runner JAR builds (concrete
data point from 2026-05-09: identical eXist SHA, +831 tests / 25x
wallclock difference between the May 9 runner and the May 10 runner
with the applyVersionHint cap-at-3.1 patch). Without metadata in the
output, CI's `compare-results` transform attributes those deltas to
the PR's source change, which is wrong.

What changed:
- New `RunnerInfo` helper reads the runner JAR's MANIFEST.MF
  (Git-Commit, Build-Tag, Build-Timestamp, ...), computes the JAR's
  sha256 via the existing `Checksum` utility, and reads the embedded
  exist-core's `META-INF/maven/org.exist-db/exist-core/pom.properties`
  for version + groupId + artifactId. All fields gracefully degrade
  to `unknown="true"` when not available (e.g., when the runner is
  launched via `sbt run` from a class directory rather than an
  assembled JAR).
- `JUnitResultsSerializerActor` captures the run's start timestamp at
  construction and the xqts-version + test count from incoming
  `TestSetResults`, then writes `<output-dir>/runner-info.xml` during
  `FinalizeSerialization`. Failures here are logged but never raised
  -- emitting metadata must not affect the run's exit status.

Companion eXist PR (the consumer of this metadata): TBD
Parent issue: eXist-db/exist#6326

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joewiz joewiz force-pushed the feature/runner-info-emission branch from 1a5c00f to c9da545 Compare May 30, 2026 19:21
@joewiz joewiz changed the base branch from feature/qt4-xquery-update to develop May 30, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants