Skip to content

ci(pr-automation): compare APIs using locked dependencies - #2071

Open
AKolenda wants to merge 2 commits into
Devolutions:masterfrom
AKolenda:fix/pr-semver-locked-dependencies
Open

AKolenda wants to merge 2 commits into
Devolutions:masterfrom
AKolenda:fix/pr-semver-locked-dependencies

Conversation

@AKolenda

@AKolenda AKolenda commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Build baseline and head rustdoc JSON with cargo rustdoc --locked, then compare those artifacts with the existing pinned cargo-semver-checks binary. Each revision keeps its own committed dependency versions. The unprivileged nobody execution, cleared environment, tool checksum, feature selection, and compatibility exit-code handling are unchanged.

Why

The original trigger is gone: picky-krb 0.12.5 added a GssApiMessageError variant that broke sspi 0.21.3, which made the API check fail for every open PR while ordinary CI passed on the lockfile's 0.12.4. #2074 capped it on master and 0.12.5 was later yanked (re-released as 0.13.0).

The cause is still there. With --baseline-rev, cargo-semver-checks builds each revision in a placeholder crate and resolves a fresh dependency graph from the registry, ignoring Cargo.lock. Any future patch release in the dependency tree that breaks the build therefore fails the API check for all PRs until someone caps or pins it on master, and because this is a pull_request_target workflow the repair only takes effect after it lands on the base branch. Building with --locked makes the API check use the same dependency versions that ordinary CI builds and tests, and makes reruns of the same SHAs reproducible. cargo-semver-checks 0.51.0 has no lockfile option, so building the rustdoc JSON ourselves and passing --baseline-rustdoc/--current-rustdoc is the way to get this.

A stale or inconsistent lockfile now fails the job (exit 101) instead of being silently re-resolved.

Validation

  • Rebased on current master (cargo-semver-checks 0.51.0).
  • actionlint 1.7.7 (with shellcheck 0.10.0 on run: blocks): no new findings; the remaining findings are identical on master (concurrency.queue and node24 are newer than actionlint knows).
  • shellcheck on the extracted inner bash -c script: clean.
  • Inner script with stubbed git/cargo/comparator: checks out base then head, builds into <root>/baseline and <root>/current, passes those JSON paths to the comparator, and propagates comparator 0/100 and cargo 101 exit codes.
  • node --test .github/pr-automation/automation.test.js: 187/187 pass.
  • Earlier, on the original revision: a real IronRDP cargo rustdoc --locked build with the full workflow feature set passed and the pinned comparator parsed and compared the JSON; fixtures showed unchanged API exits 0, a removed root function exits 100, and a stale lockfile exits 101 without modifying the lockfile. The nobody boundary was inspected, not executed locally.

The comparator's existing inability to detect some changes through dependency re-exports reproduces with both its placeholder mode and this JSON mode; that limitation is unchanged.

Copilot AI lite review requested due to automatic review settings October 2, 2026 06:34
@AKolenda
AKolenda requested review from a team as code owners October 2, 2026 06:34
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@chatgpt-codex-connector

Copy link
Copy Markdown

The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved review issues remain.

Review effort: Lite
Findings: None

What changed in this PR

Updates PR automation to compare public APIs using rustdoc artifacts generated with each revision’s locked dependencies.

Changes:

  • Builds baseline and head rustdoc JSON with cargo rustdoc --locked.
  • Compares artifacts using the pinned semver checker.
  • Documents the locked-dependency workflow.
File Summary
.github/​workflows/​pr-automation.yml Builds and compares locked rustdoc artifacts.
.github/​PR_AUTOMATION.md Documents the updated compatibility check.

@CBenoit

Benoît Cortier (CBenoit) commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

PR automation is failing because of a picky-krb 0.12.5 incompatibility, fixed on master by #2074. Please rebase on master to fix it.

Update: no rebase needed anymore. picky-krb 0.12.5 was yanked from crates.io (re-released as 0.13.0), so the API check builds again without changes to this branch. PR automation has been re-run here and passes.

@github-actions github-actions Bot added risk/low Self-contained change with no cross-crate behavioral effect triage/overlap Possible overlap with another pull request; advisory only and removed risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny automation-failed Exact-head automated classification or review failed or was unavailable labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

This pull request may overlap with #2103.

Both pull requests target the PR automation CI: this one reworks the cargo-semver-checks step in .github/workflows/pr-automation.yml, while #2103 adds retry-by-comment handling for the same automation workflow. They touch the same CI surface but address different features.

This notice is advisory only. Automated review continues as usual, and how these pull requests relate is for maintainers and authors to decide.

Note

LLM-assisted content (no human feedback).

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI-only change to the pull_request_target pr-automation workflow: the semver job now builds rustdoc JSON for baseline and head with `cargo rustdoc --locked` in an inner bash script run as nobody under `env -i`, then compares the two artifacts with the existing checksum-pinned cargo-semver-checks binary. This fixes real failures caused by the comparator's placeholder crate resolving a fresh (unlocked) dependency graph, while preserving the unprivileged boundary, cleared environment, feature selection, and exit-code handling (0/100 mapping). The accompanying PR_AUTOMATION.md note documents the locked-baseline behavior. The change is correct and safe: SHA/feature values flow through env passthrough rather than interpolation into the script text, `RUSTC_BOOTSTRAP=1` is required for the unstable rustdoc flags on the stable toolchain, and per-revision `CARGO_TARGET_DIR` values keep artifacts separated with paths matching what the comparator is given. The single accepted candidate is a low-s…

Comment thread .github/workflows/pr-automation.yml Outdated
@github-actions github-actions Bot added ai-reviewed/1 One automated review completed needs-author-action The pull request author is the current next actor labels Oct 7, 2026
@AKolenda
AKolenda force-pushed the fix/pr-semver-locked-dependencies branch from 2455b39 to c6cdee2 Compare October 10, 2026 05:17
@AKolenda
AKolenda deployed to llm-providers October 10, 2026 05:18 — with GitHub Actions Active
@github-actions github-actions Bot removed the needs-author-action The pull request author is the current next actor label Oct 10, 2026
Build each revision through a small helper called once per SHA and pass the artifact root explicitly instead of capturing CARGO_TARGET_DIR before overwriting it.
@AKolenda
AKolenda deployed to llm-providers October 10, 2026 05:20 — with GitHub Actions Active

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR reworks the semver step of the pull_request_target PR automation workflow to build baseline and head rustdoc JSON itself via `cargo rustdoc --locked`, then compares those artifacts with the pinned cargo-semver-checks binary using --baseline-rustdoc/--current-rustdoc. This is a correct and well-motivated change: cargo-semver-checks 0.51.0 has no lockfile option, so its --baseline-rev placeholder-crate mode re-resolves dependency graphs from the registry and can fail the API check on unrelated registry patch releases (the picky-krb 0.12.5 incident). Building with --locked aligns the API check with ordinary CI's dependency versions and makes reruns reproducible. The sandbox boundary is preserved: the `sudo -u nobody env -i` invocation still lists every variable explicitly, SHAs arrive only through the environment, and the comparator's 0/100/101 exit-code handling is unchanged. The PR_AUTOMATION.md doc is updated consistently. I verified the sandboxed inner script lines against the…

Push a commit after addressing these findings. If no code change is needed, you may resolve inline threads and comment @github-actions review-ready to request human review.

Comment on lines +409 to +410
SEMVER_BIN="$semver_bin" \
SEMVER_FEATURES="$CARGO_SEMVER_CHECKS_FEATURES" \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[code-compressor] Inner-script env vars SEMVER_BIN and SEMVER_FEATURES needlessly rename already-named values — low 🟡 — The added inner script receives two values under new aliases: SEMVER_BIN="$semver_bin" and SEMVER_FEATURES="$CARGO_SEMVER_CHECKS_FEATURES". SEMVER_FEATURES is a straight rename of the job-level env var CARGO_SEMVER_CHECKS_FEATURES, forcing the reader to trace a mapping that conveys no information. Passing the values under their existing canonical names preserves identical behavior: the env -i invocation already requires listing every variable explicitly, so nothing about the sandbox, exit-code handling, or feature set changes. SEMVER_ARTIFACT_ROOT is a legitimate rename and HEAD_SHA/BASE_SHA must keep their names for the existing output/notice logic. Optional compression only; no behavioral impact.

@github-actions github-actions Bot added ai-reviewed/2 Two automated reviews completed needs-author-action The pull request author is the current next actor and removed ai-reviewed/1 One automated review completed labels Oct 10, 2026

This branch was successfully deployed

1 active deployment
llm-providers — 3f0d8c48 Deployed Oct 10, 2026 by AKolenda via Classify pull request #2430
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-reviewed/2 Two automated reviews completed needs-author-action The pull request author is the current next actor risk/low Self-contained change with no cross-crate behavioral effect scope/tooling Build, CI, release, or developer tooling size/XS Size: up to 49 counted lines and 2 files triage/overlap Possible overlap with another pull request; advisory only

Development

Successfully merging this pull request may close these issues.

3 participants