ci(profile-bench): compile only the modes each matrix job benchmarks - #282
Draft
andrii-lz wants to merge 10 commits into
Draft
ci(profile-bench): compile only the modes each matrix job benchmarks#282andrii-lz wants to merge 10 commits into
andrii-lz wants to merge 10 commits into
Conversation
|
Warning This PR has more than 500 changed lines and does not include a spec. Large features and architectural changes benefit from a short spec before implementation review. If this PR is a bug fix, small cleanup, or otherwise does not warrant a spec, feel free to ignore this message. |
Documentation blast radius (advisory)These regions may need doc/spec/book updates based on changed paths. Changed files in this PR: 6
|
Benchmark Report
Detailed per-level schedule and proof-size breakdowns are available in the uploaded |
CI test timing
Run summary
Wall time spans 2 parallel nextest slice shards. Slowest tests
Regressions vs mainNo per-test regressions above the threshold. New slow testsNo new tests ≥30s vs main baseline. |
…file modes Narrow profile-ci-registry builds (one mode-* feature enabled) leave the other CI wrapper functions and their sole call chains unreferenced, which -D warnings (set globally in profile-bench.yml) promotes from a warning to a hard build error. Reachability here is a property of which mode-* features are enabled, not of the code being actually unused, so allow dead_code file-wide in the two affected example files.
CI now builds each matrix group from that group's own pcs_mode_features list, not from the umbrella profile-ci feature. A mode covered by the umbrella is not necessarily covered by its own group's narrower feature list, so a missing mode-* feature on a group would previously pass this gate and only surface as a runtime "unknown mode" failure. Add a per-group check that every bench case's mode is covered by that same group's pcs_mode_features.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
andrii-lz
force-pushed
the
worktree-profile-bench-per-mode-ci
branch
from
July 7, 2026 22:18
0ed42ee to
0bfaabc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
profile-bench.yml's bench matrix jobs all compiled the same full 8-modeprofileexample binary, even though each job only benchmarks 1-3 of those modes. That single compile unit accounted for ~94 of ~107 seconds of build time (measured locally withcargo build --timings), and it was paid redundantly by every job — twice in most jobs (PR + merge-base builds).akita-pcs's blanketprofile-ciCargo feature into 8 per-mode features + aprofile-ci-registrymarker, socrates/akita-pcs/examples/profile/modes.rs's mode registry only registers (and therefore only monomorphizes) the modes a build actually enables.profile-cistill enables all 8, unchanged for anyone using it directly.profile-bench.ymlto build with only themode-*features its own cases need, with a 3-tier fallback on the merge-base build for commits that predate this change.-D warnings(set globally in this workflow) promotes fromdead_codewarning to a hard error. Scoped#![allow(dead_code)]to the two affected example files only.scripts/check_profile_ci_features.shwith a per-group coverage check, since CI no longer builds from the umbrellaprofile-cifeature — a group missing a neededmode-*feature would previously pass this gate and only fail at runtime.Measured impact (local timing, clean builds)
Max across all jobs (the effective CI wall-clock floor for the
benchphase, since jobs runfail-fast: falsein parallel) drops from ~107s to ~48s — a ~55% reduction, not just aggregate runner-minutes.Test plan
RUSTFLAGS="-D warnings"(matching this workflow's actual flag)--features parallel,profile-cibuild still produces the same 8-mode registry and builds clean under-D warningsscripts/check_profile_ci_features.shpasses (umbrella coverage + new per-group coverage), including a deliberate-break test confirming the new per-group check catches a missing featurescripts/check_profile_ci_linkage.shpasses against a per-group binaryyaml.safe_loadprofile-bench.ymlbehavior on GitHub's runners)🤖 Generated with Claude Code