Add #[ktstr_scenario]: a test's workload as an extractable value - #43
Closed
rrnewton wants to merge 1 commit into
Closed
Add #[ktstr_scenario]: a test's workload as an extractable value#43rrnewton wants to merge 1 commit into
rrnewton wants to merge 1 commit into
Conversation
A ktstr test's workload is arbitrary Rust that happens to end in
`execute_steps(ctx, steps)`. The SHAPE of most of those bodies is
declarative — a list of steps — but nothing in the type system says
so, and nothing can recover the step list without booting a guest and
running the test. That is a shame for anything that wants to READ a
workload rather than execute it.
Introduce `ScenarioDef`: the step list (plus the optional `Assert`
override `execute_steps_with` takes) lifted out of the body into a
value that can be built, inspected and printed on the host with no VM
and no `&Ctx`. `ScenarioDef::run` dispatches to `execute_steps_with`,
so there is no second execution engine — a scenario runs down exactly
the path a hand-written body runs down.
Add `#[ktstr_scenario]` as the entrypoint that consumes one. It does
NOT fork `#[ktstr_test]`: it emits the author's function renamed, a
canonical `fn() -> ScenarioDef` builder that coerces the return value
via `Into` (so a body may return `ScenarioDef`, `Vec<Step>` or a bare
`Step`), a `ScenarioEntry` registration in the new `KTSTR_SCENARIOS`
slice, and a synthesized `fn <name>(ctx: &Ctx) -> Result<AssertResult>`
that it hands to `ktstr_test_impl` unchanged. Attribute parsing,
cross-attribute validation and every existing registration are
therefore literally the same code, and any future `#[ktstr_test]`
field is inherited by construction. `delegates_to_ktstr_test_verbatim`
pins that: the scenario expansion must CONTAIN `ktstr_test_impl`'s own
output for the runner, token for token.
The function is what is restricted, not the attribute grammar. No
parameters — in particular no `&Ctx`, which is what makes the body
host-buildable; no `async`, generics or `where` clause, which the
generated builder could not call; and no `post_vm` /
`post_vm_unconditional`, since a host-side callback is the arbitrary
Rust this entrypoint exists to exclude. Each rejection names its
reason and points at `#[ktstr_test]`.
The `&Ctx` restriction costs less than it looks. Nearly every `&Ctx`
use in an existing scenario body is `ctx.cgroup_def(n)`, which is
`CgroupDef::named(n).workers(ctx.workers_per_cgroup)` — and a
`CgroupDef` that leaves its worker count unset resolves through the
same default anyway (empty `works` -> `WorkSpec::default()` ->
`num_workers: None` -> `resolve_num_workers`'s
`unwrap_or(ctx.workers_per_cgroup)`). The two spell the same workload;
the ctx-free one just leaves it symbolic for the runner to bind, which
is the more useful form for a value that is meant to describe a
workload rather than one host's instance of it.
`Setup::Factory(fn(&Ctx) -> Vec<CgroupDef>)` remains the genuine
escape hatch, and a `ScenarioDef` may legally hold one. Rather than
pretend otherwise, `is_declarative()` reports it, so a consumer reading
scenarios as data skips such a scenario deliberately instead of
silently believing an empty cgroup list.
Port the five pure-scenario tests in `tests/ktstr_sched_tests.rs`
(`sched_basic_proportional`, `sched_cpuset_split`, `sched_dynamic_add`,
`sched_verifier_stats_populated`, `sched_perf_positive` — the last
exercising the checks path). Their attributes, topology, gates and
runtime behaviour are unchanged; the bodies lose 54 lines of `Step {
setup: .., ops: vec![], hold: .. }` boilerplate.
Also correct a stale note in that file claiming plain `#[test]`
functions in a `KtstrTestEntry`-registering binary are invisible to the
runner. `list_plain_tests` re-emits them; verified against this
binary's own `NEXTEST=1 --list` output, and the new
scenario-extraction tests at the end of the file are plain `#[test]`s
that do run.
Test: 7 `ScenarioDef` unit tests, 7 macro-expansion tests (incl. the
verbatim-delegation pin), 3 host-side extraction tests over the ported
set, 2 compile-fail fixtures. fmt / clippy (both feature sets) /
doc-strict / docs.rs-mode all clean; the full compile_fail suite passes
with no existing snapshot disturbed.
This was referenced Aug 12, 2026
Author
|
Superseded by #47, which contains this work plus the rest of the series in one branch, rebuilt on current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
A ktstr test's workload is arbitrary Rust that happens to end in
execute_steps(ctx, steps). The shape of most of those bodies is declarative— a list of steps — but nothing in the type system says so, and nothing can
recover the step list without booting a guest and running the test. That is a
shame for anything that wants to read a workload rather than execute it.
This adds
ScenarioDef(the step list, plus the optionalAssertoverrideexecute_steps_withtakes, lifted out of the body into a value) and#[ktstr_scenario], the restricted entrypoint that consumes one. Five existingtests are ported to it.
It does not fork
#[ktstr_test]That is the main design constraint.
#[ktstr_scenario]emits three items andhands the third to
ktstr_test_implverbatim:__ktstr_scenario_body_<name>, body andattributes untouched;
__ktstr_scenario_def_<name>() -> ScenarioDef, coercing (1)'s return valuevia
Into(so a body may returnScenarioDef,Vec<Step>or a bareStep), plus aScenarioEntryregistration in the newKTSTR_SCENARIOSslice;
fn <name>(ctx: &Ctx) -> Result<AssertResult> { <def>().run(ctx) }, passedto
ktstr_test_implwith the attribute list unchanged.So attribute parsing, every cross-attribute validation, and every existing
registration (
KTSTR_TESTS, the manifest stamp, both admission stamps, the#[test]wrapper) are literally the same code — any future#[ktstr_test]field is inherited by construction.
ScenarioDef::rundispatches toexecute_steps_with, so there is no second execution engine either.delegates_to_ktstr_test_verbatimpins exactly that: the scenario expansionmust contain
ktstr_test_impl's own output for the synthesized runner, tokenfor token, over a broad attribute set. That is the equivalence evidence — no
field-by-field list to keep in sync, and an attempt to intercept or rewrite the
delegation breaks the test immediately.
What is restricted is the function, not the attribute grammar
ctx: &Ctx; that is what makes the bodyhost-buildable.
async, not generic, nowhereclause — the generated builder is aplain
fn() -> ScenarioDef.post_vm/post_vm_unconditionalrejected — a host-side callback isthe arbitrary Rust this entrypoint exists to exclude. Use
#[ktstr_test].Each rejection names its reason and points at
#[ktstr_test]; two are pinnedas
compile_failfixtures.The
&Ctxrestriction costs less than it looksNearly every
&Ctxuse in an existing scenario body isctx.cgroup_def(n),which is
CgroupDef::named(n).workers(ctx.workers_per_cgroup). ACgroupDefthat leaves its worker count unset resolves through the same default anyway:
empty
works→WorkSpec::default()→num_workers: None→resolve_num_workers'sunwrap_or(ctx.workers_per_cgroup). The two spell thesame workload; the ctx-free one just leaves it symbolic for the runner to bind,
which is the more useful form for a value meant to describe a workload rather
than one host's instance of it.
Honest boundary
Setup::Factory(fn(&Ctx) -> Vec<CgroupDef>)remains the genuine escape hatch,and a
ScenarioDefmay legally hold one. Rather than pretend otherwise,is_declarative()reports it, so a consumer reading scenarios as data skipssuch a scenario deliberately instead of silently believing an empty cgroup
list.
every_registered_scenario_is_declarativeasserts it over the wholeregistry.
Naming
ktstr_scenariois new public surface and the name is open to review. I keptit over the alternatives (
ktstr_workload,declarative_test,scenario_test)because it pairs with the
ScenarioDef/ScenarioEntry/KTSTR_SCENARIOSvocabulary and reads as a sibling of
ktstr_test. Its one weakness is that itdoes not say "test" — happy to rename; it is two non-derived identifiers plus
re-blessing two
.stderrsnapshots.Test plan
ScenarioDefunit tests (src/scenario/def/tests.rs).ktstr-macros), including the verbatim-delegationpin and the rejection diagnostics.
values against independently written expectations — cgroup names and order,
disjoint cpuset indices, per-step
0.5holds, andsched_perf_positive'smin_iteration_rate/max_gap_mssurviving into the extracted checks.compile_failfixtures; the full trybuild suite passes with no existingsnapshot disturbed.
Ran green locally:
cargo checkandcargo clippyon both feature sets,cargo fmt --check, rustdoc-warnings-as-errors, the docs.rs-mode check,just compile-fail,just devdep-isolation.Not run: the KVM integration suite. My host cannot run it — ktstr's content
cache hard-requires a reflink (
cache/content.rsFICLONE, noEXDEVfallback), and here
/usr/bin/false(on/) and~/.cache/ktstr(on/home) are separate btrfs filesystems, so scheduler-artifact publicationfails with
Invalid cross-device linkbefore any test runs. I confirmed thisis not caused by this branch by running the identical command on an untouched
checkout of the base commit — it fails byte-identically. CI's KVM runners
should exercise the five ported tests normally. (
just link-checklikewiseonly fails here for a missing
mdbook; this diff touches no guide content.)Note on an incidental comment fix
tests/ktstr_sched_tests.rscarried a note claiming plain#[test]functionsin a
KtstrTestEntry-registering binary are invisible to the runner.list_plain_testsre-emits them; verified against that binary's ownNEXTEST=1 --listoutput. Corrected, and the new extraction tests are plain#[test]s in that file that do run.