tests: capture a wprof trace of sched_basic_proportional - #44
Closed
rrnewton wants to merge 6 commits into
Closed
Conversation
A ktstr test's workload is arbitrary Rust that happens to end in
`execute_steps(ctx, steps)`. The SHAPE of most of those bodies is
declarative — a list of steps — but nothing in the type system says
so, and nothing can recover the step list without booting a guest and
running the test. That is a shame for anything that wants to READ a
workload rather than execute it.
Introduce `ScenarioDef`: the step list (plus the optional `Assert`
override `execute_steps_with` takes) lifted out of the body into a
value that can be built, inspected and printed on the host with no VM
and no `&Ctx`. `ScenarioDef::run` dispatches to `execute_steps_with`,
so there is no second execution engine — a scenario runs down exactly
the path a hand-written body runs down.
Add `#[ktstr_scenario]` as the entrypoint that consumes one. It does
NOT fork `#[ktstr_test]`: it emits the author's function renamed, a
canonical `fn() -> ScenarioDef` builder that coerces the return value
via `Into` (so a body may return `ScenarioDef`, `Vec<Step>` or a bare
`Step`), a `ScenarioEntry` registration in the new `KTSTR_SCENARIOS`
slice, and a synthesized `fn <name>(ctx: &Ctx) -> Result<AssertResult>`
that it hands to `ktstr_test_impl` unchanged. Attribute parsing,
cross-attribute validation and every existing registration are
therefore literally the same code, and any future `#[ktstr_test]`
field is inherited by construction. `delegates_to_ktstr_test_verbatim`
pins that: the scenario expansion must CONTAIN `ktstr_test_impl`'s own
output for the runner, token for token.
The function is what is restricted, not the attribute grammar. No
parameters — in particular no `&Ctx`, which is what makes the body
host-buildable; no `async`, generics or `where` clause, which the
generated builder could not call; and no `post_vm` /
`post_vm_unconditional`, since a host-side callback is the arbitrary
Rust this entrypoint exists to exclude. Each rejection names its
reason and points at `#[ktstr_test]`.
The `&Ctx` restriction costs less than it looks. Nearly every `&Ctx`
use in an existing scenario body is `ctx.cgroup_def(n)`, which is
`CgroupDef::named(n).workers(ctx.workers_per_cgroup)` — and a
`CgroupDef` that leaves its worker count unset resolves through the
same default anyway (empty `works` -> `WorkSpec::default()` ->
`num_workers: None` -> `resolve_num_workers`'s
`unwrap_or(ctx.workers_per_cgroup)`). The two spell the same workload;
the ctx-free one just leaves it symbolic for the runner to bind, which
is the more useful form for a value that is meant to describe a
workload rather than one host's instance of it.
`Setup::Factory(fn(&Ctx) -> Vec<CgroupDef>)` remains the genuine
escape hatch, and a `ScenarioDef` may legally hold one. Rather than
pretend otherwise, `is_declarative()` reports it, so a consumer reading
scenarios as data skips such a scenario deliberately instead of
silently believing an empty cgroup list.
Port the five pure-scenario tests in `tests/ktstr_sched_tests.rs`
(`sched_basic_proportional`, `sched_cpuset_split`, `sched_dynamic_add`,
`sched_verifier_stats_populated`, `sched_perf_positive` — the last
exercising the checks path). Their attributes, topology, gates and
runtime behaviour are unchanged; the bodies lose 54 lines of `Step {
setup: .., ops: vec![], hold: .. }` boilerplate.
Also correct a stale note in that file claiming plain `#[test]`
functions in a `KtstrTestEntry`-registering binary are invisible to the
runner. `list_plain_tests` re-emits them; verified against this
binary's own `NEXTEST=1 --list` output, and the new
scenario-extraction tests at the end of the file are plain `#[test]`s
that do run.
Test: 7 `ScenarioDef` unit tests, 7 macro-expansion tests (incl. the
verbatim-delegation pin), 3 host-side extraction tests over the ported
set, 2 compile-fail fixtures. fmt / clippy (both feature sets) /
doc-strict / docs.rs-mode all clean; the full compile_fail suite passes
with no existing snapshot disturbed.
PR likewhatevs#43 made a test's workload a value. This makes it a value that can leave the process, so the same definition can drive the VM backend and the simulator instead of being written twice. `export_scenario` renders a `ScenarioDef` plus its `KtstrTestEntry` topology and duration as a `SourceScenario`-shaped JSON record — the input to scx-sim's `scxsim-workload-ir`, which lowers it to a restricted IR the simulator ingests. A new `export_registered_scenarios` test walks the real `KTSTR_SCENARIOS` registry and writes one record per scenario to `$KTSTR_SCENARIO_EXPORT_DIR`, doing nothing when that is unset. The records are therefore derived from the registry, not restated by hand, which is the only thing that makes a simulator run from one of them the SAME scenario. JSON rather than a Cargo dependency because ktstr and scx-sim are separate repositories and the dependency runs ktstr -> scx-sim; a path dependency between two checkouts would bake a host-specific path into a committed manifest. The schema is not maintained by agreement: the consumer deserialises with the real `SourceScenario` type, so an exporter that drifts produces a hard serde error naming the field rather than a workload that quietly means something else. Nothing is approximated here. A construct the record cannot carry faithfully is OMITTED and reported as an `ExportGap` naming what and why — unmapped work types, `Setup::Factory` setups, payloads, ops, and an `Assert` override (the record carries the workload, not the oracle). Approximation is the lowering's job, where it is classified and reported in the IR's fidelity report; an approximation invented at export time would be invisible to that machinery and would reach the simulator disguised as an exact reading of the test. `unmapped_work_type_is_a_gap_not_a_substitution` is the test that fails if someone later makes the exporter "more useful" by falling back to spin_wait. `workers_per_cgroup` is recorded explicitly rather than left implicit, because `num_workers: None` means "inherit" and two backends inheriting different defaults is exactly the divergence that makes a cross-backend comparison meaningless. The value 1 is `Ctx::builder`'s default, confirmed against a real VM run's stats sidecar. Test Results Summary: 7 new export unit tests, all pass. The dump test exports all 5 registered scenarios (4 clean, 1 reporting its Assert override as a gap). fmt and clippy clean on the workspace.
Adds `sched_basic_proportional_wprof`, a #[cfg(feature = "wprof")] sibling of
`sched_basic_proportional` that attaches wprof capture to the same workload.
A passing run writes {sidecar_dir}/{name}-{variant_hash:016x}.wprof.pb — a
Perfetto protobuf — beside the stats JSON the scx-sim calibration already
consumes, which is what makes a live-vs-simulated trace comparison possible.
Both tests now build their ScenarioDef from one shared
`sched_basic_proportional_scenario()` helper rather than restating it, so the
traced workload cannot drift from the calibrated one.
Kept as a cfg-gated sibling rather than adding `wprof` to the existing test:
the attribute requires the `wprof` cargo feature and the macro rejects it at
parse time when the feature is off, so putting it on the original would break
the default build of this test file for everyone who does not want a build
that clones and compiles wprof from GitHub.
The default -d 500 captures 500ms from guest init, which is boot, not the 12s hold. A measured default run produced a 0.495s trace containing only init/swapper/rcu/kworker and zero workload tasks.
wprof's --kthread 'Allow kernel tasks' and --idle 'Allow idle tasks' gate whether those tasks are traced. Without them the kthread accounting may be incomplete, which matters because quantifying unmodelled kernel-thread CPU time is the point of the capture.
Measured, not assumed: adding them dropped 24.0s of userspace on-CPU time from the trace and shrank it 587KB -> 189KB. Recorded at the call site so the next reader does not repeat the experiment.
Author
|
Superseded by #47, which contains this work plus the rest of the series in one branch, rebuilt on current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
sched_basic_proportional_wprof, a#[cfg(feature = "wprof")]sibling ofsched_basic_proportionalthat attaches wprof capture to the same workload. A passing run writes a Perfetto.pbbeside the stats JSON, which is what makes a live-vs-simulated trace comparison possible.Both tests build their
ScenarioDeffrom one shared helper rather than restating it, so the traced workload cannot drift from the calibrated one.Why a cfg-gated sibling rather than
wprofon the existing testThe attribute requires the
wprofcargo feature and the macro rejects it at parse time when the feature is off, so putting it onsched_basic_proportionalwould break the default build of this test file for everyone who does not want a build that clones and compiles wprof from GitHub.Two things measured rather than assumed, both recorded at the call site
The default capture misses the workload entirely.
WprofConfig::default_argsis-d 500— 500 ms — and guest init spawns the tracer at boot. A default-args run produced a 0.495 s trace containinginit,swapper,rcu_preempt,kworker,wprof-captureand not one workload task, stopping ~11.5 s before the 12 s hold. Hence-d 15000, which spans boot plus the whole hold while still finishing inside the ~19 s test so the trace is shipped host-side before teardown.--kthread --idleare not additive — they delete the workload. Their help text ("Allow kernel tasks" / "Allow idle tasks") invites the assumption that they widen the trace. Measured on this exact scenario, adding them dropped 24.0 s of userspace on-CPU time, took the trace from 587 KB to 189 KB and 10777 to 3034 packets, and moved kthread time down (4.080 → 2.745 ms). The commit history contains that experiment and its revert; the final args are the ones that contain the workload.wprof_argsreplaces the defaults rather than appending, so the ringbuf flags are restated at their default values. Sizing check: 0.495 s of boot — the busiest phase — produced 36 KB, so 15 s of mostly-steady-state spinning stays far inside the 16 MiB arena.What the trace was used for
Downstream in
sched-test, compared against the simulator running the same scenario:The trace's workload total cross-validates against the same run's stats sidecar (24012.9 ms, measured via the workers' own
CLOCK_THREAD_CPUTIMErather than wprof's sched tracing) to within 0.12%.Test plan
cargo ktstr test --features wprof --test ktstr_sched_tests sched_basic_proportional_wprof→ PASS, 1 stats sidecar + 1 wprof trace written. Requiresmoldand cargo-nextest ≥ 0.9.143. Default builds (nowproffeature) are unaffected — the new test is compiled out.