Source contracts:
sweeper/replay.py,
stack discovery, and the
built-in runner.
class RunnerFactory(Protocol):
def capabilities(self) -> RunnerCapabilities: ...
def create(self, worker_id: int) -> Runner: ...
class Runner(Protocol):
def run(
self,
spec: ReplaySpec,
*,
output_requirements: ReplayOutputRequirements | None = None,
) -> ReplayReport: ...
def close(self) -> None: ...A factory must be serializable for spawned workers. Each worker creates one
runner and reuses it across candidate replays; worker-local state must not leak
one candidate's requests or caches into another. close() releases resources.
The engine factory lazily imports the compiled runtime in its worker.
Optional factories are selected by entry-point name in
aisim.runner_factories (--stack). Resolution validates capabilities
and create; unavailable, duplicate or unloadable optional stacks raise
StackNotFoundError, DuplicateStackError or StackLoadError. A factory's
coarse capabilities are checked before runner creation; execution must still
validate finer feature combinations.
REPLAY_SPEC_API_VERSION is 1. ReplaySpec contains:
| Field | Meaning |
|---|---|
backend_deployment |
Concrete backend/version, topology, role engine arguments, worker counts, parallel shape and exact estimator identities |
workload |
Concrete source/load/stop fields; strict data crossing the process boundary |
goal |
Evaluation/optimization requirements including optional SLA |
concurrency |
Closed-loop source cap or None for open-loop arrivals |
adapters |
Adapter-owned AdapterReplaySpec values |
api_version |
Must equal the runner's replay-spec version |
execution_mode |
offline by default; online only when advertised |
runtime_hooks flattens adapter hook tuples in adapter insertion order.
Each RuntimeHookSpec declares (provider, kind, api_version, config).
Compatibility matches all three identity fields, with exact integer version
checks. Config/payload values must be strict JSON: finite numbers, strings,
booleans, null, lists and string-keyed dicts. canonical_json provides
stable ordering for serialization/cache keys; it does not license arbitrary
Python objects inside adapter payloads.
RunnerCapabilities declares backend/topology pairs, execution modes, trace
formats and hook identities. Pair/trace declarations may use "*" where the
contract allows it. More specific flags cover disaggregated attention-DP,
AgentX backends/topologies/lanes/snapshots/warmup/profiles, offload/speculation,
analytical EPD and its deferred overlay, cached-prefix tokens, state/grouped
cache and model controls.
Generic flag defaults are not qualification evidence for an optional stack.
supported_worker_types defaults to ("monolithic",); a runner must explicitly
include "afd" to accept AFD roles. The built-in engine advertises both types.
Deployment modes are agg|disagg, independent of role implementation. Ordinary
and native AFD deployments retain ReplaySpec API 1. Role-local AFD topology and
qualification artifacts use their own schema 2; this does not require downstream
runners to implement AFD or upgrade the common ReplaySpec protocol. The default
monolithic-only capability rejects AFD roles before execution.
Old afd/afd+pd deployment modes and development API-2 saved inputs must be
regenerated. API-version checks remain exact; API-2 inputs are not silently
converted to API 1.
supports_generic_speculation defaults to False; the built-in engine factory
sets it to True within the registered-scheme replay limits.
Unsupported stacks reject generic scheme inputs before runner creation or
resource preparation. supports_mtp_expected_acceptance only declares handling
of MTP/NextN acceptance inputs; it does not establish support for EAGLE or other
schemes' draft costs. A stack must explicitly declare generic speculation
support and retain those costs through its timing provider.
supports_mtp_host_offload defaults to False and declares the combined
MTP + G2 contract at any attention DP: aggregated workers with a rank-local
pool, and prefill and decode workers with a local or a shared pool. The built-in engine advertises it; existing external runners,
including Dynamo, must opt in only after their runtime and dependency pins
support the same composition. Generic speculation or G2 support alone does
not enable it.
require_compatible(spec) checks versions and coarse combinations before
execution. AgentX G2 additionally requires offline vLLM and static workers. G3
cannot be combined with AgentX G2. Speculation with G2 is limited to canonical
MTP with prefix caching and synchronous scheduling, and to MTP on a
hybrid G2 with
disable_eagle_block_drop. The feature matrix records the built-in
stack's actual boundary, including limitations stricter than a capability flag.
If every backend/topology pair fails Sweeper preflight, RunnerIncompatibleError
(subclass of NoViableParallelConfig) is raised before trials, with no
serialized SweepResult. Mixed incompatibility and model/memory/data failures
retain NoViableParallelConfig with diagnostics. The CLI reports configuration
errors as exit status 2. Candidate-specific build, replay, budget and timeout
failures are recorded as infeasible trials by Sweeper.
ReplayReport contains metrics and JSON metadata. Non-power metrics must
be finite numeric values; null is allowed only for designated power fields.
Report the normalized metric semantics used by scoring, and retain provenance
and limitations rather than substituting zero for unavailable measurements.
ReplayOutputRequirements defaults to summary output and can request raw,
per-request, telemetry, memory or timing diagnostics. A runner must reject
unsupported capture combinations. The Engine JSON runner supports telemetry for
offline vLLM/SGLang aggregated and separate prefill/decode replay without encoder
pools. capture_telemetry retains all-phase detail under
metadata["native_report"]["telemetry"]; it does not change scoring metrics.
See the telemetry contract.
A factory that sets supports_deferred_encoder_overlay accepts
defer_encoder_overlay=True. For an analytical EPD spec, the runner then
returns the normalized language report, which carries the metrics
apply_encoder_overlay reads, without applying the overlay, and rejects the
flag together with memory diagnostics. Sweeper fails a report that already
carries the overlay as a runner contract violation. It reuses a returned
language report for every encoder pool that shares it and applies the overlay
per pool; a failed, timed-out or refused replay is not reused.
An optional estimate_host_resources(workload, *, concurrency=None) factory
method participates in resource admission.
It must return ResourceEstimate API version 1 with nonnegative byte counts,
a positive known request count, and an optional positive peak estimate at least
as large as its lower bound. Unknown peak is not zero memory: guarded execution
admits at most one continuously supervised candidate when budgets permit.
Low-level run() callers own their supervision; the runner ABI itself does not
spawn a resource monitor or publish CLI artifacts.