Traffic is always expressed as three concepts:
traffic:
source: {}
load: {}
stop: {}source defines requests or sessions, load defines when they begin, and stop defines when the run
ends. The load and stop unit follows the source type.
Omitting traffic is equivalent to this concrete default:
traffic:
source:
type: synthetic
input_tokens: 1024
output_tokens: 128
load:
type: concurrency
concurrency: 10
stop:
requests: 100The default source contains independent requests with fixed input sequence length (ISL) and output
sequence length (OSL); it does not sample token-length distributions or create sessions. A supplied
traffic mapping does not merge recursively with this example. Once traffic is present, its normal
source, load, and stop validation applies.
The default run uses 100 requests at concurrency 10, or 10 times the active concurrency, following
the current SA convention.
| Knob | Default | Default Range | Preset | Rules |
|---|---|---|---|---|
traffic.source.type |
synthetic |
x |
- |
synthetic, synthetic-session, or trace. |
traffic.source.input_tokens |
1024 |
x |
- |
Positive; synthetic only. |
traffic.source.output_tokens |
128 |
x |
- |
Positive; synthetic only. |
traffic.source.images |
Unset | x |
- |
Fixed positive height, width, count (default 1), encoding (png or jpeg), identity (unique or {pool: N}) and optional min_pixels/max_pixels (the served processor's rescale budget; defaults to the checkpoint's); synthetic only. Requires engine.workers.encoder or an aggregated or prefill SGLang worker (native VL replay); the pixel budget is honored by native VL replay and native encoder pools, not by the analytical overlay. |
traffic.source.new_input_tokens_per_turn |
1024 |
x |
- |
Positive; synthetic-session only. |
traffic.source.output_tokens_per_turn |
128 |
x |
- |
Positive; synthetic-session only. |
traffic.source.session.turns |
4 |
x |
- |
At least 2. |
traffic.source.session.shared_prefix_ratio |
0 |
x |
- |
From 0 through 1. |
traffic.source.session.prefix_groups |
0 |
x |
- |
Nonnegative; positive when prefix ratio is positive. |
traffic.source.session.inter_turn_delay_ms |
0 |
x |
- |
Nonnegative. |
traffic.source.paths |
Required for trace | x |
- |
One path except dynamo, which permits multiple. |
traffic.source.format |
mooncake |
x |
- |
See Trace Format Compatibility. |
traffic.source.block_size |
512; embedded for dynamo and weka |
x |
- |
Positive. For embedded formats, an explicit value is an equality assertion. |
traffic.source.nested_timestamp_basis |
auto |
x |
- |
auto, absolute, or relative; Weka only. |
traffic.source.client |
recorded |
x |
- |
recorded or agentx; Weka only. See Weka client. |
traffic.source.chat_template |
Unset | x |
- |
Optional request_tokens, user_tokens, assistant_tokens, tail_tokens and append_tail_tokens, each a nonnegative integer, with append_tail_tokens <= tail_tokens <= request_tokens; Weka only. Unset replays the trace's in lengths and blocks. See chat template. |
traffic.load.type |
concurrency |
x |
- |
Synthetic: concurrency, poisson, constant_rate, or kv_capacity_fraction; trace: trace_timestamps or concurrency. |
traffic.load.concurrency |
10 |
- |
- |
Positive integer; explicit domains are allowed in recommend. |
traffic.load.requests_per_second |
null |
- |
- |
Positive; synthetic request open-loop load only. |
traffic.load.sessions_per_second |
null |
- |
- |
Positive; synthetic session open-loop load only. |
traffic.load.seed |
42 |
x |
- |
Nonnegative; poisson only. |
traffic.load.fraction |
null |
- |
- |
Positive finite number; kv_capacity_fraction only and may exceed 1. |
traffic.load.speedup |
1 |
- |
- |
Positive; trace timestamp load only. |
traffic.load.agentic_lanes |
null |
x |
- |
Positive integer; weka, agentic_mooncake, or agentic dynamo timestamp replay only. |
traffic.load.agentic_snapshot |
null (unset) |
x |
- |
Optional object with required unsigned 64-bit seed (0 through 2^64 - 1). Requires traffic.load.type: trace_timestamps and positive agentic_lanes; supported formats are weka, agentic_mooncake, and agentic dynamo. Unset preserves turn-zero execution. See snapshot. |
traffic.load.agentic_snapshot.initial_selections |
unset | x |
- |
Exact initial {source_play_id, t_star_ms} selections in lane order, one per lane, in the loaded graph's clock (Weka: per-play relative milliseconds). Requires corpus_cursor. Unset retains the original seed-based sampler. |
traffic.load.agentic_snapshot.corpus_cursor |
unset | x |
- |
Unsigned 64-bit next sequential corpus draw, including any skipped initial draws. Requires initial_selections and must be at least the lane count. Recycled plays start at turn zero. |
traffic.load.agentic_warmup |
false |
x |
- |
Optional boolean; true requires agentic_snapshot and positive agentic_lanes. Primes caches before measurement; see warmup. |
traffic.load.agentic_warmup_policy |
repeated_prefixes |
x |
- |
repeated_prefixes preserves the original frontier; live_trajectory_v1 advances the live dependency graph during ten additional one-token requests per lane; primers_only sends only the primers. The live policy requires agentic_warmup: true and agentic_profile, primers_only requires agentic_warmup: true, and both are offline --stack engine only. |
traffic.load.agentic_profile |
null (unset) |
x |
- |
Optional object; {} enables continuous lane replenishment with the defaults below and releases requests that depend on others in their play from dependency events alone, as AIPerf does. Requires trace_timestamps, positive agentic_lanes, and agentic_snapshot; cannot be combined with traffic.stop.max_virtual_time_seconds. Unset preserves finite replay. See continuous profile. |
traffic.load.agentic_profile.duration_seconds |
3600 when enabled |
x |
- |
Positive finite admission duration, starting at the preparation barrier or simulation start without warmup. No new workload requests or replacement plays are issued after the deadline. |
traffic.load.agentic_profile.response_grace_seconds |
30 when enabled |
x |
- |
Nonnegative finite time for already submitted requests to respond after the admission deadline; remaining client requests are then canceled. |
traffic.load.agentic_profile.cancel_drain_seconds |
10 when enabled |
x |
- |
Nonnegative finite upper bound for cancellation acknowledgements. The supported offline runtimes acknowledge synchronously; this does not guarantee server/GPU cleanup. |
traffic.load.agentic_profile.tree_idle_cap_seconds |
300 when enabled |
x |
- |
Positive finite idle cap for advancing a play's pending workload timers when that play has no outstanding requests. |
traffic.load.agentic_profile.global_idle_cap_seconds |
10 when enabled |
x |
- |
Positive finite idle cap for advancing pending workload timers when the entire client workload has no outstanding requests. Engine completions and server cleanup keep their actual timestamps. |
traffic.stop.requests |
100 for default traffic |
x |
- |
Positive integer; 10× default concurrency; synthetic request source only. |
traffic.stop.requests_per_load_unit |
null |
x |
- |
Positive; synthetic request source only. |
traffic.stop.sessions |
null |
x |
- |
Positive integer; synthetic session source only. |
traffic.stop.sessions_per_load_unit |
null |
x |
- |
Positive; synthetic session source only. |
traffic.stop.max_virtual_time_seconds |
null |
x |
- |
Positive; supported trace formats only. |
traffic:
source:
type: synthetic
input_tokens: 1024
output_tokens: 128
load:
type: poisson
requests_per_second: 8
seed: 42
stop:
requests: 100synthetic generates independent single requests. It does not accept a session mapping.
images adds the same fixed images to every request; the deployment encodes them
with an encoder pool or on an
SGLang worker.
traffic:
source:
type: synthetic-session
new_input_tokens_per_turn: 1024
output_tokens_per_turn: 128
session:
turns: 4
shared_prefix_ratio: 0.5
prefix_groups: 16
inter_turn_delay_ms: 1000
load:
type: constant_rate
sessions_per_second: 8
stop:
sessions: 100synthetic-session generates multi-turn sessions. It requires a session mapping, and requests
inside one session execute in turn order.
Both synthetic source types support closed-loop concurrency, Poisson arrivals, constant-rate arrivals, and recommendation-only KV-capacity-relative load as listed in the Traffic table.
The source-specific open-loop rate field is:
requests_per_secondforsource.type: synthetic.sessions_per_secondforsource.type: synthetic-session.
The rate is a positive finite number. concurrency is a positive integer and counts independent
requests for synthetic or active sessions for synthetic-session. At most one turn of a session is
active at a time. seed is a nonnegative integer and defaults to 42. fraction is greater than 0
and finite; it has no upper bound. fraction: 1 targets a concurrency whose estimated KV working set
equals the candidate's usable KV capacity. Values greater than 1 intentionally oversubscribe that
capacity. They remain valid because excess work can queue; they do not mean that the engine has more
physical KV memory.
The stop fields follow the source unit. The fixed-count field is a positive integer. The
load-relative field is
a positive number and resolves to max(1, round(count_per_load_unit * load_unit)). The load unit is
concurrency for concurrency and resolved kv_capacity_fraction traffic, requests per second for a
synthetic open-loop source, or sessions per second for a synthetic-session open-loop source.
For synthetic-session, a session with four turns contributes four requests but only one unit to the
load and stopping condition.
sessions_per_second controls the arrival rate of new sessions. For a multi-turn session, it schedules
the first turn; later turns follow that session's completion and inter_turn_delay_ms rules and do not
count as new load arrivals. requests_per_second schedules independent single requests.
In a recommendation input, source type, token fields, session shape, and stopping condition stay
concrete. Only traffic.load rows whose Default Range is not x can be search domains. A
kv_capacity_fraction recommendation is materialized as a concrete concurrency load in each
recommended prediction YAML.
traffic:
source:
type: trace
paths:
- traces/requests.jsonl
format: mooncake
block_size: 512
load:
type: trace_timestamps
speedup: 1.0
stop:
max_virtual_time_seconds: 300Without agentic_profile, omitting traffic.stop for a trace runs to end of trace.
max_virtual_time_seconds is trace-only and cannot be combined with agentic_profile or used for
synthetic traffic. Trace source, format, and token/session content stay concrete in a recommendation
input; only a numeric trace-load field can be a domain.
speedup: N divides authored timing by N; for example, 2 replays the timing twice as fast. For
Mooncake session traces it scales both first-turn arrival timestamps and inter-turn delays. For
agentic traces it scales root-node timestamps and the combined dependency delay and tool wait.
speedup is deliberately rejected with load.type: concurrency: concurrency replaces authored
first-arrival pacing, and inter-turn or dependency delays remain unscaled.
Rows with the same timestamp are admitted in file order. More exactly, requests that become ready at the same virtual time reach placement ordered by the file position of their session's first row, then by turn. Placement and each engine's scheduler then see them in that order. Nothing reorders a burst by prompt length, so a long prompt early in a burst is scheduled before the short prompts behind it.
A serving engine tokenizes each request before its scheduler sees it, and longer prompts take
longer. To model this, set engine.tokenization. Depending on how
many requests the engine tokenizes at once, a burst then reaches the engine either spread out in
arrival order, or with short prompts ahead of long ones.
| Format | JSONL Unit | Allowed Load | speedup |
max_virtual_time_seconds |
Other Constraints |
|---|---|---|---|---|---|
mooncake |
One request or session turn with a full prompt | trace_timestamps, concurrency |
Timestamp load only | Supported | None specific to the format. |
mooncake-delta |
One session turn; follow-up input is only the new input delta | trace_timestamps, concurrency |
Timestamp load only | Supported | Aggregated deployment only; planner.policy must be disabled. |
agentic_mooncake |
One request node in a dependency graph | trace_timestamps |
Supported | Not supported; omit it | Offline aggregated or disaggregated vLLM/SGLang Engine replay; planner.policy must be disabled. |
weka |
A raw kv-cache-tester or published AgentX JSON/JSONL corpus; directories are traversed recursively and JSONL files may contain multiple plays | trace_timestamps |
Supported | Not supported; omit it | Offline aggregated or disaggregated vLLM/SGLang Engine replay; source block size is embedded and the result is functionally qualified. |
applied_compute_agentic |
One complete session, expanded into num_turns + 1 requests |
concurrency |
Not supported; omit it | Supported | Source rows have no first-turn timestamps. |
dynamo standard trace |
Native request-trace records, possibly across multiple files | trace_timestamps, concurrency |
Timestamp load only | Supported | The embedded trace block size is authoritative. |
dynamo agentic trace |
Native agentic request-trace records, possibly across multiple files | trace_timestamps |
Supported | Not supported; omit it | Offline aggregated or disaggregated vLLM/SGLang Engine replay; planner.policy must be disabled. |
The dynamo loader detects whether its records are standard or agentic and applies the corresponding
row above. If traffic.source.block_size is supplied for dynamo or weka, it must match the
embedded block size. For the other formats, block_size is the trace hash-block size used to
reconstruct prompts.
Agentic formats are qualified on the offline engine stack; see
agentic load controls for supported runtimes. P/D workers
must share one target model. Reading a dynamo trace does not require the Dynamo stack.
Weka is the AgentX source format. AISim converts a Weka corpus into Agentic Mooncake and replays it natively; the Dynamo stack is not needed.
Weka producers use two timestamp conventions for requests inside a subagent:
raw kv-cache-tester traces are relative to the subagent marker, while published
AgentX traces are absolute. Set traffic.source.nested_timestamp_basis to
relative or absolute when you know which one applies. With auto (the
default), AISim picks relative for the whole corpus if any child request
is earlier than its subagent marker, and absolute otherwise. The chosen basis
is reported as weka_nested_timestamp_basis.
A Weka in counts message content. A chat client also sends the tokens that
the served model's chat template adds: role headers, end markers and the
generation prompt. traffic.source.chat_template adds them to every prompt:
traffic:
source:
type: trace
format: weka
paths:
- traces/agentx
chat_template:
request_tokens: 16
user_tokens: 5
assistant_tokens: 8
tail_tokens: 8
append_tail_tokens: 1Each stream, that is a conversation, a detected chain or a subagent thread,
keeps its messages as the AgentX client and AIPerf 0.13 build them from the
trace (ConversationReconstructor in AIPerf's weka_synth_buf.py):
- A stream's first request is one user message,
in + request_tokenstokens on the trace's blocks. - A later request cuts the message list at the hash blocks it shares with its
predecessor: a cut inside a message shortens it, and a cut at block 0 starts
the conversation over. It then adds the predecessor's reply, its
outrounded up to whole blocks, as an assistant message and the rest as a user message. The reply leaves the last new block to the user message when nothing else is new, and is left out whenoutis 0 or no user message is left. - A prompt is
in + request_tokens, plususer_tokensfor each further user message andassistant_tokensfor each assistant message. - A request repeats its predecessor's prompt up to the cut. A cut inside a
message's content leaves out the rest of the prompt, including its last
tail_tokens, so the shared prefix ends between hash blocks rather than on them. A request that keeps every message and adds a reply repeats all but the lastappend_tail_tokens. Blocks past the repeated prefix are private. - A forked chain or subagent thread starts a new stream, so its first request is one message, as AIPerf 0.13 and the AgentX client send it. AIPerf versions that include ai-dynamo/aiperf#1555 keep the parent's messages in a child's first request, so there a child shares more of its parent's prompt than this models.
- The blocks a later request adds are its own, so with either
client another stream shares at most the blocks of a stream's
first request. With
recorded, streams then no longer share their whole common hash prefix. - Two cases repeat a few more tokens than counted: a request that keeps every message and adds only a user message, and a cut at the end of an earlier message. Tool-call message shapes are not modeled, and a corpus that declares tool or system tokens is rejected.
Where the shared prefix ends matters for hybrid models whose prefix cache keeps recurrent state only at block ends: it decides whether a turn reaches the previous prompt's last cached state.
Measure the values by rendering plain messages with the model's chat template
and tokenizer, as AIPerf does to size its prompts. With messages u and v
from the user and a from the assistant, and p(...) a rendered prompt with
the generation prompt:
request_tokens: the length ofp(u), minus the content tokens ofu.user_tokens:p(u, v)minusp(u), minus the content tokens ofv.assistant_tokens:p(u, a, v)minusp(u, v), minus the content tokens ofa.tail_tokens: the tokens ofp(u)after the content ofu, that is its end marker and the generation prompt.append_tail_tokens: the tokens ofp(u)past its longest common prefix withp(u, a, v). It is 0 when the generation prompt is the assistant message's header, as in plain ChatML, and more when the history renders assistant messages differently, for example with an empty reasoning block.
A runner accepts chat_template only if it advertises
supports_weka_chat_template, as the engine stack does.
A Weka corpus can name several source models. Replay runs every request on
engine.model and records the source models and target under
agentic_model_projection.
traffic.source.client chooses whose replay of the corpus AISim follows.
With recorded (the default), a subagent starts at its recorded offset from
the request before it, a joining turn waits its recorded gap after each child,
and the streams of a play share their whole common hash prefix, unless a
chat template is set. agentx
follows the AgentX client, the AIPerf fork in the
behavioral reference:
- Every subagent and flattened chain is a child of the main conversation. A flattened chain is a conversation that the trace lists among its top-level requests. The spawning turn is the latest main turn listed before the child; the joining turn is the first main turn listed after it that starts at or after the child's recorded end. Subagents with the same spawning and joining turns form one group, and so do flattened chains.
- A group starts when its spawning turn returns, and each child starts at its offset from the group's earliest start. A group that started before the spawning turn's recorded end starts with that turn instead, each child at its offset from the turn's start.
- The joining turn waits for every child with no added gap, and for its own recorded delay after the previous main turn.
- Only a play's root is held to its recorded offset. Every other request is timed by its dependency edges alone, so a replay that runs faster than the recording, or whose idle gaps are capped, does not wait for recorded timestamps.
- A subagent that no main turn precedes is dropped. A flattened chain that no main turn precedes is spawned by the first one.
- Each stream is its own chat conversation: its first request is one user message, and a later request cuts the message list where its hash prefix diverges from the previous request's. Streams therefore share prompt blocks only inside their first user message, at block granularity and without chat-template header tokens. A chat template adds the template's tokens to each prompt's length and to the prefix it shares with the stream's previous request. Corpora that declare tool or system tokens are rejected.
Every runner accepts recorded. A runner accepts agentx only if it
advertises supports_weka_client, as the engine stack does.
Agentic traces (weka, agentic_mooncake and agentic dynamo) are made of
plays: dependency trees of requests from one agent session. Four
traffic.load fields control how plays are replayed. agentic_snapshot
requires agentic_lanes; agentic_warmup and agentic_profile each require
agentic_snapshot and are independent of each other, so a profile can measure
from cold or warmed caches:
traffic:
source: {type: trace, format: weka, paths: [trace.jsonl]}
load:
type: trace_timestamps
agentic_lanes: 4 # concurrent plays
agentic_snapshot: {seed: 42} # start each lane mid-play
agentic_warmup: true # fill caches before measuring
agentic_profile: {} # recycle plays for a fixed duration| Control | Without it | With it |
|---|---|---|
agentic_lanes |
Every play starts at its authored timestamp. | At most N plays run at once. |
agentic_snapshot |
Each play starts at turn zero. | Each lane starts from a sampled or imported exact cut inside its play. |
agentic_warmup |
Measurement starts with cold caches. | Caches are primed before measurement starts. |
agentic_profile |
The run ends when the selected plays finish. | Lanes keep taking new plays until a deadline. |
Supported runtimes: offline engine stack, vLLM or SGLang, aggregated or P/D
workers and HBM-only KV cache, including
speculative decoding with explicit acceptance
assumptions; ngram requires vLLM. vLLM additionally
supports G2 host offload with static workers
of any count and attention_data. Speculation with G2 is limited to canonical
MTP with prefix caching, and to MTP on a
hybrid G2 with engine.speculation.disable_eagle_block_drop.
TensorRT-LLM, G3 offload and online
P/D are rejected. Results are qualified functional_only, not hardware
accuracy. Start an AgentX simulation walks through a
complete run.
These controls only shape the workload. Forward-pass timing comes from the
worker's timing settings like any other workload; see
performance-model configuration. Replaying an
AgentX trace also does not enable conversation affinity: on the engine stack,
placement stays round-robin.
agentic_lanes: N runs at most N plays at once. Lanes limit whole plays,
not the concurrent child requests inside a play. Which plays run depends on the
other controls:
| Configuration | Plays replayed |
|---|---|
agentic_lanes only |
The whole corpus once, in order. A free lane takes the next play from one shared queue. |
+ agentic_snapshot |
Only the N initial snapshot plays, from their cuts. The run ends when they finish. |
+ agentic_profile |
Lanes keep taking plays, wrapping at the end of the corpus, until the deadline. |
For example, a 100-play corpus with 4 lanes and a snapshot replays 4 plays, not 100.
A play's client work ends when all its requests complete, or, after a failure, when all dispatched requests are terminal. Server cleanup, such as P/D source holds, can continue after the lane moves on.
Inside a play, a child request starts when the source request it depends on is
dispatched or completes, as the trace records, plus the recorded delay.
Background children run without a join. Without agentic_profile, a request
also waits for its authored offset from the play start (or snapshot cut), so a
play never runs ahead of its source clock. A
continuous profile drops that wait for requests that depend
on other requests.
agentic_snapshot: {seed: S} starts each initial lane partway through its play.
The cut is sampled uniformly between 25% and 75% of the play's span of request
start times. Requests that started before the cut become history and are not
replayed; the rest are replayed with their original timers. The same corpus,
lane count and seed always give the same cuts. S is an unsigned 64-bit
integer.
A snapshot is cut at request boundaries. It does not model partial decode progress or restore an engine checkpoint: without warmup, the caches start empty.
For a reference run, matching the seed alone is insufficient: AISim's versioned BLAKE3 sampler is not AIPerf's NumPy sampler. Import the actual cuts and next sequential corpus position instead:
agentic_snapshot:
seed: 42
initial_selections:
- source_play_id: "<source_play_id from the normalized corpus>"
t_star_ms: 40560.78627883167
corpus_cursor: 1initial_selections and corpus_cursor must be supplied together, with exactly
one selection per lane and a cursor at least as large as the selection count.
Selections are in lane order. Each source ID must exist in the exact input
corpus and each cut must lie within that play's request-start bounds. Cuts
are in the loaded graph's clock, before speedup: Weka subtracts the
earliest request timestamp in each complete play, including child streams,
whereas Agentic Mooncake retains its recorded source clock. Branch/spawn marker
times are not request starts and do not define the origin. The importer does
not guess a clock basis or convert untagged cuts; they are never re-sampled or
silently replaced. Repeated source plays still get distinct lane/cache
identities. Preserve the source corpus bytes/order, reference revision and
selection artifact alongside the replay configuration.
The sequential cursor counts source draws, not completed requests or unique
plays; it can exceed the initial lane count when the reference skipped a
source. After a tree drains, the shared cursor selects the next source play
and starts it at turn zero, with a fresh cache identity. This matches the
sequential-sampler recycling contract of
AIPerf 754356e9.
Random/shuffle recycling is not implemented by this import. Initial cuts do
not import an engine checkpoint, tokenizer/chat-template bytes, or a
post-warmup frontier; the warmup policy below remains unchanged.
To produce selections without manually computing cuts, run the optional exporter in the pinned AIPerf environment (its existing dependencies are required):
python scripts/prediction_regression/export_agentx_snapshots.py \
--aiperf-source /path/to/aiperf-at-754356e9 \
--metadata reference-dataset-metadata.json \
--source-play-map source-play-map.json --trace-format weka \
--lanes 15 --seed 42 --output snapshot-options.jsonreference-dataset-metadata.json is the reference loader's
DatasetMetadata.model_dump_json() output, with every conversation, branch,
and timestamp retained. source-play-map.json maps each root
conversation_id from that metadata to its normalized AISim
source_play_id (as recorded in agentic_snapshots). The root order in the
metadata must match the replay corpus order and each root mapping must identify
the corresponding source play; do not sort either input. The importer validates
IDs and cut bounds, but cannot infer or verify reference corpus order from a
cursor alone. A wrong mapping/order is not a valid parity comparison.
The required --trace-format selects the target loader clock explicitly:
weka subtracts the minimum request start across each selected reference tree;
agentic_mooncake leaves source-clock cuts unchanged. It uses the pinned
metadata's millisecond turn timestamps, not seconds or branch markers. The
reference and replay loaders must represent the same complete request tree
and nested timestamp interpretation. Missing children, ambiguous trees or
missing/nonfinite request timestamps in a selected tree fail explicitly;
do not invent an origin for an unmappable corpus. Existing exports made before
this conversion fix must be regenerated for nonzero-origin Weka plays, even
if their old cut happens to fall within the normalized bounds.
Paste the emitted JSON object under traffic.load.agentic_snapshot. Keep the
automatically written snapshot-options.json.provenance.json beside it: it
records the pinned revision, metadata/map/output SHA-256 hashes, root order,
clock basis and each lane's raw cut, origin and converted cut. Both output
paths must be new. For importable JSON on stdout, omit --output and provide
--provenance-out snapshot-provenance.json; the sidecar is persisted before
anything is printed to stdout.
The exporter verifies the source revision and clean source tree, invokes the
reference TrajectorySource directly. It fails on unsupported samplers, timestamp-less trajectories,
or incomplete lane selection. Neither AIPerf nor NumPy becomes an AISim
runtime dependency.
agentic_warmup: true fills the caches before measurement. The default
agentic_warmup_policy: repeated_prefixes has three stages:
| Stage | Requests | Output |
|---|---|---|
| Primer | For each conversation that has both history and remaining requests, its last historical request's full input | 1 token each |
| Warmup | 10 per lane, repeating that lane's primer inputs (or its first remaining request if it has no history) | 1 token each |
| Profile | The remaining requests of each play | Original output lengths |
Within a lane, preparation requests run one after another; lanes prepare in
parallel. Profiling starts at a barrier, after every preparation request has
succeeded and all its server work has settled, including P/D transfers. At the
barrier, worker caches and play identities are kept, and the measurement clock
starts. If any preparation request fails, the run stops before profiling:
predict exits with an error and recommend drops the candidate.
Warmup repeats the saved prefixes and does not advance the plays. Primed prefixes can still be evicted or placed on another worker, so warmup does not guarantee cache hits. See agentic warmup for the full preparation, barrier and reporting contract.
For AgentX-style live advancement, select the versioned policy explicitly:
agentic_lanes: 15
agentic_snapshot: {seed: 42} # import reference selections for matching cuts
agentic_warmup: true
agentic_warmup_policy: live_trajectory_v1
agentic_profile: {duration_seconds: 3600}This policy primes predecessor streams with globally aligned lead times, capped
by agentic_profile.global_idle_cap_seconds. After all primers settle, it runs
the actual dependency graph with zero authored idle delay and one output token
per request. Primers do not consume the additional quota of ten admitted
requests per lane. Spawns, joins, rootless children and sequential corpus
recycling continue during warmup; an exhausted lane pauses without discarding
its next request. At the all-settled barrier, the surviving frontier resumes
with original output lengths and source-clock offsets. One global minimum is
subtracted from eligible frontier offsets, preserving cross-lane spacing;
waiting at the barrier does not consume those delays.
Successful profiles exclude preparation-only requests and retired plays from
counts and latency distributions. Active partially warmed plays retain their
identities and get a formal measurement origin at the barrier. An aborted live
warmup has an empty formal cohort; its request ledger remains in
agentic_phases. See live trajectory warmup
for qualification boundaries. Unsupported CLI stacks are rejected before
--overwrite prepares the output directory; external SDK adapters must
explicitly advertise supports_agentic_live_warmup and forward the policy.
agentic_warmup_policy: primers_only models AIPerf's snapshot warmup when no
cache-pressure warmup is configured: the live policy's primers, timed the same
way, and no further preparation request. External SDK adapters must explicitly
advertise supports_agentic_primers_only and forward the policy. See
primers-only policy.
agentic_profile keeps every lane busy until an admission deadline. When a
play finishes, its lane takes the next play from the corpus, wrapping at the
end. Replacement plays start at turn zero with fresh identities.
A profile releases requests as AIPerf's agentic replay does. A request that depends on other requests in its play starts when they are dispatched or complete, plus the recorded delay; its authored offset no longer holds it back. A follow-up turn therefore starts its recorded think time after its predecessor actually completes, and a subagent starts from its parent's actual completion. A turn that joins children waits for both their completion and its own recorded delay after its predecessor. Only play roots and, after a snapshot cut, each stream's next request whose predecessor lies before the cut keep their source-clock timers.
| Field | Default | Meaning |
|---|---|---|
duration_seconds |
3600 |
Admission window, starting at the warmup barrier (or at time zero without warmup). No new requests or plays are issued after it. |
response_grace_seconds |
30 |
Time for already issued requests to finish after the deadline. Requests still running are then canceled. |
cancel_drain_seconds |
10 |
Upper bound for cancellation acknowledgements. Offline runtimes acknowledge immediately. |
tree_idle_cap_seconds |
300 |
Longest idle wait inside a play with no outstanding requests; longer authored waits are shortened. |
global_idle_cap_seconds |
10 |
Longest idle wait when no client request is outstanding. |
agentic_profile: {} uses all defaults. Durations and idle caps must be
positive; grace periods may be zero. The profile cannot be combined with
traffic.stop.max_virtual_time_seconds. Idle caps shorten client wait timers
only; engine work keeps its real timing.
Throughput uses the successful requests, including those that finish during
the grace period. The observed interval runs from their earliest arrival to
their latest response, so it can be shorter or longer than duration_seconds.
Long profiles keep lifecycle records for every play and can use much more host
memory than the trace itself. The CLI runs them under host-memory supervision;
a run that exceeds its budget stops with resource_limited. See
continuous agentic profiles for lane recycling,
idle guards, cancellation and result details.
examples/cli/agentic-profile.yaml
is a small offline example with fixed timing:
aisim predict --config examples/cli/agentic-profile.yaml \
--capture-per-request --format json --output-dir ./agentic-profileprediction.json adds these sections:
| Section | Contents |
|---|---|
agentic_snapshots |
Seed, sampled or imported exact cut, history and remaining timers for each initial lane |
agentic_phases |
Preparation requests, barrier state and time (profile_start_ms) |
agentic_play_outcomes |
Each play's terminal status (completed, failed or incomplete) and times |
agentic_profile |
Resolved profile options, deadline, recycled plays, canceled and never-issued requests, unsettled server work |
With warmup, per-request records in requests.jsonl contain only profile
requests, and their timestamps start at the barrier. Preparation is excluded
from latency, throughput and reuse metrics. wall_time_ms is host time for the
whole run, including preparation.
To check cache reuse, use first_admission_prefix_cache_reused_ratio and the
per-request reuse fields described in KV cache.
Reuse is counted in whole blocks: a resident 128-token prefix reuses 64 tokens
with vLLM's 64-token blocks and 127 tokens with SGLang's 1-token pages.
AISim's snapshot, warmup and profile behavior is modeled on AIPerf, InferenceX and the AgentX harness, implemented independently and not at full AgentX parity. Pinned revisions, licenses and known differences are listed in agentic warmup and continuous agentic profiles.
Both formats use the same row schema. Rows with the same session_id are turns in file order.
| Field | Required | Semantics |
|---|---|---|
request_id |
No | Request identity. |
session_id |
No | Groups rows into a session. An omitted value creates a one-row session whose per-request records use request_<line>; placement sees no session for it. |
input_length or input_tokens |
No | Input token count; defaults to the capacity represented by hash_ids. |
output_length or output_tokens |
Yes | Output token count. |
output_token_ids |
No | Exact output tokens; its length must equal the output token count. |
hash_ids |
Yes | Prompt hash blocks at traffic.source.block_size. |
timestamp or created_time |
Conditional | Virtual timestamp in milliseconds. Required for the first row of every session under trace_timestamps. |
delay or delay_ms |
No | Inter-turn delay in milliseconds. It must be omitted or zero on a session's first row; later rows use it instead of a timestamp difference. |
priority, strict_priority, policy_class |
No | Scheduling metadata. |
With format: mooncake, every row describes that turn's complete prompt. With
format: mooncake-delta, the first row describes the initial prompt, while each later row's input and
hash_ids describe only new input for that turn. Replay builds the next complete prompt by appending
the prior generated output and the new delta. Using mooncake-delta on a full-prompt trace would
double-count prior context.
agentic_mooncake includes all Mooncake request fields above, but request_id is required, nonempty,
and unique. Each row is an independently schedulable request node rather than a turn inferred only
from session order. It adds these fields:
| Field | Default | Semantics |
|---|---|---|
wait_for |
[] |
Request IDs that must all complete first. Unknown IDs, self-dependencies, and cycles are rejected. |
delay / delay_ms |
0 |
Delay after the last dependency completes. |
tool_wait_ms |
0 |
Additional tool wait after dependencies; scheduling delay is delay + tool_wait_ms. |
timestamp / created_time |
0 for roots |
Ready time for a node with an empty wait_for; dependent-node timestamps do not control release. |
request_kind, branches, prefix_reset, tool_events |
Empty | Producer metadata accepted by the format. Scheduling is controlled by wait_for, and cache identity is carried by hash_ids; these metadata fields do not independently change replay behavior. |
A dependent node becomes ready after the latest request in wait_for completes, plus its delay and
tool_wait_ms. speedup therefore scales both authored root arrivals and those post-dependency waits.
Each row is one session with num_turns, input_prompt_length, arrays
assistant_response_length, tool_call_output_length, and tool_call_latency, plus
final_assistant_response_length. Each array length must equal num_turns; tool latency is in
seconds. The row expands to num_turns assistant/tool turns plus one final assistant request. The
input grows cumulatively by each assistant response and tool output. Because the format has no
first-session arrival timestamps, it requires load.type: concurrency and rejects speedup.
Note
max_virtual_time_seconds limits the total simulated virtual time of one prediction or recommendation
candidate. It is not a separate processing-time limit for each path in traffic.source.paths, and it
is not a real wall-clock timeout. The limit is a soft scheduling cutoff: events at the cutoff are
processed, replay stops before the first event after the cutoff, and requests still in flight can be
reported as incomplete. The reported duration can extend slightly past the cutoff while an already
running engine pass finishes.
Trace mode normalizes the first request or session to virtual time zero. Concurrency replaces first-arrival pacing. An active multi-turn session holds its slot through all turns and think time, so concurrency caps sessions, not individual turns. Agentic lanes instead cap whole play instances and do not cap concurrent child requests. Completion dependencies, tool delays, joins, and background children determine which requests become ready.
The Python runner receives concrete ReplaySpec.workload plus concurrency;
native ReplaySpec.max_in_flight selects source-side concurrency. The SDK's
Workload uses fields such as trace_path, isl, osl, and kv_load_ratio,
which are distinct from this public YAML. See Sweeper SDK
for that mapping and candidate-relative load
for capacity-based search. Do not substitute SDK YAML for CLI YAML.