chore(inference): bump vLLM to 0.26.0 - #3166
Merged
Merged
Conversation
S1ro1
marked this pull request as ready for review
July 30, 2026 19:44
hallerite
approved these changes
Jul 30, 2026
eligotts
added a commit
that referenced
this pull request
Jul 31, 2026
Picks up 18 main commits, notably the vLLM 0.26 bump (#3166), the task-centric orchestrator config restructure (train.source.env), RAE / role-conditioned advantage estimation (#3136), and the verifiers bump that adopts runtime-on-agent and the textarena player seat (#3137). Resolutions: - serving_tokens.py: vLLM 0.26 moved the disagg serving modules under scale_out.token_in_token_out and changed mm_input() to take MultiModalKwargsItems. Adopted both (the wrapper is an API change, not an eager-path detail) while leaving out main's decode_mm_kwargs_item / MultiModalKwargsItem imports, which only serve the processed-tensor path this branch replaces with raw-ref materialization. - wordle configs: main has since adopted the textarena `player` seat and restructured to [orchestrator.train.source.env], so main's versions supersede this branch's earlier env.agent -> env.player migration. - configs/orchestrator.py + rl.py: insertion collisions only — kept the multimodal block alongside main's rollout_transport field and main's removal of the deprecated [[env]]/[sampling] shim. - tests/unit/orchestrator/test_qwen3_vl_e2e.py: stays deleted; main only refreshed its vLLM import paths, but it round-trips the eager processed features payload that v1 rejects by design. Re-pins deps/renderers (1af6589) and deps/verifiers (0e4166608), both freshly merged with their own mains.
jooooonas
added a commit
to p-doom/prime-rl
that referenced
this pull request
Jul 31, 2026
* fix(trainer): don't resolve Hub envs when parsing orchestrator configs (#3096)
* fix(trainer): don't resolve Hub envs when parsing orchestrator configs
discover_runs() builds OrchestratorConfig to read training-relevant fields, but
constructing a v1 env config eagerly resolves (imports/installs) its taskset/
harness plugins from the Environments Hub. On the trainer that 404s for private
hub envs (taskset.id = owner/name@version) — the trainer has no Hub credentials —
so get_orchestrator_config() returns None and the run is silently skipped.
Parse under verifiers' skip_plugin_install contextvar so the trainer never
touches the Hub; the env server still installs the plugin at runtime.
Depends on verifiers exposing `skip_plugin_install` — bump deps/verifiers to a
commit that includes it before this lands.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(trainer): sort imports to satisfy ruff isort (I001)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: bump verifiers to 0.2.2.dev7, adopt the multi-agent episode wire (#3104)
* feat: bump verifiers to 0.2.2.dev7, adopt the multi-agent episode wire
Companion to PrimeIntellect-ai/verifiers#1939 (multi-agent api). Re-pins
deps/verifiers to 192fd8ef0 (= 0.2.2.dev7) and deps/pydantic-config to
v0.4.2 (verifiers now requires prime-pydantic-config>=0.4.2).
- Serve wire: the env-server route run_rollout -> run, answering one
Episode envelope ({id, env, ok, errors, traces}) per env-rollout.
Env.run returns list[Rollout]; an episode that failed before minting
any trace raises (the dispatcher synthesizes its error marker), and a
not-ok episode marks its clean traces failed so a broken episode's
partial seats are never trained on.
- The episode is the pipeline currency: the dispatcher emits one episode
(list[Rollout]) per completed task onto out_q (a legacy run_group
result emits its traces as single-trace episodes), and both sinks
count episodes - never loose traces - toward group_size and eval epoch
sizes. episode_id is stamped into info so saved records keep grouping.
- ok is the success sentinel: has_error now reads it, so the dispatcher's
empty-trajectory promotion also flips ok, and error markers carry it.
- Only the trainable subset becomes training samples: a trace stamped
untrainable (a multi-agent env's frozen agent) is never tokenized,
never enters the group advantage, and never ships; it still lands in
the trace files and observation window.
- Config shape: everything env lives under the env block, following the
new vf.EnvServerConfig (env.taskset, env.agent.harness, per-agent caps
under env.agent.*). All checked-in TOMLs migrated; the spawned env
server receives the picklable env block via env_config_data.
- run_group survives only for v0/legacy bridge envs (a v1 server refuses
it and always reports requires_group_scoring=False); vf's @group_reward
removal doesn't touch prime-rl's own Algorithm.score_group.
Verified: tests/unit green (487 passed; config-loading params covered all
migrated TOMLs), ruff clean, and a 3-step reverse-text RL run on 2 GPUs
trains at 0% errors with every saved trace carrying ok, agent.name,
agent.trainable, and a distinct info.episode_id.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: trim narrative comments to the load-bearing facts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: count buffered sink progress in episodes; explicit ok=False on cancel markers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: keep untrainable traces out of the effective views and the derived avg@k
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(weight-transfer): add NIXL + ModelExpress synchronization (#3060)
* feat(weight-transfer): add NIXL ModelExpress synchronization
* refactor(weight-transfer): simplify NIXL integration
* refactor(weight-transfer): simplify NIXL worker planning
* refactor(weight-transfer): align MX update lifecycle
* feat(inference): pass through online quantization methods
* feat(weight-transfer): add GLM trainer conversion graph
* feat(weight-transfer): stream NIXL updates by layer
* feat(weight-transfer): pipeline NIXL layer reloads
* fix(weight-transfer): preserve single-buffer NIXL behavior
* fix(weight-transfer): reserve observed inference peak
* feat(weight-transfer): preserve model-declared FP32 tensors
* cleanup(weight-transfer): simplify trainer wire schema
* cleanup(weight-transfer): make trainer shard ranges explicit
* cleanup(weight-transfer): resolve model conversions through registry
* cleanup(weight-transfer): clarify lazy copy destinations
* cleanup(weight-transfer): derive lazy metadata from source
* cleanup(weight-transfer): rename weight load tracing
* cleanup(weight-transfer): name recorded tensor operations
* cleanup(weight-transfer): simplify transfer planning
* cleanup(weight-transfer): consolidate load graph
* cleanup(weight-transfer): name trainer tensor table
* cleanup(weight-transfer): clarify tensor routing
* cleanup(weight-transfer): simplify NIXL adapter
* cleanup(weight-transfer): clarify ModelExpress sessions
* cleanup(weight-transfer): clarify CUDA memory sizing
* cleanup(weight-transfer): clarify staged trainer shards
* cleanup(weight-transfer): simplify trainer broadcast guards
* cleanup(weight-transfer): clarify trainer staging state
* cleanup(weight-transfer): simplify trainer constants
* cleanup(weight-transfer): use ModelExpress client name
* cleanup(weight-transfer): clarify trainer shard discovery
* cleanup(weight-transfer): structure trainer initialization
* cleanup(weight-transfer): reuse trainer agent metadata
* cleanup(weight-transfer): reuse trainer table fragments
* cleanup(weight-transfer): simplify trainer staging setup
* cleanup(weight-transfer): simplify inference transfer planning
* cleanup(weight-transfer): split inference plan construction
* cleanup(weight-transfer): clarify inference replay methods
* cleanup(weight-transfer): localize receive arena sizing
* cleanup(weight-transfer): clarify loader recording wrapper
* cleanup(weight-transfer): clarify watcher synchronization
* cleanup(weight-transfer): unify checkpoint wait timing
* cleanup(weight-transfer): align in-memory broadcast lifecycle
* fix(weight-transfer): skip NIXL startup filesystem wait
* cleanup(weight-transfer): use declarative model conversions
* style(weight-transfer): format NIXL implementation
* fix(weight-transfer): stabilize ranks and final watcher step
* fix(tests): use instance conversion API
* cleanup(weight-transfer): scope package to NIXL
* cleanup(weight-transfer): align names and package layout
* chore: bump research-environments submodule to latest main (#3106)
* chore: bump research-environments submodule to latest main
Picks up the registry ref-cleanup wave: platform image naming convention
(openswe/tmax/terminal-lego/openthoughts-tblite/senior-swe-bench switched to
org-less platform refs, multiswe REGISTRY mapping removed) and the tmax Harbor
registry pointer bump.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: relock env versions for research-environments bump
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(monitor): add local file monitor (#3111)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat: orchestrator owns v1 tasksets, ships task data to env servers (#3043)
* feat: orchestrator owns v1 tasksets, ships task data to env servers
Companion to PrimeIntellect-ai/verifiers#2039. A v1 env's taskset is loaded once,
in the orchestrator: finite tasksets become the train source's shuffled example
table (real tasks, not index ranges); an infinite taskset's generator is pulled
per example — no server-side materialization, no idx-addressed cache, no
requirement that every pool worker regenerate the same sequence. Each dispatched
rollout ships its task's data (task.data.full_dump()) and the env server
pydantic-validates and runs it.
The legacy (v0) bridge keeps its server-side dataset and task_idx addressing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers to client-side-tasksets tip
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers to client-side-tasksets tip
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers to client-side-tasksets tip
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: plain model_dump task payload; bump deps/verifiers (full_dump removed)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat: re-land client-side tasksets on the episode wire
Companion to the rebased PrimeIntellect-ai/verifiers#2039. A v1 env's taskset is
loaded once, in the orchestrator: finite tasksets become the train source's
shuffled example table (real tasks, not index ranges); an infinite taskset's
generator is pulled per example. Each dispatched env-rollout ships its task's
data (task.data.model_dump()) and the env server validates and runs it. The
legacy (v0) bridge keeps its server-side dataset, task_idx addressing, and the
run_group route.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers (non-optional taskset); drop redundant assert
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: actually drop the redundant taskset assert
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers (content-identity resume)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers (simplified task_key)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: bump deps/verifiers to main (client-side tasksets merged, vf#2039)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor: one Env.tasks iterator instead of tasks/task_iter
Finite tasksets still materialize at start() (num_tasks needs the count and the
train source shuffles epochs), but the field is a single iterator either way —
consumers pull it uniformly and branch on num_tasks alone. Consumed once, by
TrainSource or EvalEnv.start.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(trainer): use UE8M0 FP8 scales only on SM100 (#3112)
#3022 flipped use_ue8m0 to True unconditionally to support DeepGEMM on
SM100, which also switched SM90 to power-of-2-rounded scale factors —
a pure precision loss on Hopper. On a GLM-5.2 RL prod run this doubled
trainer/inference mismatch KL (0.0038 -> 0.0059 at step 1, verified
A/B on an identical batch: same tokens and inference logprobs read
0.0059 with UE8M0 and 0.0038 with float scales).
Derive use_ue8m0 from the device (compute capability >= 10) so SM100
keeps the required UE8M0 recipe and SM90 returns to exact float scales.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(rl): pack multimodal samples with context parallelism (#3103)
* feat(rl): pack multimodal samples
Multimodal RL samples no longer force one-sample micro-batches: eager
image samples pack with text spans and compatible mm_kwargs samples from
the same run/LoRA, with per-sample boundaries kept in seq_lens. Packing
is always on — VLM training is custom-implementation-only and the custom
Qwen3.5 models advertise supports_packed_multimodal_training; a config
validator requires varlen flash attention for VLM training and
validate_multi_modal_pack fails loudly at startup for models without
packed-MM support.
Ported from #2889 minus its gating (pack_multimodal knob, resolve_pack_multimodal)
and its tokenizer packer params (removed on main since).
(cherry picked from commit 027edc2c510c35ab5666e8f9cbf160f58b1fba48)
* fix: fail loudly when RL micro batches carry multimodal data without [model.vlm]
Mirrors the SFT-side check: transported mm samples would otherwise flow
into forward unvalidated (no packability, attn, or freeze policy checks).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(rl): preserve multimodal packing across workers
Keep multimodal bins intact during worker balancing and reject malformed multimodal samples before they can stop shared multi-run packing.
* fix: sync docs to public-docs on stable release only (#3116)
* fix: sync docs to public-docs on release tags only
Change the Sync Docs to Public Docs workflow to trigger on stable v*
tag pushes instead of every commit to main that touches docs/, so the
public docs only update on releases.
Exclude v*dev* tags so dev pre-release tags never trigger a sync.
Keep workflow_dispatch for manual syncs.
On a tag push github.sha resolves to the tagged commit, so the ref sent
to public-docs remains correct.
Mirrors PrimeIntellect-ai/verifiers#2108.
* fix: trigger docs sync on release published, not on main push
Change the Sync Docs to Public Docs workflow to trigger when a stable
GitHub release is published instead of every commit to main that touches
docs/, so the public docs only update on actual stable releases.
Dev tags (v*dev*) are created via GITHUB_TOKEN which doesn't retrigger
workflows, and they don't have GitHub Releases, so using the release
event with types: [published] naturally excludes them.
Keep workflow_dispatch for manual syncs.
On a release publish, github.sha resolves to the tagged commit, so the
ref sent to public-docs remains correct.
* chore(rl): add message to multimodal packing assert (#3117)
Follow-up to #3103 review feedback: make the mm_kwargs invariant assert self-describing when it fires.
Co-authored-by: eligotts <eligotts@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(orchestrator): hold batch ship on the trainer's published version (#3050)
* fix(orchestrator): enforce target lag strictly at ship time
The dispatch gate only stops new rollouts; on tiny fast envs the sink
keeps forming batches from buffered rollouts, so the orchestrator ships
past TARGET_LAG, finishes, and tears down the watcher while the trainer
still has pending NCCL broadcasts. Those broadcasts then block forever
in the STABLE->NCCL_READY handshake (no watcher to trigger the inference
receive) and the launcher never exits.
hold_for_target_lag blocks shipping batch N until the trainer has
published v{N-1-TARGET_LAG}, bounding in-queue staleness and keeping the
orchestrator alive for every broadcast the trainer will make. Under NCCL
the last published version is max_steps-2, exactly what the final batch
requires, so the hold never waits on an unpublishable version.
Also logs a per-batch staleness decomposition (pre-queue = versions
elapsed during generation, in-queue = versions spent waiting in the
sink) under staleness/*.
Fixes RES-1077
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* remove staleness metrics from ship path
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(orchestrator): step-based staleness with demand-driven dispatch
Measure a rollout's off-policy lag against the batch that consumes it —
batch N trains on policy v{N-1}, so lag = (N-1) - generation version,
queue time included. The old off_policy_steps counter only counted
weight updates during generation, so a rollout that finished fast but
sat buffered across steps trained arbitrarily stale while reporting 0.
- max_off_policy_steps now bounds consumption lag: in-flight groups are
cancelled pre-pause (cancel_stale_train_groups) and buffered sink
survivors are swept whenever the cutoff moves (ship, weight update).
Groups share one generation version, so stale groups drop whole.
- Dispatch is demand-driven: the orchestrator injects train_demand
(batch shortfall minus buffered + partial-group + in-transit
rollouts), re-evaluated per scheduled group, replacing the lead-based
dispatch gate and the #2990 final-batch NCCL exemption it needed.
out_q backlog counts as coverage so ship-holds don't over-provision.
- Max Off-Policy in logs is now the honest consumption lag, stamped at
ship (reverse_text: steady-state max 3 vs a dishonest 0 before).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* review pass: bool dispatch contract, oversampling-aware demand, inline single-use helpers
- train_demand -> train_needed: the dispatcher only thresholds the value,
and the magnitude isn't even unit-stable across batch modes, so the
injected contract is a bool predicate.
- Demand target is max(batch_size, max_inflight_rollouts): explicit
over-provisioning (oversampling_factor > 1 or a raised max_inflight)
keeps racing batch stragglers instead of being silently capped at one
batch; capacities below the batch keep a batch_size target so the
batch can still complete.
- Reject max_off_policy_steps=0 with NCCL broadcast + max_steps: the
final batch necessarily trains one version off-policy (the trainer
never broadcasts the last versions), so a zero budget would hang the
run collecting the final batch.
- Inline single-call helpers (hold_for_target_lag, stale_cutoff, the
dispatcher lag gauges) into their call sites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* require max_off_policy_steps >= 1 at the field
The last NCCL batch necessarily trains one version off-policy (the
trainer never broadcasts the final versions), so a zero budget can never
be met and would hang the run collecting the final batch. A field floor
covers this without a conditional validator.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix CI regressions: partial-group dispatch deadlock + serialized weight applies
Two regressions surfaced by the PR's GPU CI run:
- alphabet_sort/rl_opd deadlocked at 126/128 (+3 buffered, 0 inflight):
demand hit zero mid-group, so an open group's remaining rollouts were
never scheduled — its buffered members count as batch coverage but the
group can never finalize. Already-open groups now keep scheduling
regardless of demand (try_schedule allow_fresh); only fresh groups are
demand-gated.
- reverse_text timed out (~26s/step vs ~19s on main): the ship hold
waited on the applied version, serializing every inference weight
reload into the ship path. Filesystem broadcast decouples publish from
apply and the trainer never blocks on the watcher there, so the hold
now releases on the published version (tracked in on_version_pending).
NCCL keeps waiting for the apply — the trainer blocks in the receive
handshake until then, so releasing on publish could strand its final
handshake at teardown.
Verified: reverse_text e2e clean on both broadcast types (filesystem
6-7s steps, NCCL exits clean), 485 unit tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* raise multi-run integration timeout to 600s
Resumed orchestrators now pace to the shared trainer — a batch only
ships once the trainer has produced the required policy version — so
with three runs round-robining one trainer, beta_resume's tail (steps
21-25) can legitimately exceed the old 300s wait while its versions
queue behind the other runs. Matches the 600s budget of the other RL
integration tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* retract stale-dropped rollouts from the payload-size estimate
drop_stale now mirrors what entering pending_batch added to
_payload_total/_payload_count, keeping the token-mode demand estimate
consistent with the buffered set. Also expand the allow_fresh comment
with the group-finalization invariant it protects.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* upload integration run logs as artifacts on failure
The multi-run test's failures only surface a sampled tail of one
orchestrator's output; the copied per-run orchestrator/trainer/inference
logs are what's needed to debug them (currently: beta_resume wedging on
a policy version the shared trainer never publishes after resume).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* keep CI test outputs for the failure-artifact upload
The output_dir fixture rmtree'd the whole outputs dir at session
teardown, so the upload step found nothing. Skip the cleanup under CI
(the workspace is discarded with the job) and match log files directly
instead of a directory glob.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* dispatch lookahead: cover TARGET_LAG extra batches when freshness allows
Exact-one-batch demand synchronized dispatch into one 128-rollout wave
per batch: each ship opened demand 0->128 at once, the wave stampeded
the env server together (MCP timeouts + backoff retries in CI
alphabet_sort), and inference idled ~60s per cycle while the whole wave
sat in env round-trips — tripling the step cadence vs main (artifact
logs: vLLM alternating 30s busy / 60s '0 running, 0 waiting').
With up to TARGET_LAG batches of lookahead — allowed only while a
rollout landing that far ahead still fits max_off_policy_steps —
steady-state dispatch is continuous (each arrival frees demand for the
next start), keeping the engine fed through env-phase gaps. Costs one
version of staleness on the lookahead batch (reverse_text plateau 3->4,
within the default cap of 8).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* raise reverse_text mismatch-KL budget to 0.02
Mismatch KL compares generation-time inference logprobs against the
training weights, so it includes async training's policy-version gap on
top of trainer/inference numerics. With strict pacing + dispatch
lookahead the steady-state consumption lag is 2-3 versions (visible and
bounded, vs unmeasured before), which puts the last-steps average at
~0.011-0.012 against the old 0.01 budget calibrated for main's profile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* raise alphabet_sort and multi-run integration budgets
Artifacts from the failed runs show no wedges — every process was
mid-progress at the deadline: alphabet_sort cycles at ~70s on slow vm
runners (33s on fast ones; trainer MFU varies 2-6% across the pool) and
the multi-run trainer was mid-micro-step serving three runs
round-robin. With strict pacing the orchestrators ride the trainer's
pace instead of finishing early on stale buffers, so wall time tracks
the slowest component.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* raise multi-run inner milestone waits to 600s
The wait_for_log/wait_for_file 300s defaults were calibrated for
orchestrators that sprint ahead of the shared trainer on buffered
batches; with strict pacing three orchestrators ride the trainer's
actual speed, so every internal milestone (alpha steps, checkpoint
STABLE markers, resume completion) lands slower and a different 300s
wait tripped on each CI round.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* count group-scored partial arrivals as batch coverage
buffered_count hides group-scoring envs' partial groups for the
pipeline log's honesty, but demand must count them — otherwise members
sitting between arrival and group completion invite bounded
over-dispatch. Also document why a held batch intentionally does not
count as coverage (dispatch-ahead keeps inference busy through
trainer-bound holds; staleness stays bounded by lookahead + cutoff).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* guard token-export STABLE markers against deleted run dirs
The multi-run trainer crashed with FileNotFoundError touching
run_gamma/token_exports/step_20/STABLE: the multi-run integration test
rmtree's a run's dir once its orchestrator finishes, while the trainer
is still flushing that run's tail steps. The broadcast paths already
skip deleted runs on FileNotFoundError; mark_stable now does the same.
This was the root cause of the recurring multi_run CI failure: the
crashed trainer never published the resumed run's next version, so
beta_resume's ship-hold waited forever ('failed with code None'). On
main the same crash risk exists but the orchestrator never waited on
the trainer, so nothing surfaced.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* recalibrate multi-run early-step reward floor for strict pacing
Batch N's data now comes from v{N-2}..v{N-4} instead of main's
effectively-fresher buffer profile, so reward-at-step-7 reads 1-2
policy versions staler (observed 0.14-0.20 across four CI rounds vs the
0.2 floor). The final-reward thresholds still enforce learning quality;
the early floor drops to 0.1 — comfortably above a broken run's flat
~0.05-0.08.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* raise rl_sft integration budget to 900s
Same class as the other RL integration budgets: the trainer was
SIGTERM'd at step 14 with the orchestrator already finished — killed by
the 600s deadline steps from completion on a slow runner.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* lower multi-run final reward floor to 0.6
With strict pacing the reward-vs-step-index curve sits 1-2 policy
versions lower throughout; gamma — the most trainer-contended of the
five runs — landed at 0.627 while its siblings cleared 0.69-0.75. A
broken run still reads ~0.1-0.3.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bound dispatch freshness at DISPATCH_LAG instead of the drop cap
Root cause of the reward-at-step gap vs main: the old lead gate doubled
as the dispatch freshness bound (rollouts only started at version >=
step-2, landing lag <= 2). The demand rewrite left freshness gated only
by max_off_policy_steps (8) plus a deliberately-ahead lookahead batch,
so mean consumption lag ran 2-4 versions vs main's 1-2 — visible as
Max Off-Policy 3-4, mismatch KL ~2x main's, and reward-at-step lower by
1-2 versions of learning.
Dispatch now pauses when new rollouts would train more than
DISPATCH_LAG = TARGET_LAG+1 versions behind their generation policy,
with the lookahead scaled inside that budget — the tightest bound that
still keeps inference busy during ship-holds. Landing lag returns to
<= 2 (spill 3); local e2e: Max Off-Policy plateau 4 -> 3, mismatch KL
mean ~0.012 -> ~0.008. max_off_policy_steps returns to being purely the
drop safety net.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* ci: retrigger GPU tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* tolerate trainer tail-flush when deleting run dirs in multi-run test
The fixture's rmtree races the shared trainer's token-export writes,
which mkdir the run dir back mid-deletion (OSError: Directory not
empty). Retry until the flush for the finished run drains.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* restore NIXL ModelExpress handshake signals in the version hooks
The merge of #3060 kept this branch's on_version_pending/on_new_version
bodies and dropped main's ModelExpress status updates — the READY
signal the NIXL trainer waits on before each broadcast and the
INITIALIZING reset after it. Post-startup NIXL runs would hang in the
handshake. Both signals are back, ordered as on main (READY pre-update,
INITIALIZING post-update) around this branch's staleness/wake logic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* reduce to the minimal ship-hold fix + staleness docs
Strip the demand-driven dispatch, consumption-lag drop rework, and test
recalibrations accumulated on this branch back to main's behavior, and
keep only what the spec (docs/staleness.md) strictly requires:
- ADVANCE rule: batch N ships only once the trainer has published
v{N-1-TARGET_LAG}, so the orchestrator cannot finish and tear down the
weight watcher while the trainer still has in-memory broadcast
handshakes pending (RES-1077). Main's dispatch gate (START) and
in-flight drop aging (DIE) are untouched, so generation freshness and
throughput are unchanged in both pacing regimes — the hold only moves
where a finished batch waits, never when the trainer consumes it.
- policy.version advances at publish confirmation (inference applies
immediately and pauses during the update, so published and applied
are indistinguishable to consumers), and WeightWatcher.stop() drains
an in-flight apply before cancelling — teardown can never strand the
trainer mid-handshake.
- Keep the trainer-side token-export guard, the multi-run test's
deletion-race helper, and the CI failure-artifact pipeline from the
earlier iterations; drop everything else.
Verified on the original repro (reverse_text, fast env, NCCL): 10/10
batches, clean trainer exit, 2 holds at the buffer-fed tail.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fold staleness spec into docs/training.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* drop the docs section for now, trim the hold comment
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* never delete the test output dir; clean at run start instead
The output_dir fixture deleted the whole workspace at session teardown,
racing the failure-artifact upload (and hiding logs from post-mortems
generally). Keep it unconditionally and make fresh starts responsible
for their own hygiene: alphabet_sort and reverse_text_moe now pass
--clean-output-dir like the other rl-launcher tests. Resume runs are
unaffected — validate_output_dir returns before cleaning when resuming.
The multi-run orchestrators manage per-run dirs the test creates (and
the standalone orchestrator entrypoint has no clean flag).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* stamp true consumption staleness on shipped rollouts
Batch N trains on policy v{N-1}, so a rollout generated from v{k} is
(N-1)-k versions off-policy — queue time included. The dispatcher's
in-flight counter only sees weight updates during generation, which
undercounts by the queue-wait aging and the dispatch-time baseline gap;
the ship-time stamp overwrites it so Max Off-Policy in logs and the
rollout records report the real number. The counter itself (and the
max_off_policy_steps drop rule it feeds) is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* use the deletion-retry helper for alpha's run dir too
Alpha is SIGTERM'd mid-run, so the trainer is actively flushing its
token-export tail when the dir is deleted — the recreate race is most
likely at exactly this site.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(nixl): replay strided views on inference (#3113)
* feat(ci): nightly FFT workflow (#3118)
* feat(ci): add nightly FFT workflow dispatching hosted training runs
* ci(nightly-fft): pin --image-tag to the latest published dev tag
* ci(nightly-fft): rename to reverse-text and align with pytest nightly setup
* ci(nightly-fft): pin deployment shape (2 train + 6 infer)
* ci(nightly-fft): switch to wordle mirroring examples/basic/wordle/rl.toml
* ci(nightly-fft): update wordle config to match examples/basic/wordle
* ci(nightly-fft): add explicit least-privilege permissions block
* ci(nightly-fft): drop top-level type/name — [deployment] alone triggers FFT dispatch
* ci(nightly-fft): resolve image tag from GHCR to avoid unpublished tags
* ci(nightly-fft): forward WANDB_API_KEY and HF_TOKEN into hosted payload via -e
* fix(trainer): skip no-op weight conversion for dense models (#3069)
Dense models (Qwen3, Qwen3.5, Llama) use identical key names in both
HF and PrimeRL weight formats, so is_hf_state_dict and is_prime_state_dict
both returned True for the same state dict. The auto-conversion logic in
load_dcp_from_hf would fire anyway, performing a no-op convert and
attempting to save the unchanged weights into the snapshot directory —
which crashes with OSError [Errno 30] Read-only file system when the
model cache is mounted read-only.
Fix: make is_hf_state_dict / is_prime_state_dict mutually exclusive for
dense models by having is_prime_state_dict return False (following the
pattern already used by GPT-OSS). This makes the conversion branches
naturally skip when there is no format difference, without needing
extra guards in load_dcp_from_hf.
MoE models are unaffected: their is_hf_state_dict / is_prime_state_dict
checks are already mutually exclusive (expert.0.w1 vs experts.w1 key
patterns).
* fix(trainer): skip identity HF↔PrimeRL conversion (fixes RO model-cache crash on dense Qwen3) (#3124)
* fix(trainer): skip identity HF↔PrimeRL conversion to avoid RO-cache write
* chore: ruff format
* refactor: drop inline explanation; gate branches on unambiguous snapshot format
* fix(ci): build ghcr image on release via explicit dispatch (#3125)
Releases are published with GITHUB_TOKEN, whose events do not retrigger
workflows, so build_image.yaml's `on: release` trigger never fired and
release commits never received a GHCR image. Dispatch build_image.yaml
explicitly from tag-and-release.yaml after publishing (workflow_dispatch
is exempt from the no-retrigger rule), and drop the now-dead `on: release`
trigger plus its unreachable github.event.release.* references.
* ci(nightly-fft): name RFTRun nightly-<config>-<date>-<run_id>; verify GHCR manifest exists before dispatch (#3121)
* ci(nightly-fft): add alphabet-sort, reverse-text, wiki-search, hendrycks-sanity (#3127)
* ci(nightly-fft): add alphabet-sort, reverse-text, wiki-search, hendrycks-sanity, multimodal configs
Broaden nightly-fft coverage beyond wordle. Each config mirrors wordle's
2 train + 6 infer shape and pulls model/taskset/renderer from the
corresponding configs/basic/*/rl.toml.
- multimodal-color-codeword moved from configs/ci/nightly/ (its pytest
path updated); nightly/ is now empty
- hendrycks-sanity trimmed from 5000 -> 200 steps to fit the nightly
budget
- wiki-search carries tool_call_parser = 'hermes' for tool use
* ci(nightly-fft): revert multimodal move; restore hendrycks max_steps=5000
* ci(nightly-fft): size deployments per env; add zero-advantage filter to wiki-search
- reverse-text: 1+1 GPUs, batch 128 (was 2+6/512 — overkill for a 0.6B toy loop)
- alphabet-sort: 4+4 GPUs (was 2+6 — 4B FFT needs more trainer memory)
- wiki-search: 4+4 GPUs; adds oversampling_factor=2.0 + zero_advantage
pre-batch filter mirroring examples/basic/wiki-search — without them sparse
tool rewards fill batches with all-zero groups and abort the run
* test(nightly): remove LoRA alphabet-sort and wiki-search — covered by FFT nightly workflow
* ci(nightly-fft): align hendrycks-sanity with examples/basic upstream
- drop max_completion_tokens=1024 cap so R1-Distill can produce full CoT
- group_size 16 -> 8, GPUs 2+6 -> 4+4 (upstream)
- add trainer.model.seq_len=16384 for long-context training
- add aime2024 eval block (interval=50, group_size=32)
* fix(nightly-fft): move seq_len under [orchestrator] in hendrycks-sanity
RLConfig rejects setting shared top-level seq_len alongside
trainer.model.seq_len. Mirror examples/basic/hendrycks-sanity/rl.toml:
seq_len=8192 under [orchestrator] for rollout context, seq_len=16384
under [trainer.model] for long-context training.
* revert(nightly): restore test_alphabet_sort and test_wiki_search — keep tests/nightly/ untouched
* fix(examples): set cp=4 for GLM-4.5-Air advanced examples (#3129)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix(nightly-fft): pin hendrycks attn to flash_attention_2 (#3131)
DeepSeek-R1-Distill-Qwen-1.5B is Qwen2 arch, not in prime-rl's
custom-impl support list, so impl='auto' resolves to 'hf'. The
platform default attn='flash_attention_3' requires impl='custom',
which raises ValueError and the trainer crash-loops. Pin attn to
fa2 (works with hf impl on H200).
* feat(ci): allow pinning prime-rl image_tag on nightly-fft dispatch (#3132)
* feat(ci): allow pinning prime-rl image_tag on nightly-fft dispatch
Adds an optional image_tag workflow_dispatch input. Empty (default)
keeps the existing auto-resolve behavior (newest v*.dev* with a
published manifest). Non-empty pins the tag directly, after verifying
the manifest is pullable.
* fix(ci): sanitize image_tag input to avoid shell injection
Move the workflow_dispatch input through a step env var so a
malicious value containing $(...) or embedded quotes cannot break
out of shell context during expression substitution. Validate the
result against the OCI/Docker tag charset before curl or GITHUB_OUTPUT
sees it.
* fix(nightly-fft): don't force --image-tag when no dispatch input (#3133)
* fix(nightly-fft): pin image_tag to commit-c505e07a1 in each config
The default 'main' image the platform validator ships is baked ~50
commits behind tip and predates cb74ae2be (which added the env.
wrapper on train.env entries). It rejects env.taskset with
'train.env[0].env — Extra inputs are not permitted' and doesn't know
the 'prime-qwen3' renderer name, so every nightly orchestrator
crash-loops on validation.
- Pin top-level image_tag = 'commit-c505e07a1' in each nightly-fft
toml. c505e07a1 is post-cb74ae2be and accepts the current v1 shape.
- Workflow: when no dispatch input is set, emit an empty tag and skip
--image-tag on prime train so the toml pin wins. Explicit
workflow_dispatch image_tag still overrides.
- Drop this pin once v0.7.1+ (with the v1 schema) is released and the
backend default resolves to it.
* revert: drop image_tag pin from nightly-fft configs
Keep the workflow change (only pass --image-tag when explicit input);
rely on prime-cli's built-in fallback: --image-tag > toml image_tag >
backend default. No commit-hash pin baked into the checked-in configs.
* fix(nightly-fft): restore auto-resolve latest v*.dev* when no image_tag input (#3134)
* fix(nightly-fft): restore auto-resolve latest v*.dev* when no image_tag input
The nightly's whole point is to catch regressions on tip-of-main, so a
plain 'gh workflow run "Nightly FFT"' must dispatch against the newest
built v*.dev* by default. #3133 removed that fallback and made no-input
dispatches emit an empty tag, which falls through to the platform's
default 'main' alias — currently baked ~50 commits stale — so the run
crashes on validation.
Restores the original behavior: no input → auto-resolve latest dev.
Explicit image_tag input still overrides (from #3132). Dispatch step
always passes --image-tag since the tag is always populated now.
* fix(ci): guard dev-tag resolver against pipefail
GitHub Actions runs the shell under `bash -eo pipefail` by default.
Two failure modes:
- `grep -E ...` returns 1 when no v*.dev* tags exist yet
- `head -n 10` closes the pipe after 10 lines, SIGPIPE-ing `sort`
Either aborts the step before the explicit no-candidates guard runs.
Wrap the pipeline in set +o pipefail / set -o pipefail.
* chore: bump verifiers to latest main; adopt runtime-on-agent, Reward records, textarena player (#3137)
* chore: bump verifiers to latest main; adopt runtime-on-agent, Reward records, textarena player
deps/verifiers b13ba60da → 295a06505 (+12: interleaving agents + kuhn-poker-v1,
runtime moves from the harness onto the agent (#2106), trace rewards become
Reward{score, weight} records (#2119), wordle-v1 rides the rewritten
TextArenaEnv, pluggable harness skills, agentic-judge grading knobs).
Adoption:
- every `env.<agent>.harness = { ..., runtime = ... }` config line splits into
`harness = { ... }` + `env.<agent>.runtime = ...` (configs/, examples/, docs/,
skills/)
- wordle configs address the textarena seat as `env.player`
- reward metrics read the weighted `Reward.value` (CustomMetrics gains a value
extractor); unit-test trace builders record `vf.Reward(score=...)`
- uv.lock re-locked for verifiers 0.2.2.dev29
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(tests): adopt Reward records in prime-monitor rollout builder
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(orchestrator): RAE — role-conditioned advantage estimation for self-play envs (SPIRAL) (#3136)
* feat(orchestrator): RAE — SPIRAL's role-conditioned advantage estimation for self-play envs
Add the `rae` algorithm (SPIRAL, arXiv:2506.24119): each trainable agent
keeps an EMA baseline of its own rewards, and every trace's advantage is
its reward minus its agent's pre-update baseline. Group-relative baselines
mis-credit multi-agent self-play — in a zero-sum game the group mean is ~0
whatever the policy does, and centering both agents against it turns any
structural asymmetry (first-mover edge) into permanent credit. Baselines
are keyed per (env, agent) — the paper's per (game, role) — and any
group_size works, including 1.
configs/debug/algo/rae.toml runs shared-policy Kuhn poker self-play
(kuhn-poker-v1, both agents late-bound to the run's model).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(tests): drop RAE unit tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* chore(ci): enable nightly FFT cron (#3142)
* chore: bump verifiers + research-environments to latest main (#3147)
- deps/verifiers -> d0bb0ffe5 (network-policy alignment #2124, Codex MCP tools
#2140, Ruff 0.16 / ty tooling, and everything since the last bump)
- deps/research-environments -> 6a2dee6fc (judges/swe agentic-judge setups #720,
MMMU-Pro / CharXiv tasksets, wideseek verifiers-pin fix #731)
- uv.lock relocked (adds charxiv-v1, mmmu-pro-v1; bfcl-v3-v1 0.2.0 -> 0.3.0)
No code changes needed: the runtime-on-agent config addressing and Reward-record
metrics landed with the earlier bump (#3137).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: plot overview time panels on relative wall time (#3148)
* fix(monitor): plot overview time panels on relative wall time
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: drop view_signature docstring
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com>
* feat: default rollout transport to zmq and inject host for multi-node (#3149)
* feat(config): shared [rollout_transport] propagated to trainer + orchestrator
Add rollout_transport as a shared RLConfig field (mirroring weight_broadcast /
ckpt / wandb), propagated to trainer.rollout_transport and
orchestrator.rollout_transport via propagate_shared_fields. A single
[rollout_transport] block configures both sub-configs.
auto_setup_rollout_transport (like auto_setup_weight_broadcast) resolves the
shared field from the sub-configs so the launcher's ZMQ host-injection gate is
correct whether transport is set via the shared block or directly on
trainer/orchestrator.rollout_transport (the documented fallback), and rejects
mismatched sub-config transports.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(launcher): inject rollout_transport.host for multi-node zmq transport
ZMQ rollout transport binds/connects a single rollout_transport.host (default
localhost). On multi-node trainer deployments the training-batch PUSH/PULL and
micro-batch PUB/SUB span nodes, so data ranks on non-head nodes connect to their
own localhost and hang at step 0. The launcher already injects
weight_broadcast.host=$MASTER_ADDR for nccl but had no equivalent for
rollout_transport.
When config.rollout_transport resolves to zmq, inject --rollout_transport.host
$MASTER_ADDR into the trainer torchrun and the orchestrator (mirrors the existing
weight_broadcast.host injection). Filesystem transport is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(config): default rollout transport to zmq
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: single global router in front of engines for all deployments (#3105)
* chore: single global router in front of engines for all deployments
Every deployment shape now runs one client-facing router on server.port
with engines behind it on backend_port: local single-node runs spawn a
vllm-router in the inference entrypoint, and the SLURM templates collapse
per-replica routers into a single global router on inference node 0.
router/backend_port move to InferenceConfig top level; router = "None"
runs a bare engine (used for per-rank SLURM engine processes).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: finish single-node router integration
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com>
* feat(monitor): effective reward/score in wandb overview (#3153)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: compute trainable rate over effective rollouts (#3152)
* fix(orchestrator): compute trainable rate over effective rollouts
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: drop comment
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* chore: bump renderers to 0.1.9.dev9 (#3158)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: read overview quality metrics from effective subset (#3159)
* fix: read overview quality metrics from effective subset
The all subset counts errored rollouts as zeros for num_branches,
num_turns, num_total_tokens, and is_truncated, which skews the
distributions and makes the panels uninterpretable. Point the wandb
overview panels and the eval success line at the effective subset;
has_error stays on all since effective drops errors by construction.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: reword zero-skew rationale, drop subset guidance from skill
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(orchestrator)!: rename env collections to sources, compose the verifiers config blocks (#3150)
* chore(configs)!: remove every config shim (#3156)
* feat(orchestrator): hierarchical GRPO for proposer-solver envs (#3138)
* chore: adapt to the verifiers trace record cleanups (#3157)
* rename training environments to sources
* fix CPU test config fixtures
* rename evaluation environments to sources
* chore(orchestrator)!: rename max_inflight_rollouts to max_inflight_episodes
A dispatcher permit has always bought one episode, not one rollout: for a v1
env one `run` request is one episode (however many agents play it), and a
legacy group-scoring call reserves one permit per bridged v0 rollout, each of
which is its own single-agent episode. Name the knob what it counts.
`max_inflight_rollouts` still parses as an alias, as `rollouts_per_example`
does for `group_size`.
* feat(orchestrator)!: compose the verifiers env, serve and legacy blocks
verifiers split `EnvServerConfig` into three blocks, so an env entry composes
them instead of inheriting one class:
[[orchestrator.train.source]]
env = { taskset = { id = "..." } } # what runs
serve = { pool = { ... }, address, max_concurrent } # how it's hosted
legacy = { id, args } # a classic v0 env instead
`serve.pool` sizes the spawned server (was `pool`), `serve.address` points at an
external one (was `address`), and `serve.max_concurrent` bounds one worker's
episodes in flight — usually left unset, since `max_inflight_episodes` is the
run's bound. Each moved key raises with a pointer home, including the
`num_workers` shorthand the old validator migrated silently.
BREAKING: `pool` / `address` / `num_workers` on an env entry move under `serve`,
and a v0 env's `id` / `args` / `extra_env_kwargs` move under `legacy`.
TEMPORARY: `deps/verifiers` points at the `feat/one-agent-per-episode` branch
(PrimeIntellect-ai/verifiers#2157) so the two sides can be run together; re-pin
to a merged `main` commit before this lands.
* fix the source-config readers the compose commit missed
The Prime monitor still built its run-registration payload from `env.id`, which
the composed entry doesn't have — `AttributeError` at registration whenever
Prime monitoring is on. It reads `env_id` now, the property that spans both the
v1 and the v0 shape.
The packer's fixture still used the bare `id` shorthand, so its three runs were
rejected at config parse and skipped by `discover_runs`. Full CPU unit suite:
416 passed.
Re-pins `deps/verifiers` to the branch tip of verifiers#2157, which fixes the
same class of stale access on that side (`config.legacy.id`). Still a branch
commit — re-pin to a merged `main` before this lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* spell the env annotation out instead of vf.EnvField
verifiers dropped the `EnvField` alias, which was only ever
`SerializeAsAny[EnvConfig]` under a name that collided with `env_field()`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* declare the env default with the factory verifiers exports
`vf.env_field()` is gone: it only wrapped `Field(default_factory=...)`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* inline the legacy helpers, use the config class as the default
Follows verifiers dropping the three `configs/legacy.py` helpers and the
`single_agent_env_config` factory: `is_legacy` and `env_id` are one-liners here,
the mixed-run refusal is spelled out in `validate_env`, and the `env` default is
`vf.SingleAgentEnvConfig()`. `env.max_concurrent` now works as an alias for
`env.max_concurrent_agents`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* drop the serving-key pointers here too
The same call as verifiers a0ebc408: `num_workers` / `pool` / `address` on a source
entry now fail as an ordinary `extra_forbidden` instead of naming their new home.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* bump verifiers: no max_concurrent alias on the env block
`env.max_concurrent` used to bound a served worker's agent runs across episodes, so
accepting it as `max_concurrent_agents` multiplied it by the episodes in flight. It
errors now, like every other moved key. 416 unit tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* bump verifiers to 0.2.2.dev43 (merged main)
deps/verifiers d0bb0ffe5 -> fcb48822e, which is the squash-merge of the composed
env/serve/legacy config blocks (#2157) plus everything else since the last bump:
Harbor 0.20.0 (#2161), runtime-blamed box failures (#2156), standalone MCP clients
pinned to v1 (#2155), and the v1 test/E2E split (#2127).
Lock: harbor 0.14.0 -> 0.20.0, which drops claude-agent-sdk and datasets from its
own deps and adds filelock/platformdirs, so those move with it.
Replaces the temporary branch pin this PR carried.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: spawn the env server again, and let a pip-only install resolve verifiers
Two CI failures from the compose commit.
The orchestrator stopped spawning env servers. It decides spawn-vs-connect by
`address is None`, and prime-rl's own field defaulted to None; `vf.ServingConfig`
defaults to `tcp://127.0.0.1:5000`, so every env looked external and the run waited
120s for a server nobody started. It now asks whether the address was *pinned*
(`model_fields_set`), so an unset `[serve]` spawns on a free port as before, and
pinning any address — including verifiers' own default — connects.
The slim `prime-rl-configs` wheel is installed from /tmp with no workspace context,
so `verifiers` resolves from PyPI, where `>=0.2.1` picks stable 0.2.1 — no
`SingleAgentEnvConfig`. Floor is `>=0.2.2.dev43` now, a pre-release specifier so the
dev release is eligible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: adapt to the verifiers trace record cleanups
Companion to PrimeIntellect-ai/verifiers#2172: record_* mutators, required
agent seat and RunInfo.id, last_error, and agent.name/.trainable reads.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore: re-pin verifiers to merged main (f646beb37)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: hallerite <git@hallerite.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore(inference): bump vLLM to 0.26.0 (#3166)
* chore(configs): expand inline TOML tables to sections (#3167)
Every checked-in config, example, and doc snippet now uses [section]
headers instead of key = { ... } inline tables. Purely stylistic: all
48 rewritten TOML files parse identically to their previous versions.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: minh hoang <13672394+eexwhyzee@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: hallerite <git@hallerite.com>
Co-authored-by: Matej Sirovatka <54212263+S1ro1@users.noreply.github.com>
Co-authored-by: rasdani <73563550+rasdani@users.noreply.github.com>
Co-authored-by: Mika Senghaas <mail@mikasenghaas.de>
Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>
Co-authored-by: Hubert <163992334+hubert-marek@users.noreply.github.com>
Co-authored-by: eligotts <78387377+eligotts@users.noreply.github.com>
Co-authored-by: eligotts <eligotts@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: JannikSt <JannikSt@users.noreply.github.com>
Co-authored-by: S1ro1 <matej.sirovatka@gmail.com>
Co-authored-by: Jonas Lossin <jonas.lossin@hai-login1.haicore.berlin>
eexwhyzee
added a commit
that referenced
this pull request
Jul 31, 2026
Cherry-pick of 9f0d41e (#3166) onto this branch. vLLM 0.24.0 cannot load the gpt-oss BF16 checkpoint. The FusedMoE refactor moved expert params under mlp.experts.routed_experts.*, but gpt_oss.py's _load_weights_other kept the flat params_dict lookup, so unsloth/gpt-oss-20b-BF16 fails weight loading with KeyError: 'layers.0.mlp.experts.w2_weight' (vllm#45830, fixed upstream by vllm#45818, first released in 0.25.0). 0.25.0 also moved vllm.entrypoints.serve.disagg to vllm.entrypoints.scale_out.token_in_token_out and renamed the renderer attribute, so the import adaptations and the renderers bump are required alongside the version bump. uv.lock was regenerated on this branch rather than cherry-picked; it resolves to the same versions as main for every vLLM-adjacent package, and torch/transformers/flash-attn are unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bump the pinned CUDA 12.9 vLLM wheels from 0.24.0 to 0.26.0 on x86_64 and aarch64.
Update Prime token serving for the vLLM 0.26 API:
scale_out.token_in_token_outOnlineRenderermember exposed by the newServingTokensimplementationMultiModalKwargsItemsrenderers>=0.1.9.dev10from standaloneprime-rl-configsinstallsThe lockfile was regenerated with the repository-required uv 0.11.1 and includes vLLM 0.26 transitive dependency changes.
Validation:
uvx --from uv==0.11.1 uv lock --checkuvx --from ruff==0.12.1 ruff checkon all changed Prime Python filespytest -q tests/unit/inference/test_serving_tokens.pyon the Linux vLLM 0.26-compatible runtime: 14 passed