Average planned verification depth across request rounds

Count allocated draft prefixes from validated completions and divide by
progressing decode or verification request rounds. Include rejected drafts
and zero-draft rounds while excluding padding, prefill, failed lanes, and
terminal rounds without progress. Shared accounting covers combined, split,
packed, and mixed speculation, including copy drafts.

Expose the counters through JSON and Prometheus. The verification_drafts
JSON field now measures per-request allocation instead of graph maximum
width; verification_rounds supplies its denominator. Prepared packages and
execution policy are unchanged.

Validation: 872 Helios library tests passed with 22 ignored, including
packed allocations [5, 1, 0] reporting depth 2.0 with zero acceptance,
ordinary decode, and combined MTP/copy accounting. Flare release build and
Heliopause installation passed. Full-model GPU validation was not run;
the running service has not been restarted.

Design-Signed-Off-By: Codex (GPT-6); design-review=6aeb

Author: lukealonso

docs/api.md

@@ -220,7 +220,8 @@ instantaneous gauges.
 | `decode_weight_bytes` | Bytes of weights one decode step reads for one request, as the package's recipe declares for its checkpoint; zero when undeclared. Times verification steps per second it is the weight bandwidth the model consumes, shown as WEIGHTS on the dashboards | `flare_decode_weight_bytes` |
 | `draft_steps` | Draft positions run by the learned drafter; zero without one | none |
 | `verify_steps` | Completed speculative or verification graphs | sum of `flare_speculative_rounds_total` |
-| `verification_drafts` | Draft positions verified: verification width minus one per graph | sum of `flare_speculative_rounds_total` × (`width` − 1) |
+| `verification_drafts` | Draft positions allocated for progressing request rounds, including rejected drafts and excluding padding | `flare_verification_draft_tokens_total` |
+| `verification_rounds` | Progressing decode or verification request rounds, including zero-draft rounds | `flare_verification_request_rounds_total` |
 | `accepted_drafts` | Accepted draft tokens, excluding the bonus token | `flare_accepted_draft_tokens_total` |
 | `draft_rounds` | Request rounds that offered at least one draft | `flare_draft_request_rounds_total` |
 | `active` | Model slots with a live request | `flare_active_requests` |
@@ -243,6 +244,13 @@ instantaneous gauges.
 | `host_bytes_in` | Bytes copied from the host store into GPU pages and rows | `flare_host_cache_bytes_in_total` |
 | `learned_drafter` | Boolean: the model has a learned drafter | none |
 
+Dashboard DEPTH is the change in `verification_drafts` divided by the change in
+`verification_rounds` over the sampling window. It averages each request's planned
+draft-prefix length, including zero-draft rounds, rather than graph width or batch
+capacity. For packed speculation, completion records report the prefixes allocated
+on the GPU before target verification. Prefill, resume sampling, inactive padding,
+failed lanes, and terminal rounds without progress do not enter this average.
+
 ## Conversation and tool semantics
 
 The neutral request representation in `helios/src/server/inference.rs` separates

helios/README.md

@@ -63,6 +63,10 @@ forwards inside completed speculative graphs; verification steps count completed
 speculative graphs. Copy-only proposals contribute no drafter steps. Mean draft
 acceptance length is accepted draft tokens per progressing request round with
 at least one offered draft, excluding the bonus token and width-one rounds.
+DEPTH is allocated draft positions per progressing decode or verification request
+round, including zero-draft rounds. Packed rounds contribute their allocated
+per-request prefixes; the graph's maximum width and padded batch size do not
+determine this average. Rejected drafts count toward depth.
 The display also shows active/queued requests, completed/failed requests, reused
 tokens, and cached/free KV pages. Token counters are available through `/metrics`.
 

helios/src/server/dashboard.rs

@@ -334,7 +334,7 @@ pub struct Rates {
     pub verify: f64,
     /// Accepted drafts per draft-offering request round; `None` without such rounds.
     pub acceptance: Option<f64>,
-    /// Verified draft positions per verification graph; `None` without verification.
+    /// Allocated draft positions per progressing decode or verification request round.
     pub depth: Option<f64>,
     /// Weight bytes read per second: verification graphs per second times the bytes one decode
     /// step reads, as the package declares; zero when it declares none.
@@ -377,7 +377,7 @@ impl History {
             }
         };
         let rounds = snapshot.draft_rounds.saturating_sub(before.draft_rounds);
-        let verifications = snapshot.verify_steps.saturating_sub(before.verify_steps);
+        let verifications = snapshot.verification_rounds.saturating_sub(before.verification_rounds);
         self.rates = Rates {
             prefill: per_second(snapshot.prefill_tokens, before.prefill_tokens),
             generated: per_second(snapshot.generated_tokens, before.generated_tokens),
@@ -761,7 +761,8 @@ mod tests {
             generated_tokens: 8,
             draft_steps: 7,
             verify_steps: 1,
-            verification_drafts: 5,
+            verification_drafts: 7,
+            verification_rounds: 3,
             accepted_drafts: 6,
             draft_rounds: 2,
             ..Default::default()
@@ -772,15 +773,16 @@ mod tests {
             generated_tokens: 10,
             draft_steps: 8,
             verify_steps: 2,
-            verification_drafts: 8,
+            verification_drafts: 10,
+            verification_rounds: 5,
             accepted_drafts: 7,
             draft_rounds: 3,
             ..Default::default()
         };
         history.sample(start + Duration::from_secs(2), last);
         assert_eq!(history.rates.prefill, 150.0);
         assert_eq!(history.rates.generated, 5.0);
-        assert_eq!(history.rates.depth, Some(4.0));
+        assert_eq!(history.rates.depth, Some(2.0));
         assert_eq!(history.rates.verify, 1.0);
         assert_eq!(history.rates.acceptance, Some(7.0 / 3.0));
         for tick in 3..=130 {
@@ -795,6 +797,7 @@ mod tests {
             start + Duration::from_secs(131),
             Snapshot {
                 verify_steps: last.verify_steps + 1,
+                verification_rounds: last.verification_rounds + 2,
                 ..last
             },
         );
@@ -822,6 +825,7 @@ mod tests {
                     draft_steps: tick * 12,
                     verify_steps: tick * 3,
                     verification_drafts: tick * 12,
+                    verification_rounds: tick * 6,
                     accepted_drafts: tick * 7,
                     draft_rounds: tick * 3,
                     ..cache

helios/src/server/metrics.rs

@@ -76,8 +76,10 @@ pub struct Snapshot {
     pub draft_steps: u64,
     /// Completed speculative or verification graphs, including terminal no-op rounds.
     pub verify_steps: u64,
-    /// Draft positions those graphs verified: verification width minus one per graph.
+    /// Draft positions allocated for progressing request rounds, including rejected drafts.
     pub verification_drafts: u64,
+    /// Progressing decode or verification request rounds, including zero-draft rounds.
+    pub verification_rounds: u64,
     /// Accepted draft tokens from progressing request rounds, excluding the bonus token.
     pub accepted_drafts: u64,
     /// Progressing request rounds that offered at least one draft.
@@ -203,6 +205,8 @@ pub(super) struct Metrics {
     generated_tokens: AtomicU64,
     accepted_drafts: AtomicU64,
     draft_rounds: AtomicU64,
+    verification_drafts: AtomicU64,
+    verification_rounds: AtomicU64,
     pub admitted: AtomicU64,
     pub http_requests: [AtomicU64; 5],
     pub completed: AtomicU64,
@@ -268,12 +272,15 @@ impl Metrics {
         }
     }
 
-    pub fn progress(&self, prefill: u64, generated: usize, accepted: u64, rounds: u64) {
+    pub fn progress(&self, prefill: u64, generated: usize, accepted: u64, rounds: u64,
+        verification_drafts: u64, verification_rounds: u64) {
         for (counter, value) in [
             (&self.prefill_tokens, prefill),
             (&self.generated_tokens, generated as u64),
             (&self.accepted_drafts, accepted),
             (&self.draft_rounds, rounds),
+            (&self.verification_drafts, verification_drafts),
+            (&self.verification_rounds, verification_rounds),
         ] {
             if value != 0 {
                 counter.fetch_add(value, Ordering::Relaxed);
@@ -288,6 +295,8 @@ impl Metrics {
             generated_tokens: self.generated_tokens.load(Ordering::Relaxed),
             accepted_drafts: self.accepted_drafts.load(Ordering::Relaxed),
             draft_rounds: self.draft_rounds.load(Ordering::Relaxed),
+            verification_drafts: self.verification_drafts.load(Ordering::Relaxed),
+            verification_rounds: self.verification_rounds.load(Ordering::Relaxed),
             active: self.active_requests.load(Ordering::Relaxed),
             pending: self.pending.load(Ordering::Relaxed),
             completed: self.completed.load(Ordering::Relaxed),
@@ -310,10 +319,9 @@ impl Metrics {
             ..Snapshot::default()
         };
         for widths in &self.speculative {
-            for (drafts, counts) in widths.iter().enumerate() {
+            for counts in widths {
                 let rounds = counts.rounds.load(Ordering::Relaxed);
                 snapshot.verify_steps += rounds;
-                snapshot.verification_drafts += rounds * drafts as u64;
             }
         }
         snapshot
@@ -625,6 +633,16 @@ impl Metrics {
                 "Progressing live request rounds offering at least one draft; denominator for mean acceptance length.",
                 &self.draft_rounds,
             ),
+            (
+                "verification_draft_tokens_total",
+                "Draft positions allocated for progressing live request rounds, including rejected drafts and excluding padding.",
+                &self.verification_drafts,
+            ),
+            (
+                "verification_request_rounds_total",
+                "Progressing live decode or verification request rounds, including zero-draft rounds; denominator for mean planned verification depth.",
+                &self.verification_rounds,
+            ),
             (
                 "requests_admitted_total",
                 "Requests admitted into a physical model slot.",
@@ -998,7 +1016,6 @@ mod tests {
         for learned in [false, true] {
             let snapshot = metrics.snapshot(learned);
             assert_eq!(snapshot.verify_steps, 1);
-            assert_eq!(snapshot.verification_drafts, 7);
         }
         let text = metrics.render();
         for line in [
@@ -1024,7 +1041,6 @@ mod tests {
         let snapshot = metrics.snapshot(true);
         assert_eq!(snapshot.draft_steps, 7);
         assert_eq!(snapshot.verify_steps, 2);
-        assert_eq!(snapshot.verification_drafts, 2);
         let text = metrics.render();
         assert!(text.contains("flare_proposal_rounds_total{batch=\"4\",depth=\"7\"} 1"));
         assert!(text.contains("flare_proposal_gpu_nanoseconds_total{batch=\"4\",depth=\"7\"} 700"));
@@ -1034,7 +1050,7 @@ mod tests {
     #[test]
     fn snapshot_json_round_trips_beside_the_prometheus_counters() {
         let metrics = Metrics::new(100, 128, 0);
-        metrics.progress(40, 3, 2, 1);
+        metrics.progress(40, 3, 2, 1, 5, 2);
         metrics.scheduler(SchedulerStats {
             active_requests: 2,
             free_pages: 10,
@@ -1044,6 +1060,8 @@ mod tests {
         let snapshot = metrics.snapshot(true);
         let json = snapshot.json(true);
         assert_eq!(json["generated_tokens"], 3);
+        assert_eq!(json["verification_drafts"], 5);
+        assert_eq!(json["verification_rounds"], 2);
         assert_eq!(json["free_pages"], 10);
         assert_eq!(json["learned_drafter"], true);
         assert_eq!(serde_json::from_value::<Snapshot>(json).unwrap(), snapshot);
@@ -1057,6 +1075,8 @@ mod tests {
         let text = metrics.render();
         for line in [
             "flare_generated_tokens_total 3",
+            "flare_verification_draft_tokens_total 5",
+            "flare_verification_request_rounds_total 2",
             "flare_prefill_tokens_total 40",
             "flare_active_requests 2",
             "flare_free_kv_pages 10",

helios/src/server/runtime.rs

@@ -1319,10 +1319,13 @@ impl<'a> Worker<'a> {
         let mut prefill = 0;
         let mut accepted = 0;
         let mut draft_rounds = 0;
+        let mut verification_drafts = 0;
+        let mut verification_rounds = 0;
         if matches!(
             kind,
             GraphKind::Prefill
                 | GraphKind::ScoredPrefill
+                | GraphKind::Decode
                 | GraphKind::Speculative
                 | GraphKind::Verification
                 | GraphKind::PackedSpeculative
@@ -1343,9 +1346,11 @@ impl<'a> Worker<'a> {
                         .saturating_sub(live.consumed_tokens)
                         .min(lane.tokens as u64);
                 } else if record.generated_tokens > live.generated_tokens as i64 {
+                    verification_rounds += 1;
                     if let Some(vector) = record.vector {
                         let offered =
                             (vector.offered_drafts as usize).min(lane.tokens.saturating_sub(1));
+                        verification_drafts += offered as u64;
                         if offered > 0 {
                             accepted += (vector.accepted_drafts as usize).min(offered) as u64;
                             draft_rounds += 1;
@@ -1377,7 +1382,8 @@ impl<'a> Worker<'a> {
                     completion.speculative_survival.as_ref().map(|rows| rows.as_slice()))?
         };
         self.metrics
-            .progress(prefill, generated.len(), accepted, draft_rounds);
+            .progress(prefill, generated.len(), accepted, draft_rounds,
+                verification_drafts, verification_rounds);
         self.metrics.replay_completed(completion.timing);
         if prefill > 0 && self.store.is_some() {
             self.observe_prefill_rate(completion.fence, prefill);

helios/src/server/runtime/phase_tests.rs

@@ -430,6 +430,9 @@ fn worker_observes_prefill_decode_and_prerequisite_timings() {
         frame.timing.wall_ns = 350;
         worker.complete(kind, frame).unwrap();
         assert_eq!(worker.scheduler.estimated_duration_ns(&step), Some(619));
+        let metrics = fixture.metrics.snapshot(false);
+        assert_eq!(metrics.verification_drafts, 0);
+        assert_eq!(metrics.verification_rounds, u64::from(kind == GraphKind::Decode));
     }
 }
 
@@ -903,6 +906,8 @@ fn speculative_dashboard_counts_live_rounds_and_batched_forward_passes() {
                 .is_err()
         );
         assert_eq!(fixture.metrics.snapshot(!copy).generated_tokens, 1);
+        assert_eq!(fixture.metrics.snapshot(!copy).verification_rounds, 0);
+        assert_eq!(fixture.metrics.snapshot(!copy).verification_drafts, 0);
         worker.complete(GraphKind::Speculative, frame).unwrap();
         let profile = worker
             .scheduler
@@ -916,11 +921,90 @@ fn speculative_dashboard_counts_live_rounds_and_batched_forward_passes() {
         let metrics = fixture.metrics.snapshot(!copy);
         assert_eq!((metrics.prefill_tokens, metrics.generated_tokens), (13, 4));
         assert_eq!((metrics.accepted_drafts, metrics.draft_rounds), (2, 1));
+        assert_eq!((metrics.verification_drafts, metrics.verification_rounds), (3, 1));
         assert_eq!(metrics.verify_steps, 1);
         assert_eq!(metrics.draft_steps, if copy { 0 } else { 3 });
     }
 }
 
+#[test]
+fn packed_dashboard_depth_uses_allocated_prefixes_and_live_request_count() {
+    use crate::model::contract::{GraphOperation, ValueSource};
+    let mut fixture = Fixture::new();
+    fixture.config.speculative_widths = (1..=8).collect();
+    let mut manifest = crate::model::testing::manifest(&fixture.config);
+    let name = "Speculative-4-8";
+    let graph = manifest.graphs.iter_mut().find(|g| g.name == name).unwrap();
+    graph.kind = GraphKind::PackedSpeculative;
+    graph.draft_tokens = 7;
+    graph.packed_token_capacity = 9;
+    let contract = fixture.config.contract.graphs.iter_mut().find(|g| g.graph == name).unwrap();
+    contract.kind = GraphOperation::PackedSpeculative;
+    for input in &mut contract.inputs {
+        if input.source == ValueSource::QueryLength {
+            input.source = ValueSource::SpeculativePrefixLimit;
+        }
+    }
+    let mut scheduler = RequestScheduler::new(fixture.config.identity, fixture.config.limits,
+        crate::model::scheduler_capabilities(&fixture.config)).unwrap();
+    let graphs: Vec<_> = manifest.graphs.iter().filter(|g|
+        g.kind != GraphKind::Speculative || g.token_capacity == 1).cloned().map(|mut g| {
+            g.estimated_duration_ns = match g.kind {
+                GraphKind::Speculative | GraphKind::Decode => 100_000,
+                GraphKind::PackedSpeculative => 10_000,
+                _ => 100,
+            };
+            g
+        }).collect();
+    scheduler.measured_graph_durations(&graphs).unwrap();
+    scheduler.enable_speculation(&(1..=8).collect::<Vec<_>>()).unwrap();
+    scheduler.split_speculation(8).unwrap();
+    let (sender, receiver) = mpsc::channel();
+    let mut generations = Vec::new();
+    for token in 0..3 {
+        let (incoming, generation) = fixture.incoming(vec![token; 13], 16);
+        sender.send(vec![incoming]).unwrap();
+        generations.push(generation);
+    }
+    let mut worker = Worker::new(&fixture.config, &fixture.frontend, &fixture.stop,
+        &fixture.metrics, scheduler, receiver, 4);
+    worker.inputs = Some(crate::model::Inputs::new(&fixture.config, &manifest).unwrap());
+    assert_eq!(worker.admit().unwrap(), ControlFlow::Continue(true));
+    let mut sequence = 0;
+    let step = loop {
+        sequence += 1;
+        assert!(sequence < 20);
+        let (plan, _) = worker.plan(0).unwrap();
+        let step = plan.unwrap();
+        worker.reserve_and_pack(&step, Fence(sequence)).unwrap();
+        if step.kind == GraphKind::PackedSpeculative { break step; }
+        let frame = completion(&worker, &step, sequence, step.kind.is_prefill().then_some(0));
+        worker.complete(step.kind, frame).unwrap();
+    };
+    assert_eq!((step.batch_capacity, step.token_capacity, step.lanes.len()), (4, 8, 3));
+    let mut frame = completion(&worker, &step, sequence, Some(1));
+    frame.proposal_confidence = Some([[0.5; 7]; crate::execution::REQUEST_LANES]);
+    for (lane, offered) in step.lanes.iter().zip([5, 1, 0]) {
+        let record = &mut frame.records[lane.request.slot as usize];
+        record.consumed_tokens = lane.start_position as i64 + 1;
+        record.vector = Some(crate::execution::TokenVector {
+            emitted_count: 1, offered_drafts: offered, accepted_drafts: 0,
+            verification_width: 8, draft_cache_consumed: lane.start_position as i64,
+            tokens: [1, -1, -1, -1, -1, -1, -1, -1], ..Default::default()
+        });
+    }
+    worker.complete(GraphKind::PackedSpeculative, frame).unwrap();
+    let metrics = fixture.metrics.snapshot(true);
+    assert_eq!((metrics.verification_drafts, metrics.verification_rounds), (6, 3));
+    assert_eq!((metrics.accepted_drafts, metrics.draft_rounds), (0, 2));
+    assert_eq!((metrics.draft_steps, metrics.verify_steps), (7, 1));
+    let start = std::time::Instant::now();
+    let mut history = super::super::dashboard::History::new(start, Default::default());
+    history.sample(start + std::time::Duration::from_secs(1), metrics);
+    assert_eq!(history.rates().depth, Some(2.0));
+    assert_eq!(history.rates().acceptance, Some(0.0));
+}
+
 #[test]
 fn shared_decoded_history_reaches_copy_and_mtp_uploads_after_response_completion() {
     use crate::model::contract::{GraphOperation, InputIndexing, ValueSource};
@@ -2203,7 +2287,7 @@ async fn metrics_route_serves_prometheus_or_json_without_the_api_key() {
     use std::io::{Read, Write};
     let (sender, _receiver) = mpsc::sync_channel(4);
     let state = http_state(sender, Some("serving-key".into()));
-    state.model.metrics().progress(10, 3, 0, 0);
+    state.model.metrics().progress(10, 3, 0, 0, 0, 0);
     let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
     let address = listener.local_addr().unwrap();
     let router = super::super::router(state, &[], 65536);