WIP: plan phase-specific expert policy

[drunkcoding]

Description

Plan-only design for separate prefill/decode expert admission, prefetch, and eviction policy over one shared cache.

Motivation

Prefill and decode have different expert working sets but currently share one undifferentiated policy.

Scope

• Explicit phase propagation and disabled compatibility
• Single ExpertResidencyManager authority
• Deferred per-device capacity configuration
• Deterministic eligibility, leases, transactions, and telemetry

Dependencies / limitations

This shared residency manager is the prerequisite substrate for adaptive mixed precision. No separate P/D GPU pools.

Type of Change

• Documentation update
• Production implementation

Performance / support evidence

• Not applicable — workload-specific TTFT/TPOT matrix only.

Checklist

• Router semantics unchanged
• One-flag rollback specified
• No production files changed

[mfethe1]

Hi — we maintain a replay harness that measures cache-policy ceilings for expert residency on real routing traces (Kimi E3 30B-A3B / Qwen3-A3B class models, CPU+slow-device expert paging). Since #179 is plan-only, here are measured numbers from our side that bound the design space before you commit to an implementation:

1. Policy-ceiling magnitudes (decode, replay harness). Across admission/prefetch/eviction policy arms we measured a +2–8% e2e decode ceiling; the ceiling scales with demand coverage f: f=0.70 → +4.5%, f=1.00 → +12.8%. One measurement correction that matters for your TTFT/TPOT matrix: score policy gains against demand-miss I/O, not total I/O — total-I/O accounting inflates apparent policy wins by counting prefetch traffic as saved I/O (we initially made this mistake; it's a systematic bias, not noise).

2. Negative result you should design around: decode-side admission gating is inert on our traces. We tested phase-split admission (decode-phase priority knobs over the same residency authority) on a full decode trace: 0/9 cells passed a +3pp threshold; the decode-priority knob moved the target metric exactly 0.000pp; decode-only admission lost 0.3–7.0pp (it evicts warm prefill-admitted experts). Our read: in #179, the phase-split value has to come from prefill-side residency/prefetch planning (prefill sees the whole chunk's expert set at once — admission there is genuinely informative), not from gating during decode, where per-token expert demand is nearly memoryless. If your traces show otherwise we'd love to see the delta — that's exactly the kind of cross-check the plan stage is for.

3. Architectural constraint we hit that your invariants anticipate: with a single ExpertResidencyManager authority, keep expert lookup O(1) hash + no mutex held across the device read. We measured a peer implementation where a global mutex + linear slot scan serialized loads to QD1 — more RAM made it slower (longer slot scan). Your "single authority, no second residency authority" invariant is right; add "O(1) lookup, QD≥4" to it.

Happy to share the routing trace + ceiling numbers as calibration cases for the #179 TTFT/TPOT matrix — measurement-first, happy to trade. (Same numbers also bound #184: precision×residency is a cross-term, and the policy ceiling caps what adaptive precision residency can recover on top of a good static policy.)