Every measurement in this project that changed our minds, in the order it changed them — including the ones that made us undo work.
The numbers here are copied from
docs/BENCHMARKS.md, which is the record. This file exists for a different reason: BENCHMARKS says what was measured, and this says what we expected before measuring it. The entries worth reading are the ones where those two differ.Machine: M-series Mac, 32 GB, Xcode 26, iPhone 17 Pro simulator on iOS 26.5, unless an entry says otherwise.
A benchmarks file is a list of facts and reads like one. It cannot tell you that a number was a surprise, and the surprises are the only part with teaching in them. Most of the entries below reversed a decision we had already made and in several cases already shipped.
The pattern in every one of them is the same, and it is not "we were careless". It is that a plausible mechanism is not evidence, and the cost of finding out was always lower than the cost of being wrong.
What we believed: that a local model judging failed steps was worth measuring, and that 64% on the first population was a modest but real signal.
What we measured: the population was 30 stop against 5 wait. Always
answering stop scores 86%. The model scored 64% — worse than a constant.
What changed: nothing about the model. The number had never been about the
model. Every accuracy figure computed on a skewed population is a fact about the
fixture set wearing the model's name, and scripts/score-rulings.mjs now prints
the majority-class baseline above every accuracy it reports and refuses to let
one be read without the other.
The general form: an accuracy without its baseline is not a measurement. This cost a week of believing a component was mediocre when we had no idea.
What we believed: that warming the model at startup would make the first judgement faster, and that when first-call latency did not improve the warm-up was not working.
What we measured: it was working. The old warm-up was a throwaway
respond() — it loaded the same weights the hard way and paid a full generation
to do it, so the cost had simply moved earlier rather than disappearing.
prewarm() is the supported call and does not generate: ~880 ms cold against
~600 ms warm.
What changed: the axis. We had been measuring wall-clock to first answer, which mixes weight residency with generation. Residency is process- and system-managed and outlives the session, which is also why a fresh session per judgement is affordable at all.
What we believed: that the rule separating an iOS large title (chrome) from the content beneath it was fixed, because it had been validated against a real non-Apple app.
What we measured: on the React Native testbed, the same 62.9 pt inset produced opposite answers on two screens. The "is this a real inset" test compared against the screen's median row gap: a 24-row list gives a median of 0 and the rule fires; four rows above a tab bar give a median of 414 and it does not. A system-drawn title was chrome or content depending on how many rows happened to sit below it.
What changed: both bounds became absolute platform constants.
TOKEN_RULES_VERSION went to 8.
The general form: a threshold derived from the data it is classifying will pass every test written on one screen. The testbed found this on its first day of existence, which is most of the argument for having built it.
What we believed: that once the fingerprint could see a screen's title, two screens with different titles would stop colliding.
What we measured: 0.50 similarity against a 0.36 threshold — still a match. The name was one token out of six, so it was outvoted by the structure.
What changed: a chrome label became decisive rather than merely present. Two readings that both name themselves, and name themselves differently, are not the same screen. Two where either side is silent are still compared normally, because silence is not disagreement.
What we believed: that the wedged-device problem was a CoreSimulator bug we would have to work around.
What we measured: one log held 670 damage-callback registrations and zero unregistrations. Registration was not idempotent, and every recovery attempt added one.
What changed: unregisterChangeCallback() is now called from every path that
re-registers.
What is still not proven, and must not be claimed: the capture soak that followed had zero capture failures, so it never exercised the recovery path at all. The mechanism is established; the fix is not verified against the symptom. Writing this down is the point of this file.
What we believed: that a graph built by scanning an app's source — screens, routes, destinations — could remove a large share of the model calls that navigation costs.
What we measured: 9% of escalations in the log are ones such a graph could have answered. Not zero, and not the transformation it felt like.
What changed: the idea is worth a prototype and is not worth a phase, and the other two angles it opens — element names and the destructive barrier — look stronger than navigation. Measuring the prize before building the solution is now the default; two phase premises in a row had been false.
What we believed: that 18 collected rulings were 18 data points.
What we measured: 14 of them came from the collection script's own plumbing — setup steps failing, not fixtures.
What changed: the scaffold runs with supervisor: 'none' so it cannot
generate rulings about itself.
What we believed: that a fixture which ran without error had reached the screen it claimed to be testing.
What we measured: three of four rulings in one run were taken on step 1 of a three-step form while the fixture's label said step 3.
What changed: fixtures assert arrival. Worth noting precisely because
scripts/eval-fingerprint.mjs already had exactly this check and it had not been
carried over — the lesson existed in the repository and still had to be learned
twice.
And a third time, the same day this file was written. The fingerprint tour
waited for a network page load with {"pause": 3000}. On a hosted runner that
is sometimes not enough: one reading of example.com had a stop button where the
others had refresh and no content tokens at all, scored 0.31 against a 0.46
bar, and turned CI red under the message "the threshold no longer has the
clearance this bar states". The threshold was fine. Every fixed sleep in that
tour is now an arrival assertion — which the project's own rules had required
since Phase 11.
What we believed: that off-screen elements were filtered out of the element map.
What we measured: every filter in the project checked y and ignored x. A
filter chip at x = 422 on a 402 pt screen counted as visible, and scrollTo
reported "'Assigned to Me' is in view at 422,277 already" — confidently wrong
about the one question it exists to answer.
What changed: both axes. And the cost was not the wrong answer: it was that the wrong answer was confident, so the recovery was hand-tuned swipes and two overshoots. Reported as the most expensive finding of that session.
What we believed: that fixing the large-title rule would visibly improve the fingerprint distributions.
What we measured: the gap did not move. 0.48 either way.
What changed: how we describe the fix. The collision it prevents has never happened on this machine — it happened on a CI runner, where a sparse OCR-only reading of two nameless screens matched exactly. The change removes the precondition for a failure, not a measured regression, and saying otherwise would have been a number-shaped claim with no number behind it.
"we can of course do our own test with a chosen model… and decide based on the numbers rather than speculations." — 2026-09-11
What we believed, and recommended: that we should not spend time comparing a larger local model, because the three-word vocabulary makes capacity nearly irrelevant.
That recommendation was overruled, correctly, and the numbers are below.
Most small-model errors are invalid-output faults, not judgement faults. Apple's guided generation eliminates those at the sampling layer — constraints are enforced by logit masking, so a fourth word is unrepresentable rather than rejected afterwards (WWDC25 301; Tech Report §7). An unconstrained challenger would lose on formatting and we would read it as losing on judgement.
So every model arm is:
- given the same briefing, which is not a second copy of the prompt but is
read out of
native/supervise.swiftat run time bysrc/ollama.js, because two hand-maintained copies of a prompt is two arms answering different questions and the difference would be invisible in the numbers; - constrained to the same three words by a JSON schema whose
decisionis an enum, which Ollama enforces at sampling exactly as Apple does; - asked not to deliberate (
think: falsefor Qwen3), because the Apple arm does not either and a supervisor that answers in twenty seconds has already lost the argument it is here to have.
Model-without-briefing is dropped with cause, not for convenience: asked cold it scored about one in four and called a list that was plainly still arriving a dead end.
Five fixtures on the in-tree React Native testbed, each with a known correct
answer, seeded so the same failure happens on demand. wait/retry differ only
in how long they settle, so both satisfy a situation whose answer is "wait".
22 rulings from one device pass (6 seeds x 5 fixtures, 30 judged steps, 0
skipped), balance 10 wait / 12 stop, so the majority-class baseline is 55%.
Every arm answered the identical 22 situations, three times.
| arm | accuracy | median | deterministic |
|---|---|---|---|
| always the commonest answer | 55% | — | — |
stillMs > 3000ms -> stop |
95% | 0 ms | yes |
| Apple Foundation Models (~3B) | 77 / 82 / 86% | 634–641 ms | no |
qwen3:8b (4-bit, 5.2 GB) |
91% | 919 ms | yes |
qwen3:14b (4-bit, 9.3 GB) |
82% | 1,489 ms | yes |
We believed capacity would help a little, and that the interesting question was how much. The 14B is 79% larger than the 8B, 62% slower, and scored lower — 82% against 91%, on identical inputs, both deterministic at temperature 0. Whatever this task is hard at, it is not hard in a way more parameters fix.
We believed a headline accuracy would rank the arms. It ranks them backwards. Every arm errs in exactly one direction, and the directions differ:
| arm | errors | direction |
|---|---|---|
| Apple | 4 | all wait/retry where stop was right |
qwen3:8b |
2 | all stop where wait was right |
qwen3:14b |
4 | all stop where wait was right |
A wrong wait costs a settle and one re-run. A wrong stop abandons a plan
that would have worked. So the best-scoring arm fails in the expensive
direction and the worst-scoring arm fails in the cheap one. No single accuracy
figure shows that, and a comparison that had reported only the percentages
would have recommended the wrong model.
We did not expect the shipped arm to be non-deterministic. Asked the same 22 questions three times, Apple gave 77%, 82% and 86% — a spread of two rulings. Both Ollama arms returned byte-identical answers every time. That is a fact about reproducibility, not about quality, and it means any single Apple number in this project carries ±2 rulings of noise that was never being reported.
stillMs > 3000ms was fixed on the previous population, before this one
existed, so 95% here is genuinely out of sample. Second independent
confirmation that a one-line comparison on a number the daemon already computes
outperforms every model arm at no latency.
The reason to still distrust it, which matters more than the number:
sweeping the threshold gives 91% at 2,000 ms, 95% at 2,500 and 3,000, 82% at
3,500 and 77% at 4,000. The plateau is about 2,100–3,200 ms, and the blocked
fixtures sit at 3,225 / 3,250 / 3,401 / 3,699 ms — just above it, which is why
3,500 collapses. A rule whose correctness depends on a 1.1-second window that
the fixture design happens to straddle may be separating the fixtures
rather than the world.
So the honest reading is not "the supervisor is unnecessary". It is:
- for these five fixture shapes, a free threshold is at least as good as any of three models, and the burden of proof has moved onto the models;
- the next measurement that would change anything is a population nobody designed — not a bigger model, which this table has now tested and which did not help;
- and the error-direction result stands independently of all of it, because it does not depend on the balance, the threshold, or the fixture design.
The reason for a local model was never accuracy. It was latency: a model consulted between steps should not cost a network round trip. That framing deserves its own table, because the one above ranks the arms against each other and not against the thing they exist to replace.
| median | accuracy | |
|---|---|---|
| a model round trip in the field | 10,000–16,000 ms | — |
qwen3:14b, local |
1,489 ms | 82% |
qwen3:8b, local |
919 ms | 91% |
| Apple ~3B, on-device | 634 ms | 77–86% |
| a threshold, no model | 0 ms | 95% |
The thesis holds: local is 7–25x faster than going out. Three things follow that the thesis did not anticipate.
More parameters are a straight loss on the latency axis. The 14B costs 2.3x the latency of the 3B and is no more accurate. A design motivated by latency should reach for the smallest model that clears the bar, which is the opposite of the instinct a disappointing accuracy number produces.
A local model contends with the app under test. The medians above are from a
quiet machine. Run while the simulator was being driven, the same calls measured
5,191 ms for qwen3:8b and 7,250 ms for qwen3:14b — against 842 ms for
Apple, which runs on the Neural Engine rather than fighting the app for the GPU.
That is one observation and not a controlled measurement, and it is flagged
rather than tabulated for that reason. But if it holds, a local Qwen under
load lands in the same order of magnitude as the network round trip it was
chosen to avoid, and the latency argument for it disappears exactly when the
machine is busy — which is always, because the machine is busy running the app
you are testing. This is the next thing to measure, and it matters more than
another point of accuracy.
And for this particular decision the latency argument is moot, because a
zero-cost threshold already answers it better than any model. That is not an
argument against local models; it is an argument about which decisions deserve
one. Where the thesis pays undiminished is a decision with no cheap rule
available — the seek container ranker is 564 ms warm against the same
10–16 s round trip, ~20x cheaper, and nothing free replaces it.
The owner's design, and the right shape: answer free where you can, pay for the on-device model only where you cannot, and pay a round trip only after that. Measured on the same 22 situations, letting the rule abstain in a band around its own threshold and handing those to the model:
| abstain band | escalated | cascade accuracy |
|---|---|---|
| none — the rule alone | 0/22 | 95% |
| ±250 ms → Apple | 2/22 | 91% |
| ±500 ms → Apple | 3/22 | 91% |
| ±1,000 ms → Apple | 5/22 | 86% |
| ±1,500 ms → Apple | 8/22 | 82% |
±1,500 ms → qwen3:8b |
8/22 | 91% |
Every escalation made it worse or left it unchanged. Never better. And the reason is more useful than the table.
The cascade needs a tier that can say "I don't know", and no tier has one. The threshold is a comparison: it always answers. The supervisor's vocabulary is three words and none of them is an abstention. So the question a cascade really asks is not "does falling through help" but "what is the abstain signal" — which is item 100, arrived at from a completely different direction.
And the obvious abstain signal is the wrong one here. Proximity to the
decision boundary is what you would reach for first. The rule's single error
sits 3,534 ms from its own threshold — further out than the median row's
1,610 ms — so a band wide enough to catch it would escalate 20 of the 22 rows
first. The error is not a close call. It is a detail screen still fetching
after 6.5 s of stillness, which is wrong for a semantic reason that no
confidence band around a duration can see.
That is the finding to carry: the shape of the cascade is right and the hard part is not the shape. A tier that knows when it does not know is worth more than a tier that is slightly more accurate, and nothing here measures confidence yet. n=22 on five designed fixture shapes, so this rules the simple version out rather than settling the idea.
What we believed: that an abstain token was the obvious next move — three
populations in a row had errors in one direction only, and a judge that can
decline is strictly safer than one that cannot. Item 100 has been near the top
of the list for weeks.
What we measured, on the same 22 situations, with a second @Generable type
in the same binary so one build answers both vocabularies:
| arm | three words | four words | abstentions |
|---|---|---|---|
| Apple ~3B | 77 / 82 / 86% | 45 / 50 / 55% | 0 of 22 |
qwen3:8b |
91% | 95% | 0 of 22 |
Apple never abstained once and lost about a third of its accuracy for having been told it could. The brief was the three-word brief plus one paragraph, built from it by concatenation so the two could not drift; nothing else changed. So the paragraph changed how it answered the original question. That is prompt sensitivity, not judgement, and it is invisible without a before/after on identical inputs.
What changed: the fourth word stays off for the Apple arm, and item 100 is no longer "add the token" — it is "find a model that will actually use it". The capability stays in the tree because it is the apparatus for the next attempt, and nothing here tested a model that abstains, because neither of these did.
The general form, and it is the one this file keeps finding: a change that is obviously right can be wrong in a way that has nothing to do with the argument for it. The argument for abstention is still sound. The model just cannot be told about it for free.
What was proposed: run the local model, check whether its answer was right, and call Claude if it was not.
Why the direct version is impossible: a checker that can tell you an answer
is wrong is the correct answer. If something local could verify stop, that
thing should have been asked in the first place.
What is possible, and is already running: you cannot check the decision, but
you can check the outcome. Acting on wait or retry re-runs the step, and
whether it worked is the verdict on the ruling — still_failed in the ruling
log is exactly "the model was checked and was wrong", and it escalates. On the
22 live rulings that loop handled 9 of 22 with no round trip and escalated
13, which at 10-16 s a round trip is roughly 90-145 s saved. That, and not
accuracy, is what the supervisor should be judged on.
And the asymmetry is the part worth keeping. stop produces no evidence —
you stopped, so you never learn whether continuing would have worked. It already
means "hand back to Claude with the unattempted steps" rather than "give up", so
it is an escalation and not an abandonment. But it is the one word whose
correctness nothing local can ever confirm, which is why the direction a model
errs in matters more than how often it errs.
Whether acting on a ruling actually recovered the flow is a fact about the
device at that moment and belongs to whichever arm was live. Only the Apple arm
was. On the live pass it recovered or correctly stopped 13 of 22 — 59%,
against the same 55% baseline — and the gap between its decisions and its outcomes is the same
one the previous population showed: the detail fixture takes the right word
almost every time and recovers about a third of the time, because wait settles
for a fixed budget and that screen wants longer. A correct ruling is not
sufficient.
What we believed: that a warm transition graph was the main reason a second
run of a flow is faster than the first. It is the feature the whole memory layer
exists for, and two field rounds appeared to confirm it — 33 sim_do calls on
the first pass, 12 on the second.
Why those two rounds proved nothing. The second pass had a warm graph and an operator who had already driven the app once. Both changed together, so the 21-call gap could be attributed entirely to either one, and we had been attributing it to the graph because the graph is the part we built.
The experiment: a third session, on the same app and the same task family, with an operator who had never seen it, and the graph left warm from pass 1. One variable held still.
| pass | operator | graph | sim_do calls |
|---|---|---|---|
| 1 | fresh | cold | 33 |
| 3 | fresh | warm | 24 |
| 2 | experienced | warm | 12 |
What it says. The graph is worth ~27% — nine calls of the thirty-three. Operator knowledge is worth the remaining twelve, which is half of what was left after the graph had taken its share. The thing we built is real and it is the smaller half.
The corroboration is the best in this file. Peer 2 had no way to isolate the graph and estimated its contribution indirectly, from escalation rate per step, at ~23%. The direct measurement says 27%. Two methods, three points apart, neither knowing the other's number.
What changed because of it. Nothing was undone, which makes this the rare entry where the measurement confirmed the feature and still redirected the work. The larger share belongs to what an operator learns and simframe cannot yet carry between sessions — which route through a screen is the fast one, which control is a trap, what this app calls the thing you are looking for. That is now the case for the next faculty, and it is a case made of a number rather than of an intuition.
Caveats, stated because the number will be quoted. n=1 app. One operator per
pass, so operator variance and operator experience are not separated from each
other. sim_do calls are round trips, not seconds; they correlate with waiting
but a round trip on a slow screen and one on a fast screen count the same.
What we believed, from entry 14's own closing paragraph: that what an operator carries between sessions is "which route through a screen is the fast one, which control is a trap, what this app calls the thing you are looking for." Entry 14 measured that this knowledge is worth 12 of 24 round trips. It did not measure which of those three things it was, and the list was written as prose rather than as a finding.
Counting before building, on two real-app graphs.
The first item is fully recorded already. Every node that offers a route choice
has both exits timed — 34 of 34 on one app, 17 of 17 on the other — because
record() has been storing settles per edge all along (97% and 85% of edges
carry samples). The median gap between the fastest and slowest exit at a choice
point is 1287 ms and 773 ms. Nothing reads it: route() is a plain BFS
over hop count.
So the obvious first increment is to route by measured time. It is not worth building for this reason, and that is the result:
| reachable pairs | fastest route differs from fewest-hops | saved when it differs | |
|---|---|---|---|
| app A | 65 | 6 (9%) | median 1853 ms |
| app B | 27 | 0 (0%) | — |
| bench | 408 | 29 (7%) | median 615 ms, max 6406 ms |
Fewer than one route in ten changes, it saves about a second when it does, and it saves seconds rather than round trips. Route choice does not cause an escalation, so it cannot be any part of the twelve calls entry 14 measured. A real and unused signal, pointed at the wrong prize.
Where the calls actually are. On the two real-app devices the escalation log is dominated by one reason:
| device | verification_failed |
ambiguous_intent |
unknown_screen |
|---|---|---|---|
| app A | 58 of 67 | 8 | 1 |
| app B | 42 of 46 | 4 | 0 |
Not naming, and not planning. The model is being consulted because simframe could not confirm that a step did what it was supposed to. And the addressable share is large: 37% (app A) and 66% (app B) of those escalations happened on a screen the graph already knew. Memory had the screen and still had to ask.
Nondeterminism is recorded and is the plausible mechanism — 19% and 7%
of edges have changedOutcomes > 0, meaning the same action has reached more
than one screen from the same place.
What this changes. The next faculty is not route knowledge. It is what an operator knows about outcomes: which unexpected-looking result on a known screen is in fact correct. Entry 14's prose named three things and the countable one turned out to be the least valuable of them.
Caveats. The logs span nine sessions per device over several days and cannot be narrowed to entry 14's three passes, which ran as peer sessions elsewhere — so this locates the escalations, it does not prove these particular ones are the twelve. Edge cost is the median of recorded settles, which is a proxy for wall time and ignores everything between steps. n=2 apps.
17. The verification calls are not recoverable from memory — they are no-visible-change in novel territory
The question §16 left, and the obvious next build: of the
verification_failed escalations that dominate the real-app logs, how many
happened on a (screen, action) the graph had already seen behave consistently?
If most, then remembering what an edge did last time removes them.
Almost none. Joining every verification_failed escalation to the graph by
screen_fingerprint and matching the intent against the edge signature:
| app A (n=58) | app B (n=42) | |
|---|---|---|
| screen not in the graph at all | 36 (62%) | 14 (33%) |
| screen known, this action never seen | 12 (21%) | 28 (67%) |
| edge seen exactly once | 4 (7%) | 0 |
| edge known to be nondeterministic — rightly asked | 3 (5%) | 0 |
| edge repeated and consistent — addressable | 3 (5%) | 0 (0%) |
Three calls out of a hundred. Remembering an edge's prior outcome is not the lever, and building that store would have been a phase spent on 3%.
What they actually are. Classified against a closed verdict vocabulary:
| verdict | app A | app B |
|---|---|---|
no-visible-change |
36 (62%) | 28 (67%) |
| wait-timeout | 7 | 5 |
| settle-timeout | 4 | — |
| unexpected-screen | 3 | — |
| ambiguous | 3 | 1 |
Two thirds of every verification escalation is the step ran and nothing visibly changed — and 83% and 100% of them are on a screen or an action the graph has never seen. Memory of any kind cannot answer a question about a place it has never been.
What this does NOT refute, and the distinction is the whole value of the entry. It does not say operator knowledge is worthless — §14 measured that at 12 calls and the measurement stands. It says the mechanism is not recall of what an edge does. The novel-territory dominance is in fact consistent with §14: an operator who knows the app goes straight to the screen they want, while a fresh one explores, lands somewhere unrecorded, and escalates. The value is in not being there at all, not in knowing what happens once you are.
So the candidate faculty is task-level: what this app calls the thing you want,
and which screen it lives on — the third item in §14's prose list, not the
first two. And the sharpest lever on the remaining calls is not memory at all:
it is the accuracy of no-visible-change, which DEFERRED item 4 already
measured as blind to small-delta taps and which the graph-recording code's own
comment says a toggle reads as.
Caveats. The graph is read as it is now, so an edge matched here may have been learned after the escalation — which makes the 5%/0% an upper bound on what memory could have answered, and it is already near zero. The logs span nine sessions per device and are not §14's three passes. n=2 apps.
A hygiene note, because it happened here. The first version of this analysis
printed the detail prefix of each escalation to classify verdicts, which put a
client's screen labels and a customer email address into a terminal. detail is
app content. The script now classifies against a closed vocabulary and never
emits the raw string — the same rule phase17-corpus.mjs states and this
analysis briefly forgot.
Three habits, each bought with a reversal above.
Measure the prize before building the solution. Entries 6 and 1. Two phase premises in a row were false, and both were countable in minutes.
An accuracy without its baseline is not a measurement, and a threshold
chosen after seeing the answers is not a threshold. Entry 1, and the reason
scripts/score-rulings.mjs was written before the population it first scored.
A confident wrong answer costs more than a refusal. Entry 9 is the clearest case, and it is why the supervisor may only say three words: the answer space is the safety property, not a confidence threshold.
Two variables that always move together have never been measured. Entry 14. Two field rounds agreed with each other and with us for a day, and neither could have told us we were crediting the wrong half. The isolating run cost one peer session; the belief it corrected had already shaped a phase plan.
What we believed: that the wedged display is a CoreSimulator condition we
suffer rather than cause, and — since 2026-09-13 — that it is triggered by
switching apps. Both beliefs were load bearing: the first decided that detection
plus revive is the whole answer, and the second was written into DEFERRED as
"the first reproducible trigger anyone has had".
Why the first belief deserved a test. Entry 5 in this file records believing it once before, and then measuring 670 damage-callback registrations and zero unregistrations — our own daemon wedging the device. That was fixed, and entry 5 says plainly that the fix was never verified against the symptom. It still has not been. An error raised by CoreSimulator is not evidence that the cause is CoreSimulator, which is the same inference that once reported a failed accessibility read as "idb is not installed".
The experiment: the same workload — launch Settings, screenshot, launch Safari, screenshot, terminate, screenshot — forty rounds on a spare device, run twice. Once with no simframe daemon attached, once with the daemon capturing. Our daemon holds three private-framework connections the workload otherwise would not: an IOSurface damage callback, an Indigo HID session, an AXPTranslator bridge.
| arm | rounds | screenshots | failures | median | max |
|---|---|---|---|---|---|
| no daemon | 40 | 120 | 0 | 299 ms | 892 ms |
| daemon attached | 40 | 120 | 0 | ~280 ms | — |
The result is negative in both arms, so it does not discriminate, and saying so is the point of writing it down. Forty rounds of app-switching did not reproduce a wedge that hits the bench device several times a day under real flows. What it does establish is narrower and still useful: app switching plus screenshots, at this rate, is not sufficient to cause it. The trigger claimed on 2026-09-13 is therefore withdrawn — it was an inference from where the symptom was noticed in a long script, not from a controlled run.
What the same afternoon did establish, from the daemon's own log of 22 real
wedges. Each one ends with capture is wedged and both recoveries are spent.
Classifying every one by the failure immediately before it:
| what preceded the wedge | count |
|---|---|
no frame was available from the display for 600ms |
21 |
the device exposes no active display port |
1 |
the display surface could not be read |
0 |
Almost every wedge is a display that reads fine and delivers no frames — and the daemon's own message for that case says "this is usually transient; a display that has stopped rendering says 'the display surface could not be read' instead". The mode we have been naming in every error string, every DEFERRED entry and every CI comment — the dead surface — preceded none of them.
A correction, because the first version of this entry got it wrong and the mistake is instructive. It reported two equal populations of 21 and 21. That came from grepping each preceding line for either phrase, and the frame-starvation message quotes the dead-surface message inside its own explanatory text — so one line matched both patterns and every wedge was counted twice. 22 wedges classified as 43. The fix is to classify a line by what it is rather than by what it mentions, which is the same distinction this file keeps rediscovering one level up.
DEFERRED 126 called this one cause wearing five symptoms, and on this evidence it is one cause — just not the one named. A live surface that stops delivering frames is the shape most likely to be ours, because a damage callback that stops firing is exactly the defect entry 5 found and never verified the fix for.
The obvious next question, asked by the owner: hosted runners have a fraction of a laptop's capacity, so is the wedge a capacity problem? The premise turns out to be wrong in an interesting way.
| this laptop | hosted runner | |
|---|---|---|
| capture time, p50 | 21.8 ms | 55.4 ms |
| capture time, p95 | 41.1 ms | 94.0 ms |
| wedges | 22 in ~1.5 days of use | 1 run in 6 |
The runner is consistently ~2.5× slower at identical work, so capacity is real and measurable. But it does not show up as wedging: of six recent CI runs, one carried a wedge signature at all. Locally, 22 wedges in a day and a half.
So the wedge is not primarily a CI phenomenon — it is primarily a local one, and that is good news twice over. It is happening on the machine where it can actually be debugged, and the CI red we have been chasing is mostly not it.
What CI fails on instead is latency. simctl openurl exceeding its own
internal timeout, a settle not completing in budget, and a tap landing before a
screen is ready — the fingerprint tour that never left the Settings root is that
last one exactly. Every one of those is what a 2.5× slowdown produces, and none
of them is a wedge.
That splits the work cleanly, which is the whole value of having asked:
- CI needs timing tolerance and fewer timing-fragile vehicles.
openurldrags in simctl, LaunchServices, a Safari cold start and a network fetch to assert a thing a swipe would assert — see DEFERRED 133. Curing wedges harder will not help, because wedges are not what is failing. - The wedge needs instrumentation and a soak, on this laptop, where it reproduces roughly fifteen times a day.
Caveat, because the numbers will be quoted. Six CI runs is a small sample for a rate, and a local "daemon session" is not the same unit as a CI run — the comparison is indicative of direction, not a measured ratio.
What to do next, and what not to. Not another threshold and not another
retry. Instrument the frame-starvation case — the daemon should say whether the
surface was readable when frames stopped — and soak it locally with damage
callback registrations counted, which is the one mechanism this project has
already caught wedging a device once. Until that exists, revive stays the cure
and the CI guard stays the mitigation, and neither is a diagnosis.