Decide whether one failing matrix cell should fail the whole nightly run

[widgetii]

Split out from #2036, which is now closed — the 404 flakiness that motivated it is fixed, but this follow-up was never picked up.

Question

build.yml builds ~99 matrix cells. The matrix is fail-fast: false, so one failing cell does not cancel the others — but the run still reports conclusion: failure. One board out of 99 turns the nightly red.

Is that the signal we want?

Why it matters

The failure classes are not equivalent, and today they are indistinguishable at a glance:

Class	Example	Should it fail the run?
Real breakage, broad	2026-08-12 — majestic-webui fixup broke all 99 boards	Yes
Real breakage, one board	2026-04-06 — hi3516ev300_neo kernel drivers/net/mdio	Probably yes
Transient, survived retry	404 absorbed by the 7-attempt backoff	Already invisible — correct
Transient, exhausted retry	>10min CDN 502 storm on one board	Arguably no
Runner infrastructure	GHA "Server Error"	No

After the #2036 work the nightly sits at 1 failure in 30 days, so the red/green signal is currently meaningful and this is not urgent. It was urgent when 30% of runs were red and everyone had learned to ignore the colour. Worth settling before that recurs, not after.

Things to weigh

• Alerting on red is only useful if red is rare. It is rare now. That argues for leaving it alone.
• A single flaky board can still mask a real one. With 99 cells, "the nightly is red" does not tell you whether one board or fifty broke. A summary in the job output (counts by failure class) may solve the real problem without touching pass/fail at all.
• Per-board history is the more useful signal. "hi3516ev300_neo has been red 3 nights running" is actionable; "the nightly is red" is not.
• Interaction with the CI umbrella gate (shipped 2026-06-07) and with branch protection — changing the workflow conclusion has consequences beyond the nightly.

Possible outcomes

Closing this as "current behaviour is correct" is a legitimate result. The point is to make it a decision rather than an accident of the default.

• Decide whether transient-classified failures should report without failing the run.
• Consider a per-run summary: N passed / N failed, grouped by failure class, in the run summary rather than buried in 99 job logs.
• Consider per-board streak tracking so a persistently broken board is louder than a one-off.