Skip to content

63786101 - Add issue-filing scaffolding for the log-error skill - #48

Open
Danswar wants to merge 43 commits into
developfrom
63786101-error-issue-scaffolding
Open

63786101 - Add issue-filing scaffolding for the log-error skill#48
Danswar wants to merge 43 commits into
developfrom
63786101-error-issue-scaffolding

Conversation

@Danswar

@Danswar Danswar commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

EN:
Four additive pieces toward letting the log-error skill actually take its "open an Issue with repro" branch, all deterministic — no model judgment in any of the grouping or throttling logic. A template_fingerprint groups error variants that differ only by chain and/or asset ticker under one template, masking those names only as whole tokens so ordinary prose like "Based" or "RESOLVE" is never mistaken for one. A new error_issue_act module finds-or-creates a GitHub issue per template, handles every pending row of one template in a single pass, tracks the variants in a machine-owned delimited section of the issue body, and announces genuinely new ones in one comment. Two throttles sit in front: burst detection, counted over templates never filed before so a backlog draining after downtime is not mistaken for a burst, and a cooldown checked against local history that a dry_run preview never opens.

DE:
Vier additive Bausteine, damit die log-error-Skill ihren Zweig "Issue mit Repro öffnen" tatsächlich nutzen kann, durchgehend deterministisch — keine Modell-Beurteilung in der Gruppierungs- oder Drossel-Logik. Ein template_fingerprint gruppiert Fehler-Varianten, die sich nur durch Chain und/oder Asset-Ticker unterscheiden, unter einer Vorlage und maskiert diese Namen nur als ganze Tokens, damit gewöhnlicher Text wie "Based" oder "RESOLVE" nie dafür gehalten wird. Ein neues Modul error_issue_act findet oder legt ein GitHub-Issue pro Vorlage an, verarbeitet alle offenen Zeilen einer Vorlage in einem Durchgang, führt die Varianten in einem maschinenverwalteten, abgegrenzten Abschnitt des Issue-Bodys nach und kündigt wirklich neue in einem Kommentar an. Davor liegen zwei Drosseln: eine Burst-Erkennung, gezählt über noch nie gemeldete Vorlagen, damit ein nach einer Störung ablaufender Rückstau nicht als Burst gilt, und eine Abklingzeit gegen die lokale Historie, die ein dry_run nie eröffnet.

Details

Grouping

template_fingerprint()/template_signature() in errors.py are additive — the existing fingerprint()/stack_sig() identity used for error.seen count/last_seen tracking is unchanged, so per-variant dedup stays exact. The chain and payment-rail names and asset tickers masked in the template signature come from the platform's own live enum (checked against production, not guessed from log samples); the asset list is cut to tickers seen on 2+ chains plus two single-chain tickers confirmed present in real production errors.

Both lists are matched longest-first and only at word boundaries. Without those anchors Base matched inside "Based", SOL inside "RESOLVE", COMP inside "COMPLETE" and DAI inside "DAILY", which both mangled the template hash of unrelated errors and labelled them with a token that had nothing to do with them. Ordering still matters for a name containing punctuation: USDC inside USDC.e passes a word boundary because . is not a word character, so the longer name is tried first and a glued USDC.e_balance yields nothing at all — the same answer as USDC_balance. Matching is deliberately case-sensitive: lowercase usd in "token tether -> usd" is prose, not a ticker, and three tests pin that. service, class and environment are percent-escaped before the join so two different field tuples cannot serialize to one fingerprint.

Issue handling

error_issue_act.py mirrors error_fix_act.py's deterministic pattern (find-or-create, runner-injected gh calls for testability). Dedup uses a label plus a hidden marker carrying a salted digest of the template fingerprint, matched against the issue body rather than handed to gh --search, whose full-text matching can both miss the marker and return an issue that does not carry it. The marker is re-checked on the body that is actually written, so a marker a hand edit moved inside the machine-owned section is never spliced away. The digest is salted with the device id because it sits in a public issue while the fingerprint behind it stays raw and injective — grouping must not merge two tenants, and an unsalted digest would let a reader confirm a guessed stream label. The same salting and the same encode guard apply to the burst row digest.

The variant table lives between delimited markers and is spliced in place — nothing outside that section is ever touched. A body with a lone or duplicated marker is treated as damaged and fails loud rather than gaining a second section, and every body sent to gh passes one shared ceiling check before the call. The table is capped, so normal growth cannot walk an issue into that ceiling and wedge every later update; the dropped count stays visible.

All pending rows of one template are processed together, so two variants seen in one run land in one issue write. Handled row by row, a second variant seen in the same run was silently dropped — exactly the Arbitrum/USDC vs Arbitrum/WBTC case this change exists to group. The edit is written before the comment so the durable record lands first and a retry can never double-file a variant; a failed comment is reported on the row rather than discarding a successful edit.

The burst issue reuses the same splice mechanism with its own marker. Only templates with no prior filing count toward the threshold — a backlog draining after downtime is a volume spike, not a burst — and a template already folded into an open burst is not filed again, judged from the issue body and from local fold history keyed by issue number so a fold into a since-closed burst is not credited to an unrelated open one. The cooldown reads local history rather than calling gh, is scoped to the issue repo, ignores rows that were themselves skipped, ignores dry_run previews, ignores rows that name no issue, and treats a future-dated timestamp as not in cooldown.

Validation

Verified end-to-end against real production data (not just synthetic tests): replayed a batch of real error.seen rows from a live instance through this code locally in dry_run mode. Before the asset-ticker masking, 30 raw incidents grouped into 25 templates; after, into 11 — mostly collapsing repeated low-balance alerts that only varied by token. Separately measured real volume over a 7-day production window: ~353 raw error lines/day collapsing to a steady state of ~14–29 new templates/day (a single very-noisy known issue accounted for 26% of all raw lines but correctly collapsed to one template) — this is the basis for the burst-detection default and the reason a cooldown was worth building before this goes anywhere near live.

Local suite green: 665 passed, 1 skipped. Every fix in this branch carries a mutation check confirming its test actually catches the regression rather than passing by construction.

Out of scope

Not included, intentionally: CLI wiring (adding error.issue to the agent activity add allowlist, an agent watch error-issue command, and the issue_repo/dry_run/cooldown_minutes/storm_threshold config fields) and the human-already-filed-a-duplicate fuzzy-match check. Both are separate, larger surfaces that deserve their own focused review. The deferral is recorded in DESIGN.md §21.6, and §21.3 documents the error.seen payload this writes.

Also deliberately left for follow-up: the pre-existing fingerprint() joins its fields without escaping, the same ambiguity fixed here for template_fingerprint; changing it would re-key already-stored error.seen rows, so it does not belong in this change.

@Danswar
Danswar force-pushed the 63786101-error-issue-scaffolding branch from 432662f to 14a61e5 Compare September 1, 2026 00:54
@Danswar

Danswar commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

EN:
Ready after 25 review passes.
Adds deterministic scaffolding that groups error variants differing only by chain or asset ticker under one template and files or updates a single GitHub issue per template, behind a burst fold and a per-template cooldown.

DE:
Bereit nach 25 Review-Durchläufen.
Fügt deterministisches Scaffolding hinzu, das Fehler-Varianten, die sich nur durch Chain oder Asset-Ticker unterscheiden, unter einer Vorlage gruppiert und pro Vorlage genau ein GitHub-Issue anlegt oder aktualisiert, gedrosselt durch eine Burst-Faltung und eine Abklingzeit je Vorlage.

Details

Review passes

25 passes, each two parallel lanes (conformance and logic) over the full diff at a fresh head, plus five cross-vendor gate runs. Passes 3, 7, 16 and one lane of 20 ended partial or unavailable and were re-run rather than counted: two because the branch moved while a lane was reading it, one because a lane did not cover its full brief, one on a transient CLI auth failure and one on a dropped connection.

Findings that changed the code, by pass:

  • 1 — the cooldown dropped a second variant of the same template seen in one scan (reproduced: the Arbitrum/USDC vs Arbitrum/WBTC case this feature exists to group); a dry_run counted as a real touch and suppressed the following real run for the whole window; chain and asset names matched as substrings, so Base matched inside "Based", SOL inside "RESOLVE" and COMP inside "COMPLETE"; burst detection counted every template rather than only new ones; a body with one marker gained a second section instead of failing; an unbounded issue body; a failed comment discarded an already-successful edit; a placeholder repo name in 27 test fixtures.
  • 2 — the body ceiling was enforced when splicing but not when creating.
  • 3 — the marker was matched by gh --search and the returned issue's body was never checked; gh errors were caught but a missing binary was not; a dead wrapper; missing payload documentation.
  • 4 — no truncation guard on the issue list, so a full page read as "no issue exists" and would have filed a duplicate.
  • 6 — a burst fold remembered locally was not tied to the issue it was folded into, so a fold into a since-closed burst was credited to an unrelated open one.
  • 8 — the burst row label was not unique: two templates differing only by environment shared a row, and a pipe in a field shifted which field was read.
  • 12 — the variants header was recognised by prefix, so a real row whose name began with variant was silently dropped on parse.
  • 13 — untrusted log text could move the section boundary, break out of its code fence, or render as a live mention; history was not scoped to the issue repo; internal helpers were public.
  • 18 — the marker was validated on the body that was read but not on the body that was written.
  • 21 — a created issue's URL was never validated, so a success without one counted as a touch naming no issue.
  • 22 — the public marker digest was taken over unredacted stream labels; a corrupted row could open a cooldown window with nothing behind it.
  • 24 — redacting before hashing (the pass-22 fix) collapsed two tenants into one template; a row with a damaged timestamp read as "never filed" and could be folded into a burst, orphaning its open issue; an unencodable fingerprint crashed the whole scan and would have recurred on every later one.
  • 25 — the burst row digest was still unsalted and unguarded, so both of those held on the burst path.

Reasoned rejections, each against a verified repo fact: case-insensitive masking (three existing tests require lowercase usd not to match); cross-process races (§21.1 fixes a single-device model and the module already holds a per-device lock); the "prod" wording (already used plainly in DESIGN.md and the existing tests); the first-chain/first-asset variant heuristic (documented best-effort); a literal <CHAIN> in a log line colliding with the mask (the same theoretical ambiguity as the pre-existing UUID and digit substitutions); and automatic reconciliation after an ambiguous gh transport error (the row is marked error and visible; no duplicate and no silent gap).

The CLI wiring is a declared deviation from end-to-end completeness, granted by a human reviewer and recorded in DESIGN.md §21.6 rather than implemented.

Gates at this head

  • pytest green on 17c36b3; that is the only check the configuration produces for this PR (test.yml on pull_request, no path filters), and develop has neither classic protection nor a ruleset requiring checks.
  • Local suite 665 passed, 1 skipped. Every fix carries a mutation check confirming its test catches the regression.
  • mergeable: MERGEABLE against develop.
  • No open comments: issue comments, reviews, inline comments and review threads are all empty.
  • All 30 commits verified.

@Danswar
Danswar marked this pull request as ready for review September 1, 2026 13:01

@TaprootFreakAI TaprootFreakAI left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EN:
Do not file GitHub issues for grouped errors; keep the grouping in the local store.

DE:
Keine GitHub-Issues für gruppierte Fehler anlegen; die Gruppierung gehört in den lokalen Store.

Details

The useful part of this change is template_fingerprint: variants that differ only by chain or asset belong to one template. That identity already has a home — error.seen in the local store, which already tracks fingerprint, count, and last_seen.

GitHub issues are the wrong second store. error_issue_act exists to treat an issue body as a database (delimited variant table, hidden markers, salted public digest, burst issue, cooldown against local history). That complexity is the symptom: issue text is untrusted, public, and not a row. Cooldown and variant counting already live locally; the issue only mirrors them, with a large surface (log excerpts in public bodies, marker splicing, a truncated gh list read as "no issue exists").

The human-facing next step on an eligible error is already specified: error.fix → draft pull request (DESIGN §21.5). An extra issue in between is not a work item the store and the draft PR do not already provide.

Please keep template_fingerprint on error.seen if grouping is still wanted, and drop the GitHub issue-filing path (error_issue_act, error.issue, DESIGN §21.6 filing behaviour). Do not wire agent watch error-issue.

@Danswar
Danswar marked this pull request as draft September 1, 2026 13:58
Danswar and others added 25 commits September 1, 2026 13:50
@Danswar
Danswar force-pushed the 63786101-error-issue-scaffolding branch from 17c36b3 to cc91a88 Compare September 1, 2026 17:07
Section 21.3 still described the deleted issue-marker mechanism and
pointed at section 21.6 for detail that no longer exists there.
Three docstrings still described the deleted GitHub-issue mechanism;
the functions and tests they document are unrelated leftovers that
stay exactly as they are.
The module comment above _KNOWN_CHAINS still described the deleted
issue-filing feature.
@Danswar

Danswar commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

EN:
Ready after 5 review rounds. Removes the GitHub-issue-filing path (error_issue_act.py, the error.issue activity type) that a reviewer flagged as an unnecessary external dependency, while keeping the approved template_fingerprint grouping.

DE:
Bereit nach 5 Review-Durchläufen. Entfernt den GitHub-Issue-Filing-Pfad (error_issue_act.py, den error.issue-Activity-Typ), den ein Reviewer als unnötige externe Abhängigkeit bemängelte, und behält die genehmigte template_fingerprint-Gruppierung bei.

Details

Addresses the CHANGES_REQUESTED review directly: error_issue_act.py and tests/test_error_issue_act.py are deleted, along with the corresponding DESIGN.md §21.6 text and activity-catalog row. template_fingerprint/template_signature grouping in errors.py is untouched — it was never contested.

Review rounds (Grok then Codex, quality + logic each):

  1. Removal commit — both dimensions found a dangling DESIGN.md §21.3 reference to the deleted issue-marker mechanism.
  2. Fixed; both Grok dimensions approved.
  3. Codex found: (a) a real doc-coherence gap (same class as round 1, a second instance), (b) a suggestion to delete known_chain_in/known_asset_in as dead production code — rejected that specific suggestion since both functions are exercised by ~10 regression tests covering the live masking-regex edge cases; fixed the docstrings only, kept the functions and tests.
  4. Grok found one more stale comment the prior pass missed (errors.py:45-49).
  5. Fixed; full repo-wide sweep confirms zero residual references. All four gates approved at the final head.

Local suite: 693 passed, 1 skipped throughout (was 771 before this branch removed test_error_issue_act.py's ~78 tests along with the module itself).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants