docs: add parity-sweep rubric - #6242
Conversation
…struction and blank-ship divergences
…chdog/probe and pending-subscription divergences
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5b610690e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| Resolved 2026-08-13 (remove after PR #6276 merges): the TLON-6322 "unverified small twins" entry — mention @p validation (n/a on Hermes, residual shared-converter gap filed as TLON-6334), blank-ship rejection (equivalent guarantee, declared divergence above), authorless cite paths/gating (verified; three small divergences fixed). | ||
|
|
||
| Resolved 2026-08-13 (remove after PR #6282 merges): the TLON-6319 SSE reap-detection/watchdog entry — Hermes gains event-id regression + per-action floor + genesis-action detectors and a probe-driven staleness watchdog with validated `TLON_SSE_STALE_THRESHOLD_SECONDS`/`TLON_SSE_WATCHDOG_INTERVAL_SECONDS`; pending-subscription retry assessed not-applicable (declared divergence above). |
There was a problem hiding this comment.
Until PR #6282 is actually merged, moving TLON-6319 out of the standing-gap list and declaring the Hermes watchdog behavior an intentional divergence suppresses the exact status transition this sweep is meant to report. In this commit's parent, the Hermes driver exposes only SSE stream error and configures only TLON_SSE_READ_TIMEOUT_SECONDS; the stated stale/watchdog knobs are not present. A weekly run before the follow-up rubric edit will therefore treat the gap as resolved, and when the implementation lands it can be ignored as declared behavior rather than emitted as a standing-gap update. Keep this entry in the open list until the implementation is visible at the pinned head.
Useful? React with 👍 / 👎.
| - **Reaction-based approvals** (👍/👎/🛑 on approval DMs) are OpenClaw-legacy and intentionally not ported — A2UI approval cards supersede them. | ||
| - **Hermes-only surfaces** accepted as reverse divergence: `/channel-access`, extended owner-listen modes, `/tlon status` diagnostics, native block-list pre-check, in-package `image_search`. | ||
| - **OpenClaw-only** session/route persistence machinery (webchat-leak prevention) is architecture-specific and has no Hermes analogue by design. | ||
| - **Outbound story construction** is OpenClaw-only (`src/urbit/story.ts` text→story conversion). Hermes sends raw text through the `tlon` CLI, which converts via the shared `packages/api` markdown converter (out of sweep scope). Fixes to OpenClaw's converter have no Hermes twin; shared-converter gaps are filed against `packages/api` (e.g. TLON-6334). |
There was a problem hiding this comment.
Do not classify every change to OpenClaw's story converter as having no Hermes twin. Hermes sends the same model-authored Markdown through tlon, whose packages/tlon-skill/scripts/markdown.ts delegates to packages/api's separate markdownToStory implementation, so an OpenClaw fix for a parsing or rendering case can reveal that Hermes still produces different user-visible output for the same input. Because the triggering OpenClaw commit is inside the swept scope, this blanket exemption creates false negatives beyond the acknowledged inability to list shared-package commits; the sweep should inspect the shared converter and report a possible twin there when the fixed behavior is not equivalent.
Useful? React with 👍 / 👎.
|
|
||
| Intentional behavioral divergence is documented in the shared e2e harness's per-driver expectations (`packages/tlon-bot-e2e/src/drivers/{openclaw,hermes}.ts` and driver-branched scenarios in `src/scenarios/shared/common.ts`) and summarized here. The prose summary below is the working list; when it and the harness disagree, flag the discrepancy rather than picking a side. | ||
|
|
||
| - **Reply/thread placement**: the runtimes anchor replies and reaction acks differently; Hermes runs with `reply_in_thread` off. |
There was a problem hiding this comment.
Add the existing reaction prompt-text difference to this declared-divergence list, not just the acknowledgement-placement difference. In src/scenarios/shared/common.ts, all three reaction scenarios explicitly require OpenClaw's model input to contain the reacting ship while omitting that assertion for Hermes because Hermes carries the actor only in MessageEvent.source.user_id. A swept change to either reaction handler can otherwise be classified as a new feature gap, or repeatedly produce a rubric-discrepancy proposal, even though the cited harness already treats this separate model-visible behavior as intentional.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds
docs/parity-sweep.md, the classification rubric for a planned weekly parity sweep: a hosted bot will fetch this doc (pinned to a resolved head SHA), sweep new commits topackages/openclawandpackages/hermes-tlon-adapter, and post a drift digest to an internal ops channel. When the sweep misclassifies, the fix is a PR to this doc.Changes
docs/parity-sweep.mdcovering dispositions and finding types (feature gap, twin bug, bar shrink), declared divergences, standing open gaps (status changes only), evidence-phrasing rules, and the digest format.packages/tlon-skill,packages/api,packages/tlon-bot-e2e) are out of scope for the sweep, documented as an accepted blind spot restated in each digest.