Skip to content

feat: expose seat model and declared Jcode catalog - #908

Merged
davidfarah2003 merged 22 commits into
mainfrom
life/905-seat-provenance
Aug 29, 2026
Merged

feat: expose seat model and declared Jcode catalog#908
davidfarah2003 merged 22 commits into
mainfrom
life/905-seat-provenance

Conversation

@davidfarah2003

@davidfarah2003 davidfarah2003 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Show each managed seat's model and optional requested variant in the default cotal ps row.
  • Keep variant provenance honest: an omitted value means no override was recorded. Cotal does not infer an effective provider default it cannot observe.
  • Keep --wide focused on extra operational facts instead of repeating model and variant from the compact identity row.
  • Expose Jcode providers that enable model_catalog = true in the operator config.toml, with bare Jcode model ids and declared reasoning efforts.
  • Mark Jcode effort tiers as declared config, not provider-verified capabilities, and fail loud on unreadable, malformed, or empty enabled catalogs.
  • Keep connector model-id forms explicit: OpenCode and Codex catalogs use provider/model; Jcode catalogs use bare ids because provider selection comes from Jcode config.
  • Gate the focused provenance smoke at the true append-only tail after current main, preserving every existing suite's shard assignment.

Variant finding

The filing-time fleet-wide zero was a survivorship result from a working recorder. A seat given an effort tier its provider refuses dies during launch and never reaches ps. At the filing moment, no surviving seat had an accepted variant. Later read-only snapshots contained accepted variants on 5 of 46 rows and 5 of 48 rows. The sets were compared by managed identity, not count. Every variant-bearing row used gpt-5.6-sol.

The same-root positive control traced one accepted request end to end: COTAL_VARIANT=high in the process environment, "variant":"high" in ps --json, and (high) in human output. A running seat with no override had no environment value, no JSON key, and no invented display value. The resolve to launch record to ps serialization path therefore works.

The remaining field semantics are two-sided. For a running seat, absence conflates "did not ask" with "could not express an override for this model". Presence proves that a request was accepted at the launch boundary, not that the provider applied the tier. A requested tier that the provider refuses is a launch failure and cannot appear in ps; that visibility belongs to #828 rather than this change.

A signal that is absent rather than wrong can produce a green result nobody earned. Review verdicts recently required re-grading after their effort provenance was discovered to be lower than expected, while the original verdicts looked identical to correctly graded work. This change makes the model and any accepted requested override visible by default without inventing effective state.

Effective effort open question

A future schema could carry requested and effective effort separately, but it needs a truthful effective source. The current Harness API does not provide one. It exposes set_reasoning_effort, while get_runtime_info and its corresponding event report provider, model, and routes only. No reply or event returns the applied tier. The local config catalog is also demonstrably non-authoritative, so it cannot fill that gap. This PR keeps variant as the requested override and leaves an effective field for a future provider-backed API.

Catalog accuracy limitation

The host config contains 51 Jcode model entries, but its declared reasoning efforts are not authoritative about provider acceptance. Live launch observations all returned provider code invalid_request:

  • opus-5 declares low, medium, high, xhigh, and max, but refused all five.
  • claude-opus-4-8 declares the same ladder, but refused max.
  • glm-5.3 declares low, high, and max, but refused all three.
  • gemini-3.7-flash declares low, medium, and high, but refused low and high.
  • grok-4.6 declares low, medium, high, and xhigh, but refused medium and high.

Declared tiers were verified to work for gpt-5.6-sol and deepseek-v4-pro. For that reason cotal models exposes configured efforts as declarations and renders variants (declared, not provider-verified) on the same line as each tier list. Provider validation at launch remains the authority. The provenance, authoritative, and warning metadata fields exist to drive that human rendering; a repository-wide consumer census found no other production reader.

A fleet may also enforce a routing policy such as excluding a model family from review work. Provenance makes that policy auditable after a verdict is produced: the model string identifies the rail that ran, while requested effort distinguishes an explicit accepted request from no recorded override. This PR does not add a policy field or claim more than those recorded facts.

Compatibility was measured across every persona on the host. Of 30 personas declaring an agent: pin, 20 agreed with the harness already running and none disagreed; the other 10 had no live seat. The only distinct pin values were claude (8) and jcode (22), and both are registered connectors. Honoring these pins therefore changes the harness for zero currently running seats and introduces no unresolved connector name.

Current-main integration

Released main 87bee50d37f737c96b5a884eb1ae5ce69d7461ff was merged without rebase. The successor preserves the public PR lineage, persona harness pins, gated web smokes, repaired component-health mutation anchor, interactive regrant retirement, published manager runtime dependencies, current 0.34.0 versions, and the released lockfile. The original 16-file provenance and Jcode catalog feature remains intact, with three additional source-documentation closure paths: extensions/connector-core/src/tool-specs.ts, generated docs/mcp-tools.md, and docs/run-a-mesh.md. The overlapping surfaces were resolved as follows:

  • bin/smoke/ci-suites.txt preserves the complete 408-suite current-main prefix byte-for-byte, then appends smoke:ps-provenance as suite 409. No existing suite changes shard.
  • docs/cli.md preserves the web merge's --host documentation and describes model and variant as operational descriptors rather than claiming a display name uniquely identifies an owner+actor seat.
  • docs/connect-jcode.md now names requested reasoning effort in the supported beta path instead of contradicting its own model section and the implementation.
  • cotal_spawn now documents the implemented harness precedence as explicit agent > persona agent: > caller COTAL_DEFAULT_AGENT > manager COTAL_DEFAULT_AGENT > product default. docs/mcp-tools.md, docs/run-a-mesh.md, and the bundled docs carry the same order.
  • extensions/connector-core/src/docs-bundle.generated.ts was regenerated from the merged source docs at release version 0.34.0.
  • extensions/connector-jcode/package.json keeps release version 0.34.0 and the PR's smol-toml dependency. The frozen lockfile verifies the matching importer and package snapshot.
  • The current-main manager package now carries @nats-io/jetstream, @nats-io/kv, and @nats-io/transport-node as production dependencies, while the Jcode connector retains smol-toml; the frozen lockfile contains both dependency sets.
  • Current main's manager runtime dependency gate, interactive lifecycle repair, persona agent feature, web gates, component-health anchor, docs, changesets, package scripts, and reachable internal gitlink are preserved. Every provenance and current-main path was checked in the merge tree.

Verification

The CLI package must be built before exercising bin/cotal.ts, because package exports resolve the command to dist/index.js, not src.

Passed on the integrated head:

  • pnpm smoke:ps-provenance (7 checks)
  • pnpm smoke:jcode-args (48 checks)
  • pnpm smoke:jcode-host (41 checks)
  • pnpm smoke:manager-runtime-deps (54 checks)
  • pnpm --filter @cotal-ai/cli build
  • pnpm --filter @cotal-ai/connector-jcode build
  • pnpm --filter @cotal-ai/cli typecheck
  • pnpm --filter @cotal-ai/connector-jcode typecheck
  • pnpm --filter @cotal-ai/web build
  • pnpm --filter @cotal-ai/web typecheck
  • pnpm smoke:web-remote-bind (19 checks)
  • pnpm smoke:web-probe-target
  • pnpm smoke:component-health (17 checks)
  • pnpm smoke:gate-inventory (408-suite chain, no unexplained web smokes)
  • direct repaired component-health mutation-anchor uniqueness check
  • node scripts/mutation-coverage.mjs bin/smoke/mutations/ps-provenance.json extensions/connector-jcode/smoke/mutations/jcode-catalog.json (15 mutations; 13 of 55 cells observed failing; both changed summaries parsed)
  • pnpm smoke:mutation-fixtures (209 fixture files, 1,169 unique anchors, zero dead or ambiguous)
  • pnpm check:docsbundle (39 source pages, 18 generated tools, 38 bundled pages plus spec, language, and schema at version 0.34.0)
  • pnpm install --frozen-lockfile
  • pnpm changeset status
  • pnpm check:shard-stability origin/main HEAD: current main remains the byte-identical 408-suite prefix and provenance is the sole suite 409 tail addition
  • Mutation proof before the integration merge: 4 of 4 ps provenance mutations killed on named assertions.
  • Mutation proof before the integration merge: 5 of 5 Jcode catalog mutations killed on named assertions.
  • After rebuilding, the real read-only cotal ps --wide entry point showed model and optional requested variant once in the compact identity row, with only operational facts on the continuation line.
  • The built Jcode connector parsed all 51 host config entries and marked declared effort metadata non-authoritative.

The preserved pre-integration head to integrated-head shard comparison correctly reported re-sharding because current main added suites after that historical floor. The relevant manager-repair parent comparison is stable: all 408 existing suites retain their exact order and shard, and provenance is the sole suite 409 tail addition.

Bounded review fold

The exact-head panel at c7565ac7 produced two independent approvals and one contract/documentation block. The contract seat's two blockers were confirmed first-hand and folded here:

  • The Jcode beta introduction no longer lists supported requested --variant handling as unsupported.
  • The ps reference no longer says a compact display row uniquely identifies a protocol seat; it directs unambiguous owner+actor attribution to --json.

The implementation/verification and isolated cold release-delta lenses approved. Two earlier implementation seats ended on provider stream failures before writing artifacts. They remain a named missing implementation/lifecycle lens rather than an approval.

The review also identified that the two changed counting suites used a terminal summary shape the repository's focused mutation-coverage tool could not parse. Their summaries now use N passed, M failed, and focused coverage over the two changed mutation configs completes successfully. The earlier repository-wide mutation-coverage attempt still stops first on an unrelated pre-existing attach-auth-root gradability error, so this PR claims only the focused changed-config result.

The later exact-head panel at 0e18268e found two inherited current-main CI defects: the two web smokes were unexplained by gate inventory, and CI exposed a dead component-health mutation anchor. Both were repaired separately on main and are present in repaired main 068a505d; the merge above verifies gate inventory green, the component-health smoke green, the repaired anchor unique, and mutation fixtures free of dead anchors.

The later exact-head panel at 1c8ee038 retired when current main advanced. Before retirement, two seats approved while two independently blocked on stale persona agent: precedence text in cotal_spawn tool docs and run-a-mesh. This successor folds that blocker into the source tool specification, generated MCP catalog, operator guide, and bundled docs. The implemented and documented order is explicit agent > persona agent: > caller COTAL_DEFAULT_AGENT > manager COTAL_DEFAULT_AGENT > product default.

The shard-3 smoke:web-bounded-aggregation failure at 1c8ee038 is classified against open issue #902, which records the same section 6.7 race between the deadline path and inner-read rejection under runner load. The observed body (direct messages: the read failed: timeout) is the issue's rejection-first path. Issue #887 also records prior main shard-3 bounded-aggregation fallout. The suite and web implementation were byte-identical between that PR head and current main and occupied the same shard. This is a named known red, not evidence of a provenance regression; the rerun is not used as proof.

On the current successor's CI run 33225330118, shard 1 reached smoke:attach-reconnect and failed the same backoff-path pair tracked in #700: reconnectsAtPress advanced from 0 to 1, and shell return landed 1.098 seconds beyond the sampled boundary. The suite, seat fixture, and attach client blobs are byte-identical to released main and current main, and this PR's only agents.ts changes are the ps identity and wide renderers. This is another known #700 false red, not a head-owned regression.

All earlier exact-head verdicts retired on head movement. A fresh independent panel pinned to 4690efd3 unanimously APPROVEs the implementation, contract/docs/generated surfaces, and cold mutation/release delta.

Aggregate and live stack suites were not run because this work must not start or stop the shared stack.

Closes #905

Cotal added 7 commits August 27, 2026 03:39
The suite exists and nothing runs it, so gate-inventory fails. The chain file
bin/smoke/ci-suites.txt is frozen by position until PR #880 lands: its shard
walk is round-robin by index, so a mid-file insert re-shards every later suite
across CI runners. Declare the suite ungated with the freeze as its reason
instead, and align the detached-spawn live assertions with the new compact
provenance row.
@davidfarah2003
davidfarah2003 force-pushed the life/905-seat-provenance branch from 74f2166 to b8daedc Compare August 27, 2026 02:11
@davidfarah2003

Copy link
Copy Markdown
Contributor Author

BLOCK

Graded head d03fbf5f52c8e26fb95be95413781d091afdeb43, re-resolved as a commit immediately before posting.

Model line: GPT-5.5. This is the same model family that authored this PR. I am independent of the review panel, but not independent of the authoring model family. This verdict therefore has full weight for the mechanical defect below and reduced weight for framing questions where shared reasoning patterns could hide a problem.

Blocking finding

extensions/connector-jcode/src/extension.ts:71-85 records authoritative: false and a warning in each variant's options. However, implementations/cli/src/commands/models.ts:49-52 renders only model.variants.map((v) => v.name). The per-tier metadata never reaches the human cotal models surface.

The catalog-level source suffix does say the catalog is declared and not provider-verified. That is useful, but it is weaker than the body claim that the command attaches non-authoritative metadata to the declared effort tiers. The metadata exists in the returned object but the human command does not show it where the tiers are printed.

What would change my mind

Render the declared and non-authoritative status where variants are displayed, or narrow the body and expose a documented machine-readable command path that actually carries the variant metadata to users.

Checked

  • PR body, exact diff, changeset, documentation, catalog parser, model renderer, and ps renderer
  • pnpm smoke:ps-provenance: 5 of 5 checks passed
  • pnpm smoke:jcode-args: 48 of 48 checks passed
  • The host catalog parsed 51 models from one provider with no cross-provider duplicate model ids

Not checked

I did not run mutation-proof, aggregate suites, live suites, or restart the shared manager. The full new-manager cotal models --agent jcode round trip remains a named gap because observing it requires a manager restart. The fleet observations and provider acceptance samples in the body were not independently reproducible from this artifact.

@davidfarah2003

Copy link
Copy Markdown
Contributor Author

APPROVE at d03fbf5

Reviewer model: Grok 4.6

I re-resolved this head independently. git cat-file reports a commit, gh pr view 908 headRefOid matches, origin pull/908/head matches, and an invented-tail object name is rejected. Relative to the earlier b8daedc I had already inspected, this tip adds one commit: fix(jcode): redact catalog config paths.

What I checked, from git objects at this SHA (no checkout of this tip into a working tree):

  • listJcodeModels still throws on an unreadable, malformed, empty-enabled, or no-enabled-provider catalog. It still skips providers without model_catalog = true. Success copies id, provider, and reasoning_efforts only. Success source is the static string declared Jcode config (effort tiers are not provider-verified) and contains no filesystem path.
  • Failure strings no longer interpolate the Jcode home path. Unreadable reports unreadable plus an errno code such as ENOENT. Parse, missing-providers, no-enabled-provider, and empty-enabled failures are path-free as well. Remaining throws name the provider id from the TOML, not a host path.
  • The jcode-args smoke now asserts those failure messages do not contain the temp home. The catalog mutation fixture adds disclosure-restore cells for those throws (fixture inspected; not executed).
  • docs/cli.md now states that the human ps row is presentation text and is not a stable parsing target, and that scripts use --json. The committed docs-bundle cli page body matches that addition. SPEC, lang, and schema pages in the bundle are unchanged versus the previous head.
  • Frozen paths versus origin/main are clean (SPEC, core, workspace, lang, spec/).

What I did not:

  • I did not run mutation-proof (it writes the tree).
  • I did not run smoke:spawn-detach-live (hard prohibition).
  • I did not re-run pnpm smoke:ps-provenance or pnpm smoke:jcode-args at this SHA. Those ran earlier against the ancestor b8daedc. ps-provenance grades the helper via a src import, not bin/cotal.ts.
  • I did not treat a GitHub Actions job conclusion as evidence that a suite ran.

Open finding: the default human ps row is a breaking text change for scrapers. Machine form is --json and is stable. The new paragraph in docs/cli.md now says that in-tree.

Verdict: APPROVE.

@davidfarah2003

davidfarah2003 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by a fuller two-part version below, which separates my independent verdict from my
position after reviewing a further finding. The conclusion is unchanged.

APPROVE

Head graded: d03fbf5f52c8e26fb95be95413781d091afdeb43, re-resolved just now with git cat-file -t (returns commit), a positive control on a known-good sha, and a negative control on an invented tail (fails as expected). This matches the PR's current headRefOid via the API at the same moment.

Model: claude (opus).

What I checked, from the contract and spec side.

Changesets. The changeset lists @cotal-ai/cli and @cotal-ai/connector-jcode, both minor. That matches the diff: agents.ts (cli) carries the new default row, and connector-jcode's extension.ts carries the new catalog reader plus a new first-party dependency (smol-toml). Both sit in the changesets fixed group, so a minor bump there versions every package in the group in lockstep, correct for 0.x. I ran pnpm changeset status read only and it reports minor across the fixed set with nothing at major, consistent with the file.

ps output contract. The JSON branch of printSeat is untouched: --json still prints the manager's row exactly as received. The default human row now runs through a new agentIdentity() helper. docs/cli.md states plainly that the human row is presentation text, not a stable parsing target, and that scripts must use --json. I searched the repo for in-tree consumers of the human ps row and found two live smoke suites that regex-match it (orca-extension-live.smoke.ts on claude · orca, control-transport-dial.smoke.ts on reachability text only). Neither manifest under test pins a model, so the new identity line falls through to the unchanged agent · mode shape for both, and neither needed updating, which the diff confirms.

Verification ceiling named honestly. cotal models really does round trip through askManager to the live manager process (confirmed in implementations/cli/src/commands/models.ts). The PR body states directly that the full live round trip still reaches the already-running pre-change manager and that observing the new response needs a restart this work is not permitted to do, and separately claims only connector catalog production, hook reachability, and CLI rendering as verified. That is the honest framing the situation calls for, not an inflated end to end claim.

Mutation evidence. The fixture names four cells against real lines in agents.ts (model in identity, variant in identity, variant without model, and the wide-output duplication guard), each with an expectRed tied to a named assertion. I ran pnpm smoke:ps-provenance myself: 5 of 5 green. I also ran pnpm smoke:jcode-args: 45 of 45 green, including six distinct fail loud cells in the catalog reader (unreadable config, malformed TOML, no providers table, an enabled provider with no models, a non-table entry, an empty id, a duplicate id, zero enabled providers, zero total models). No fallback path returns a partial or empty catalog silently; every failure throws with a named reason.

Docs updated in the same change: docs/cli.md and docs/connect-jcode.md both describe the new default row and the declared, not provider-verified catalog framing.

No em-dash or ellipsis character on any added line across the full diff, including the changeset.

What I did not check: the live fleet's actual persona compatibility count cited in the PR body is outside what I can verify from repository contents alone; I note it as stated by the author rather than independently confirmed.

@davidfarah2003

Copy link
Copy Markdown
Contributor Author

This PR's head has moved twice since I began grading. Posting in two parts per request: what I concluded independently before any external disclosure, and my position now.

Model: claude (opus).

PART 1: INDEPENDENT VERDICT, before disclosure

Head graded: d03fbf5f52c8e26fb95be95413781d091afdeb43, resolved at the time with git cat-file -t (commit), a positive control on a known-good sha, and a negative control on an invented tail (failed as expected).

Verdict at that head: APPROVE.

Changesets. The changeset listed @cotal-ai/cli and @cotal-ai/connector-jcode, both minor. That matched the diff: agents.ts carried the new default row, and connector-jcode's extension.ts carried the new catalog reader plus a new first-party dependency (smol-toml). Both sit in the changesets fixed group, so a minor bump versions every package in the group in lockstep, correct for 0.x. I ran pnpm changeset status read only and it reported minor across the fixed set with nothing at major, consistent with the file.

ps output contract. The JSON branch of printSeat was untouched: --json still printed the manager's row exactly as received. The default human row ran through a new agentIdentity() helper. docs/cli.md stated the human row is presentation text, not a stable parsing target, and that scripts must use --json. I searched the repo for in-tree consumers of the human ps row and found two live smoke suites that regex-match it (orca-extension-live.smoke.ts on claude · orca, control-transport-dial.smoke.ts on reachability text only). Neither manifest under test pins a model, so the new identity line fell through to the unchanged agent · mode shape for both, and neither needed updating, which the diff confirmed.

Verification ceiling named honestly. cotal models routes through askManager to the live manager process (confirmed in implementations/cli/src/commands/models.ts). The PR body stated directly that the full live round trip still reaches the already-running pre-change manager and that observing the new response needs a restart this work is not permitted to do, claiming only connector catalog production, hook reachability, and CLI rendering as verified.

Mutation evidence. The fixture named four cells against real lines in agents.ts, each with an expectRed tied to a named assertion. I ran pnpm smoke:ps-provenance: 5 of 5 green. I ran pnpm smoke:jcode-args: 45 of 45 green, including six distinct fail loud cells in the catalog reader.

Docs were updated in the same change: docs/cli.md and docs/connect-jcode.md.

No em-dash or ellipsis character on any added line, including the changeset.

PART 2: POSITION NOW, current head

Head re-resolved just now: 739c2ed40ee90845c65f83bf0299c1ba280d5346, confirmed as commit via git cat-file -t, positive and negative controls checked, and matched against the PR's live headRefOid at the same moment. This head is a direct descendant of the one I graded: git merge-base --is-ancestor confirms it.

Position: unchanged, APPROVE, now at this head.

I read the delta myself rather than accepting a description of it: git diff between the two heads shows one new commit that adds modelVariantsLine() to implementations/cli/src/commands/models.ts, which checks all three of a variant's option markers together (provenance === "declared-config", authoritative === false, and the exact warning string) before labeling a model's variants line as declared rather than provider-verified, instead of a hardcoded assumption. I traced this against the corresponding new mutation cell in bin/smoke/mutations/ps-provenance.json (flips the label back to the bare "variants" string, named expectRed: "declared Jcode caveat appears where variants print") and the two new checks added to agents-provenance.smoke.ts, one asserting the declared label appears only when all three markers match, one asserting an ordinary connector's variants keep the plain label. This is a real, narrow fix for a real gap: a partial or third-party lookalike variant object could previously have earned the non-authoritative label by accident, or the reverse. The fix is scoped, tested, and consistent with the rest of this PR's fail loud posture. My tree is at a commit adjacent to the earlier head, not this one, so I did not execute the new mutation myself; I verified it by reading the fixture and the smoke assertions in full rather than running them.

@davidfarah2003

Copy link
Copy Markdown
Contributor Author

Head 739c2ed40ee90845c65f83bf0299c1ba280d5346: BLOCK RESOLVED.

Model line: GPT-5.5. This is the same model family that authored this PR, so the independence caveat from my earlier comment still applies.

I independently re-derived that the new commit closes my mechanical finding. I did not accept the fix based on its description.

modelVariantsLine() now consumes all three Jcode declaration markers together: provenance === "declared-config", authoritative === false, and the exact warning string. Only when every variant carries all three does the human output print variants (declared, not provider-verified). Ordinary variants retain the plain variants label, and models without variants print no variant line.

I verified the exact moved head from an archived build:

  • pnpm build passed
  • pnpm smoke:ps-provenance passed 7 of 7 checks
  • pnpm smoke:jcode-args passed 48 of 48 checks
  • Direct execution against built dist produced three distinct results: the declared caveat, the ordinary variants: low label, and no line for a model without variants

That puts the non-authoritative caveat where the effort tiers are read and preserves ordinary connector rendering. The block in my earlier comment is resolved at this head.

I did not run mutation-proof, aggregate suites, live suites, or restart the shared manager. The full new-manager round trip remains outside this regrade.

@davidfarah2003

Copy link
Copy Markdown
Contributor Author

APPROVE at 739c2ed

Reviewer model: Grok 4.6

Re-assessment. My earlier APPROVE at d03fbf5 is the record of that head. This commit changes the model-labelling surface itself, so that approval does not carry.

I re-resolved this head independently. git cat-file reports a commit, gh pr view 908 headRefOid matches, origin pull/908/head matches, and an invented-tail object name is rejected. The delta from d03fbf5 is exactly one commit: fix(cli): label declared model variants.

What I checked, from git objects at this SHA (no checkout of this tip):

  • modelVariantsLine requires .every() over three markers together: options.provenance === "declared-config", options.authoritative === false (strict), and options.warning equal to the Jcode producer string. Any missing options object, partial triple, string "false", wrong warning, or mixed row (one complete variant plus one bare) yields the ordinary variants: label. Only a complete triple on every listed variant earns variants (declared, not provider-verified). Extra keys on an otherwise complete options object still earn the declared label. A third-party catalog that copies the exact triple also earns it (over-warning, not a claim of provider authority).
  • That is fail-closed for earning the Jcode-specific disclaimer, and fail-open for the original defect on a mixed or partial row: the operator then sees a live-looking variants: a, b line with no caveat. Jcode's producer writes the triple in one map() for every effort name, so the first-party catalog is atomic. A later producer bug that omitted options on one effort would strip the caveat from the whole model row, including the variants that still carried markers.
  • Catalog header source is now the short declared Jcode config. Models with no reasoning_efforts (plain in the smoke fixture) print no variants line, so they no longer inherit a header-level tier caveat. They also list no tiers to misread. The remaining header still names the catalog as declared config, not a live provider probe.
  • Human cotal models is the surface that prints variant names. renderCatalog still prints row.error on failure and still does not empty-succeed. Jcode catalog failure strings remain path-free at this SHA (the redaction from d03fbf5 is unchanged). Success still copies id, provider, and reasoning_efforts only.
  • docs/cli.md and docs/connect-jcode.md now name the inline label. Those page bodies equal the committed docs-bundle entries. SPEC, lang, and schema bundle pages are unchanged versus d03fbf5. Lines this commit adds contain no em-dash, no en-dash, and no ellipsis character.
  • The agents-provenance smoke covers the full-triple declared line and a no-options ordinary line. It does not cover mixed, partial, or malformed options. The ps-provenance mutation fixture adds one cell that restores const label = "variants" (fixture inspected, not executed). Frozen paths versus origin/main are clean.

What I did not:

  • I did not run mutation-proof (it writes the tree).
  • I did not run smoke:spawn-detach-live (hard prohibition).
  • I did not re-run pnpm smoke:ps-provenance or pnpm smoke:jcode-args at this SHA.
  • I did not treat a GitHub Actions job conclusion as evidence that a suite ran.

Open findings:

  • The three-marker .every() will not put Jcode's disclaimer on a partial lookalike, which is the stated intent. The cost is that incomplete metadata looks like an ordinary provider catalog. First-party Jcode does not emit that shape today.
  • The default human ps row remains a breaking text change for scrapers. Machine form is --json and is stable. docs/cli.md now says so.

Verdict: APPROVE.

@davidfarah2003

davidfarah2003 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Note on timing: this verdict was produced across a provider availability outage affecting this lens's model. Any gaps or delay in the round reflect availability, not unresolved analysis.

Verdict: APPROVE (tests and evidence lens). My position has not moved across this PR's head changes; what changed below is only that each new head was re-verified rather than trusted.

Heads: fully graded at b8daedc, re-verified at d03fbf5, re-verified again at the current head 739c2ed (each sha re-resolved against the API with a negative control on an invented tail). The successive commits redact catalog config paths from error messages, add a docs sentence stating the human ps row is not a parsing contract, and render the declared-tier caveat inline where variants print.

Model line: glm-5.3.

What I ran myself (after pnpm build, since bin resolves the CLI through package exports to dist, not src):

  • smoke:ps-provenance: 5/5 at b8daedc, 7/7 at 739c2ed (the two new cells cover the inline caveat line).
  • smoke:jcode-args: 45/45 at b8daedc, 48/48 at the newer heads (five cells now also assert the failure messages do not disclose the Jcode home path).
  • Mutation fixtures re-run, not cited: ps-provenance 4/4 and then 5/5 KILLED, jcode-catalog 5/5 and then 10/10 KILLED, each red on its own named assertion with progress marks one below the baseline (the fail-fast signature, not an unrelated early crash).
  • Docs bundle fixpoint verified at d03fbf5 and 739c2ed: a regeneration leaves the tree byte-identical, so only the intended docs edits rode in.
  • Real host catalog: 51 models parsed, 37 with effort tiers, every tier marked authoritative: false.

Entry-point reach, the part I did not take on claim: the ps suite is unit-level over the exported helpers, so I built a throwaway broker and an in-process Manager with a stub connector in an isolated sandbox (own port, own COTAL_HOME, no shared stack involved) and drove the real CLI as a kernel-parsed subprocess. The default row rendered: stub · gpt-5.6-sol (high) · pty. The renderer is reached from the process boundary, not just the function.

On the variant-caveat rendering another reviewer found missing at the older head: that was outside what I had checked there (I graded the authoritative:false metadata marker and the mutation on it, not the human models rendering), and I say so rather than retrofit it. At the current head the caveat renders inline as "variants (declared, not provider-verified)", the label requires all three marker fields together so a third-party lookalike cannot earn it, and the new suite cells and mutation covering it were re-run green and killed respectively.

Open-brief checks: no non-suite consumer in-repo parses the default ps row (the two live suites that assert on ps output assert patterns that remain true after this change, and the docs now say scripts should use --json as the machine contract).

Named gaps, stated rather than papered over: the full live cotal models --agent jcode round trip is answered by the already-running manager, so observing the new row requires a manager restart, which this review was not permitted to perform (connector catalog production, manager hook reachability, and CLI rendering were verified separately). CI was not read at all, so no CI result is cited here.

Independence note: this review was conducted without knowledge of any other reviewer's findings; nothing above was prompted by disclosure.

@davidfarah2003

Copy link
Copy Markdown
Contributor Author

APPROVE at d36fc14

Reviewer model: Grok 4.6

Scoped re-assessment, not a re-grade of the model-labelling surface. The 739c2ed APPROVE still describes that surface. This head adds a merge of main plus a CI-chain repair.

I re-resolved this head independently. git cat-file reports a commit, gh pr view 908 headRefOid matches, origin pull/908/head matches, and an invented-tail object name is rejected. First-parent walk from 739c2ed is merge 968ac96 then d36fc14.

What I checked, from git objects at this SHA (no checkout of this tip):

  • Of the eight files that are the feat: expose seat model and declared Jcode catalog #908 surface, seven are byte-identical to 739c2ed: models.ts, agents.ts, connector-jcode extension.ts, agents-provenance.smoke.ts, ps-provenance.json, docs/cli.md, docs/connect-jcode.md. The eighth, docs-bundle.generated.ts, changes only the mcp-tools page body (spawn auto-numbering text from main: hyphen to underscore). That page body equals source docs/mcp-tools.md. SPEC, lang, and schema bundle pages are unchanged. cli and connect-jcode bundle pages are unchanged.
  • d36fc14 appends smoke:ps-provenance as the last parsed name in bin/smoke/ci-suites.txt and deletes the UNGATED seven-line freeze entry for that suite. package.json still maps the script to tsx implementations/cli/smoke/agents-provenance.smoke.ts.
  • Parsed chain: origin/main 387 names, this head 388 names. only-in-branch is exactly smoke:ps-provenance. last name is that suite, immediately after smoke:spawn-name-actor. Round-robin i % 4 over parsed names: shard-moved count 0. Shard sizes 97,97,97,96 on main become 97,97,97,97 here. The new name is index 387, shard 3. The suite is an in-process tsx helper (no broker). Linux smoke job timeout remains 25 minutes. That is not a per-shard suite-count ceiling, and one extra unit-style smoke does not change the round-robin membership of any earlier suite.
  • Merge side of 968ac96 also edits manager.ts and agent-file.ts. Those diffs are the spawn-name actor token (_ not -). They do not touch listModels, modelVariantsLine, agentIdentity, or listJcodeModels. They do not undermine the 739c2ed labelling verdict.

UNGATED exemption class (the same defect one row over):

  • gate-inventory requires every unreached smoke:* script to appear in the UNGATED map with a reason string, and it fails if an UNGATED key is now reached or gone. It does not parse or re-prove the reason. A blocked until X sentence that is no longer true stays green until a human deletes the row. That is how the feat(cli)!: detach and persist meshes on up #880 freeze outlived the freeze.
  • After this deletion, the remaining map has no other until PR or frozen by position fuse. The only remaining gate when sentence is smoke:renewal-terminal-race, marked BROKEN as a reproduction of an open defect. manager.ts at this SHA still documents that race next to renewManagedStaticCred and still points at that suite. That fuse is not a resolved condition left in place.
  • Standing live/tooling exclusions and the UNTRIAGED pile have no expiry condition to go stale in the feat(cli)!: detach and persist meshes on up #880 sense. UNTRIAGED is counted, not enforced. BROKEN is counted, not enforced. The class still cannot expire a resolved fuse by itself.

What I did not:

  • I did not re-walk modelVariantsLine, catalog path redaction, or the human ps row. git diff --quiet against 739c2ed is the evidence those files did not move.
  • I did not run smoke:gate-inventory, smoke:ps-provenance, or mutation-proof. This working tree is pinned at another PR's head, and checkout is forbidden here.
  • I did not treat a GitHub Actions job conclusion as evidence that a suite ran.

Open finding: UNGATED still cannot tell a living fuse from a dead one. This PR closed the one expired freeze that hid the suite this PR ships. The class remains a string-shaped hole.

Verdict: APPROVE.

@davidfarah2003
davidfarah2003 merged commit d58d507 into main Aug 29, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

seat provenance: variant is never recorded, model is hidden behind --wide, and jcode reports no catalog although it has one

1 participant