Skip to content

[agent] preserve acceptance requirements with explicit length validation - #522

Merged
r4yharris-agent-coder[bot] merged 4 commits into
mainfrom
agents/acceptance-fidelity
Oct 11, 2026
Merged

r4yharris-agent-coder[bot] merged 4 commits into
mainfrom
agents/acceptance-fidelity

Conversation

@r4yharris-agent-coder

Copy link
Copy Markdown
Contributor

fix: preserve acceptance requirements with explicit length validation

Model

GPT-6.1-Sol; GHCP context used/output unknown,1M declared capacity.

Summary

Refs #521 and #489. FEATURE_SPEC5.3/5.4/5.5, honoring3/7. Actual E2 local-41186ddf913d86b2 exposed checks silently truncated mid-word at240characters. Introduce shared explicit acceptance validation across slice JSON, TASK and PLAN parsing, child outlines and receipts. Retain240character normalized bound but reject oversize with actionable correction feedback; title shortening unchanged. All planner prompts describe the bound. No added dependency/store, authority widening, policy/workflow/vendor edits or private routing changes.

Verification

Two initial red regressions established silent acceptance and missing correction. Final affected Node20 planner/planning/layout selection94total93pass1existing skip0fail,78.7s. Exact240/241 boundary, whitespace normalization, four ingress paths, JSON and written-TASK correction preserve original Ask and complete concise requirements. No test threshold weakened.

node --test --test-concurrency=4 tests\planner.acceptance.test.mjs tests\planner-task.test.mjs tests\planner.test.mjs tests\plan-mode.test.mjs tests\test-layout.test.mjs

Latest actual easy E2 before this repair reviewed pass after one repair571636ms, GLM served for requested Qwen; not efficient or comparable local-Qwen proof. Higher/concurrent tests and human acceptance remain gated. #520 unrelated memory PR trusted1720/1720green but pinned publisher stale-head defect blocks its merge; this fresh scoped PR does not duplicate its commits or waive that blocker.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

AI-Agent: r4yharris-agent-coder
AI-Model: GPT-6.1-Sol

1|github-copilot|GPT-6.1-Sol@-|x|-/1000000|-|ghcp-20261011-115104|acceptance-fidelity

## Model

GPT-6.1-Sol; GHCP context used/output unknown,1M declared capacity.

## Summary

Refs #521 and #489. FEATURE_SPEC5.3/5.4/5.5, honoring3/7. Actual E2 local-41186ddf913d86b2 exposed checks silently truncated mid-word at240characters. Introduce shared explicit acceptance validation across slice JSON, TASK and PLAN parsing, child outlines and receipts. Retain240character normalized bound but reject oversize with actionable correction feedback; title shortening unchanged. All planner prompts describe the bound. No added dependency/store, authority widening, policy/workflow/vendor edits or private routing changes.

## Verification

Two initial red regressions established silent acceptance and missing correction. Final affected Node20 planner/planning/layout selection94total93pass1existing skip0fail,78.7s. Exact240/241 boundary, whitespace normalization, four ingress paths, JSON and written-TASK correction preserve original Ask and complete concise requirements. No test threshold weakened.

node --test --test-concurrency=4 tests\planner.acceptance.test.mjs tests\planner-task.test.mjs tests\planner.test.mjs tests\plan-mode.test.mjs tests\test-layout.test.mjs

Latest actual easy E2 before this repair reviewed pass after one repair571636ms, GLM served for requested Qwen; not efficient or comparable local-Qwen proof. Higher/concurrent tests and human acceptance remain gated. #520 unrelated memory PR trusted1720/1720green but pinned publisher stale-head defect blocks its merge; this fresh scoped PR does not duplicate its commits or waive that blocker.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

AI-Agent: r4yharris-agent-coder
AI-Model: GPT-6.1-Sol
AI-Run: 1|github-copilot|GPT-6.1-Sol@-|x|-/1000000|-|ghcp-20261011-115104|acceptance-fidelity
@r4yharris-agent-coder
r4yharris-agent-coder Bot marked this pull request as ready for review October 11, 2026 15:50
## Model

GPT-6.1-Sol.

## Summary

Refs #521 and #522. Spec5.3/5.4/5.5, honoring3/7. Trusted full CI exposed one genuine compatibility issue: an existing human task with a long check must remain readable for source-validated evidence continuation. Do not shorten or weaken that fixture. Parsing now preserves full normalized text; new model-authored TASK and PLAN proposals still enforce240characters in planFromTask, with slice JSON, outlines and receipts sharing the same strict guard. Titles unchanged. Existing64KiB task bound remains. Update docs and obsolete checklist comment.

## Verification

Node20 affected planner/context/checklist/planning/layout142total141pass1existing skip0fail81.4s. Tests cover exact240/241, legacy readback retains suffix, new proposal rejects oversize, two bounded correction paths retain original Ask. Existing long-check context regression passes unchanged. Prior trusted full matrix1719pass1failure was solely that compatibility defect. No new deps/store/policy/workflow/vendor edits or human acceptance fabricated.

node --test --test-concurrency=4 tests\planner.acceptance.test.mjs tests\context.test.mjs tests\planner-task.test.mjs tests\planner.test.mjs tests\plan-mode.test.mjs tests\test-layout.test.mjs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

AI-Agent: r4yharris-agent-coder
AI-Model: GPT-6.1-Sol
## Model

GPT-6.1-Sol.

## Summary

Refs #521/#522/#489, Spec5.3/5.4/5.5 respecting3/7. Actual strict-fidelity E2 local-d1d1627b14fefa5a stopped100108ms before coder: retained original proposal had safe covering/gives instructions/indexes content phrasing rejected by verb validator. Accept those document-content verbs, retain all command/action success assertions and active clauses. Preserve exact original Ask, no silent truncation, no execution grant. This directly closes the observed plain-text acceptance regression on the reviewed fidelity branch.

## Verification

Node20 targeted planner/task/check-fidelity/context/layout68/68pass. Added exact observed safe phrasings and negative command/deployment-success variants. Latest prior compatibility-repair trusted full CI is green; original legacy long-check fixture unchanged. No dependencies/store, policy/workflow/vendor edits, private config changes or fabricated evaluation.

node --test --test-concurrency=4 tests\planner-task.test.mjs tests\planner.acceptance.test.mjs tests\planner.test.mjs tests\context.test.mjs tests\test-layout.test.mjs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

AI-Agent: r4yharris-agent-coder
AI-Model: GPT-6.1-Sol
@r4yharris-agent-coder

Copy link
Copy Markdown
Contributor Author

Acceptance fidelity repair and actual easy results

#522 head f208b8c is App-authored, open and non-draft.
Final trusted Linux Node20 CI: 1720/1720 pass,0 failures,0 skips,52333ms runner time; trailer check passes.
Spec5.3/5.4/5.5 respecting3/7, scoped #521.
Shared strict validation rejects oversize NEW planner checks rather than silently truncating at240characters.
Reading existing tasks preserves full normalized checks, including legacy long acceptance-continuation obligations.
Initial full CI exposed that genuine compatibility gap; fixed without changing the original long-check fixture.
Affected Node20 planning/context/checklist/layout142total141pass1existing skip0fail81.4s.
Final observed document-content wording coverage plus boundaries/context/layout68/68pass.
Covering, gives instructions and indexes content phrasing now recognized; command/action-success assertions remain denied.

Actual easy E2 evidence, all retained rather than cherry-picked:

  • local-41186ddf913d86b2 on [agent] retain planner memory across planning modes #520: review pass after one repair571636ms. Independent readback shows design and concrete GitHub upload/Pages instructions. Not efficient; not human acceptance.
  • local-d1d1627b14fefa5a on strict-check repair: planner stopped100108ms before coder on safe content wording; exact redacted TASK and checks captured, then validator repaired.
  • local-e5a92d0bc5987230 on final [agent] preserve acceptance requirements with explicit length validation #522: malformed write_file JSON arguments after bounded retry62541ms; no coder/reviewer.
  • local-c21ee2959a02b0ef diagnostic: reached600s task deadline during repeated invalid long-check proposals. Clone-stream diagnostic promise aborted and the external driver exited before final observation save. This is a failed run with incomplete diagnostic evidence, not successful delivery. Live log and TASK remain; driver now catches/reports capture failures immediately to avoid losing the final observation on future aborts. No fresh run claimed from that driver fix.

Gateway served GLM for requested Qwen throughout these replays. No private config, endpoint, routing or timeout changed; no comparable local-Qwen speed claim.

Both #520 and #522 remain publication-blocked by pinned reviewed contracts SDK immediate post-push stale-head checks.
Real App follow-up commits pushed; no dummy/duplicate commit, direct merge, human fallback, vendor patch or policy/workflow change.
SDK lacks resume-only mode. Separate dependency repair authorization was unavailable, not assumed.
Installed persistent clean checkout remains5202946 (#518); neither green PR is claimed merged/installed.
Overall reliable easy delivery, efficient end-to-end functionality, human AI-Eval, cross-platform and harder/concurrent gates remain incomplete.

## Model

GPT-6.1-Sol.

## Summary

Refs #521/#522/#489; FEATURE_SPEC5.3/5.4/5.5 respecting3/7. Actual local-c21ee2959a02b0ef consumed600s during repeated oversized TASK proposals. Give this specific length failure its existing single validation correction immediately, rather than spending the general turn budget rewriting the same invalid contract. Still-invalid correction explicitly stops before coder/publication. Preserve all existing changed-Ask, scope, locked-model and other rewrite behavior. No parsing waiver, requirement truncation or increased budget.

## Verification

New red20-turn regression confirmed prior repeated rewrite exhaustion. Final Node20 planner/acceptance/task/plan-mode/layout63/63pass,4.8s. Exactly two calls for unchanged invalid oversized draft, successful written and JSON corrections retained; existing Ask/protected-path/selected-model refusals unchanged. Last full trusted CI1720/1720green; new head requires CI. No private config, runtime deps/store, workflow/policy/vendor edit, dummy commit or invented evaluation.

node --test --test-concurrency=4 tests\planner.acceptance.test.mjs tests\planner.test.mjs tests\planner-task.test.mjs tests\plan-mode.test.mjs tests\test-layout.test.mjs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

AI-Agent: r4yharris-agent-coder
AI-Model: GPT-6.1-Sol
@r4yharris-agent-coder

Copy link
Copy Markdown
Contributor Author

Verified bounded oversized-check correction

#522 newest App head3d7aedc088f8bcb0c6a0308cd8000a184f5fcbda: trusted Linux Node20 full1721/1721pass,0fail,0skip,64211ms runner time. Trailer check passes.
New exact20-turn regression established oversized TASK rewriting consumed the full budget.
Only that observed length-error path now uses its existing single validation correction immediately; still-invalid correction stops after two requests.
Existing changed-Ask/protected-path/selected-model rejection and successful rewrite contracts unchanged.
Targeted Node20 planner/acceptance/task/plan-mode/layout63/63pass4.8s.
Spec5.3/5.4/5.5 respecting3/7, no runtime deps/store, private config or authority changes.

Newest actual easy E2 local-4c41ceabafca57db completed in396751ms with review pass after one repair.
Independent readback confirms design layout/copy/style/accessibility and setup guide with HTML starter, concrete Netlify manual-upload controls and GitHub push/Pages branch-root settings, public URLs, DNS/HTTPS, costs and illustrative placeholders.
Every rendered check is complete and within240characters; original Ask remains unchanged.
This is an observed reviewed delivery, not human acceptance or reliable corpus-wide performance.
Mixed responses remain visible: planner/coder Qwen, reviewer GLM, earlier requests also GLM aliases. Missing coder usage stays unknown; no single-model speed comparison.
All earlier failures remain recorded, including malformed62541ms and prior600s diagnostic abort.
Capture driver now reports clone-stream errors immediately so diagnostic aborts cannot become unhandled promises; no aborted run is relabeled success.

#520 and #522 remain OPEN/non-draft with green checks, not merged/installed.
App publisher again pushed genuine repair then rejected immediate stale PR-head response; no resume-only mode.
No duplicate/dummy commit, human or direct merge, vendor patch or policy/workflow bypass.
User unavailable to authorize separate contracts-repository repair; no permission inferred.
Persistent installed checkout remains clean5202946 (#518).
Higher/concurrent difficulty and human evaluation gates remain unsatisfied.

@r4yharris-agent-coder
r4yharris-agent-coder Bot merged commit 418101a into main Oct 11, 2026
2 checks passed
@R4yHarris
R4yHarris deleted the agents/acceptance-fidelity branch October 20, 2026 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants