Skip to content

Add corrupt success case study - #41

Open
marsloting wants to merge 2 commits into
vectara:mainfrom
marsloting:sb-awesome-vectara-corrupt-success
Open

Add corrupt success case study#41
marsloting wants to merge 2 commits into
vectara:mainfrom
marsloting:sb-awesome-vectara-corrupt-success

Conversation

@marsloting

Copy link
Copy Markdown

This adds a case study for a failure mode that is easy to miss in agent workflows: corrupt success.

The agent reports done and the producer-side check passes, but the downstream consumer never receives the work because proof was collected upstream of the point where the failure surfaces.

Why this fits this list:

  • It is a concrete verification and termination failure, not just a hallucination.
  • It connects a production workflow failure pattern to the 2026 "corrupt success" research thread.
  • It adds a practical mitigation: evidence for done/written/shipped claims must come from the consumer endpoint, not from an upstream proxy.

Changes:

  • Added a new case study: docs/case-studies/agent-corrupt-success-consumer-verification.md
  • Added a README entry under Autonomous Agent Failures
  • Added the example to docs/failure-modes/verification-termination.md

Validation:

  • git diff HEAD~1..HEAD --check
  • Scanned changed files for accidental private identity/internal-context terms before publishing

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new case study documenting the “corrupt success” verification/termination failure mode, where an agent’s producer-side “done” evidence passes checks but the downstream consumer never receives the work. This expands the repository’s catalog of concrete agent workflow failure patterns and mitigation guidance.

Changes:

  • Added a new case study on consumer-end verification for “done/written/shipped” claims.
  • Linked the new case study from the main README under “Autonomous Agent Failures”.
  • Added the case study as a real-world example under Verification & Termination Failures.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
README.md Adds a new “Autonomous Agent Failures” entry pointing to the corrupt success case study.
docs/failure-modes/verification-termination.md Adds a new real-world example describing the corrupt success verification failure pattern.
docs/case-studies/agent-corrupt-success-consumer-verification.md Introduces a new case study write-up with scenario, impact, and mitigation guidance focused on consumer-end proof.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +5 to +7
**Agent**: AI coding and operations agent<br>
**Operator**: Marsloting agent workflow<br>
**Date**: June 2026<br>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants