Skip to content

watch: no verification gate - round success is the child process exit code, nothing builds, tests, or lints the resulting work #1712

Open

Description

@swigerb

Summary

Nothing in the squad watch path verifies that the work an agent produced actually builds, passes tests, or lints. A round is considered successful purely on the spawned child process exit code.

Evidence

packages/squad-cli/src/cli/commands/watch/capabilities/execute.ts:204-212:

const result = await executeAll(eligible, context, timeout);

return {
  success: result.success,
  summary: result.success
    ? `agent dispatched with ${eligible.length} issues`
    : `agent failed: ${result.error}`,
  data: { dispatched: eligible.length, success: result.success },
};

result.success comes from spawnAgent() in watch/agent-spawn.ts:190-211, which resolves { success: true } on child exit code 0 and { success: false, error } otherwise.

Grepping the whole watch/ tree for npm test, lint, or any build invocation returns only unrelated buildAgentCommand / buildAgentPrompt identifier matches. No capability in any of the four phases (pre-scan, post-triage, post-execute, housekeeping) runs the repository's tests, build, or linter.

Impact

  • The agent CLI exits 0 whenever it completed its turn, regardless of whether the resulting code compiles. "Agent finished talking" is being treated as "work is good".
  • An unattended overnight run can push a series of green-looking rounds while leaving the branch broken.
  • The repo already declares its own feedback loops in package.json (test, lint, lint:eslint, build), so the signal is available and free - it is simply never consulted.

Proposal

Add an opt-in verification gate capability that runs after execute in the post-execute phase:

  1. Resolve the project's declared checks. Start with a verify block in watch config (list of shell commands), defaulting to auto-detection from package.json scripts (build, test, lint) when present.
  2. Run them in the agent's working directory after the agent turn completes.
  3. Treat a non-zero result as a failed round for that work item, not a successful one - surface it in reportBoard() and feed it into failure tracking.
  4. Make it configurable (verify: false to opt out) so existing users are not broken.

This turns "the child process exited" into "the repository still works", which is the difference between an autonomous loop and an unattended one.

Activity

  1. added 3 commits that reference this issue on Oct 31, 2026
    075ebde
    c6a16a5
    02ac551
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    wave1:P5Wave 1 PR group P5wave:1-nextWave 1 — immediate next sprint

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions