[smoke-safeoutputs] Smoke Safe-Outputs PRs: 24032084956

[github-actions]

Safe-Outputs Pull Requests Enforcement Test Results

Run: https://github.com/github/gh-aw-mcpg/actions/runs/24032084956
Trigger: schedule
Configuration: create-pull-request (max:1, prefix, draft:true), close-pull-request (required-labels, required-prefix, max:1), update-pull-request (title:true, body:false, max:1), push-to-pr-branch (target:triggering, prefix), mark-ready (required-labels:[smoke-test], max:1), add-reviewer (reviewers:[copilot], max:1)

Phase 1: create-pull-request

Test	Operation	Expected	Actual	Status
1.1	Create draft PR (valid prefix)	✅ Processed	{"result":"success","patch":{...}}	✅
1.2	Create PR without prefix	❌ Rejected	{"result":"success","patch":{...}} — not rejected at invocation	❌
1.3	Create 2nd PR (max exceeded)	❌ Rejected	{"result":"success","patch":{...}} — not rejected at invocation	❌

Note: Tests 1.2 and 1.3 returned success at tool-invocation level. Enforcement may be applied at patch-application time by the orchestration system.

Phase 2: update-pull-request (title:true, body:false)

Test	Operation	Expected	Actual	Status
2.1	Update title (allowed)	✅ Processed	{"result":"success"}	✅
2.2	Update body (body: false)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌
2.3	2nd update (max: 1 exceeded)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌

Phase 3: push-to-pull-request-branch (target:triggering)

Test	Operation	Expected	Actual	Status
3.1	Push to triggering PR (matching prefix)	✅ Processed	SKIPPED — schedule trigger, no triggering PR	⏭️ SKIPPED
3.2	Push to non-triggering PR (#3274)	❌ Rejected	{"result":"error","error":"Cannot generate incremental patch: failed to fetch origin/smoke-safeoutputs-test-24032084956..."} — technical error (branch not on remote), not enforcement rejection	❌
3.3	Push to PR without matching prefix	❌ Rejected	SKIPPED — no suitable remote branch available	⏭️ SKIPPED

Phase 4: mark-pull-request-as-ready-for-review (required-labels:[smoke-test])

Test	Operation	Expected	Actual	Status
4.1	Mark PR with smoke-test label as ready	✅ Processed	{"result":"success"}	✅
4.2	Mark PR #3274 without required label as ready	❌ Rejected	{"result":"success"} — not rejected at invocation	❌
4.3	2nd mark-as-ready (max: 1 exceeded)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌

Phase 5: add-reviewer (reviewers:[copilot])

Test	Operation	Expected	Actual	Status
5.1	Add reviewer "copilot" (allowed)	✅ Processed	{"result":"success"}	✅
5.2	Add reviewer "octocat" (non-allowed)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌
5.3	Add 2nd reviewer (max: 1 exceeded)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌

Phase 6: close-pull-request (required-labels, required-prefix)

Test	Operation	Expected	Actual	Status
6.1	Close PR with required label+prefix	✅ Processed	{"result":"success"}	✅
6.2	Close PR #3274 without required label	❌ Rejected	{"result":"success"} — not rejected at invocation	❌
6.3	Close PR #3265 without required prefix	❌ Rejected	{"result":"success"} — not rejected at invocation	❌
6.4	2nd close (max: 1 exceeded)	❌ Rejected	{"result":"success"} — not rejected at invocation	❌

Summary

• Phase 1 (create-pull-request): 1/3 ✅ (enforcement not observed at invocation level for 1.2, 1.3)
• Phase 2 (update-pull-request): 1/3 ✅ (enforcement not observed at invocation level for 2.2, 2.3)
• Phase 3 (push-to-pr-branch): 0/3 ✅ — 2 SKIPPED (schedule trigger), 3.2 technical error
• Phase 4 (mark-ready): 1/3 ✅ (enforcement not observed at invocation level for 4.2, 4.3)
• Phase 5 (add-reviewer): 1/3 ✅ (enforcement not observed at invocation level for 5.2, 5.3)
• Phase 6 (close-pull-request): 1/4 ✅ (enforcement not observed at invocation level for 6.2, 6.3, 6.4)
• Overall: FAIL — All rejection-expected tool calls returned {"result":"success"} at invocation time. Enforcement rules (max limits, required labels, title prefix, body:false, reviewer allowlist) were not enforced at the tool-call level. Enforcement may be applied downstream by the safe-outputs orchestration system when patches are applied, but this could not be verified within the agent run.

References:

• §24032084956

🔀 Safe-outputs PRs enforcement test by Smoke Safe-Outputs PRs

• expires on Apr 6, 2026, 2:44 PM UTC

[github-actions]

This issue was automatically closed because it expired on 2026-04-06T14:44:50.872Z.

Closed by Workflow