Skip to content

chore(skills): benchmark-driven cleanup of repo-local skills & commands - #3240

Closed
kaeba0616 wants to merge 1 commit into
rtk-ai:developfrom
kaeba0616:chore/skill-cleanup
Closed

kaeba0616 wants to merge 1 commit into
rtk-ai:developfrom
kaeba0616:chore/skill-cleanup

Conversation

@kaeba0616

Copy link
Copy Markdown

What

Cleanup of repo-local .claude/ skills and commands, driven by a controlled benchmark of overlapping/duplicated skills.

Method

Ran each overlapping skill against the same planted task, scored outputs against answer keys, and verified every produced artifact with cargo test --offline (fixtures present, lazy_static used, tests passing). Key finding: on these tasks, detection accuracy was at ceiling — skills differed in cost, judgment conservatism, and report structure, not correctness.

Changes

  • Remove rtk-tdd — scored 11/12 in the TDD benchmark because it used synthetic fixtures (the anti-pattern cli-testing.md explicitly warns against), vs tdd-rust's 12/12 at lower token cost. TDD is now consolidated on tdd-rust.
  • Rename ship → rtk-release — the repo ship skill was shadowed by gstack's ship in the skill list and was effectively unreachable. Renamed the directory (history preserved), added an explicit name: field, and updated the /ship invocation hint to /rtk-release.
  • Consolidate worktree commands — they existed twice (.claude/commands/ root and tech/) with diverged content. Kept the richer root versions (they document --check/--fast, .worktreeinclude, manual force-remove), deleted the tech/ duplicates, and moved remove-worktree into root so the whole worktree family lives in one place.

tech/ now holds only its unique commands (audit-codebase, codereview).

Not included (deliberately)

  • pr-review vs pr-triage — benchmark showed their behavior converges (pr-triage drifted into per-PR deep review), but which to keep is a workflow preference, so left for maintainer judgment.
  • investigate — flagged as redundant with systematic-debugging, but it's a global gstack skill (out of this repo's scope; regenerated on gstack upgrade).

Verification

All benchmark arm outputs compiled and passed cargo test --offline. No source code changed — this touches only .claude/ skill/command definitions. Dangling-reference check passed (no stray rtk-tdd or /tech:worktree references).

🤖 Generated with Claude Code

Ran a controlled benchmark (same task per overlapping skill, scored against
answer keys, outputs verified with `cargo test --offline`). Applied the verdict:

- Remove rtk-tdd: 11/12 in the TDD benchmark (used synthetic fixtures, the
  anti-pattern cli-testing.md warns against) vs tdd-rust's 12/12 at lower cost.
  TDD is now consolidated on tdd-rust.
- Rename ship -> rtk-release: the repo `ship` skill was shadowed by gstack's
  `ship` in the skill list and was unreachable. Renamed the dir, added an
  explicit `name:`, and updated the `/ship` invocation hint to `/rtk-release`.
- Consolidate worktree commands: they existed twice (root and tech/) with
  diverged content. Kept the richer root versions (they document --check/--fast,
  .worktreeinclude, manual force-remove), deleted the tech/ duplicates, and moved
  remove-worktree into root so the whole worktree family lives in one place.

tech/ now holds only its unique commands (audit-codebase, codereview).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Jul 27, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@rtk-release-bot

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale due to 90 days of inactivity. If this pull request is still relevant, please leave any comment (for example, "bump"), and we'll keep it open. Your contribution is very much appreciated — we're sorry we haven't been able to review it yet.

@rtk-release-bot

Copy link
Copy Markdown
Contributor

Closing this pull request after a prolonged period of inactivity. If this is still relevant, please ask for it to be reopened. Thank you!

@rtk-release-bot rtk-release-bot Bot closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants