Skip to content

feat: improve skill scores for 5 core Rosetta skills - #50

Open
rohan-tessl wants to merge 1 commit into
griddynamics:mainfrom
rohan-tessl:improve/skill-review-optimization
Open

feat: improve skill scores for 5 core Rosetta skills#50
rohan-tessl wants to merge 1 commit into
griddynamics:mainfrom
rohan-tessl:improve/skill-review-optimization

Conversation

@rohan-tessl

@rohan-tessl rohan-tessl commented Apr 8, 2026

Copy link
Copy Markdown

Hey @isolomatov-gd 👋

I ran your skills through tessl skill review at work and found some targeted improvements.

image

Here's the full before/after:

Skill Before After Change
coding-agents-prompt-adaptation 10% 84% +74%
requirements-authoring 10% 87% +77%
requirements-use 10% 77% +67%
load-context 25% 68% +43%
reasoning 36% 65% +29%

This PR is intentionally scoped to 5 skills to keep it reviewable — more can be improved in follow-ups or via automated review on future PRs.

Note: The source skills live in instructions/r2/core/skills/ and are propagated to plugins/ via scripts/plugin_generator.py. All changes target the source files only — plugin copies will regenerate automatically on the next sync.

Changes summary

All 5 skills:

  • Fixed duplicate tags fields in YAML frontmatter (caused validation failures and 10% scores for 3 skills)
  • Ensured description uses quoted string format without XML-like tags
  • Removed generic <role> preamble sections (the agent doesn't need motivational framing)

coding-agents-prompt-adaptation:

  • Restructured into explicit 8-step numbered workflow (detect → read → load KB → identify → HITL → adapt → validate → deliver)
  • Consolidated best_practices into workflow steps to reduce redundancy
  • Compressed pitfalls into categorized single-line entries

requirements-authoring:

  • Added explicit 9-step numbered workflow replacing the scattered <authoring_flow> and <initialization> sections
  • Consolidated SRP/DRY/KISS/MECE rule sections into a single <core_principles> block
  • Merged <conflict_checks>, <gap_checks>, and <validation_rules> into a unified <validation_checklist>
  • Combined <resources> and <templates> (they listed the same assets)
  • Compressed <language_constructs> into a dense paragraph preserving all rules

requirements-use:

  • Added explicit 7-step numbered workflow integrating the <process> steps with HITL gates
  • Consolidated <requirement_usage_rules> and <ambiguity_and_conflict_rules> into the workflow and core_concepts
  • Merged <resources> and <templates> sections

load-context:

  • Expanded from 2 lines to a full skill with proper frontmatter (tags, license, metadata)
  • Added 5-step workflow with clear blocking prerequisite semantics
  • Added validation checklist and pitfalls section

reasoning:

  • Restructured 7D flow from flat bullet lists into an explicit numbered workflow
  • Made confidence feedback loop actionable: specific formula for weighted confidence, targeted retry of weakest sub-problem (not entire process)
  • Consolidated best_practices into the workflow and pitfalls

Honest disclosure — I work at @tesslio where we build tooling around skills like these. Not a pitch - just saw room for improvement and wanted to contribute.

Want to self-improve your skills? Just point your agent (Claude Code, Codex, etc.) at this Tessl guide and ask it to optimize your skill. Ping me - @rohan-tessl - if you hit any snags.

Thanks in advance 🙏

Hey @isolomatov-gd 👋

I ran your skills through `tessl skill review` at work and found some targeted improvements. Here's the full before/after:

| Skill | Before | After | Change |
|-------|--------|-------|--------|
| coding-agents-prompt-adaptation | 10% | 84% | +74% |
| requirements-authoring | 10% | 87% | +77% |
| requirements-use | 10% | 77% | +67% |
| load-context | 25% | 68% | +43% |
| reasoning | 36% | 65% | +29% |

This PR is intentionally scoped to 5 skills to keep it reviewable — more can be improved in follow-ups or via automated review on future PRs.

Note: The source skills live in instructions/r2/core/skills/ and are propagated to plugins/ via scripts/plugin_generator.py. All changes target the source files only — plugin copies will regenerate automatically on the next sync.

Changes:
- Fixed duplicate tags fields in YAML frontmatter (caused validation failures)
- Ensured description uses quoted string format without XML-like tags
- Removed generic role preamble sections
- Added explicit numbered workflows to all 5 skills
- Consolidated redundant rule sections (SRP/DRY/KISS into core_principles)
- Merged overlapping validation/conflict/gap checks into unified checklists
- Expanded load-context from 2 lines to full skill with workflow
- Made reasoning confidence feedback loop actionable with weighted formula

Signed-off-by: rohan-tessl <rohan-tessl@users.noreply.github.com>
@rohan-tessl
rohan-tessl marked this pull request as ready for review April 8, 2026 08:42
@omaiesh
omaiesh requested a review from isolomatov-gd April 9, 2026 15:21
@isolomatov-gd

Copy link
Copy Markdown
Contributor

Thank you for the review, we have a skill we use internally to improve prompts which we did not apply on those files. Thank you for the PR, we will harden those prompts

@isolomatov-gd isolomatov-gd added invalid This doesn't seem right needs more work labels Jun 17, 2026
@isolomatov-gd
isolomatov-gd marked this pull request as draft August 18, 2026 19:23
@isolomatov-gd
isolomatov-gd marked this pull request as ready for review August 18, 2026 19:23
@github-actions

Copy link
Copy Markdown
Contributor

Rosetta Triage Review

Summary: Proposes rewording/restructuring 5 instructions/r2/core/skills/*/SKILL.md files (KTLO release) based on scores from an external tool ("tessl skill review"): coding-agents-prompt-adaptation, load-context, reasoning, requirements-authoring, requirements-use.

Findings:

  • GitHub reports this PR as mergeable: CONFLICTING / mergeStateStatus: DIRTY — it cannot be auto-merged as-is.
  • The duplicate top-level tags: frontmatter key is a real, currently-present bug in requirements-authoring/SKILL.md and requirements-use/SKILL.md on main (confirmed by direct inspection) — the fix for those two files is legitimate. reasoning and load-context have no tags: key at all, so the PR's "duplicate tags caused 10% scores for 3 skills" claim doesn't hold for those two.
  • The PR's base commit is ~4 months old; the r2 skills tree has since been restructured independently on main: coding-agents-prompt-adaptation no longer exists there (its function appears folded into coding-agents-prompt-authoring/references/pa-adapt.md), and load-context/reasoning on main already contain more advanced content (explicit Plugin/MCP/Adhoc mode-detection workflow; 8-step reasoning flow with updated model list) than either this PR's base or its proposed replacement. Merging as-is would resurrect a removed file and clobber those independent improvements.

Caveats:

  • The PR body's percentage claims (e.g. 10%→84%) come from an external, non-public scoring tool with no visible rubric — not verifiable from repo content alone.
  • Author discloses affiliation with an adjacent prompt-tooling vendor (Tessl) and promotes it in the PR body — noted for maintainer awareness, not a defect.
  • Per docs/ARCHITECTURE.md, R2 is "backported fixes only." The duplicate-tags fix fits that scope; the broader restructuring (numbered <workflow> blocks, consolidated principle sections, expanded load-context) goes beyond a narrow backport and duplicates evolution already happening independently on main.

Suggestions:

  • Rebase against current main and regenerate the diff — several target files have moved on since this was opened.
  • Consider splitting the uncontested duplicate-tags YAML fix from the structural rewrite.

Questions:

  • After rebasing onto current main, which of these changes are still intended, given load-context/reasoning have already evolved past this PR's baseline and coding-agents-prompt-adaptation no longer exists as a standalone skill?

Automated triage by Rosetta agent

@github-actions github-actions Bot added the instructions skills, subagents, rules, workflows, commands for AI agents label Aug 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📋 Prompt Quality Validation Report

❌ Validation Failed

The full markdown report and raw JSON output are available in the workflow artifacts for 5 days.


Files With Issues

  • instructions/r2/core/skills/coding-agents-prompt-adaptation/SKILL.md: 3 issue(s)
  • instructions/r2/core/skills/load-context/SKILL.md: 5 issue(s)
  • instructions/r2/core/skills/reasoning/SKILL.md: 4 issue(s)
  • instructions/r2/core/skills/requirements-authoring/SKILL.md: 6 issue(s)
  • instructions/r2/core/skills/requirements-use/SKILL.md: 3 issue(s)

📄 instructions/r2/core/skills/coding-agents-prompt-adaptation/SKILL.md

⚠️ Issues Found

Severity Gate Details
High Precision & Explicitness Problem:
The deleted <best_practices> block removed the only two rules that protected section order and edit size: "Preserve original structure and section order" and "Keep diffs surgical and traceable". Nothing in the new file replaces them. The closest surviving rules are #5 "Maintain file names and sub-paths exactly as-is" (paths only), #11 "Keep everything else AS-IS including unknowns", and #12 "MUST NOT rewrite lines in your own way" (wording only) — none forbids moving or merging whole sections.
Reason:
An agent can reorder or merge sections and still pass every remaining rule, because reordering is not rewriting a line. Section order in these prompts is load-bearing, so this silently destroys the source's sequence.
Solution:
Add two lines to the boundaries list in <adapt_transformations>: preserve source structure and section order; keep diffs surgical and traceable.
Medium Rosetta Problem:
The frontmatter description was extended with "Use when porting prompts between agents/IDEs, adapting KB prompts to local context, or migrating rules between formats" — the exact sentence that already opens <when_to_use_skill>. The same duplication pattern was introduced in the descriptions of reasoning, requirements-authoring, and requirements-use in this PR.
Reason:
Descriptions stay in context for every turn of every session across all agents, so duplicated text is paid for repeatedly, and long descriptions get cut off in agent menus where they drive skill selection.
Solution:
Trim each description to a short call-to-action and drop the duplicated "Use when..." clause, since <when_to_use_skill> already carries it. Apply the same trim to the other three skills changed in this PR.
Medium Output Contract Problem:
The <templates> block was deleted. It held the only statement of the output shape: "Adapted prompt follows the same format as the source prompt. No additional templates — adaption preserves source structure." The new file only says "store in target IDE/Agent/OS format and location", which is about destination, not shape.
Reason:
With no output shape stated, the agent may re-template the prompt instead of producing a format-preserving copy, which is the opposite of what ADAPT is for.
Solution:
Keep one line stating that the adapted prompt keeps the source format and section order, and that no new templates are introduced.

📄 instructions/r2/core/skills/load-context/SKILL.md

⚠️ Issues Found

Severity Gate Details
Very High Dependency Management Problem:
get_context_instructions is now hardcoded in six places — the frontmatter description, workflow steps 1 and 2, the fallback line, the validation checklist, and the pitfalls. The base file named it once, guarded by "(if available)".
Reason:
This one file ships unchanged to MCP, plugin, and local modes. Naming one transport's tool as the only path makes the skill wrong in the other two.
Solution:
Use the mode-neutral alias (EXECUTE PREP STEPS / GET PREP STEPS) in the body, and keep the raw tool name only inside the MCP branch.
Very High Failure Handling Problem:
The new fallback reads "If get_context_instructions is unavailable or fails, immediately ask the user for help — this is critical and unexpected." It merges two different situations. In plugin and local mode the tool being absent is the normal case, not an anomaly.
Reason:
As written, a plugin-mode session can open by interrupting the user with a false alarm instead of loading context. That blocks the work and trains users to dismiss genuine escalations.
Solution:
Split the two cases: tool absent means fall through to the active mode rule's prep steps and continue silently; tool present but failing means stop and ask the user.
Very High Decision Branching Problem:
The base line was "load current context using get_context_instructions tool (if available)". New workflow step 1 drops "(if available)" and makes the call unconditional: "Call get_context_instructions — this is a blocking prerequisite; do not respond, call other tools, or process the user message until this completes". No branch covers plugin or local mode, where no such callable tool exists — plugin-files-mode.md line 58 only maps the name to "already loaded bootstrap-* instructions, continue with prep steps 2 and 3".
Reason:
plugin-files-mode.md line 85 invokes this skill at session start ("USE SKILL load-context, if available"), so in plugin mode the agent's first instruction is to call something that is not a tool. Behavior then varies run to run — some agents resolve the alias, others stall — and non-determinism at session start is a reliability defect.
Solution:
Restore the availability branch in step 1: if the tool exists, call it; otherwise execute the prep steps already loaded by the active mode rule, then continue.
High Bloat Control Problem:
The body grew from two lines to about forty, and much of the new text repeats always-on bootstrap policy — "Skipping prep steps because the task seems simple", "Calling other tools before get_context_instructions", and "do not respond, call other tools, or process the user message until this completes" all restate the <must> block in bootstrap.md.
Reason:
This skill loads at the start of every session, so duplicated policy is paid for on every request while adding no instruction the agent does not already have in context.
Solution:
Keep the numbered workflow and the validation checklist; drop the pitfalls that restate always-on bootstrap policy.
High Rosetta Problem:
The rewritten skill restates prep-step mechanics that the always-on bootstrap rules already own, and contradicts them. plugin-files-mode.md line 58 says the tool call is already satisfied and the agent should "continue with prep steps 2 and 3", while this skill now says the call itself is a blocking prerequisite. The checklist also hardcodes "All three prep steps".
Reason:
When a skill and an always-on rule disagree about the first action of a session, the agent picks one arbitrarily, so the session either stalls or skips context loading.
Solution:
Keep the skill mode-neutral: point at the active mode rule for prep-step mechanics, and say "all prep steps" instead of "all three prep steps".

📄 instructions/r2/core/skills/reasoning/SKILL.md

⚠️ Issues Found

Severity Gate Details
High Decision Branching Problem:
The new workflow intro says "For simple questions, skip to step 5 (DELIVER) with a direct answer", but steps 6 DESIGN and 7 DEBRIEF still follow step 5 in the same numbered list and nothing says to stop. The base kept this branch out of the numbering entirely: "For simple questions, skip deep decomposition and answer directly".
Reason:
A step-following agent lands on 5 and continues into 6 and 7, so the cheap path for a trivial question becomes the full 7D flow on some runs and not others.
Solution:
State the exit explicitly at the branch point, for example: simple question means DELIVER only, then stop.
High Precision & Explicitness Problem:
Step 6 DESIGN lost its enumerated quality-attribute list. The base named "security, performance, reliability, maintainability, scalability, testability, observability, compliance, backward compatibility, and cost"; the new step says only "include NFR/quality attributes".
Reason:
Agents cover the dimensions that are named and skip the ones left implicit, so security, compliance, and backward-compatibility checks now drop out of every design pass.
Solution:
Keep the enumerated attribute list inside step 6, as a compact inline series after "include NFR/quality attributes".
Medium Rosetta Problem:
The deleted <best_practices> block removed the hook "Challenge first answer for blind spots". The new <pitfalls> cover evidence separation and conflicting signals, but nothing tells the agent to challenge its own first answer.
Reason:
Settling on the first plausible answer is the dominant failure mode this skill exists to counter, so losing that line removes its main safeguard.
Solution:
Add "challenge the first answer for blind spots" as one line in <boundaries> or <pitfalls>.
Medium Output Contract Problem:
Steps 4 and 5 lost three conditional output items: "Define acceptance criteria with EARS when relevant", "Generate scenarios, testing approach, and test data when relevant", and feasibility checks. The new steps keep only success criteria.
Reason:
These were the parts of the reasoning output that downstream planning and testing consumed; without them the result narrows to prose plus a confidence number.
Solution:
Re-add the acceptance-criteria, scenarios/test-data, and feasibility items to steps 4 and 5 as short conditional clauses.

📄 instructions/r2/core/skills/requirements-authoring/SKILL.md

⚠️ Issues Found

Severity Gate Details
High Safety Boundaries Problem:
The deleted <authoring_flow> removed the local anti-assumption gate: "Explicit approval, do not assume approval, user questions/comments do not mean it was approved". New workflow steps 8 and 9 only say "seek explicit unit-level approval (HITL required)" and "get explicit user sign-off".
Reason:
The always-on bootstrap-hitl-questioning.md still forbids assuming approval, so this is a loss of defense-in-depth rather than an outright break. But this skill writes requirement files to disk on approval, and that is where the reinforcement mattered most.
Solution:
Add one line to the HITL gates in <core_concepts>: user questions, comments, or edits are review, not approval.
High Precision & Explicitness Problem:
The <id_rules> block was collapsed and lost "Never renumber existing IDs" — the new <requirement_unit> keeps only "Never reuse retired IDs". The <dry_rules> block was deleted entirely, so "Reference IDs, not copies" and "Centralize shared definitions / shared constraints" are gone; <core_principles> now names DRY as an abbreviation without any of its content.
Reason:
Renumbering IDs and copy-pasting requirement text are exactly the edits that break traceability across INDEX.md and tests, and neither is forbidden anywhere in the new file.
Solution:
Add "never renumber existing IDs" to the ID format line, and restore "reference IDs, not copies; centralize shared definitions and constraints" in <core_principles>.
Medium Output Contract Problem:
The base Default output sections list named "Open Questions" as a required output section. The new Output artifacts list has only intent capture, requirement units, validation pack, and traceability matrix; questions survive only as a sub-field inside the intent-capture bullet.
Reason:
Open questions are how unresolved ambiguity reaches the user. Demoted to a sub-field, they get folded into intent capture and the ambiguity is quietly resolved by the agent instead.
Solution:
Add open questions back to the output artifacts list as its own item.
Medium Decision Branching Problem:
The deleted <core_principles_to_enforce> removed "Defer by keeping Draft status", which was the only stated handling for a requirement the user has not approved yet. The new file defines Draft as a status value but never says to use it for deferral.
Reason:
With no defined deferral path the agent either drops unapproved requirements or writes them as though approved.
Solution:
Add "defer unapproved items by keeping status Draft" to the HITL gates or <core_principles>.
Medium Self-Validation Problem:
The <validation_rules> block was deleted and the surviving <validation_checklist> does not cover "Validate feasibility with constraints", "Validate consistency across files", or "Validate non-redundancy across files".
Reason:
Requirements span many files, so dropping the cross-file checks lets contradictions between files survive into implementation.
Solution:
Add one checklist line covering cross-file consistency, non-redundancy, and feasibility against stated constraints.
Medium Conflict Resolution Problem:
The nine-item <conflict_checks> block was reduced to one checklist line covering duplicate IDs, contradictory shall clauses, incompatible thresholds, and circular dependencies. Dropped: duplicate statements, mismatched terminology, ordering issues, conflicting actors and responsibilities, and ambiguity.
Reason:
Only the named conflicts get checked, so semantically duplicated requirements and inconsistent terminology now pass the quality gate unnoticed.
Solution:
Extend that checklist line to also name duplicate statements, mismatched terminology, and conflicting actors or responsibilities.

📄 instructions/r2/core/skills/requirements-use/SKILL.md

⚠️ Issues Found

Severity Gate Details
High Safety Boundaries Problem:
The deleted <ambiguity_and_conflict_rules> removed "Stop and escalate via HITL" and "Wait for explicit user decision". New workflow step 3 says only "escalate via HITL with options and tradeoffs" — the stop and wait verbs are gone.
Reason:
The always-on bootstrap-hitl-questioning.md still requires waiting at HITL gates, so this is a loss of local reinforcement rather than a full break. But escalate-and-continue is the exact failure this gate guards: the agent implements its own interpretation and the user's answer arrives after the code is written.
Solution:
Add "stop and wait for the explicit user decision" to step 3.
High Output Contract Problem:
The base Default output sections list required "Execution Plan" and "Open Questions". The new Output artifacts list has only scope capture, traceability matrix, validation pack, and change log — neither the plan nor the questions section survives anywhere in the file.
Reason:
The execution plan is what turns approved requirement IDs into sequenced work. Without it in the contract, the skill can return coverage tables with no plan for the requester to act on.
Solution:
Add execution plan and open questions back to the output artifacts list.
Medium Failure Handling Problem:
The deleted <best_practices> removed "Raise blockers immediately" and "Use small batches for approvals". The one surviving mention of batching is the new pitfall "Batching matrix updates instead of continuous tracking", which is about the coverage matrix, not approvals.
Reason:
Without a batching rule the agent presents all coverage at the end, which is both too large to review and too late to change direction.
Solution:
Add "raise blockers immediately; request approvals in small batches" to <core_principles>.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

instructions skills, subagents, rules, workflows, commands for AI agents invalid This doesn't seem right needs more work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants