feat: improve skill scores for 5 core Rosetta skills - #50
Open
rohan-tessl wants to merge 1 commit into
Open
Conversation
Hey @isolomatov-gd 👋 I ran your skills through `tessl skill review` at work and found some targeted improvements. Here's the full before/after: | Skill | Before | After | Change | |-------|--------|-------|--------| | coding-agents-prompt-adaptation | 10% | 84% | +74% | | requirements-authoring | 10% | 87% | +77% | | requirements-use | 10% | 77% | +67% | | load-context | 25% | 68% | +43% | | reasoning | 36% | 65% | +29% | This PR is intentionally scoped to 5 skills to keep it reviewable — more can be improved in follow-ups or via automated review on future PRs. Note: The source skills live in instructions/r2/core/skills/ and are propagated to plugins/ via scripts/plugin_generator.py. All changes target the source files only — plugin copies will regenerate automatically on the next sync. Changes: - Fixed duplicate tags fields in YAML frontmatter (caused validation failures) - Ensured description uses quoted string format without XML-like tags - Removed generic role preamble sections - Added explicit numbered workflows to all 5 skills - Consolidated redundant rule sections (SRP/DRY/KISS into core_principles) - Merged overlapping validation/conflict/gap checks into unified checklists - Expanded load-context from 2 lines to full skill with workflow - Made reasoning confidence feedback loop actionable with weighted formula Signed-off-by: rohan-tessl <rohan-tessl@users.noreply.github.com>
rohan-tessl
marked this pull request as ready for review
April 8, 2026 08:42
rohan-tessl
requested review from
YevheniiaLementova,
k-bieniek and
scappuccino-grid
as code owners
April 8, 2026 08:43
Contributor
|
Thank you for the review, we have a skill we use internally to improve prompts which we did not apply on those files. Thank you for the PR, we will harden those prompts |
isolomatov-gd
marked this pull request as draft
August 18, 2026 19:23
isolomatov-gd
marked this pull request as ready for review
August 18, 2026 19:23
Contributor
Rosetta Triage ReviewSummary: Proposes rewording/restructuring 5 Findings:
Caveats:
Suggestions:
Questions:
Automated triage by Rosetta agent |
Contributor
📋 Prompt Quality Validation Report❌ Validation FailedThe full markdown report and raw JSON output are available in the workflow artifacts for 5 days. Files With Issues
📄
|
| Severity | Gate | Details |
|---|---|---|
| High | Precision & Explicitness | Problem: The deleted <best_practices> block removed the only two rules that protected section order and edit size: "Preserve original structure and section order" and "Keep diffs surgical and traceable". Nothing in the new file replaces them. The closest surviving rules are #5 "Maintain file names and sub-paths exactly as-is" (paths only), #11 "Keep everything else AS-IS including unknowns", and #12 "MUST NOT rewrite lines in your own way" (wording only) — none forbids moving or merging whole sections.Reason: An agent can reorder or merge sections and still pass every remaining rule, because reordering is not rewriting a line. Section order in these prompts is load-bearing, so this silently destroys the source's sequence. Solution: Add two lines to the boundaries list in <adapt_transformations>: preserve source structure and section order; keep diffs surgical and traceable. |
| Medium | Rosetta | Problem: The frontmatter description was extended with "Use when porting prompts between agents/IDEs, adapting KB prompts to local context, or migrating rules between formats" — the exact sentence that already opens <when_to_use_skill>. The same duplication pattern was introduced in the descriptions of reasoning, requirements-authoring, and requirements-use in this PR.Reason: Descriptions stay in context for every turn of every session across all agents, so duplicated text is paid for repeatedly, and long descriptions get cut off in agent menus where they drive skill selection. Solution: Trim each description to a short call-to-action and drop the duplicated "Use when..." clause, since <when_to_use_skill> already carries it. Apply the same trim to the other three skills changed in this PR. |
| Medium | Output Contract | Problem: The <templates> block was deleted. It held the only statement of the output shape: "Adapted prompt follows the same format as the source prompt. No additional templates — adaption preserves source structure." The new file only says "store in target IDE/Agent/OS format and location", which is about destination, not shape.Reason: With no output shape stated, the agent may re-template the prompt instead of producing a format-preserving copy, which is the opposite of what ADAPT is for. Solution: Keep one line stating that the adapted prompt keeps the source format and section order, and that no new templates are introduced. |
📄 instructions/r2/core/skills/load-context/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| Very High | Dependency Management | Problem:get_context_instructions is now hardcoded in six places — the frontmatter description, workflow steps 1 and 2, the fallback line, the validation checklist, and the pitfalls. The base file named it once, guarded by "(if available)".Reason: This one file ships unchanged to MCP, plugin, and local modes. Naming one transport's tool as the only path makes the skill wrong in the other two. Solution: Use the mode-neutral alias ( EXECUTE PREP STEPS / GET PREP STEPS) in the body, and keep the raw tool name only inside the MCP branch. |
| Very High | Failure Handling | Problem: The new fallback reads "If get_context_instructions is unavailable or fails, immediately ask the user for help — this is critical and unexpected." It merges two different situations. In plugin and local mode the tool being absent is the normal case, not an anomaly.Reason: As written, a plugin-mode session can open by interrupting the user with a false alarm instead of loading context. That blocks the work and trains users to dismiss genuine escalations. Solution: Split the two cases: tool absent means fall through to the active mode rule's prep steps and continue silently; tool present but failing means stop and ask the user. |
| Very High | Decision Branching | Problem: The base line was "load current context using get_context_instructions tool (if available)". New workflow step 1 drops "(if available)" and makes the call unconditional: "Call get_context_instructions — this is a blocking prerequisite; do not respond, call other tools, or process the user message until this completes". No branch covers plugin or local mode, where no such callable tool exists — plugin-files-mode.md line 58 only maps the name to "already loaded bootstrap-* instructions, continue with prep steps 2 and 3".Reason: plugin-files-mode.md line 85 invokes this skill at session start ("USE SKILL load-context, if available"), so in plugin mode the agent's first instruction is to call something that is not a tool. Behavior then varies run to run — some agents resolve the alias, others stall — and non-determinism at session start is a reliability defect.Solution: Restore the availability branch in step 1: if the tool exists, call it; otherwise execute the prep steps already loaded by the active mode rule, then continue. |
| High | Bloat Control | Problem: The body grew from two lines to about forty, and much of the new text repeats always-on bootstrap policy — "Skipping prep steps because the task seems simple", "Calling other tools before get_context_instructions", and "do not respond, call other tools, or process the user message until this completes" all restate the <must> block in bootstrap.md.Reason: This skill loads at the start of every session, so duplicated policy is paid for on every request while adding no instruction the agent does not already have in context. Solution: Keep the numbered workflow and the validation checklist; drop the pitfalls that restate always-on bootstrap policy. |
| High | Rosetta | Problem: The rewritten skill restates prep-step mechanics that the always-on bootstrap rules already own, and contradicts them. plugin-files-mode.md line 58 says the tool call is already satisfied and the agent should "continue with prep steps 2 and 3", while this skill now says the call itself is a blocking prerequisite. The checklist also hardcodes "All three prep steps".Reason: When a skill and an always-on rule disagree about the first action of a session, the agent picks one arbitrarily, so the session either stalls or skips context loading. Solution: Keep the skill mode-neutral: point at the active mode rule for prep-step mechanics, and say "all prep steps" instead of "all three prep steps". |
📄 instructions/r2/core/skills/reasoning/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Decision Branching | Problem: The new workflow intro says "For simple questions, skip to step 5 (DELIVER) with a direct answer", but steps 6 DESIGN and 7 DEBRIEF still follow step 5 in the same numbered list and nothing says to stop. The base kept this branch out of the numbering entirely: "For simple questions, skip deep decomposition and answer directly". Reason: A step-following agent lands on 5 and continues into 6 and 7, so the cheap path for a trivial question becomes the full 7D flow on some runs and not others. Solution: State the exit explicitly at the branch point, for example: simple question means DELIVER only, then stop. |
| High | Precision & Explicitness | Problem: Step 6 DESIGN lost its enumerated quality-attribute list. The base named "security, performance, reliability, maintainability, scalability, testability, observability, compliance, backward compatibility, and cost"; the new step says only "include NFR/quality attributes". Reason: Agents cover the dimensions that are named and skip the ones left implicit, so security, compliance, and backward-compatibility checks now drop out of every design pass. Solution: Keep the enumerated attribute list inside step 6, as a compact inline series after "include NFR/quality attributes". |
| Medium | Rosetta | Problem: The deleted <best_practices> block removed the hook "Challenge first answer for blind spots". The new <pitfalls> cover evidence separation and conflicting signals, but nothing tells the agent to challenge its own first answer.Reason: Settling on the first plausible answer is the dominant failure mode this skill exists to counter, so losing that line removes its main safeguard. Solution: Add "challenge the first answer for blind spots" as one line in <boundaries> or <pitfalls>. |
| Medium | Output Contract | Problem: Steps 4 and 5 lost three conditional output items: "Define acceptance criteria with EARS when relevant", "Generate scenarios, testing approach, and test data when relevant", and feasibility checks. The new steps keep only success criteria. Reason: These were the parts of the reasoning output that downstream planning and testing consumed; without them the result narrows to prose plus a confidence number. Solution: Re-add the acceptance-criteria, scenarios/test-data, and feasibility items to steps 4 and 5 as short conditional clauses. |
📄 instructions/r2/core/skills/requirements-authoring/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Safety Boundaries | Problem: The deleted <authoring_flow> removed the local anti-assumption gate: "Explicit approval, do not assume approval, user questions/comments do not mean it was approved". New workflow steps 8 and 9 only say "seek explicit unit-level approval (HITL required)" and "get explicit user sign-off".Reason: The always-on bootstrap-hitl-questioning.md still forbids assuming approval, so this is a loss of defense-in-depth rather than an outright break. But this skill writes requirement files to disk on approval, and that is where the reinforcement mattered most.Solution: Add one line to the HITL gates in <core_concepts>: user questions, comments, or edits are review, not approval. |
| High | Precision & Explicitness | Problem: The <id_rules> block was collapsed and lost "Never renumber existing IDs" — the new <requirement_unit> keeps only "Never reuse retired IDs". The <dry_rules> block was deleted entirely, so "Reference IDs, not copies" and "Centralize shared definitions / shared constraints" are gone; <core_principles> now names DRY as an abbreviation without any of its content.Reason: Renumbering IDs and copy-pasting requirement text are exactly the edits that break traceability across INDEX.md and tests, and neither is forbidden anywhere in the new file. Solution: Add "never renumber existing IDs" to the ID format line, and restore "reference IDs, not copies; centralize shared definitions and constraints" in <core_principles>. |
| Medium | Output Contract | Problem: The base Default output sections list named "Open Questions" as a required output section. The new Output artifacts list has only intent capture, requirement units, validation pack, and traceability matrix; questions survive only as a sub-field inside the intent-capture bullet.Reason: Open questions are how unresolved ambiguity reaches the user. Demoted to a sub-field, they get folded into intent capture and the ambiguity is quietly resolved by the agent instead. Solution: Add open questions back to the output artifacts list as its own item. |
| Medium | Decision Branching | Problem: The deleted <core_principles_to_enforce> removed "Defer by keeping Draft status", which was the only stated handling for a requirement the user has not approved yet. The new file defines Draft as a status value but never says to use it for deferral.Reason: With no defined deferral path the agent either drops unapproved requirements or writes them as though approved. Solution: Add "defer unapproved items by keeping status Draft" to the HITL gates or <core_principles>. |
| Medium | Self-Validation | Problem: The <validation_rules> block was deleted and the surviving <validation_checklist> does not cover "Validate feasibility with constraints", "Validate consistency across files", or "Validate non-redundancy across files".Reason: Requirements span many files, so dropping the cross-file checks lets contradictions between files survive into implementation. Solution: Add one checklist line covering cross-file consistency, non-redundancy, and feasibility against stated constraints. |
| Medium | Conflict Resolution | Problem: The nine-item <conflict_checks> block was reduced to one checklist line covering duplicate IDs, contradictory shall clauses, incompatible thresholds, and circular dependencies. Dropped: duplicate statements, mismatched terminology, ordering issues, conflicting actors and responsibilities, and ambiguity.Reason: Only the named conflicts get checked, so semantically duplicated requirements and inconsistent terminology now pass the quality gate unnoticed. Solution: Extend that checklist line to also name duplicate statements, mismatched terminology, and conflicting actors or responsibilities. |
📄 instructions/r2/core/skills/requirements-use/SKILL.md
⚠️ Issues Found
| Severity | Gate | Details |
|---|---|---|
| High | Safety Boundaries | Problem: The deleted <ambiguity_and_conflict_rules> removed "Stop and escalate via HITL" and "Wait for explicit user decision". New workflow step 3 says only "escalate via HITL with options and tradeoffs" — the stop and wait verbs are gone.Reason: The always-on bootstrap-hitl-questioning.md still requires waiting at HITL gates, so this is a loss of local reinforcement rather than a full break. But escalate-and-continue is the exact failure this gate guards: the agent implements its own interpretation and the user's answer arrives after the code is written.Solution: Add "stop and wait for the explicit user decision" to step 3. |
| High | Output Contract | Problem: The base Default output sections list required "Execution Plan" and "Open Questions". The new Output artifacts list has only scope capture, traceability matrix, validation pack, and change log — neither the plan nor the questions section survives anywhere in the file.Reason: The execution plan is what turns approved requirement IDs into sequenced work. Without it in the contract, the skill can return coverage tables with no plan for the requester to act on. Solution: Add execution plan and open questions back to the output artifacts list. |
| Medium | Failure Handling | Problem: The deleted <best_practices> removed "Raise blockers immediately" and "Use small batches for approvals". The one surviving mention of batching is the new pitfall "Batching matrix updates instead of continuous tracking", which is about the coverage matrix, not approvals.Reason: Without a batching rule the agent presents all coverage at the end, which is both too large to review and too late to change direction. Solution: Add "raise blockers immediately; request approvals in small batches" to <core_principles>. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey @isolomatov-gd 👋
I ran your skills through
tessl skill reviewat work and found some targeted improvements.Here's the full before/after:
This PR is intentionally scoped to 5 skills to keep it reviewable — more can be improved in follow-ups or via automated review on future PRs.
Note: The source skills live in
instructions/r2/core/skills/and are propagated toplugins/viascripts/plugin_generator.py. All changes target the source files only — plugin copies will regenerate automatically on the next sync.Changes summary
All 5 skills:
tagsfields in YAML frontmatter (caused validation failures and 10% scores for 3 skills)descriptionuses quoted string format without XML-like tags<role>preamble sections (the agent doesn't need motivational framing)coding-agents-prompt-adaptation:
requirements-authoring:
<authoring_flow>and<initialization>sections<core_principles>block<conflict_checks>,<gap_checks>, and<validation_rules>into a unified<validation_checklist><resources>and<templates>(they listed the same assets)<language_constructs>into a dense paragraph preserving all rulesrequirements-use:
<process>steps with HITL gates<requirement_usage_rules>and<ambiguity_and_conflict_rules>into the workflow and core_concepts<resources>and<templates>sectionsload-context:
reasoning:
Honest disclosure — I work at @tesslio where we build tooling around skills like these. Not a pitch - just saw room for improvement and wanted to contribute.
Want to self-improve your skills? Just point your agent (Claude Code, Codex, etc.) at this Tessl guide and ask it to optimize your skill. Ping me - @rohan-tessl - if you hit any snags.
Thanks in advance 🙏