Skip to content

perf(icp-cli): reduce skill body by 33 lines for token efficiency#199

Merged
raymondk merged 1 commit into
mainfrom
perf/icp-cli-token-efficiency
May 31, 2026
Merged

perf(icp-cli): reduce skill body by 33 lines for token efficiency#199
raymondk merged 1 commit into
mainfrom
perf/icp-cli-token-efficiency

Conversation

@marc0olo
Copy link
Copy Markdown
Member

Summary

  • Remove alternative installation methods (Homebrew / shell script installer) from the Installation section — agents should recommend npm install -g, and the alternatives added noise without value
  • Inline pitfall 4's verbose wrong-example YAML block (cargo build commands) into a single sentence — the point is "use recipes", not "here's the cargo invocation to avoid"

Result: 405 → 372 lines (−8%), skill body 4,037 tokens (down from ~4,500).

Context

Found during a skill-creator testrun done as part of reviewing #198. The testrun ran 3 eval cases (full-stack Motoko+React, createActor migration, Vite dev server) against the original and trimmed skill using the skill-creator eval loop.

Eval results

Iteration-2 benchmark (trimmed vs original)
Metric Original (405 lines) Trimmed (372 lines) Delta
Pass rate 100% (18/18) 100% (18/18) 0
Avg time 42.6s 43.1s +0.5s
Avg tokens/run 25,015 24,803 −212

All 18 assertions pass on the trimmed skill. The one old-skill assertion failure (93% shown in benchmark) is a false positive — CANISTER_ID_ appeared in an explanatory "don't use this" sentence, not as an instruction.

Remove alternative installation methods (brew/shell script) and
inline pitfall 4's verbose wrong-example block. Quality verified via
skill-creator eval loop: 18/18 assertions pass, avg -212 tokens/run.
@marc0olo marc0olo requested review from a team and JoshDFN as code owners May 29, 2026 20:03
@github-actions
Copy link
Copy Markdown

Skill Validation Report

Validating skill: /home/runner/work/icskills/icskills/skills/icp-cli

Structure

  • Pass: SKILL.md found
  • Pass: all files in references/ are referenced

Frontmatter

  • Pass: name: "icp-cli" (valid)
  • Pass: description: (507 chars)
  • Pass: license: "Apache-2.0"
  • Pass: metadata: (2 entries)

Markdown

  • Pass: no unclosed code fences found

Tokens

File Tokens
SKILL.md body 4,948
references/binding-generation.md 1,031
references/dev-server.md 690
references/dfx-migration.md 2,620
Total 9,289

Content Analysis

Metric Value
Word count 2,602
Code block ratio 0.19
Imperative ratio 0.12
Information density 0.16
Instruction specificity 0.83
Sections 17
List items 56
Code blocks 27

References Content Analysis

Metric Value
Word count 2,133
Code block ratio 0.25
Imperative ratio 0.13
Information density 0.19
Instruction specificity 0.80
Sections 18
List items 33
Code blocks 12

Contamination Analysis

Metric Value
Contamination level low
Contamination score 0.12
Primary language category shell
Scope breadth 3
  • Warning: Language mismatch: config, javascript (2 categories differ from primary)

References Contamination Analysis

Metric Value
Contamination level low
Contamination score 0.03
Primary language category javascript
Scope breadth 2
  • Warning: Language mismatch: shell (1 category differ from primary)

Result: passed

Project Checks


✓ Project checks passed for 1 skills (0 warnings)

@raymondk raymondk merged commit 5ccd559 into main May 31, 2026
6 checks passed
@raymondk raymondk deleted the perf/icp-cli-token-efficiency branch May 31, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants