Better Skill Creator

Better Skill Creator helps you build Claude Code skills — small, focused agents that Claude loads automatically when it recognises a matching request. It measures two things: whether Claude loads your skill at the right moment, and how well it performs once loaded.

Contents by question

Question	Where to read
What is this for, and how does it build a skill?	How it works · What makes it different
How do I install it and run my first check?	Quick start · Setup
Why can the same skill behave differently across Claude models?	Model behavior and evidence
How should I write for a specific model or tier?	Model guidance
Which Claude Code features does the evaluator rely on?	Verified Claude Code contracts
How do I design and validate a skill?	Design a skill · Validate it
How do I test missed triggers, false triggers, and near misses?	Trigger tests · Near misses
How are model runs, outputs, and credentials kept separate?	Run isolation
How do I test what the skill actually does?	Behavior tests · Paired baseline grading
What do blind judges see, and how do I read the cost report?	Blind judging and cost
How do I build and evaluate a skill with fewer model calls?	Low-cost authoring workflow
How do I compare results across versions or models and tune descriptions?	Comparison and optimization
How do baselines, blind comparison, and held-out cases work?	Evaluation evidence
What evidence do I need before sharing a skill?	Independent review · Packaging
What goes into the package, and which files are excluded?	Safe packaging
Where are the commands and troubleshooting steps?	Command index · Troubleshooting
What shipped, what's planned, and why?	Release notes · Roadmap · Decision record

Quick start

1. Install the Claude Code plugin.

   claude plugin marketplace add rolling-codes/better-skill-creator
   claude plugin install skill-creator@skill-creator-local
   

2. Open Claude Code and describe what you want your skill to do. Claude will walk you through naming it, defining when it should activate, and writing the initial draft. The remaining commands use a local checkout of this repository — clone it and run from its root. You can also install the optional pre-commit hook:

   python bsc.py hooks install
   

3. Tell the validator which prompts should activate your skill (positive cases) and which shouldn't (negative cases). See examples/release-notes/tests/should_trigger.yaml for the format, then validate your skill folder:

   python bsc.py check skills/my-skill
   

4. Preview the trigger evaluation and its model-call estimate.

   python bsc.py eval skills/my-skill
   

5. After checking your allowance and overage settings, run the live evaluation.

   python bsc.py eval skills/my-skill --live
   

Requirements

• Claude Code, installed and authenticated
• Python 3.12+ and PyYAML (pip install pyyaml) for the repository commands

How it works

1. Define the skill's outcome, boundaries, and when Claude should load it.
2. Run the same task with the skill and with a comparable baseline. Grade the saved outputs or grade behavior from a saved transcript.
3. Test positive and negative trigger cases separately. The description optimizer selects a candidate on validation cases and measures it on held-out cases.
4. For substantial changes, record independent review before packaging. Run python bsc.py package skills/my-skill to build a distributable skill.

What makes it different

• Design analysis separates what the task requires from what the skill is authorized to do.
• Independent review checks outcomes, scope, and structure. A completion reviewer challenges the evidence before the packaging gate accepts it.
• Model-aware comparisons keep trigger results separate by model and surface failed evaluations separately from missed triggers.

Roadmap

• Model-assisted near-miss proposals with semantic review. Today, the generator proposes cases from neighboring skills and reviewers approve the labels.
• Repeated trigger and behavior benchmarks across supported model tiers. Today's scheduled canary covers a small trigger suite on Haiku and Sonnet.

See the roadmap and documentation notes.

This is the beginning of the README. Read the rest at the original. Read the full README ↗