how it works
What the score means, and how the evals actually run.
Most skill lists rank by GitHub stars. Stars measure popularity, not whether a skill makes your agent better. SkillRank ranks by measured impact — and where a skill has been benchmarked, the number is a real, reproducible eval, not a vibe.
the SkillRank score (0–100)
Blends three signals: community (GitHub stars, log-scaled), usage (real installs), and our eval lift (measured token/success impact).
Until a skill is evaluated its score is provisional and capped at 75 — popularity alone can never reach the top tier. That cap is the whole point: a skill has to be measured to break out.
evaluated skills
Once a skill is benchmarked it leaves provisional and its score reflects the measurement. We lead with token efficiency: a skill that measurably saves tokens rises; one that costs tokens for no success gain drops — even if it has hundreds of thousands of stars. Success lift, when a task is hard enough to show it, adds on top.
how an eval runs
- Paired, forced-mode trials. The exact same task is run two ways: control (the agent with no skill) and treatment (the agent with the skill loaded). The only variable is the skill.
- A reproducible suite. Each suite is a fixture repo pinned to a commit + a task instruction + a verifier. Everything is content-addressed, so a run is repeatable.
- Hidden verification. The agent sees the task spec, not the tests. The pass/fail verifier is injected only afterthe agent process exits — so a skill can't be gamed by peeking at the checks, and correctness comes from reasoning.
- A pinned reference model. Control and treatment use the same agent + model (e.g. Claude Haiku 4.5), recorded in the result. Results are only comparable within the same model / agent / suite — never mixed across them.
- N trials per arm. We run several trials each side and report the deltas: token delta (negative = the skill saves tokens) and success delta (percentage points). Low-N results are flagged as directional.
trust tiers
Results carry a tier and are never blended across tiers:
self-reported — run locally in a worktree (weakest isolation).
community-reported — run in container isolation, plausibility-checked against the official baseline.
official— audited on skillrank's reference harness.
what we won't do
We don't fabricate numbers. A skill with no eval shows "pending", not a made-up score. We publish the losers too — if a popular skill costs tokens for no gain, that's exactly the finding worth surfacing. Every eval is reproducible; anyone can re-run it.
reproduce it yourself
The same eval the registry runs is one CLI command against a public suite:
$ skillrank eval <skill>--suite <id> --model claude-haiku-4-5-20251001