git404hub

what is skills-evals fr?

ahnafyy/skills-evals — explained in plain English

Analysis updated 2026-05-18

0JavaScriptAudience · developerComplexity · 2/5Setup · easy

tl;dr

A testing tool that checks whether your AI agent skills, instructions, and rules still trigger and work correctly, catching regressions in CI.

vibe map

mindmap
  root((skills-evals))
    What it does
      Tests agent skills
      Catches trigger regressions
      Validates instructions
    Tech stack
      JavaScript
      Node.js
      Zero dependencies
    Use cases
      Discover skills automatically
      Catch broken triggers
      Test prompt routing
      Run scheduled behavioral checks
    Audience
      Developers
      AI tooling teams
    Setup
      Run via npx
      Install setup skill globally
      Wire into CI

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

what do people make with this?

VIBE 1

Automatically discover every AI agent skill, instruction file, and rule in your repository.

VIBE 2

Catch when a code change breaks a previously working skill or agent trigger.

VIBE 3

Test whether realistic user prompts route to the correct skill instead of a competing one.

VIBE 4

Run scheduled checks where a real AI agent follows your instructions to confirm they still work.

what's the stack?

JavaScriptNode.js

how it stacks up fr

ahnafyy/skills-evals00kaku/gallery-slider-block3rd-eden/ircb.io
Stars0
LanguageJavaScriptJavaScriptJavaScript
Last pushed2021-05-192016-11-16
MaintenanceDormantDormant
Setup difficultyeasyeasyeasy
Complexity2/52/52/5
Audiencedevelopergeneraldeveloper

Figures from each repo's GitHub metadata at analysis time.

how do i run it?

Difficulty · easy time til it works · 30min

Requires Node.js 18.17 or newer, behavioral tests need access to an AI model.

in plain english

skills-evals is a command line tool that checks whether the instructions you give an AI coding agent are still working. If you write skills, custom agents, rules, or prompt files for tools like Claude Code, GitHub Copilot, or Cursor, this tool tests whether those artifacts still trigger correctly and behave as intended, especially after your codebase or the instructions themselves change. It scans your repository for these files automatically, covering formats like Claude and Copilot skill files, Copilot instructions and custom agents, Claude custom agents, Cursor rules, and prompt files. It has no external dependencies and needs Node version 18.17 or newer to run. The checks run in three tiers. The first tier is structural, checking things like file formatting and description limits, and runs for free in continuous integration. The second tier tests whether realistic prompts actually route to the right skill or agent, also for free, by ranking how well descriptions match sample prompts. The third tier is behavioral: it has a real AI agent follow the instructions and checks, using a grading model, whether the agent actually did the right thing. This tier is meant to run on a schedule, like nightly or weekly, using a cheap and fast model. A key feature is regression detection. You save a baseline snapshot of test results, and on every later run the tool compares against it. If a previously working trigger suddenly fails on a file that was not changed, that signals another skill is now stealing its triggers, and the tool flags it as an error so your team catches the problem in a pull request rather than in production. The project also ships an installable setup skill that can walk another AI agent through configuring skills-evals in any repository automatically, including writing test cases and wiring up continuous integration. The project describes itself as compatible with the eval case format used by Anthropic's own skill creation tooling.

prompts (copy fr)

prompt 1
Help me set up skills-evals in this repository to test my Claude Code skills.
prompt 2
Explain the difference between the three testing tiers in skills-evals.
prompt 3
Show me how to write an eval case file for one of my custom agents.
prompt 4
Help me configure a nightly CI job to run behavioral evals with a cheap model.
prompt 5
Walk me through creating a regression baseline for my skill files.

Frequently asked questions

what is skills-evals fr?

A testing tool that checks whether your AI agent skills, instructions, and rules still trigger and work correctly, catching regressions in CI.

What language is skills-evals written in?

Mainly JavaScript. The stack also includes JavaScript, Node.js.

How hard is skills-evals to set up?

Setup difficulty is rated easy, with roughly 30min to a first successful run.

Who is skills-evals for?

Mainly developer.

peek the repo → explain another one

This repo across BitVibe Labs

double-check against the repo, no cap.