ai-driven-dev · blafourcade · Jun 11, 2026 · Jun 11, 2026
diff --git a/GOVERNANCE.md b/GOVERNANCE.md
@@ -64,7 +64,7 @@ none object.
 ## Plugins, breaking changes, conflicts
 
 - **New plugin**: lands via PR following [`docs/CREATE_PLUGIN.md`](docs/CREATE_PLUGIN.md)
-  (description on every skill, ≥1 `evals/scenarios.json`, registered in
+  (description on every skill, registered in
   `marketplace.json` + `release-please-config.json`, a Habilité owner). Starts
   `experimental` → `release candidate` (one external success) → `stable` (Habilité
   review).

diff --git a/UPGRADE.md b/UPGRADE.md
@@ -20,7 +20,7 @@ This guide tells you exactly what disappears, what each old command becomes, and
 
 1. **Delivery: marketplace, not clone.** In `v3` an external CLI copied the whole repo into each project and generated per-tool copies (Claude Code, Cursor, Copilot). In `v4` you point Claude Code at the marketplace and install plugins on demand.
 2. **Split into 6 plugins.** Each plugin owns one slice of the SDLC and ships its own version.
-3. **Commands became skills.** Every former `/command` is a skill with structured frontmatter, references, assets and evals. A skill can auto-trigger from your intent or be invoked by name.
+3. **Commands became skills.** Every former `/command` is a skill with structured frontmatter, references, and assets. A skill can auto-trigger from your intent or be invoked by name.
 4. **Many commands were merged into routers.** Related v3 commands (the three `assert_*`, the two `review_*`, `performance` + `security_refactor`, the three debug-family commands, the five `generate_*`) collapsed into a single skill that routes to the right sub-action. See the mapping in section 4.
 5. **Agents, hooks, templates moved into their owning plugin** (for example `plugins/aidd-dev/agents/`).
 
@@ -42,7 +42,7 @@ Each plugin ships:
 - `.claude-plugin/plugin.json` (manifest + version)
 - `skills/NN-action-name/SKILL.md` (one skill per former command or command family)
 - `CATALOG.md` (auto-generated index of the plugin's skills)
-- its own assets, agents, references and evals
+- its own assets, agents, and references
 
 ---
 

diff --git a/aidd_docs/CONTRIBUTING.md b/aidd_docs/CONTRIBUTING.md
@@ -8,7 +8,7 @@ Use the generator skills to scaffold new content that follows the framework stru
 
 | Skill                              | Creates              |
 | ---------------------------------- | -------------------- |
-| `aidd-context:03-context-generate` | New skill, agent, or rule (router-based, with actions and evals) |
+| `aidd-context:03-context-generate` | New skill, agent, or rule (router-based, with actions) |
 | `aidd-context:05-learn`            | New memory or rule capturing a learning                          |
 
 Generator skills consume the templates inside their `assets/` folder and write the output to the correct location for your AI tool (Claude Code, Cursor, Copilot, Codex, OpenCode).
@@ -19,7 +19,7 @@ All templates live alongside the skill that owns them, under `plugins/<plugin>/s
 
 | Where                                              | What it scaffolds                                        |
 | -------------------------------------------------- | -------------------------------------------------------- |
-| `aidd-context:03-context-generate/assets/skills/`  | `SKILL.md`, action, evals templates                      |
+| `aidd-context:03-context-generate/assets/skills/`  | `SKILL.md`, action templates                             |
 | `aidd-context:03-context-generate/assets/agents/`  | Agent file template                                      |
 | `aidd-context:03-context-generate/assets/rules/`   | Rule file template                                       |
 | `aidd-pm:03-prd/assets/`                           | PRD body template                                        |
@@ -38,7 +38,6 @@ When tools differ in syntax (frontmatter, slash command name, references), follo
 
 - Open a pull request for any new skill, agent, rule, or template. Visible changes that affect how the AI behaves on the project deserve team review.
 - Keep skills router-pure: SKILL.md holds no business logic; everything lives inside actions.
-- Add evals (`evals/scenarios.json`) for every auto-trigger skill so router behavior stays correct over time.
 - Stay within 5 to 10 percent deviation from a template structure. Beyond that, update the template first, then derive the new content from it.
 
 ## Conventions

diff --git a/aidd_docs/GUIDELINES.md b/aidd_docs/GUIDELINES.md
@@ -124,5 +124,4 @@ For each increment:
 - Anthropic, "Prompt engineering overview": <https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview>
 - Anthropic, "Claude Code memory": <https://docs.claude.com/en/docs/claude-code/memory>
 - OpenAI, "Prompt engineering best practices": <https://help.openai.com/en/articles/6654000-best-practices-for-prompting>
-- OpenAI Cookbook, "Eval-driven development": <https://cookbook.openai.com/examples/evaluation/use-cases/evalsapi_tools_evaluation>
 - GitHub Docs, "Repository custom instructions for Copilot": <https://docs.github.com/en/copilot/how-tos/configure-custom-instructions/adding-repository-custom-instructions-for-github-copilot>
diff --git a/aidd_docs/README.md b/aidd_docs/README.md
@@ -138,7 +138,6 @@ Inside the synchronous path, `aidd-dev:00-sdlc` is the Dev SDLC orchestrator tha
 - Skills must have an `## Available actions` table, `## Default flow`, `## Transversal rules`.
 - Actions must contain only `## Inputs`, `## Outputs`, `## Process`, `## Test`.
 - Tests must be observable: command, artifact check, or side effect.
-- Evals (`evals/scenarios.json`) ship for every auto-trigger skill.
 
 ---
 

diff --git a/aidd_docs/memory/architecture.md b/aidd_docs/memory/architecture.md
@@ -44,7 +44,6 @@ flowchart TD
     Skills --> SkillMd["SKILL.md"]
     Skills --> Actions["actions/NN-action.md"]
     Skills --> Assets["assets/"]
-    Skills --> Evals["evals/scenarios.json"]
 ```
 
 ## Services Communication

diff --git a/aidd_docs/memory/project-brief.md b/aidd_docs/memory/project-brief.md
@@ -28,7 +28,6 @@ AI-assisted software development tooling. The framework ships workflows, not cod
 | Memory | Structured context file loaded at conversation start | context file |
 | SDLC | Software Development Life Cycle — the end-to-end pipeline from idea to deployed PR | |
 | Marketplace | Central registry listing available plugins with version metadata | |
-| Eval | Scenario JSON file verifying a skill's routing logic | test |
 
 ## Features & Use-cases
 

diff --git a/aidd_docs/memory/testing.md b/aidd_docs/memory/testing.md
@@ -2,20 +2,19 @@
 
 ## Tools and Frameworks
 
-- **Playwright MCP**: browser automation available via `.playwright-mcp/` config — used for manual or AI-driven UI testing on downstream projects
-- **Evals** (`evals/scenarios.json`): per-skill JSON scenarios that verify routing logic (which action a skill dispatches to for a given input)
+- **Playwright MCP**: browser automation available via `.playwright-mcp/` config, used for manual or AI-driven UI testing on downstream projects
 
 ## Testing Strategy
 
 - No unit test runner configured at framework level
-- Skills are tested via eval scenarios; router behavior must be covered by `evals/scenarios.json` for every auto-trigger skill
+- Skills are validated by running each action's `## Test` end-to-end against a real environment
 - Framework correctness validated by running actual skills against a real project (integration)
 
 ## Test Execution Process
 
-- Evals: evaluated by the AI tool's eval runner or manually reviewed
+- Each action declares a `## Test` (a command to run, an artifact check, or an observable side-effect) that must pass before the next action runs
 - `scripts/build-dist-verification.md` documents how to verify the build output
 
 ## Mocking and Stubbing
 
-Not applicable — the framework has no runtime; all logic is markdown interpreted by an LLM.
+Not applicable: the framework has no runtime; all logic is markdown interpreted by an LLM.
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md
@@ -66,8 +66,7 @@ plugins/<plugin>/
 │       ├── README.md       # human-facing skill landing
 │       ├── actions/        # atomic actions invoked by the router
 │       ├── assets/         # templates and static files
-│       ├── references/     # extended docs the skill links into
-│       └── evals/          # scenario fixtures
+│       └── references/     # extended docs the skill links into
 ├── agents/                 # named AI agents          (optional)
 ├── commands/               # slash commands           (optional)
 ├── hooks/hooks.json        # lifecycle hooks          (optional)

diff --git a/docs/CREATE_PLUGIN.md b/docs/CREATE_PLUGIN.md
@@ -156,7 +156,6 @@ When you change SKILL.md or actions, run `/reload-plugins` in the same Claude Co
 ## Step 5 - document, test, ship
 
 - Run `pnpm exec lefthook run pre-commit` to confirm JSON validity, YAML validity, and SKILL.md frontmatter checks pass. The pre-commit hook also regenerates your `plugins/aidd-example/CATALOG.md` automatically.
-- Add at least one scenario under `skills/01-hello/evals/scenarios.json` if you expect future evaluation runs.
 - Open a PR using the project template and pick the right commit scope (`feat(aidd-example): ...`). Listing the scope in `commitlint.config.cjs` is encouraged (an unlisted scope warns but does not block).
 
 ## Step 6 - release

diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md
@@ -16,7 +16,7 @@ Per-project context files under `aidd_docs/memory/` (architecture, conventions,
 
 ## Skill
 
-A self-contained workflow under `plugins/<plugin>/skills/<NN-name>/`. Triggered by a user phrase, a slash command, or an explicit `Use skill <id>` invocation. A skill owns a `SKILL.md` router, one or more atomic actions, optional `assets/` and `evals/`. The `SKILL.md` `name:` is the folder slug (`00-onboard`); the invocation id is `<plugin>:<folder>`, for example `aidd-context:00-onboard`.
+A self-contained workflow under `plugins/<plugin>/skills/<NN-name>/`. Triggered by a user phrase, a slash command, or an explicit `Use skill <id>` invocation. A skill owns a `SKILL.md` router, one or more atomic actions, and optional `assets/` and `references/`. The `SKILL.md` `name:` is the folder slug (`00-onboard`); the invocation id is `<plugin>:<folder>`, for example `aidd-context:00-onboard`.
 
 ## Router-based skill
 

diff --git a/lefthook.yml b/lefthook.yml
@@ -62,26 +62,6 @@ pre-commit:
             exit 1
           fi
         done
-    evals-validity:
-      glob: "plugins/*/skills/*/evals/scenarios.json"
-      run: |
-        for f in {files}; do
-          [ -f "$f" ] || continue
-          if ! jq empty "$f" >/dev/null 2>&1; then
-            echo "❌ invalid JSON: $f"
-            exit 1
-          fi
-          type=$(jq -r 'type' "$f")
-          if [ "$type" != "array" ]; then
-            echo "❌ $f: root must be an array, got $type"
-            exit 1
-          fi
-          invalid=$(jq '[.[] | select((.prompt == null) or ((has("expect_action") | not) and (has("expect_skip") | not) and (has("expect") | not)))] | length' "$f")
-          if [ "$invalid" -gt 0 ]; then
-            echo "❌ $f: $invalid scenario(s) missing required keys (prompt + at least one of expect_action/expect_skip/expect)"
-            exit 1
-          fi
-        done
     summarize-plugin-catalogs:
       run: |
         [ -d plugins ] || exit 0

diff --git a/plugins/aidd-context/CATALOG.md b/plugins/aidd-context/CATALOG.md
@@ -8,7 +8,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 
 - [`.claude-plugin`](#claude-plugin)
 - [`hooks`](#hooks)
-  - [`hooks/routing`](#hooksrouting)
 - [`skills`](#skills)
   - [`skills/00-onboard`](#skills00-onboard)
   - [`skills/01-bootstrap`](#skills01-bootstrap)
@@ -33,21 +32,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | [hooks.json](hooks/hooks.json) |
 | [update_memory.js](hooks/update_memory.js) |
 
-#### `hooks/routing`
-
-| Group | File |
-|-------|------|
-| `bin` | [build-index.js](hooks/routing/bin/build-index.js) |
-| `bin` | [build-on-session.js](hooks/routing/bin/build-on-session.js) |
-| `bin` | [eval.js](hooks/routing/bin/eval.js) |
-| `bin` | [measure-follow-rate.js](hooks/routing/bin/measure-follow-rate.js) |
-| `bin` | [test-prompts.js](hooks/routing/bin/test-prompts.js) |
-| `lib` | [bm25.js](hooks/routing/lib/bm25.js) |
-| `lib` | [paths.js](hooks/routing/lib/paths.js) |
-| `lib` | [tokenize.js](hooks/routing/lib/tokenize.js) |
-| `-` | [routing-hint.js](hooks/routing/routing-hint.js) |
-| `tests` | [held-out-prompts.json](hooks/routing/tests/held-out-prompts.json) |
-
 ### `skills`
 
 #### `skills/00-onboard`
@@ -58,7 +42,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | `actions` | [02-recommend-next.md](skills/00-onboard/actions/02-recommend-next.md) | - |
 | `actions` | [03-execute-or-handoff.md](skills/00-onboard/actions/03-execute-or-handoff.md) | - |
 | `assets` | [state-matrix.md](skills/00-onboard/assets/state-matrix.md) | - |
-| `evals` | [scenarios.json](skills/00-onboard/evals/scenarios.json) | - |
 | `-` | [README.md](skills/00-onboard/README.md) | - |
 | `-` | [SKILL.md](skills/00-onboard/SKILL.md) | `Detect the current project's state and open a hub of project actions - understand the project, set up or refresh the memory bank, or continue the AIDD development journey. Silently inspects the project, the AIDD setup, and which AIDD plugins are installed, then adapts the menu to that context. Use when the user says "where do I start", "onboard me", "onboard me to this project", "what should I run next", "what should I work on next", "what's the state of this project", "guide me through aidd", "guide me through aidd-context", or invokes `aidd-context:00-onboard`. Do NOT use to enumerate every installed surface from raw user intent (the discovery skill in this plugin handles that).` |
 
@@ -73,7 +56,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | `actions` | [05-write-install-md.md](skills/01-bootstrap/actions/05-write-install-md.md) | - |
 | `assets` | [checklist.md](skills/01-bootstrap/assets/checklist.md) | - |
 | `assets` | [install-template.md](skills/01-bootstrap/assets/install-template.md) | - |
-| `evals` | [scenarios.json](skills/01-bootstrap/evals/scenarios.json) | - |
 | `-` | [README.md](skills/01-bootstrap/README.md) | - |
 | `references` | [stack-heuristics.md](skills/01-bootstrap/references/stack-heuristics.md) | - |
 | `-` | [SKILL.md](skills/01-bootstrap/SKILL.md) | `Imagine and validate the technical architecture of a new SaaS through interactive Q&A, candidate-stack comparison, multi-agent audit, and an INSTALL.md output. Use when starting a new SaaS project, choosing a stack, designing the architecture pattern (monolith vs microservices vs serverless), or producing a project's INSTALL.md. Do NOT use for editing an existing project's stack, database schema design, or scaffolding actual files (this skill produces docs only, no code).` |
@@ -92,7 +74,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | `assets` | [golden-principles.md](skills/02-project-init/assets/golden-principles.md) | - |
 | `assets` | [GUIDELINES.md](skills/02-project-init/assets/GUIDELINES.md) | - |
 | `assets` | [README.md](skills/02-project-init/assets/README.md) | - |
-| `evals` | [scenarios.json](skills/02-project-init/evals/scenarios.json) | - |
 | `-` | [README.md](skills/02-project-init/README.md) | - |
 | `references` | [mapping-ai-context-file.md](skills/02-project-init/references/mapping-ai-context-file.md) | - |
 | `-` | [SKILL.md](skills/02-project-init/SKILL.md) | `Initialize or refresh the project memory bank and ensure AI context files contain the project memory block. Use when running `aidd init` for the first time, bootstrapping a new project, or re-running the init flow on an existing project. Do NOT use for updating individual memory files after they exist - use `aidd-context:05-learn` instead; do NOT use for editing a single rule - edit the file directly.` |
@@ -101,7 +82,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 
 | Group | File | Description |
 |-------|------|---|
-| `evals` | [scenarios.json](skills/03-context-generate/evals/scenarios.json) | - |
 | `-` | [README.md](skills/03-context-generate/README.md) | - |
 | `references` | [ai-mapping.md](skills/03-context-generate/references/ai-mapping.md) | - |
 | `references` | [command.md](skills/03-context-generate/references/command.md) | - |
@@ -117,7 +97,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | Group | File | Description |
 |-------|------|---|
 | `actions` | [01-mermaid.md](skills/04-mermaid/actions/01-mermaid.md) | - |
-| `evals` | [scenarios.json](skills/04-mermaid/evals/scenarios.json) | - |
 | `-` | [README.md](skills/04-mermaid/README.md) | - |
 | `references` | [mermaid-conventions.md](skills/04-mermaid/references/mermaid-conventions.md) | - |
 | `-` | [SKILL.md](skills/04-mermaid/SKILL.md) | `Generate high-quality Mermaid diagrams from markdown content using a structured plan-validate workflow.` |
@@ -131,7 +110,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | `actions` | [03-sync.md](skills/05-learn/actions/03-sync.md) | - | - |
 | `assets` | [adr-template.md](skills/05-learn/assets/adr-template.md) | `Architecture Decision Record template` | - |
 | `assets` | [decision-template.md](skills/05-learn/assets/decision-template.md) | `Individual decision record template` | `<title>` |
-| `evals` | [scenarios.json](skills/05-learn/evals/scenarios.json) | - | - |
 | `-` | [README.md](skills/05-learn/README.md) | - | - |
 | `-` | [SKILL.md](skills/05-learn/SKILL.md) | `Capture and store project-level learnings, conventions, and decisions surfaced during work into memory, decisions, or rules. Use proactively when the user states a durable project rule or convention ("for next", "always do X", "from now on", "going forward", "rule:", "convention:"), records a technical decision and its rationale, deprecates something, or notes an insight that should outlive the current task. Do NOT use for personal or AI-preference reminders (those belong to user memory), routine code edits, minor fixes, or anything already captured.` | - |
 
@@ -147,7 +125,6 @@ Auto-generated index of skills, agents, references and assets shipped by the `ai
 | `actions` | [06-find-rule.md](skills/06-discovery/actions/06-find-rule.md) | - |
 | `actions` | [07-find-hook.md](skills/06-discovery/actions/07-find-hook.md) | - |
 | `actions` | [08-find-memory.md](skills/06-discovery/actions/08-find-memory.md) | - |
-| `evals` | [scenarios.json](skills/06-discovery/evals/scenarios.json) | - |
 | `-` | [README.md](skills/06-discovery/README.md) | - |
 | `references` | [ai-mapping.md](skills/06-discovery/references/ai-mapping.md) | - |
 | `-` | [SKILL.md](skills/06-discovery/SKILL.md) | `Enumerate installed surfaces of the AI tool (skills, agents, commands, plugins, MCP servers, rules, hooks, memory files) and recommend the best match for the user's stated intent. Use proactively whenever the user asks the model to list, show, enumerate, find, or pick among any of these surfaces - including imperative phrasings ("list hooks", "show me the rules", "enumerate skills", "find a memory file", "which agent reviews code"), question phrasings ("what's available?", "what hooks do we have?", "which rule applies here?", "what memory files do we have?"), and indirect phrasings ("what can I use for X?", "do we have something that does Y?"). Always pick this skill over scanning the filesystem with grep, find, ls, or reading action files directly when the user is enumerating a surface. Do NOT use for picking a specific item inside one plugin (the plugin's own onboard handles that), creating a new surface, or executing a recommended item (this skill only points; the user invokes).` |

diff --git a/plugins/aidd-context/hooks/hooks.json b/plugins/aidd-context/hooks/hooks.json
@@ -6,22 +6,6 @@
           {
             "type": "command",
             "command": "node ${CLAUDE_PLUGIN_ROOT}/hooks/update_memory.js"
-          },
-          {
-            "type": "command",
-            "command": "node ${CLAUDE_PLUGIN_ROOT}/hooks/routing/bin/build-on-session.js",
-            "timeout": 30
-          }
-        ]
-      }
-    ],
-    "UserPromptSubmit": [
-      {
-        "hooks": [
-          {
-            "type": "command",
-            "command": "node ${CLAUDE_PLUGIN_ROOT}/hooks/routing/routing-hint.js",
-            "timeout": 5
           }
         ]
       }