Skip to content

feat(framework)!: remove evals system end-to-end#261

Merged
blafourcade merged 1 commit into
mainfrom
worktree-remove-evals
Jun 11, 2026
Merged

feat(framework)!: remove evals system end-to-end#261
blafourcade merged 1 commit into
mainfrom
worktree-remove-evals

Conversation

@blafourcade

Copy link
Copy Markdown
Contributor

What

Removes the evals concept across the whole framework end-to-end.

Why

The evals fixtures were not isolated test data: every evals/scenarios.json doubled as the BM25 training corpus for the auto-routing hook. Removing evals therefore means removing the routing engine they fed, the authoring path that produced them, and the gates/docs that referenced them.

Scope

  • Fixtures — delete all 31 plugins/*/skills/*/evals/scenarios.json.
  • Routing engine — delete aidd-context/hooks/routing/ (bin, lib, routing-hint, tests); unwire its SessionStart + UserPromptSubmit hooks in hooks.json (update_memory.js kept).
  • Authoring — drop 02-design-evals action + evals-template asset from 03-context-generate; renumber the skills action chain to 01-05 and the tool-resolution gate to R10.
  • Gates/docs — remove evals-validity from lefthook.yml; scrub evals references from docs, memory, and per-plugin CATALOG.md.

Out of scope / preserved

  • CHANGELOG.md files untouched (release-please owns them).
  • False positives kept: evaluate/evaluation prose and the eval() injection warning in the security rule.

⚠️ Breaking change

Auto-routing is removed. Skills become manual-invoke only; the prompt-to-skill routing hint no longer runs.

Verification

  • 0 scenarios.json / evals/ left on disk; dangling-ref sweep clean.
  • hooks.json valid JSON; lefthook.yml valid YAML.
  • 05-review ran: 2 renumber leftovers found (stale 06-validate include, R11 gate gap) and fixed.
  • 77 files changed, 58 insertions, 2280 deletions.

🤖 Generated with Claude Code

Remove the evals concept across the whole framework: trigger-test
fixtures, the BM25 routing engine they fed, the skill-authoring path
that produced them, and every gate and doc that referenced them.

- delete 31 plugins/*/skills/*/evals/scenarios.json fixtures
- delete aidd-context hooks/routing engine (bin, lib, routing-hint,
  tests) and unwire its SessionStart + UserPromptSubmit hooks; keep
  update_memory.js
- drop the design-evals action and evals-template asset from
  03-context-generate; renumber the skills action chain to 01-05 and
  the R-rules gate to R10
- remove the evals-validity check from lefthook.yml
- scrub evals references from docs, memory, and per-plugin CATALOGs

CHANGELOGs left untouched (release-please owns them).

BREAKING CHANGE: auto-routing is removed. Skills are now manual-invoke
only; the prompt-to-skill routing hint no longer runs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Baptiste LAFOURCADE <baptiste.lafourcade@gmail.com>
@blafourcade blafourcade marked this pull request as ready for review June 11, 2026 06:19
@blafourcade blafourcade requested a review from a team as a code owner June 11, 2026 06:19
@blafourcade blafourcade merged commit 9a3c1b8 into main Jun 11, 2026
10 checks passed
@aidd-bot aidd-bot Bot mentioned this pull request Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant