refactor: remove redundant assessors, realign tiers, rebalance weights#464
refactor: remove redundant assessors, realign tiers, rebalance weights#464jwm4 wants to merge 2 commits into
Conversation
Remove 4 assessors per accepted ADR (Proposals C and E): - RepomixConfigAssessor (T3, 2%): single-vendor, not universal - CodeSmellsAssessor (T4, 1%): redundant with complexity/size/SoC - BranchProtectionAssessor (T4, 0.5%): perpetual stub - ConciseDocumentationAssessor (T2, 3%): redundant with length/size Move 2 assessors between tiers: - DesignIntentAssessor: T3 (2%) -> T2 (3%) - ProgressiveDisclosureAssessor: T4 (1%) -> T3 (2%) Redistribute freed 4% to highest-evidence T1 attributes: - test_execution: 10% -> 12% - type_annotations: 8% -> 10% New tier totals: 59/27/12/2 (was 55/27/14/4), 24 attributes (was 28). Self-score: 74.2 Silver (was 80.0 Gold) due to heavier T1 weighting. Closes #458 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
📝 WalkthroughWalkthroughRemoves four assessors, replaces README conciseness with InlineDocumentationAssessor, reassigns two assessors across tiers, and updates default weights, config examples, docs, and tests to produce the v2.1.0 24-attribute set. ChangesAssessor Set Consolidation and Tier Rebalancing
Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📉 Test Coverage Report
Coverage calculated from unit tests only |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/agentready/assessors/__init__.py`:
- Line 63: The docstring in src/agentready/assessors/__init__.py incorrectly
states "24 attributes" but the actual tiered list sums to 25 (after adding
inline_documentation and removing repomix_config, code_smells,
concise_documentation, branch_protection); update that docstring to "25
attributes" and then make the same count correction wherever the old "24
attributes" text appears (src/agentready/data/default-weights.yaml,
docs/attributes.md, CLAUDE.md) so all references are consistent, or
alternatively remove one assessor if you intend to keep 24—ensure changes
reference the assessor names (e.g., inline_documentation) and the tier breakdown
in the same files.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 5d89c22c-4a2a-46f1-ae58-c7759266365f
📒 Files selected for processing (16)
CLAUDE.mddocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/assessors/code_quality.pysrc/agentready/assessors/documentation.pysrc/agentready/assessors/patterns.pysrc/agentready/assessors/repomix.pysrc/agentready/assessors/testing.pysrc/agentready/data/.agentready-config.example.yamlsrc/agentready/data/default-weights.yamlsrc/agentready/github/review_formatter.pytests/e2e/test_critical_paths_simplified.pytests/unit/test_assessors_code_quality.pytests/unit/test_assessors_go.pytests/unit/test_assessors_patterns.pytests/unit/test_repomix.py
💤 Files with no reviewable changes (8)
- tests/unit/test_repomix.py
- tests/unit/test_assessors_code_quality.py
- src/agentready/assessors/repomix.py
- src/agentready/assessors/code_quality.py
- tests/unit/test_assessors_go.py
- src/agentready/assessors/documentation.py
- src/agentready/github/review_formatter.py
- src/agentready/assessors/testing.py
branch_protection was never in default-weights.yaml (intentionally omitted as a stub), so removing it from the code does not reduce the weighted attribute count. 28 - 3 (concise_documentation, repomix_config, code_smells) = 25. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/agentready/data/default-weights.yaml (1)
10-13:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftTier distribution conflicts with repository’s required default weights.
The updated config applies 59/27/12/2, but this repo’s policy for
src/agentready/data/default-weights.yamlrequires 55/27/15/3. This creates a compliance mismatch in the canonical scoring baseline and should be reconciled before merge.As per coding guidelines: "
src/agentready/data/default-weights.yaml: Use Tier weights from data/default-weights.yaml: Tier 1 (55%), Tier 2 (27%), Tier 3 (15%), Tier 4 (3%)."Also applies to: 28-59
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/agentready/data/default-weights.yaml` around lines 10 - 13, Update the tier weight percentages in src/agentready/data/default-weights.yaml so they match the repository policy: set Tier 1 to 55%, Tier 2 to 27%, Tier 3 to 15%, and Tier 4 to 3% (replace the current 59/27/12/2 values); also search the same file for the duplicated tier-distribution block and update any other occurrences to the same 55/27/15/3 values to keep the canonical scoring baseline consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/agentready/data/default-weights.yaml`:
- Around line 10-13: Update the tier weight percentages in
src/agentready/data/default-weights.yaml so they match the repository policy:
set Tier 1 to 55%, Tier 2 to 27%, Tier 3 to 15%, and Tier 4 to 3% (replace the
current 59/27/12/2 values); also search the same file for the duplicated
tier-distribution block and update any other occurrences to the same 55/27/15/3
values to keep the canonical scoring baseline consistent.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: d4c35ecf-0757-4dd7-9617-c7398a511304
📒 Files selected for processing (4)
CLAUDE.mddocs/attributes.mdsrc/agentready/assessors/__init__.pysrc/agentready/data/default-weights.yaml
jwm4
left a comment
jwm4
left a comment
There was a problem hiding this comment.
Review: PR #464 — Remove redundant assessors, realign tiers, rebalance weights
Overall this is a clean, well-motivated PR. CI passes, all 1064 unit tests pass, and assessments run correctly on three sample repos (agentready self, Flask, FastAPI) with no crashes or unexpected behavior. A few items to address before merge:
1. Stale attribute count in docs/attributes.md
The fix commit (ee504cb) caught most "24 → 25" references but missed two in docs/attributes.md:
- Line 6: Still says "all 28 agent-ready attributes" (should be 25)
- Line 1167: Says "All 24 assessors" (should be "All 25 assessors")
2. Stale tier weights in CLAUDE.md
Lines 54-57 still show the old tier distribution (55/27/15/3) instead of the new one (59/27/12/2).
3. Self-score discrepancy: 74.2 claimed vs 73.3 observed
Running agentready assess . on the PR branch (detached HEAD at ee504cb, editable install) produces 73.3/100 Silver, not 74.2. Manual calculation confirms the scorer IS using the new YAML weights correctly (0.12 for test_execution, 0.10 for type_annotations), so this is not a weight loading issue. The difference must come from individual attribute scores. Can you reproduce 74.2 on a clean checkout of this branch?
4. Class-level default_weight not updated for reweighted attributes
The YAML weights for test_execution and type_annotations were updated (0.10 → 0.12 and 0.08 → 0.10), but the default_weight in the assessor classes was not:
src/agentready/assessors/testing.py:37— stilldefault_weight=0.10src/agentready/assessors/code_quality.py:40— stilldefault_weight=0.08
The scorer uses the YAML value when present (line 135: weights.get(attr_id, finding.attribute.default_weight)), so scoring is correct. But the class values serve as fallbacks, appear in JSON output, and should stay in sync.
Testing on sample repos
| Repo | Score | Tier | Assessed/Skipped | Errors |
|---|---|---|---|---|
| agentready (self) | 73.3 | Silver | 23/6 | None |
| Flask | 48.4 | Bronze | 23/6 | None |
| FastAPI | 54.4 | Bronze | 24/5 | None |
Posted by Bill Murdock with assistance from Claude Code.
1 participant
Summary
Implements Proposals C and E from the accepted ADR.
Self-score change: 80.0 Gold -> 74.2 Silver (heavier T1 weighting exposes this repo's test coverage gap at 37%).
Related issues
This is the first of six implementation PRs for the ADR:
Test plan
black . && isort . && ruff check .passespytest tests/unit/passes (1064 passed, 17 skipped)agentready assess .runs successfully (74.2/100 Silver)Closes #458
Posted by Bill Murdock with assistance from Claude Code.
Summary by CodeRabbit
Refactor
New Features
Documentation
Tests