Skip to content

Commit 9d30b2c

Browse files
authored
feat(chaos-engine): classifier NPath helpers + Codacy Complexity gate (#5747) (#5751)
## Summary Closes #5747. Encodes the ChaosEngine learning from Waves D/E: treat Codacy **Complexity** `ACTION_REQUIRED` as unit-red urgency, and land new classifier kinds as kind-family helpers / rule tables before opening interaction PRs. - **Portable checklist** `chaos-engine/references/codacy-complexity-gate.md` + Level-1 catalog / router / GitHub playbook wiring - **Soft PreToolUse hook** reminder on `ElementClassifier` / interaction mutations (non-blocking `additionalContext`) - **Classifier helpers:** `TAG_KINDS` / `ROLE_KINDS` maps for `classifyByTag` / `classifyByRole` (NPath-safe; `TEXT_ROLES` remains the text-role source) ## Checks ```bash python3 -m unittest tests.scripts.test_chaos_engine_codacy_complexity_gate_5747 tests.scripts.test_chaos_engine_wave_b_router_catalog -q python3 scripts/ci/validate_agent_setup.py --skip-external mvn -pl shaft-engine -Dtest=ElementClassifierTest -Dallure.automaticallyOpen=false -DheadlessExecution=true test ``` Local: harness 12 tests OK; `ElementClassifierTest` 45/0; validate_agent_setup exit 0. ## Continuation Watch CI / Codacy Complexity; Complexity `ACTION_REQUIRED` is unit-red — fix helpers first if it fires.
2 parents d63622f + 8edf288 commit 9d30b2c

8 files changed

Lines changed: 280 additions & 27 deletions

File tree

chaos-engine/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -346,7 +346,7 @@ usable without Mermaid; unknown source entries fail the inventory validator.
346346
| --- | --- | --- | --- | --- | --- | --- | --- |
347347
| argparse | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py, chaos-engine/dependencies.py, chaos-engine/draft_skill_pr.py, chaos-engine/heuristics.py, chaos-engine/hooks/reflection.py, chaos-engine/install.py, chaos-engine/learning.py, chaos-engine/learning_session.py, chaos-engine/meta_optimize.py, chaos-engine/phase_ledger.py, chaos-engine/retrieve.py, chaos-engine/significance.py, chaos-engine/silent_verify.py, chaos-engine/skill_compress_audit.py, chaos-engine/skills/freetoken/scripts/probe.py, chaos-engine/skills/local-coding-delegate/scripts/probe_hardware.py, chaos-engine/skills/omniroute/scripts/runner.py, chaos-engine/wake_pack.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
348348
| base64 | Portable runtime standard-library dependency. | chaos-engine/hosts.py, chaos-engine/install.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
349-
| collections | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py, chaos-engine/hooks/lifecycle.py, chaos-engine/mcp_policy.py, chaos-engine/silent_verify.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
349+
| collections | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py, chaos-engine/hooks/guard.py, chaos-engine/hooks/lifecycle.py, chaos-engine/mcp_policy.py, chaos-engine/silent_verify.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
350350
| concurrent | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
351351
| contextlib | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py, chaos-engine/dependencies.py, chaos-engine/hooks/guard.py, chaos-engine/hooks/kernel.py, chaos-engine/hooks/lifecycle.py, chaos-engine/hosts.py, chaos-engine/install.py, chaos-engine/learning.py, chaos-engine/learning_session.py, chaos-engine/silent_verify.py, chaos-engine/skills/omniroute/scripts/runner.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
352352
| ctypes | Portable runtime standard-library dependency. | chaos-engine/bootstrap.py, chaos-engine/dependencies.py, chaos-engine/hosts.py, chaos-engine/skills/local-coding-delegate/scripts/probe_hardware.py | required | Windows, Linux, macOS | resolved latest stable Python | Python runtime | affected command fails closed |
@@ -692,7 +692,7 @@ reports, caches, runtime indexes, or `graphify-out/`.
692692
## Read next
693693

694694
- [Delivery phase gates](references/delivery-phase-gates.md) — hooks vs skills map + research-before-mutation.
695-
695+
- [Codacy Complexity gate](references/codacy-complexity-gate.md) — classifier helpers; Complexity ACTION_REQUIRED == unit red.
696696

697697
- [Zero-LLM catalog](references/zero-llm-catalog.md) — deterministic doctor/install/repair paths.
698698
- [Context firewall](references/context-firewall.md) — research/explore subagent isolation; distillate only.

chaos-engine/hooks/guard.py

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
import re
1414
import shlex
1515
import sys
16+
from collections.abc import Mapping
1617
from pathlib import Path
1718

1819
def _load_sibling(module_name: str):
@@ -647,6 +648,69 @@ def _record_denial_with_significance(event: dict, event_name: str, tool_name: st
647648
)
648649

649650

651+
# Soft Codacy Complexity reminder for classifier / interaction mutations (#5747).
652+
_CLASSIFIER_PATH_MARKERS = (
653+
"elementclassifier",
654+
"/interaction/",
655+
"\\interaction\\",
656+
)
657+
_CLASSIFIER_NAME_MARKERS = (
658+
"elementclassifier",
659+
"classifymobilenative",
660+
"classifywindowsdesktop",
661+
"classifybytag",
662+
"classifybyrole",
663+
"classifyinput",
664+
)
665+
CODACY_COMPLEXITY_GATE_HINT = (
666+
"Codacy Complexity gate (#5747): kind-family helpers / rule tables before "
667+
"fat classify* arms; Complexity ACTION_REQUIRED == unit red. "
668+
"Checklist: references/codacy-complexity-gate.md"
669+
)
670+
671+
672+
def _mutation_path_blobs(tool_name: str, tool_input: object, commands: tuple[str, ...]) -> str:
673+
"""Flatten Write/Edit/patch/shell targets for classifier-path matching."""
674+
chunks: list[str] = []
675+
if isinstance(tool_input, Mapping):
676+
for key in ("file_path", "filePath", "path", "notebook_path", "notebookPath"):
677+
value = tool_input.get(key)
678+
if isinstance(value, str) and value.strip():
679+
chunks.append(value)
680+
for key in ("old_string", "new_string", "content", "patch", "input", "command", "cmd"):
681+
value = tool_input.get(key)
682+
if isinstance(value, str) and value.strip():
683+
chunks.append(value)
684+
elif isinstance(tool_input, str) and tool_input.strip():
685+
chunks.append(tool_input)
686+
chunks.extend(commands)
687+
compact = tool_name.casefold() if tool_name else ""
688+
if compact:
689+
chunks.append(compact)
690+
return "\n".join(chunks).casefold()
691+
692+
693+
def classifier_complexity_gate_hint(
694+
event_name: str,
695+
*,
696+
mutation: bool,
697+
tool_name: str,
698+
tool_input: object,
699+
commands: tuple[str, ...] = (),
700+
) -> str | None:
701+
"""Return soft checklist hint when mutating classifier / interaction surfaces."""
702+
if event_name != "PreToolUse" or not mutation:
703+
return None
704+
blob = _mutation_path_blobs(tool_name, tool_input, commands)
705+
if not blob:
706+
return None
707+
path_hit = any(marker in blob for marker in _CLASSIFIER_PATH_MARKERS)
708+
name_hit = any(marker in blob for marker in _CLASSIFIER_NAME_MARKERS)
709+
if path_hit or name_hit:
710+
return CODACY_COMPLEXITY_GATE_HINT
711+
return None
712+
713+
650714
def _run_event(event: dict, _host: str) -> int:
651715
tool_input = event.get("tool_input", {}) if isinstance(event, dict) else {}
652716
tool_name = str(event.get("tool_name", "")) if isinstance(event, dict) else ""
@@ -717,6 +781,16 @@ def _run_event(event: dict, _host: str) -> int:
717781
return 2
718782
if event_name == "SessionStart":
719783
print(json.dumps({"additionalContext": _event_context(event_name, token)}))
784+
return 0
785+
complexity_hint = classifier_complexity_gate_hint(
786+
event_name,
787+
mutation=mutation,
788+
tool_name=tool_name,
789+
tool_input=tool_input,
790+
commands=commands,
791+
)
792+
if complexity_hint:
793+
print(json.dumps({"additionalContext": complexity_hint}))
720794
return 0
721795

722796

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Codacy Complexity gate
2+
3+
Learning from issue #5747 (Waves D/E of epic #5732): Codacy **Complexity /
4+
NPath** repeatedly blocked merge after functional CI was already green.
5+
Pattern: element-classifier growth lands as one fat `classify*` method, then a
6+
second pass splits it.
7+
8+
## Iron rule
9+
10+
Treat Codacy **Complexity** `ACTION_REQUIRED` as a first-class red gate **equal
11+
to a failing unit test**. Do not wait for GitHub unit jobs to finish when Codacy
12+
Complexity already failed — extract helpers and re-push.
13+
14+
`scripts/agents/watch_pr_checks.py` already classifies `ACTION_REQUIRED` as
15+
RED (Codacy and similar apps). Agent behavior must match that: triage Complexity
16+
red immediately, same urgency as unit red.
17+
18+
## Checklist (before opening or pushing interaction / classifier PRs)
19+
20+
- [ ] New mobile / desktop / web kind branches land as **kind-family helpers**
21+
or first-match rule tables — not more sequential `if`/`return` arms on a
22+
single `classify*` method
23+
- [ ] Prefer `Set` / `Map` buckets plus a loop over rules (NPath multiplies
24+
across sequential branches even when each guard is trivial)
25+
- [ ] Codacy Complexity `ACTION_REQUIRED` == unit red: fix before waiting on
26+
other checks
27+
- [ ] Local proof includes the touched classifier unit tests (and any
28+
interaction strategy tests that share those kinds)
29+
30+
## Hook
31+
32+
Portable PreToolUse soft reminder (non-blocking `additionalContext`) when a
33+
mutation targets an `ElementClassifier` path or interaction `classify*` surface.
34+
See [`hooks/guard.py`](../hooks/guard.py). Hosts that ignore soft context still
35+
owe this checklist via the Level-1 catalog and router row.
36+
37+
## Boundaries
38+
39+
- Soft reminder never denies a tool call; Complexity red is still a delivery gate.
40+
- Do not weaken Codacy findings or delete tests to clear the gate.
41+
- Portable overlay owns the policy; host adapters stay thin pointers.

chaos-engine/references/level-1-catalog.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ sorted. Each row is a name, ≤2-line Use-when, and a path — no workflow dumps
2525
| Retrieve-first | A store can shorten discovery; one bounded attempt | [`retrieve-first.md`](retrieve-first.md) |
2626
| Research receipt | Before implementation mutation; triage scales depth | [`research-receipt.md`](research-receipt.md) |
2727
| Delivery phase gates | Phase ledger / research-before-mutation enforcement | [`delivery-phase-gates.md`](delivery-phase-gates.md) |
28+
| Codacy Complexity gate | Classifier / interaction PRs; Complexity ACTION_REQUIRED == unit red | [`codacy-complexity-gate.md`](codacy-complexity-gate.md) |
2829
| Eval-parity fixtures | Cross-host CE policy fixture suite | [`eval-parity-fixtures.md`](eval-parity-fixtures.md) |
2930
| Self-improve | Learning Session dual-track harness + product observations | [`../skills/self-improve/SKILL.md`](../skills/self-improve/SKILL.md) |
3031
| Self-improve master plan | Next-wave self-improve roadmap + Top 10 (post-Learning Session Stop gate); profile may extend under `profiles/<product>/references/` | [`self-improve-master-plan.md`](self-improve-master-plan.md) |

chaos-engine/references/work-github-playbook.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,10 @@ and focused proofs observed; it must not represent remote checks as green.
196196
branch, or merge the fetched configured upstream default branch for a
197197
conflict or stale head, then return to watch. Never force-push away
198198
owner-visible history. Any new push restarts the comment gate before
199-
auto-merge may remain armed.
199+
auto-merge may remain armed. Codacy **Complexity** `ACTION_REQUIRED` is
200+
unit-red urgency ([codacy-complexity-gate](codacy-complexity-gate.md)):
201+
extract kind-family helpers immediately; do not wait for unit jobs when
202+
Complexity already failed.
200203
8. **Confirm** remotely that `mergedAt` is non-null; armed is not merged.
201204

202205
## 8. Report

chaos-engine/skills/chaos-engine/SKILL.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -216,6 +216,7 @@ return here for the next.
216216
| Eliminate waste | Token optimization: drop hops that do not change the next decision | [eliminate-waste](../../references/eliminate-waste.md) |
217217
| No proxy | Never install a traffic proxy | [no-proxy](../../references/no-proxy.md) |
218218
| GAP-EXIT2 UX | Grok/Copilot may not honor exit-2 hard blocks | [host-parity-matrix](../../references/host-parity-matrix.md) checklist |
219+
| Codacy Complexity | Classifier / interaction PRs; Complexity ACTION_REQUIRED == unit red | [codacy-complexity-gate](../../references/codacy-complexity-gate.md) checklist |
219220

220221
Routing also orders applicable knowledge retrieval before broad manual
221222
discovery. One bounded attempt is enough; never retry, repair, refresh, mine,

shaft-engine/src/main/java/com/shaft/gui/element/internal/interaction/ElementClassifier.java

Lines changed: 31 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -332,24 +332,29 @@ private static ElementSignals fromWebElement(WebElement element) {
332332
safeDom(element, "autocomplete"));
333333
}
334334

335+
/** Tag → kind for web tags decided before input handling (NPath-safe map). */
336+
private static final Map<String, ElementKind> TAG_KINDS = Map.of(
337+
"select", ElementKind.SELECT,
338+
"textarea", ElementKind.TEXT_LIKE,
339+
"iframe", ElementKind.IFRAME,
340+
"frame", ElementKind.IFRAME,
341+
"a", ElementKind.LINK,
342+
"button", ElementKind.BUTTON);
343+
335344
/**
336345
* Tag-driven kinds (and tag/role pairs that are decided before input handling).
337346
* Returns null when the tag does not decide the kind.
338347
*/
339348
private static ElementKind classifyByTag(String tag, String role) {
340-
if ("select".equals(tag)) {
341-
return ElementKind.SELECT;
342-
}
343-
if ("textarea".equals(tag)) {
344-
return ElementKind.TEXT_LIKE;
345-
}
346-
if ("iframe".equals(tag) || "frame".equals(tag)) {
347-
return ElementKind.IFRAME;
349+
ElementKind fromTag = tag == null ? null : TAG_KINDS.get(tag);
350+
if (fromTag != null) {
351+
return fromTag;
348352
}
349-
if ("a".equals(tag) || "link".equals(role)) {
353+
// Role-only link/button when the tag did not already decide.
354+
if ("link".equals(role)) {
350355
return ElementKind.LINK;
351356
}
352-
if ("button".equals(tag) || "button".equals(role)) {
357+
if ("button".equals(role)) {
353358
return ElementKind.BUTTON;
354359
}
355360
return null;
@@ -407,23 +412,25 @@ private static ElementKind classifyButtonOrTextInput(String type) {
407412
return ElementKind.UNKNOWN;
408413
}
409414

415+
/** ARIA role → kind for the role-only fallthrough path (NPath-safe map). */
416+
private static final Map<String, ElementKind> ROLE_KINDS = Map.of(
417+
"checkbox", ElementKind.CHECKBOX,
418+
"switch", ElementKind.CHECKBOX,
419+
"radio", ElementKind.RADIO,
420+
"slider", ElementKind.RANGE,
421+
"combobox", ElementKind.COMBOBOX,
422+
"listbox", ElementKind.COMBOBOX);
423+
410424
private static ElementKind classifyByRole(String role) {
411-
if ("checkbox".equals(role) || "switch".equals(role)) {
412-
return ElementKind.CHECKBOX;
413-
}
414-
if ("radio".equals(role)) {
415-
return ElementKind.RADIO;
416-
}
417-
if ("slider".equals(role)) {
418-
return ElementKind.RANGE;
419-
}
420-
if ("combobox".equals(role) || "listbox".equals(role)) {
421-
return ElementKind.COMBOBOX;
425+
if (role == null) {
426+
return ElementKind.UNKNOWN;
422427
}
423-
if (role != null && TEXT_ROLES.contains(role)) {
424-
return ElementKind.TEXT_LIKE;
428+
ElementKind kind = ROLE_KINDS.get(role);
429+
if (kind != null) {
430+
return kind;
425431
}
426-
return ElementKind.UNKNOWN;
432+
// TEXT_ROLES stays the single source for textbox/searchbox/spinbutton.
433+
return TEXT_ROLES.contains(role) ? ElementKind.TEXT_LIKE : ElementKind.UNKNOWN;
427434
}
428435

429436
private static boolean isContentEditable(ElementSignals signals) {
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
"""#5747: Codacy Complexity gate checklist + soft classifier-edit hook."""
2+
3+
from __future__ import annotations
4+
5+
import importlib.util
6+
import io
7+
import json
8+
import sys
9+
import tempfile
10+
import unittest
11+
from contextlib import redirect_stdout
12+
from pathlib import Path
13+
14+
ROOT = Path(__file__).resolve().parents[2]
15+
GATE = ROOT / "chaos-engine/references/codacy-complexity-gate.md"
16+
LEVEL1 = ROOT / "chaos-engine/references/level-1-catalog.md"
17+
SKILL = ROOT / "chaos-engine/skills/chaos-engine/SKILL.md"
18+
PLAYBOOK = ROOT / "chaos-engine/references/work-github-playbook.md"
19+
GUARD = ROOT / "chaos-engine/hooks/guard.py"
20+
21+
22+
def load(path: Path, name: str):
23+
spec = importlib.util.spec_from_file_location(name, path)
24+
module = importlib.util.module_from_spec(spec)
25+
sys.modules[spec.name] = module
26+
spec.loader.exec_module(module)
27+
return module
28+
29+
30+
class CodacyComplexityGate5747Tests(unittest.TestCase):
31+
@classmethod
32+
def setUpClass(cls):
33+
cls.guard = load(GUARD, "ce_guard_codacy_5747")
34+
35+
def test_checklist_documents_unit_red_parity_and_helpers(self):
36+
text = GATE.read_text(encoding="utf-8")
37+
self.assertIn("ACTION_REQUIRED", text)
38+
self.assertIn("unit", text.casefold())
39+
self.assertIn("kind-family", text.casefold())
40+
self.assertIn("classify*", text)
41+
self.assertIn("#5747", text)
42+
43+
def test_catalog_and_router_point_at_checklist(self):
44+
level1 = LEVEL1.read_text(encoding="utf-8")
45+
skill = SKILL.read_text(encoding="utf-8")
46+
playbook = PLAYBOOK.read_text(encoding="utf-8")
47+
self.assertIn("codacy-complexity-gate.md", level1)
48+
self.assertIn("Codacy Complexity", level1)
49+
self.assertIn("codacy-complexity-gate.md", skill)
50+
self.assertIn("| Codacy Complexity |", skill)
51+
self.assertIn("codacy-complexity-gate.md", playbook)
52+
self.assertIn("ACTION_REQUIRED", playbook)
53+
54+
def test_hint_fires_for_element_classifier_edit(self):
55+
hint = self.guard.classifier_complexity_gate_hint(
56+
"PreToolUse",
57+
mutation=True,
58+
tool_name="Edit",
59+
tool_input={
60+
"file_path": (
61+
"shaft-engine/src/main/java/com/shaft/gui/element/"
62+
"internal/interaction/ElementClassifier.java"
63+
),
64+
"old_string": "a",
65+
"new_string": "b",
66+
},
67+
)
68+
self.assertIsNotNone(hint)
69+
self.assertIn("ACTION_REQUIRED", hint)
70+
self.assertIn("codacy-complexity-gate.md", hint)
71+
72+
def test_hint_skips_unrelated_mutation(self):
73+
hint = self.guard.classifier_complexity_gate_hint(
74+
"PreToolUse",
75+
mutation=True,
76+
tool_name="Write",
77+
tool_input={"file_path": "README.md", "content": "hello"},
78+
)
79+
self.assertIsNone(hint)
80+
81+
def test_hint_skips_non_mutation_and_non_pretool(self):
82+
self.assertIsNone(
83+
self.guard.classifier_complexity_gate_hint(
84+
"PreToolUse",
85+
mutation=False,
86+
tool_name="Edit",
87+
tool_input={"file_path": "ElementClassifier.java"},
88+
)
89+
)
90+
self.assertIsNone(
91+
self.guard.classifier_complexity_gate_hint(
92+
"PostToolUse",
93+
mutation=True,
94+
tool_name="Edit",
95+
tool_input={"file_path": "ElementClassifier.java"},
96+
)
97+
)
98+
99+
def test_pretool_use_emits_soft_context_without_blocking(self):
100+
with tempfile.TemporaryDirectory() as temporary:
101+
event = {
102+
"hook_event_name": "PreToolUse",
103+
"session_id": "codacy-5747-soft",
104+
"cwd": temporary,
105+
"tool_name": "Edit",
106+
"tool_input": {
107+
"file_path": (
108+
"shaft-engine/src/main/java/com/shaft/gui/element/"
109+
"internal/interaction/ElementClassifier.java"
110+
),
111+
"old_string": "x",
112+
"new_string": "y",
113+
},
114+
}
115+
buffer = io.StringIO()
116+
with redirect_stdout(buffer):
117+
code = self.guard._run_event(event, "claude")
118+
self.assertEqual(0, code)
119+
payload = json.loads(buffer.getvalue().strip().splitlines()[-1])
120+
self.assertIn("additionalContext", payload)
121+
self.assertIn("ACTION_REQUIRED", payload["additionalContext"])
122+
self.assertNotEqual("block", payload.get("decision"))
123+
124+
125+
if __name__ == "__main__":
126+
unittest.main()

0 commit comments

Comments
 (0)