Skip to content

Commit 15f6a3d

Browse files
igerberclaude
andcommitted
Strip shell-only grep directive from local-mode adapted prompt
The CI prompt at `.github/codex/prompts/pr_review.md:54` contains a literal "Command to check: \`grep -n \"pattern\" diff_diff/*.py\`" directive under Pattern Consistency. The local Responses-API path has no shell access, so the model would either hallucinate having run grep or silently skip the pattern-consistency check while still claiming completeness. This bug pre-dates PR #404 (the grep directive has been in the prompt for as long as the local script has existed). PR #404's `9b76cd4` follow-up addressed shell-access claims inside the Mandate substitution specifically, but the older, standalone `Command to check:` line was never neutralized. Reverting #404 didn't introduce this — but neither does it fix it. Adds a `_SUBSTITUTIONS` entry that replaces the directive with a static-context note: "Verify by inspecting the loaded source files (no shell access in this path; do not claim to have run \`grep\`)". Also adds `test_strips_shell_grep_directive_from_real_prompt` that verifies the directive IS in the unadapted CI prompt and IS NOT in the local-adapted output, with "no shell access" wording in its place. 170 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent cbe3396 commit 15f6a3d

2 files changed

Lines changed: 26 additions & 0 deletions

File tree

.claude/scripts/openai_review.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -911,6 +911,15 @@ def estimate_cost(
911911
"If the PR fixes a pattern bug",
912912
"If the changes fix a pattern bug",
913913
),
914+
# Strip the shell-only `grep` directive — the local Responses API path
915+
# has no shell access and would either hallucinate having run grep or
916+
# silently skip the check. Replace with a static-context directive that
917+
# scopes the check to the loaded source files.
918+
(
919+
' - Command to check: `grep -n "pattern" diff_diff/*.py`',
920+
" - Verify by inspecting the loaded source files (no shell access "
921+
"in this path; do not claim to have run `grep`)",
922+
),
914923
(
915924
"the PR has prior AI review comments",
916925
"there is a previous review",

tests/test_openai_review.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,23 @@ def test_all_substitutions_apply_to_real_prompt(self, review_mod, capsys):
221221
captured = capsys.readouterr()
222222
assert "Warning: prompt substitution did not match" not in captured.err
223223

224+
def test_strips_shell_grep_directive_from_real_prompt(self, review_mod):
225+
"""The local path has no shell access — the literal `grep` directive
226+
in pr_review.md must be neutralized so the model doesn't claim to
227+
have run it."""
228+
assert _SCRIPT_PATH is not None
229+
repo_root = _SCRIPT_PATH.parent.parent.parent
230+
prompt_path = repo_root / ".github" / "codex" / "prompts" / "pr_review.md"
231+
if not prompt_path.exists():
232+
pytest.skip("pr_review.md not found")
233+
source = prompt_path.read_text()
234+
# Sanity: the directive IS present in the unadapted CI prompt.
235+
assert 'Command to check: `grep -n "pattern" diff_diff/*.py`' in source
236+
# After local adaptation: directive is gone, no-shell-access note is in.
237+
adapted = review_mod._adapt_review_criteria(source)
238+
assert 'Command to check: `grep' not in adapted
239+
assert "no shell access" in adapted
240+
224241

225242
# ---------------------------------------------------------------------------
226243
# compile_prompt

0 commit comments

Comments
 (0)