fix: auto-resolve question tool in non-interactive contexts#937
fix: auto-resolve question tool in non-interactive contexts#937sahrizvi wants to merge 2 commits into
Conversation
`Question.ask()` awaits an Effect Deferred that only resolves on a TUI click. When `altimate-code run` is invoked as a subprocess (Claude Code's Bash tool, CI, plugin host) and a skill that uses `question` fires, nobody can ever click — the deferred awaits forever and the parent eventually TaskStops the subprocess. The symptom is indistinguishable from a hang: 0% CPU, no log activity, no error. In non-interactive contexts (no TTY, or explicit env-var opt-in), auto-resolve `question` with a conservative-by-default policy and flag the auto-answer in the tool result so the calling LLM can adapt instead of treating it as a real user choice. Resolution policy (env-var controlled): - Detect non-interactive: `!process.stdin.isTTY`. Overrides: ALTIMATE_FORCE_INTERACTIVE=1 — keep the original interactive Deferred path even when isTTY is false. ALTIMATE_NON_INTERACTIVE=1 — force non-interactive even when isTTY is true (useful for tests + CI assertions). - Default in non-TTY (ALTIMATE_AUTO_ANSWER=last): pick the option whose label/description contains a safe keyword (skip, cancel, no, abort, profile only, decline, deny, stop); fall back to the last option (UX convention: safer/cancel typically sits at end). - ALTIMATE_AUTO_ANSWER=first / =skip / =<exact label>: explicit overrides for callers who want a specific behavior. Tool result prefix reflects mode — "Running in non-interactive mode (no TTY). Auto-answered with safe defaults: ..." vs the original "User has answered your questions: ..." — so the agent knows the choice was not a real user answer. Tests: 6 new bun:test cases covering safe-keyword selection, last-option fallback, each ALTIMATE_AUTO_ANSWER mode, and the prefix wording. Existing 2 legacy tests gated with ALTIMATE_FORCE_INTERACTIVE=1 so they preserve their original intent under non-TTY CI. Closes #936
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughQuestionTool now detects non-interactive runs (env + TTY) and, when non-interactive, computes deterministic answers via ALTIMATE_AUTO_ANSWER (first/last/exact/skip) instead of awaiting Question.ask, and it prefixes output to indicate auto-answering. ChangesNon-interactive auto-answer support
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
packages/opencode/src/tool/question.ts (2)
39-57: 💤 Low valueConsider logging when ALTIMATE_AUTO_ANSWER doesn't match any known mode or option label.
When
ALTIMATE_AUTO_ANSWERis set to a value that doesn't match "skip", "first", "last", or any option label (case-insensitive), the function silently returns empty arrays (Unanswered). This is safe but could make debugging harder if a user misspells a mode or label.Consider logging a warning in this case to help users diagnose configuration issues.
📝 Optional: Add debug logging for unmatched modes
// exact label match for explicit answers, e.g. ALTIMATE_AUTO_ANSWER="Profile only" const match = q.options.find((o) => o.label.toLowerCase() === mode) + if (!match && mode !== "skip" && mode !== "first" && mode !== "last") { + console.warn(`ALTIMATE_AUTO_ANSWER="${mode}" did not match any option label; returning Unanswered`) + } return match ? [match.label] : []🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/opencode/src/tool/question.ts` around lines 39 - 57, The autoAnswer function silently returns empty answers when ALTIMATE_AUTO_ANSWER contains an unknown mode or when the explicit-label match fails; add a warning log to aid debugging: detect when mode is not "skip"/"first"/"last" and no q.options label matches the lowercase mode, and emit a single warning (e.g., console.warn or the module's logger) including the provided ALTIMATE_AUTO_ANSWER value and question id/label (use Question.Info properties) so users know the env value didn't match any known mode or option; place this check inside autoAnswer just before returning an empty array for unmatched cases and reference autoAnswer, ALTIMATE_AUTO_ANSWER, and q.options in the change.
86-92: 💤 Low valueConsider mode-specific prefix wording for accuracy.
The prefix says "Auto-answered with safe defaults" but this is only accurate when
ALTIMATE_AUTO_ANSWERis "last" (default). When the mode is "first" or an exact label match, the selection isn't necessarily using safe defaults—it's just picking the first option or the specified label.While the key information ("Running in non-interactive mode") is accurate and sufficient, you might consider mode-specific wording for precision:
- "last" → "Auto-answered with safe defaults"
- "first" → "Auto-selected first option"
- exact match → "Auto-selected option: {mode}"
- "skip" → "Auto-skipped (Unanswered)"
This is a minor clarity improvement; the current wording is acceptable since the primary goal is signaling non-interactive execution.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/opencode/src/tool/question.ts` around lines 86 - 92, The prefix message built in question.ts (variable prefix) misstates the non-interactive auto-answer behavior by always saying "Auto-answered with safe defaults"; update the logic that sets prefix (referencing isNonInteractive() and the ALTIMATE_AUTO_ANSWER mode) to choose mode-specific wording: "Auto-answered with safe defaults" for "last", "Auto-selected first option" for "first", "Auto-selected option: {mode}" for exact label matches, and "Auto-skipped (Unanswered)" for "skip", while keeping the existing "Running in non-interactive mode" text and preserving the interactive branch "User has answered your questions".
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/opencode/src/tool/question.ts`:
- Around line 39-57: The autoAnswer function silently returns empty answers when
ALTIMATE_AUTO_ANSWER contains an unknown mode or when the explicit-label match
fails; add a warning log to aid debugging: detect when mode is not
"skip"/"first"/"last" and no q.options label matches the lowercase mode, and
emit a single warning (e.g., console.warn or the module's logger) including the
provided ALTIMATE_AUTO_ANSWER value and question id/label (use Question.Info
properties) so users know the env value didn't match any known mode or option;
place this check inside autoAnswer just before returning an empty array for
unmatched cases and reference autoAnswer, ALTIMATE_AUTO_ANSWER, and q.options in
the change.
- Around line 86-92: The prefix message built in question.ts (variable prefix)
misstates the non-interactive auto-answer behavior by always saying
"Auto-answered with safe defaults"; update the logic that sets prefix
(referencing isNonInteractive() and the ALTIMATE_AUTO_ANSWER mode) to choose
mode-specific wording: "Auto-answered with safe defaults" for "last",
"Auto-selected first option" for "first", "Auto-selected option: {mode}" for
exact label matches, and "Auto-skipped (Unanswered)" for "skip", while keeping
the existing "Running in non-interactive mode" text and preserving the
interactive branch "User has answered your questions".
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 2e6c3623-6d27-4e80-adc6-f385f03948eb
📒 Files selected for processing (2)
packages/opencode/src/tool/question.tspackages/opencode/test/tool/question.test.ts
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 2 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
❌ Tests — Failures DetectedTypeScript — 15 failure(s)
Next StepPlease address the failing cases above and re-run verification. cc @sahrizvi |
The earlier non-interactive policy in this branch scanned option label text for "safe" keywords (skip/cancel/no/abort/...) and fell back to the last option. That tried to recover semantics the LLM already knew at construction time, and false-positived on substrings — "no" matched inside "Snowflake", "Annotate", "Knowledge", "Honor". The fix: don't guess. Return Unanswered for every question when no TTY is present and let the agent decide. The agent has full context — it knows what action it was about to take and why it asked. It can pick a safe path from that context or report that user input is required. Pretending a decision was made that wasn't is the worse failure mode. Changes: - Drop SAFE_KEYWORDS and the label-text scan entirely. - Default non-interactive behavior returns [] (renders as "Unanswered" via the existing format()). - Cache isNonInteractive() once at execute() entry so the result prefix can't disagree with the path that produced the answer. - Non-interactive prefix tells the agent how to proceed AND names the escape hatch (ALTIMATE_AUTO_ANSWER=first|last|<label>) so it can surface it to the user when reporting back. - Keep =first / =last / =<exact label> as explicit user opt-ins for callers who genuinely want a default. Drop =skip — it's the new default. Addresses cubic-dev-ai review feedback on #937 by removing the heuristic that needed the word-boundary fix in the first place. Tests: 9 pass / 0 fail in packages/opencode/test/tool/question.test.ts. Refs #936 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
New commit: redesigned around the actual problemPushed Delete the label-text heuristic entirely. The LLM that constructed the question already knew at construction time which option meant "cancel" — the keyword scan was trying to recover information that was never lost, and false-positived on substrings ( What this commit does
Addresses cubic review feedback
Tests + verification
Honest tradeoffThe prior policy (auto-pick "last" or safe-keyword) tried to make forward progress when no human is present. The new policy says: forward progress on behalf of someone who didn't choose is the worse failure mode — let the agent (which has full context) decide. The escape hatch in the prefix gives the user explicit knobs if they disagree. |
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
1 similar comment
|
👋 This PR was automatically closed by our quality checks. Common reasons:
If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you. |
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/opencode/src/tool/question.ts">
<violation number="1" location="packages/opencode/src/tool/question.ts:85">
P2: Non-interactive output contradicts itself by saying no user answered, then telling the agent to continue with the user's answers. This can mislead downstream agent behavior in the exact flow this change introduces.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| // altimate_change start — make the non-interactive case unambiguous to | ||
| // the agent so it doesn't treat "Unanswered" as a real user choice. | ||
| const prefix = nonInteractive | ||
| ? `Running in non-interactive mode (no TTY). No user was available to answer. Either pick a safe path from the context of the action you were about to take, or report that user input is required to proceed — the user can set ALTIMATE_AUTO_ANSWER=first|last|<exact option label> to pre-answer questions in this mode. Result: ` |
There was a problem hiding this comment.
P2: Non-interactive output contradicts itself by saying no user answered, then telling the agent to continue with the user's answers. This can mislead downstream agent behavior in the exact flow this change introduces.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/opencode/src/tool/question.ts, line 85:
<comment>Non-interactive output contradicts itself by saying no user answered, then telling the agent to continue with the user's answers. This can mislead downstream agent behavior in the exact flow this change introduces.</comment>
<file context>
@@ -83,11 +79,10 @@ export const QuestionTool = Tool.define("question", {
+ // altimate_change start — make the non-interactive case unambiguous to
+ // the agent so it doesn't treat "Unanswered" as a real user choice.
+ const prefix = nonInteractive
+ ? `Running in non-interactive mode (no TTY). No user was available to answer. Either pick a safe path from the context of the action you were about to take, or report that user input is required to proceed — the user can set ALTIMATE_AUTO_ANSWER=first|last|<exact option label> to pre-answer questions in this mode. Result: `
: `User has answered your questions: `
// altimate_change end
</file context>
Centralized-test bot failures — likely shared infra, not PR-causedThe Strong signal these are shared fault-injection-harness failures, not regressions introduced by this PR. Flagging here so the pattern is on record across the three PRs; happy to dig into the harness config if the bot owner wants. |
suryaiyer95
left a comment
suryaiyer95
left a comment
There was a problem hiding this comment.
One workflow-affecting concern (verified against the codebase) — posting just this; the rest were quality/description nits.
| function isNonInteractive(): boolean { | ||
| if (process.env["ALTIMATE_FORCE_INTERACTIVE"] === "1") return false | ||
| if (process.env["ALTIMATE_NON_INTERACTIVE"] === "1") return true | ||
| return !process.stdin.isTTY |
There was a problem hiding this comment.
Likely regression for server / IDE (headless-but-interactive) mode — !process.stdin.isTTY is the wrong signal for "no human is listening."
Question.ask() does not only resolve on a TUI keypress. Answers also arrive over HTTP: Question.reply({ requestID, answers }) is exposed at POST /question/:requestID/reply (packages/opencode/src/server/routes/question.ts), with GET /question to list pending ones. So when altimate-code runs as a server with a frontend/IDE client (VS Code / JetBrains / web), process.stdin.isTTY is false, yet a real human can answer via the route.
With this guard, that existing interactive flow is misclassified as non-interactive: execute() short-circuits to autoAnswer() → returns Unanswered immediately and never publishes the question for the client to reply to. The UI user loses the ability to answer, out of the box — ALTIMATE_FORCE_INTERACTIVE=1 exists but server deployments won't have it set.
Suggestion: gate on whether an answer channel actually exists, not on TTY. E.g. treat it as interactive when the server/question clients are connected (a pending-question listener is registered), and reserve the auto-answer path for true headless runs (run subprocess / CI). At minimum, default server mode to interactive and require explicit ALTIMATE_NON_INTERACTIVE=1 to opt into auto-answer, so the altimate-code run-as-subprocess fix doesn't also silently disable the HTTP reply path.
dev-punia-altimate
left a comment
dev-punia-altimate
left a comment
There was a problem hiding this comment.
🤖 Code Review — OpenCodeReview (Gemini) — 1 finding(s)
- 1 anchored to a line (posted inline when the comment stream is on)
- 0 without a line anchor
All findings (full text)
1. packages/opencode/src/tool/question.ts (L34)
[🟠 MEDIUM] There's a potential null pointer exception here. Depending on the runtime environment (e.g., embedded environments or child processes without standard streams), process.stdin could be undefined. Using optional chaining process.stdin?.isTTY is a safer approach and adheres to the null checks guideline.
Suggested change:
return !process.stdin?.isTTY
| function isNonInteractive(): boolean { | ||
| if (process.env["ALTIMATE_FORCE_INTERACTIVE"] === "1") return false | ||
| if (process.env["ALTIMATE_NON_INTERACTIVE"] === "1") return true | ||
| return !process.stdin.isTTY |
There was a problem hiding this comment.
[🟠 MEDIUM] There's a potential null pointer exception here. Depending on the runtime environment (e.g., embedded environments or child processes without standard streams), process.stdin could be undefined. Using optional chaining process.stdin?.isTTY is a safer approach and adheres to the null checks guideline.
Suggested change:
| return !process.stdin.isTTY | |
| return !process.stdin?.isTTY |
🤖 Code Review — OpenCodeReview (Gemini) — 1 finding(s)
All findings (full text)1.
|
3 participants
Closes #936
Summary
packages/opencode/src/tool/question.tscallsQuestion.ask(), which awaits an EffectDeferredthat only resolves on a TUI click. Whenaltimate-code runis invoked as a subprocess (Claude Code's Bash tool, CI,subprocess.run, plugin host) and a skill that uses thequestiontool fires, nobody can ever click — the deferred awaits forever and the parent eventuallyTaskStops the subprocess. Symptom: 0% CPU, no log activity, no error, indistinguishable from a hang.This PR short-circuits the
questiontool in non-interactive contexts with a conservative-by-default auto-answer policy.Resolution policy
!process.stdin.isTTY. Overrides:ALTIMATE_FORCE_INTERACTIVE=1— keep the original interactive Deferred path even when isTTY is false.ALTIMATE_NON_INTERACTIVE=1— force non-interactive even when isTTY is true (useful for tests + CI assertions).ALTIMATE_AUTO_ANSWER=last): pick the option whose label/description contains a safe keyword (skip,cancel,no,abort,profile only,decline,deny,stop); fall back to the last option (UX convention: safer/cancel typically sits at the end).ALTIMATE_AUTO_ANSWER=first— always pick first option.ALTIMATE_AUTO_ANSWER=skip— returnUnansweredfor all questions.ALTIMATE_AUTO_ANSWER="<exact label>"— exact-match an option's label (case-insensitive)."Running in non-interactive mode (no TTY). Auto-answered with safe defaults: ..."vs the original"User has answered your questions: ..."— so the agent knows the choice was not a real user answer and can adapt strategy.Why not just always pick "cancel"?
Picking cancel/abort blindly fails open in the opposite direction: skills that ask permission to do reasonable work would always get a no, breaking the user's actual intent. The safe-keyword scan tries to match the question author's intent (these are typically "may I do destructive thing X?" prompts) without blocking legitimate flows.
Where this lives
Two reasonable choices:
packages/opencode/src/tool/question.tsshort-circuits before callingQuestion.ask().Question.ask()itself, so any caller ofQuestion.ask(not just the tool) benefits.Option 1 is what this PR ships; the scope is contained and the diff is reviewable. Option 2 is the deeper, more invasive change and may be the right long-term home. Happy to refactor if reviewers prefer.
Test plan
bun test test/tool/question.test.ts→ 8 pass, 0 fail, 12 expect() calls. Verified locally before pushing.ALTIMATE_FORCE_INTERACTIVE=1inbeforeEachso they preserve their original intent under non-TTY CI).ALTIMATE_AUTO_ANSWER=first,=skip,=<exact label>, non-interactive prefix wording.questionfrom Claude Code's Bash tool — expect completion in seconds with auto-answered output, not a hang.Diff size
The patch is roughly 70 lines of source change plus tests. Larger than #935 (the stdin-wedge guard) because the resolution policy has real branching to implement; still bounded to one tool file.
Risk
Low. Default behavior under TTY is unchanged (the new branch only fires when
!isTTYorALTIMATE_NON_INTERACTIVE=1). The non-TTY auto-answer surfaces explicitly in the tool result, so a downstream agent treating the choice as a real user click is impossible.ALTIMATE_FORCE_INTERACTIVE=1provides an escape hatch for any consumer that wants the original behavior even in non-TTY.Links
data-parityskill's PII consent question wedge)plugin-skill-experiments/03-issues-and-fixes.mdIssue chore(deps): Bump @gitlab/gitlab-ai-provider from 3.6.0 to 4.1.0 #5run.ts— same surface area, different code path)Summary by cubic
Fixes hangs in the
questiontool when no TTY is present by returning Unanswered by default and clearly marking non-interactive mode, addressing #936. Interactive behavior is unchanged.!process.stdin.isTTY; overrides:ALTIMATE_FORCE_INTERACTIVE=1,ALTIMATE_NON_INTERACTIVE=1.ALTIMATE_AUTO_ANSWER=first,ALTIMATE_AUTO_ANSWER=last, orALTIMATE_AUTO_ANSWER="<exact label>".ALTIMATE_AUTO_ANSWERso agents can proceed safely or request input.Written for commit f981199. Summary will update on new commits.
Summary by CodeRabbit
New Features
Tests