GPT-5.6 often serializes independent Code Mode calls; explicit batching reduced weighted usage by 27–45%

[MakerOfToys]

What version of the Codex App are you using (From “About Codex” dialog)?

26.721.30844

What subscription do you have?

ChatGPT Pro for the personal-project tests; ChatGPT Business for the work-project tests.

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

Related to #32503, but this report adds controlled same-model benchmarks, credit-weighted usage measurements, quality comparisons, and a tested workaround across two unrelated codebases.

I began investigating this independently after noticing an unusually high number of recent complaints about Codex usage depletion. I had not personally noticed an obvious long-term reduction in my limits, so I inspected my own .codex session logs rather than assume that allowances had been silently changed.

Only after reaching the batching hypothesis and completing my initial tests did I search for similar reports and find #32503. Its author had already identified essentially the same Code Mode parallelism problem from a long observational trace, including only 5 of 739 GPT-5.6 exec cells using Promise.all. The tests below appear to provide the controlled validation that report said was still missing.

I tested two unrelated repositories:

• one large proprietary work codebase;
• one smaller personal finance application.

Each repository had one fixed read-only investigation task. Within each comparison, the subscription plan, repository state, task prompt, model, reasoning effort and permissions were unchanged. Only the presence of the batching instruction differed. No code was modified during the investigations.

Main results

Dataset	Sample	Model cycles	Raw tokens	Weighted usage	Same-credit capacity
Large work codebase	4 batched runs vs 3 controls, High	52.1% fewer	63.1% lower	45.0% lower	81.8% more
Personal finance app	4 matched pairs, High/XHigh	54.7% fewer	52.6% lower	27.0% lower	37.0% more
Personal finance app, Max	1 matched pair, adjusted	80.4% fewer	79.5% lower	47.4% lower	90.2% more

The Max result is a single pair and needs replication. It still covered all required areas and aligned on the core conclusions, but the batched report referenced fewer unique files than its control, so I treat both its efficiency and quality result cautiously. The repeated High/XHigh results are the stronger basis for the current conclusion.

The exact improvement varied substantially by workload. I am not claiming that every Codex task will gain 37–82% more capacity.

“Weighted usage” is the Codex-credit equivalent calculated from the returned uncached-input, cached-input and output-token categories. Reasoning tokens are included in output tokens.

Most of the raw-token reduction came from repeated context processing:

• cached input fell by 56–65% in the repeated datasets;
• uncached input fell by 5–26%;
• output changed by only about 0–12%.

This is consistent with batching reducing outer model round trips rather than simply reducing the amount of investigation performed.

Normal-usage observation

At work I deleted my .codex directory and allowed it to rebuild from scratch.

By July 23, the fresh logs had accumulated roughly 1,600 calls that a post-hoc Codex analysis classified as independent and batchable. Only a little over 100 had actually been grouped by the agent.

This was not part of the controlled benchmark, but it suggests that GPT-5.6 sometimes discovers batching on its own while doing so very inconsistently without concrete instructions.

Quality

I did not find an apparent systematic quality reduction.

Across the Finance comparisons:

• both conditions covered every required investigation area;
• the main conclusions aligned;
• the aggregate performed 106 batched inspections versus 105 control inspections.

On the large codebase, the batched runs performed substantially more shell operations than the controls while still using much less.

Batched runs sometimes found additional useful details. Subtle quality differences may still exist, and implementation-heavy or mutation-heavy tasks may behave differently from these read-only investigations.

---

What steps can reproduce the bug?

1. Choose a read-heavy repository task involving several independent searches, file reads or metadata inspections.

2. Freeze the repository state.

3. Start fresh GPT-5.6 Sol sessions using the same:

   • task prompt;
   • reasoning effort;
   • permissions;
   • repository state.

4. Run one condition without custom batching guidance.

5. Run the other with this exact instruction in either repository AGENTS.md or the Codex App custom instructions:

In Code Mode, within each bounded stage, run independent, functions.exec-available tool calls concurrently in one functions.exec call. Use await Promise.allSettled([...]) when partial results are useful, and inspect every result; use await Promise.all([...]) only when any failure should abort the batch. Keep dependencies, waits/resumes, approvals, conflicting or interdependent mutations, and adaptive investigations where each result may change the next step sequential. Do not split otherwise batchable inspections across outer tool calls.

6. Compare the session logs:

   • outer model/tool cycles;
   • direct tool calls and exec cells;
   • nested calls per exec cell;
   • use of Promise.all or Promise.allSettled;
   • cached, uncached and output tokens;
   • task coverage and final conclusions.

The controlled repository comparisons used AGENTS.md. I also used the same instruction through app-level custom instructions and observed the same multi-call Code Mode batching pattern.

Despite using older functions.exec terminology, it reliably caused GPT-5.6 Sol to group multiple nested operations into concurrent Code Mode stages.

---

What is the expected behavior?

GPT-5.6 Code Mode should receive concrete guidance to:

• group independent, non-conflicting calls into one bounded exec stage;
• use Promise.all or Promise.allSettled for nested concurrency;
• keep dependent, adaptive, approval-sensitive and conflicting operations sequential;
• avoid repeated one-call exec cells when multiple independent calls are already known;
• avoid expanding investigation scope only because more calls can be launched concurrently.

The model-facing exec description should include an example containing multiple nested tools.* calls inside Promise.all(...) or Promise.allSettled(...).

It would also help if telemetry distinguished outer model cycles from nested tool calls and showed quota-weighted usage more directly.

---

Additional information

GPT-5.2 through GPT-5.5 were explicitly instructed to parallelize common independent reads using multi_tool_use.parallel.

GPT-5.6 moved to the Responses Lite and Code Mode path. Native top-level parallel calls are disabled there, while JavaScript inside exec is the practical replacement. The runtime supports nested concurrency, but the shipped GPT-5.6 guidance is generic and the exec example demonstrates only a single nested call.

I may be missing an internal constraint or implementation detail, but the public source, local traces, controlled behavior and the independent findings in #32503 are consistent with an instruction/tool-interface regression during that transition.

I do not see evidence that this was an intentional allowance reduction. OpenAI added a replacement batching mechanism; the model appears not to use it reliably without concrete guidance.

I also tested several rewritten instructions using the current exec and tools.* terminology. One essentially matched the original instruction on the large codebase, but none consistently matched its Finance results. Some expanded investigation scope, while others produced one-call Promise.allSettled cells that provided little real batching. One overly aggressive variant consumed 26.8% more weighted credits than its control.

Limitations

These tests covered read-heavy repository investigations.

The large-codebase result used grouped averages rather than matched simultaneous pairs. The Max result and each revised instruction variant currently have only one pair. Model sampling, service load and differences in peripheral investigation scope remain sources of variance.

“Same-credit capacity” means more comparable completed work from the same allowance. It does not mean the allowance necessarily lasts the same additional percentage in literal wall-clock time, because the batched tasks also tend to finish faster.

Additional sanitized per-run measurements are available on request.

[MakerOfToys]

Update: output-truncation caveat

A raw-log audit found that some batched exec responses exceeded Code Mode’s shared output limit and were truncated. This may have overstated the measured usage savings and introduces a quality risk, although the reduction in model/tool cycles remains clear and the repeated comparisons still showed aligned required coverage and core conclusions.

I am retesting with a minimally revised instruction that bounds combined output and narrowly retrieves any missing evidence. Until then, please treat the 27–45% savings range as provisional.

[ForrestGrump]

Independent replication on a different setup (Codex CLI 0.145.0, WSL2,
ChatGPT Pro auth, gpt-5.6-sol), plus two findings I haven't seen in this
thread or #32503: the effect is effort-gated, and ultra effort is a
separate order of magnitude.

Setup

Controlled A/B, only variable = a batching instruction (the
Promise.allSettled paragraph from this issue) vs a length-matched placebo
instruction, injected via AGENTS.md (synthetic repos) or
-c developer_instructions=... (real repos, read-only sandbox). Fresh
session per trial, alternating arm order, MCP/plugins/memories disabled,
fixed model + effort per run. Metrics parsed from the
~/.codex/sessions rollout JSONL (token_count events = model cycles;
custom_tool_call name="exec" cells inspected for tools.*( counts and
Promise.all/allSettled). Harness: https://github.com/ForrestGrump/codex-batch-bench.

Results — synthetic 48-module repo, effort sweep (no-instruction ÷ instructed)

effort	model cycles	raw tokens	weighted credits*
low	1.00×	1.11×	0.98×
medium	1.14×	1.01×	0.90×
high	1.06×	0.96×	0.89×
xhigh	1.50×	1.56×	1.16×
max	1.33×	1.36×	0.99×
ultra	4.83×	4.44×	3.65×

* at the published 125/12.5/750 per-1M rate card.

At low–high, the uninstructed model batches on its own (often via shell
loops in plain exec_command calls — zero Code Mode cells). At xhigh/max it
degenerates to one tool call per model cycle. A fix that only tests at
high effort will not see this bug.

Results — three real repos, xhigh, 2 trials/arm each

Raw-token multipliers (uninstructed ÷ instructed): 1.41× / 3.15× / 2.66×
(mean ≈ 2.4×, consistent with the 2.1–2.7× in the OP). Worst trial: 25
cycles / 1.71M raw tokens uninstructed vs 7 cycles / 400k instructed for the
identical task. Deliverable quality was equivalent in all trials, both arms
(graded on the synthetic repos: 48/48 modules + 40/40 config keys at every
effort level including ultra).

Ultra is its own problem

At ultra, the uninstructed runs spawned 23–25 sub-sessions despite
features.multi_agent=false, used Promise.all* zero times across ~50
exec cells, and consumed 7.5–17.6M raw tokens (267–605 credits) per
read-only investigation task — the worst single trial moved a Pro weekly
meter a full 1%. Filed separately: #35177.

Also verified: sub-agents inherit the parent's resolved config (including
-c developer_instructions overrides); they do not re-read
~/.codex/config.toml.

Repro

git clone https://github.com/ForrestGrump/codex-batch-bench && cd codex-batch-bench
python3 bench.py setup
python3 bench.py run --preset large --effort xhigh   # ~15 min, ~1.1M raw tokens

Happy to provide raw rollout logs / metrics CSVs for any trial.

[MakerOfToys]

Further truncation-audit context

A more detailed comparison shows that roughly 80k of the batched-side omitted tokens came from two ALL_TOOLS catalogue dumps rather than repository evidence. Non-batched controls also truncated broad compound shell outputs.

Excluding those catalogue dumps and the single Max pair, the four repeated High/XHigh comparisons omitted 63.6k repository-output tokens with batching versus 20.9k in controls. Restoring all explicitly omitted repository output once changes the repeated weighted-saving estimate from 27.0% to approximately 23.9%; the large reduction in model cycles and cached-input usage remains.

Truncation is therefore still a real output-reliability issue, but the detailed logs do not suggest that it explains most of the measured efficiency gain or caused an apparent systematic collapse in answer quality. A cleaner output-bounded replication is still needed.

[ForrestGrump]

@MakerOfToys I ran your truncation audit against my 52-trial replication (methodology upthread; harness: https://github.com/ForrestGrump/codex-batch-bench, audit script now in the repo as audit_ingestion.py). Metric: characters of all tool output entering each trial's transcript (function_call_output + custom_tool_call_output payloads in the rollout JSONL), split unique vs exact-duplicate.

run	no-instruction	batching	ingestion ratio	raw-token ratio
synthetic 48-module @ xhigh	55.8k	55.9k	1.00×	1.56×
real repo A @ xhigh	92.2k	70.4k	1.31×	1.41×
real repo B @ xhigh	245.9k	169.7k	1.45×	3.15×
real repo C @ xhigh	109.8k	70.7k	1.55×	2.66×
synthetic @ ultra	849k	420k	2.02×	4.44×

Three observations relevant to "a cleaner output-bounded replication is still needed":

1. The synthetic-xhigh pair is that replication. Fixed 48-file corpus, generated ground truth (answers graded 48/48 modules + 40/40 config keys in both arms), and ingestion parity to within 0.2% — yet a 1.56× raw-token and 1.50× cycle gap. On an evidence-bounded task the entire gap is round-trip overhead; zero truncation component.

2. On real repos the batched arm did ingest 24–36% less tool output, consistent with your concern — but the consumption gaps (1.41–3.15×) and cycle gaps (1.50–2.79×) substantially exceed the ingestion gaps, and the dominant cost term (cached-input re-reads scaling with cycle count) is independent of output volume. Deliverables were equivalent in every pair.

3. At ultra, the uninstructed arm was the over-ingester: the swarm read the ~250KB corpus ≈3.4× over (vs ≈1.7× batched) across 23–25 sub-sessions (gpt-5.6-sol ultra effort spawns multi-agent swarms despite features.multi_agent=false, consuming 250–600 credits per read-only task without batching guidance #35177) — redundant re-reading, not deeper evidence; both arms graded perfect.

Caveat matching yours: exact-duplicate share was only 0–2% by hash, but overlapping re-reads via different commands/line ranges hash as unique, so byte-level dedup understates redundancy. Net: truncation is a real output-reliability concern on open-ended tasks, but on this dataset it cannot account for the majority of the measured gap, and on the bounded task it accounts for none of it.

[MakerOfToys]

@ForrestGrump Thank you for doing all this work and for continuing to test the different explanations. The ingestion audit is especially useful - it’s reassuring to see a clean case where both arms received essentially the same amount of tool output while the batching run still used substantially fewer tokens. Really appreciate you publishing the harness and results too.

[avoethan]

Edited after a source and benchmark audit (July 26, 2026).

I still think a description-only PR is the right first experiment, but I would narrow the causal claim and make the output-limit guidance more precise.

Why this scope still fits the current source

• gpt-5.6-sol uses Responses Lite and code_mode_only.
• Responses Lite disables top-level parallel_tool_calls, so JavaScript concurrency inside Code Mode exec is the practical batching path.
• The current model-facing exec description shows a single awaited nested call and does not explain Promise.all(...) or Promise.allSettled(...).
• The runtime test suite demonstrates that two nested tool calls can overlap. That proves the mechanism exists, but not that every tool, mutation, approval path, or failure mode is universally safe to parallelize.

Proposed PR scope

Change only the model-facing Code Mode exec description and focused tests. Do not change request serialization, parallel_tool_calls, model metadata, runtime scheduling, telemetry, or approval behavior in this first patch.

Suggested guidance:

When multiple nested tool calls for the current stage are already known, mutually independent, and safe to run without ordering, approval, or shared-state conflicts, run a small bounded group concurrently in one exec call. Prefer Promise.allSettled(...) when partial results are useful, inspect every settled result, and explicitly handle rejections or truncated results. Use Promise.all(...) only when any failure should abort the whole batch. Keep adaptive, approval-sensitive, waiting, potentially conflicting, or state-changing work sequential unless the tool contract explicitly guarantees parallel safety. Do not broaden the task merely because concurrency is available.

The outer // @exec max_output_tokens value is a shared budget for the combined exec result. Keep batches bounded, request narrow fields or ranges, use nested tool-specific output limits where supported, and set the outer budget deliberately. If evidence is truncated, recover only the missing evidence with a narrow follow-up read.

A compact example could be:

// @exec: {"max_output_tokens": 3000}
const tasks = [
  ["status", tools.exec_command({ cmd: "git status --short", max_output_tokens: 800 })],
  ["diff", tools.exec_command({ cmd: "git diff --stat", max_output_tokens: 1200 })],
];

const settled = await Promise.allSettled(tasks.map(([, promise]) => promise));
const checked = settled.map(([name], index) => {
  const result = settled[index];
  return result.status === "fulfilled"
    ? { name, ok: true, value: result.value }
    : { name, ok: false, error: String(result.reason) };
});

text(JSON.stringify(checked));

The exact example can be adjusted to match the repository's preferred tool and platform-neutral style.

Tests I would add

1. Direct assertions on the rendered exec description for:
   • Promise.allSettled guidance;
   • inspection of every result;
   • sequencing exclusions;
   • shared outer-output-budget wording.
2. A runtime test with one successful and one failed nested call under Promise.allSettled, confirming that the successful result remains available and the failure is explicit.
3. A combined-output-budget test confirming that concurrent nested results still obey the single outer exec budget.
4. Existing Responses Lite request snapshots should remain unchanged.

Validation before calling it fixed

Compare stock and patched binaries with the public harness using:

• stock placebo vs patched placebo as the primary comparison;
• explicit batching instructions as a positive-control ceiling;
• XHigh as the primary effort tier, Max as secondary;
• Low–High as non-regression tiers;
• Ultra reported separately because its automatic sub-session behavior is tracked in gpt-5.6-sol ultra effort spawns multi-agent swarms despite features.multi_agent=false, consuming 250–600 credits per read-only task without batching guidance #35177.

I would look for a material reduction in XHigh outer model cycles and singleton exec stages, no more than a 5% regression at Low–High, no increase in runtime failures or unrecovered truncation, and exact task grading rather than only checking module-name and unused-key presence.

This remains a bounded, testable first fix rather than a claim that description text is the sole root cause. I am happy to prepare the focused patch and validation if a Codex-team member agrees with the approach and explicitly invites a PR under the repository's contribution policy.

[ForrestGrump]

@avoethan Strong +1 on this scope from the replication side — the data says a description-only fix is aimed at the right layer:

• The runtime path already works when the model knows about it. Across 30 instructed trials (six effort tiers + three real repos), the model used Promise.allSettled inside exec in every xhigh/max/ultra trial with zero runtime failures — the capability is fine; only the contract is silent about it. A prompt-contract fix matches the failure mode.
• "Teach the capability and decision boundary rather than force maximum fan-out" is exactly what the effort curve demands. At high and below the model already groups reads on its own (often via shell compounds), and my always-batch instruction was mildly negative there (0.89× weighted credits vs placebo). Unconditional "batch everything" wording risks regressing the healthy tiers; conditional wording ("already-known, independent, non-mutating, one bounded stage") produced the wins without measurable quality cost (graded 48/48 + 40/40 in both arms at every tier).
• Your item 3 (output-safety guidance) is empirically motivated too: in my real-repo runs the batched arm ingested 24–36% less tool output — "inspect every settled result, request narrow ranges, set appropriate max_output_tokens" is the right mitigation for exactly the truncation concern discussed above.

To help review such a PR: the harness now supports --codex-bin, so a patched build can be benchmarked against stock with identical prompts — compare the placebo arms across the two runs (no explicit instruction anywhere; any delta isolates the built-in description change), with the batching arm as a positive-control ceiling. Recipe in the README: https://github.com/ForrestGrump/codex-batch-bench#validating-a-tool-description-pr-binary-vs-binary

One review-criterion suggestion: validate at xhigh/max (and ideally ultra, where the orchestrator + sub-agents inherit the same description — see #35177), and check high-and-below for non-regression. A description change validated only at default efforts will not exercise the failure mode this issue is about.

Happy to run the pre/post sweep on a candidate branch and post numbers.