[BUG] Hook output over 10K silently dropped from context with zero signal — memory plugins lose injected data invisibly

[Alex-Fleet]

Problem
Hooks that emit more than 10,000 characters of additionalContext (internal persistHookOutput threshold, since v2.1.89) are persisted to a temp file and never injected into the model context. The hook receives no error, no warning, and no way to know its output was discarded.

Why this is worse than just a size cap
For long-term-memory plugins this is silent data loss on every session start:

• Hook writes ~38KB of context → process exits 0 → nothing in context → the model starts with no memory.
• No stderr warning, no marker in the hook JSON output, no log line — a plugin author cannot detect this from the hook side.
• The user sees a normal-looking session and has no idea the injected context was dropped.

Requested
When output exceeds the threshold, give the hook (and the user) an explicit signal:

1. A warning line on stderr and/or in the hook result, and/or
2. A structured field in the hook JSON (e.g. additionalContextFile + truncated: true), and/or
3. A persistent log entry.

That at least turns invisible loss into an observable condition that plugins can handle (chunk their own output, fail loudly, or surface it to the user).

Related

• SessionStart hook output saved to disk at ~10K instead of documented 50K threshold #42369 (docs say 50K, implementation is 10K) — closed as NOT_PLANNED
• [BUG] SessionStart hook output silently truncated at 10KB — model never sees the missing content #70460 (SessionStart silently truncated) — open, but marked duplicate/stale

Environment: Claude Code 2.1.216, VS Code extension, macOS.

[JhouCode]

Confirming the behaviour on a different client and a different event, with one factual correction and a workaround that may help people stuck on this today.

Environment: Claude Code desktop CLI 2.1.229, Linux. UserPromptSubmit hook — not only SessionStart, which answers your open question about other hook types: it affects both.

Correction: the content is not discarded entirely. A ~2,000-character preview is injected, along with the file path. I measured 18,057 bytes emitted and ~2,000 delivered — 11% arrived. For the briefing/memory case that is worse than total loss: the model receives the beginning of the document and proceeds as if it had the whole thing, so the symptom is not "no memory", it is confidently partial memory. In my case half of the method layer arrived and none of the operational-rules layer.

Confirmed on the runtime side: the mechanism strings ("Output too large (…). Full output saved to: …", "Preview (first …)") are present in the build, and the preview-size constant resolves to 2000.

How this stayed invisible to me: three green instruments at once — the hook success status, the hook's own log, and its self-test. All three were answering "did the hook run?". The question that mattered was "did the content arrive?". Only a byte-for-byte diff between the script's output and the delivered context separated them. Which is exactly your request #1.

A workaround that works today, without waiting for a fix: the cap applies per registered command, not per event. Registering the same script N times, each instance emitting one slice under the cap, delivers everything in the same event. Measured with two 5,149-character probes registered together: both arrived intact — 10,298 characters in a single turn. It is the "chunk their own output" you mention, minus the need for a signal, and the docs do not make it obvious.

Two caveats for anyone using it: arrival order is not guaranteed (blocks arrive out of order, so each one has to identify itself), and chunking does not solve the problem you raised — it only avoids falling into it. Without a signal, anyone whose payload grows past the cap still finds out the hard way.

On the 50K discrepancy you cite from #42369: the current docs already state 10,000 (JSON output section), so that part seems to have been corrected in the text. What is still missing is item 1 of your request — that documented behaviour be signalled at runtime.

[JhouCode]

Adding the report lineage, since it is scattered and no single issue lists it. This is offered as triage context, not as a complaint about handling.

The same behaviour has been reported independently at least five times since January:

issue	date	what it reported	outcome
#17944	Jan 13	BASH_MAX_OUTPUT_LENGTH ignored — large output spills to disk regardless of the setting	closed not planned by the stale bot
#41799	Apr 1	[DOCS] hooks docs omit the spill-to-disk + preview behaviour	still open, no replies
#42369	Apr 2	SessionStart output spills at ~10K, not the documented 50K	flagged duplicate by bot → closed inactive → locked
#70460	—	SessionStart output silently truncated at 10KB	closed by the stale bot
#84021	Aug 5	this one	open

Adjacent, from the memory side: #82056 (a session cannot tell whether its index loaded whole, truncated, or not at all).

Two things stand out from the set, and both are mechanical rather than editorial:

None of the closed ones were closed on technical grounds. They aged out. The bot flags, warns, closes, and eventually locks. So "already reported and rejected" would be the wrong reading — it was never assessed either way.

None of the reporters found each other. Five people hit the same failure independently, each one alone on their thread, each discovering it the hard way. That is the strongest argument for request #1 in this issue: a runtime signal would have made every one of those five a five-minute diagnosis instead of a private investigation.

I have no visibility into triage priorities and I am not arguing about them. I am adding the history because it is the piece that was missing, and because activity is what keeps this thread from meeting the same automated end as the other four.

[JhouCode]

Correcting my own lineage comment above: there are seven, not five. I searched by title and missed two that are only referenced inside another thread's comments. Both are @permaevidence's, and they are the most directly on point.

issue	date	what it reported	outcome
#17944	Jan 13	BASH_MAX_OUTPUT_LENGTH ignored — large output spills to disk regardless of the setting	closed not planned by the stale bot
#41799	Apr 1	[DOCS] hooks docs omit the spill-to-disk + preview behaviour	open, no maintainer reply
#42369	Apr 2	SessionStart spills at ~10K, not the documented 50K	flagged duplicate → closed inactive → locked
#50571	Apr 19	"Is there a way to configure the persistHookOutput 10,000-character threshold?"	closed not planned by the stale bot, May 31
#51537	Apr 21	[FEATURE] raise, remove or make the 10,000-char cap configurable	open, labelled duplicate + stale
#70460	—	SessionStart output silently truncated at 10KB	closed by the stale bot
#84021	Aug 5	this one	open

Two things those two add that I did not have.

The "is it configurable?" question was asked formally, and closed without an answer. That is #50571. So when several of us report "I could not find an env var or a settings key", that is not for lack of looking — someone asked the question directly, on the tracker, and the thread aged out. Worth knowing before anyone re-derives it.

The constant has been read independently, four months apart, under two different names. @permaevidence identified it as PE4 = 1e4 in v2.1.89 (comment on #41799, Apr 21), noting it was renamed but unchanged through v2.1.116. I read it as BTK = 1e4 in a current build, with the preview size at 2000, without knowing that comment existed. Same constant, same value, minified name changed between versions — which is a useful detail for anyone trying to verify this from a shipped binary and finding a different symbol.

That also revises what I wrote earlier about the 50K/10K discrepancy being settled: the documentation now says 10,000, but the underlying request in #51537 — raise it, remove it, or make it configurable — is still open and still labelled stale, and it is a different ask from the signalling requested here. Both can be true: the value is correctly documented now, and it is still both unsignalled (this issue) and unconfigurable (#51537).

Seven reports, one author with two of them, no maintainer reply on any, and every closure automated. Posting the corrected set so the next person does not have to reassemble it.

[colinsurprenant]

Confirming on macOS, 2.1.237, SessionStart: same numbers @JhouCode measured. ~12.3KB emitted, ~2KB preview plus the file path delivered.

Adding a consumer-side workaround that works today, complementary to the chunking above: put a short instruction at the very top of the emitted block telling the model that if what it received is a persisted-output rendering (a "saved to" path plus preview instead of the full text), it must read that file before doing anything else. Since the preview is the first ~2,000 characters, the instruction survives truncation by construction. In practice the model complies and recovers the full payload; this recovered a 12.3KB session-state block for me this morning, and I run this in production (Director, SessionStart rehydration; the exact injected wording is here)

Caveats: it costs a tool round-trip, it depends on the model following the instruction rather than on any contract, and it spends preview budget on self-defense which is really an argument for request 1: producers can chunk, consumers can self-instruct, but both exist only because the runtime gives no signal.

[JhouCode]

@colinsurprenant — thank you for reproducing it. Confirmed on macOS 2.1.237 is worth considerably more than measured once on Linux 2.1.229.

Two things from running the producer-side split in production since, in case they save you time.

Ordering is not incidental — it is guaranteed to bite. Hooks on one event run in parallel, so the blocks arrive shuffled. Four parts of one document, same event, one real compaction:

10:03:48.959  part 2  (305ms)
10:03:48.968  part 1  (314ms)
10:03:48.972  part 4  (317ms)
10:03:48.974  part 3  (319ms)

So every chunk has to name itself. Mine carry "part N of 4" in the first line, plus an explicit line telling the reader not to trust arrival position. A consumer that assumes order reassembles the document wrong — which is worse than truncation, because nothing looks missing.

Your self-instruction and chunking do not compete; I would run both. Chunking keeps each payload under the cap so nothing spills. Your instruction recovers the case where one chunk grows past the cap anyway — which is exactly the case a producer stops noticing, since the block that outgrew the limit is the one you stopped reading months ago.

The reason I would not rely on chunking alone: the cap is per output string, so N registrations give N caps. That quietly means my payload budget grows every time I add a command and shrinks when I merge two — with no signal in either direction. I measured two 5,149-character probes registered together arriving intact, 10,298 characters in one turn. That is a real affordance, but it is one you can only find by experiment.

Neither of these is a fix. Both exist because the runtime spends 2,000 characters telling the model where the content went, instead of telling it that content went missing.

[colinsurprenant]

@JhouCode Thanks for the ordering data, that is the kind of failure I would have found the hard way, months from now. Taking both caveats as requirements: the delivery-check instruction stays, and if I move to producer-side chunking, parts will self-identify and any change in the number of registrations gets re-measured rather than assumed, since the budget quietly moves with it. Agreed on the bottom line: chunking, self-identification, and self-instruction are all scaffolding around the same missing one-liner. If the event said "content elided, N bytes at path", none of this thread would need to exist. +1 on request 1.

[JhouCode]

@colinsurprenant, @Alex-Fleet — a thank-you, one production number that I think is the most useful thing I have from three months of living with this, and an offer.

The thanks first. @Alex-Fleet, naming persistHookOutput and the threshold is what let four of us find each other on one thread instead of each running a private investigation, which is what the seven-report lineage shows happened before. @colinsurprenant, reproducing on macOS turned my single-platform measurement into something anyone could rely on, and your consumer-side self-instruction is the half I did not have — I have been running producer-side chunking, which only avoids the trap, while yours recovers from it.

The number, and it is a warning about my own mitigation. I split one document across four registered commands so each stays under the cap. Measured just now, the largest part:

part 1    9,242 bytes    8,911 UTF-16 units    8,904 characters

That is 89% of the cap, or 92%, depending on which unit the runtime counts — and I cannot tell which. The document grows every week. So the mitigation for the silent cap is itself sitting a few hundred characters from the silent cap, and the first symptom will be the thing @colinsurprenant described exactly: the part that outgrew the limit is the one I stopped reading months ago.

Which raises a question this thread has not asked. Over on #82056, four of us spent a week establishing that the memory index cap counts UTF-16 units while the docs and the runtime warning both label it KB — a unit count with a byte label, invisible on ASCII and 3× off on CJK. Nobody has established the unit for this cap. It is documented as "10,000 characters". If it is actually units, an index-style block full of emoji markers costs double what its author counts; if it is bytes, any accented text costs more than its character count. My three numbers above differ by 4%, and I have no way to find out which one the threshold compares against.

That is the same disease as request 1, one layer down: the constant exists, the runtime knows it, and nobody outside can see the unit it is measured in — let alone get told when they cross it.

The offer. I am a solo operator; there is no team here to catch this for me, and these silent caps are the current ceiling on what I can build rather than an annoyance I route around. Both of you have solved parts of it I had not. If either of you wants to compare notes — how you budget, how you detect growth before it bites, what you do about the unit question — I am glad to, here or wherever suits. I would rather trade than each of us re-derive it alone, which is what the lineage above is a record of.

[colinsurprenant]

@JhouCode Your unit question was answerable by experiment, so I ran it. The cap counts UTF-16 code units. Your binding number is 8,911, which is 89.1% of the cap.

Method: five SessionStart probes on 2.1.246 (macOS, headless claude -p), each a payload chosen so the three candidate units disagree about whether it crosses 10,000. The probe characters are the two cases you raised: the emoji 😀 counts as 1 character, 2 UTF-16 units, 4 UTF-8 bytes; the accented é counts as 1 character, 1 unit, 2 bytes.

payload	characters	UTF-16 units	UTF-8 bytes	result
A × 10,000	10,000	10,000	10,000	inline, intact
A × 10,001	10,001	10,001	10,001	persisted
😀 × 5,001	5,001	10,002	20,004	persisted
😀 × 4,999	4,999	9,998	19,996	inline, intact
é × 9,999	9,999	9,999	19,998	inline, intact

Reading: é × 9,999 arriving intact at 19,998 bytes rules out bytes. 😀 × 5,001 spilling at 5,001 characters rules out characters. The two emoji rows bracket the boundary at exactly 10,000 UTF-16 units, and the two ASCII rows show the comparison is inclusive: 10,000 exactly arrives, 10,001 spills.

This is consistent with the shipped code: the size function is JavaScript's .length on the output string (UTF-16 units by definition), and the 2,000 preview is .slice(0, 2000), also units. So the #82056 finding generalizes: same runtime, same idiom, and every one of these caps measured by .length will count units regardless of what the docs call them.

Two things I did not expect:

The runtime's own size report has the unit bug. Both spilled probes reported Output too large (9.8KB). That is the unit count divided by 1024 with a KB suffix: the emoji payload was actually 20,004 bytes, 19.5KB. So the one place the runtime does tell you a size, the number is a unit count wearing a byte label: 2× under on emoji, 3× under on CJK, exact only on ASCII.

The confusion reaches Anthropic's own telemetry. In the current binary, the tool-result persistence path (the sibling of this cap, readable as plain text in the build; the hook module itself is compiled to bytecode now, which is why your BTK symbol is no longer visible) emits a telemetry event whose field is literally named originalSizeBytes, fed from .length. Whoever dashboards that number in-house is reading units labeled bytes too, which may explain why no doc has ever stated the unit: internally it is mislabeled at the source.

For budgeting, the practical rules: measure s.length in JS or sum(2 if ord(c) > 0xFFFF else 1 for c in s) in Python. Pure ASCII can budget in bytes (they coincide). Accented Latin text costs 1 unit per character but 2 bytes, and CJK 1 unit but 3 bytes, so byte-based budgeting over-reserves on both. Emoji cost 2 units per character, so an index full of emoji markers under-counts if budgeted in characters — exactly your case.

Repro is ~15 lines if anyone wants to re-run on a future version: a --settings file registering a SessionStart hook that prints the payload, claude -p "ok" --settings probe.json, then grep the session transcript under ~/.claude/projects/ for persisted-output. Happy to paste the script if useful.

On the offer: gladly. My budgeting is currently one producer (Director's SessionStart block) measured in units at emit time, with the delivery-check line as the backstop, and after this thread the number it checks against is 10,000 units, not 10KB.

[zizi4n5]

Additional measurements on 2.1.251 (native installer, macOS / Darwin 25.6.0, interactive CLI). Three data points, one of which I have not seen reported elsewhere: the size figure in the "Output too large" notice counts UTF-16 code units but is labelled KB/MB, so non-ASCII payloads read up to ~30% smaller than their byte size.

1. Behaviour on the interactive CLI surface at 2.1.251

A SessionStart hook emitting large stdout gets: the full output persisted intact (file ends at a natural terminator, nothing lost on disk) to ~/.claude/projects/<project>/<session-id>/tool-results/hook-<uuid>-stdout.txt, and an injected notice of the form Output too large (<size>). Full output saved to: <path> followed by Preview (first 2KB).

So on this surface the user-side signal requested here partially exists (announced size + path + small preview). The hook-side signal — the core ask of this issue — is still absent: the hook process gets no indication its output was diverted.

This also extends the reproduction range: the issue reports 2.1.216; the behaviour is alive at 2.1.251.

2. The notice's size figure is UTF-16 code units in mislabelled binary units

Two persisted hook outputs, very different non-ASCII (Japanese) fractions:

file	bytes	chars (wc -m)	notice said	chars ÷ 1024ⁿ
A (JP-heavy)	3,513,560	2,472,447	2.4MB	2,472,447 / 1048576 = 2.36
B (mixed)	412,791	390,597	381.4KB	390,597 / 1024 = 381.44

• The figure reconciles exactly with character count ÷ 1024 (labelled KB) or ÷ 1024² (labelled MB) — and with neither file's byte count.
• Cross-validated by pre-registered prediction: from file B's notice alone (381.4KB at 0.1 resolution), the character count was predicted to lie in [390,502, 390,605] before it was measured; measured 390,597.
• The byte/char ratios (1.42 vs 1.06) are just each file's UTF-8 Japanese fraction — so the understatement is payload-dependent, not a constant factor.
• The counted quantity behaves like a JS string length (UTF-16 code units; astral-plane characters would count as 2).

Practical effect: a CJK-heavy payload's notice reads about a third smaller than the file it points to. If the notice is kept as part of the signal path proposed here, labelling the figure correctly (or computing bytes) would help the reader trust it.

3. Reconciliation with the persisted file

Byte-level check on file A: 16,940 lines, ends at its writer's natural final line — the persistence path loses nothing; only the in-context injection is reduced to the ~2KB preview. "Saved to disk" is accurate; "shown to the model" is ~0.06% of it.