Summary
nanobot has a provider-agnostic fallback (LLMProvider._strip_image_content, added in
c4628038): when a model returns a non-transient error on image_url input, the image
blocks are stripped and the turn is retried as text. This fallback is correct and
desirable — but the text it substitutes is [image: <path>] (or [image omitted]),
which reads like a live, available image.
So when an image is not actually delivered to the model, the model:
- acts as if it saw the image and describes contents that aren't there,
- tries to
read_file the leaked path, and
- is exposed to an internal server file path it never needs.
This is core behavior in nanobot/providers/base.py, inherited by every provider — it
is not specific to any one model or backend.
Observed behavior
A user sent the plain black-and-white line drawing below (people talking on top,
people doing crafts on the bottom — no color, no characters) and asked whether the bot
could see it. The transcript:
User: (attaches the image) Can you see this image?
Bot: Yes, I can see it. The image shows a yellow character that looks like a
KakaoFriends character on a black background, with text below it reading "파프리카를
먹으면 화가 나는데…" ("when I eat paprika I get angry…").
User: That's a lie.
Nothing the bot described — the black background, the yellow character, the text — was
in the picture. The bot never received the image, but instead of saying so it fabricated
a completely different one.
The actual image that was sent:
Observed with a text-only model behind an OpenAI-compatible endpoint that rejects
image_url. The model/backend is not the cause — it correctly rejects images it
cannot process; the misleading placeholder is produced by nanobot's fallback. Any
non-vision / gateway / unknown model that rejects image input triggers the same path.
Root cause
nanobot/providers/base.py, _strip_image_content / _strip_image_content_inplace:
path = (b.get("_meta") or {}).get("path", "")
placeholder = image_placeholder_text(path, empty="[image omitted]")
[image: <path>] is a neutral "an image was here" marker. It does not signal that
the image is gone and unviewable, so a model that just lost the pixels treats the
marker as a live reference and fills in the blank. The embedded path additionally
invites read_file attempts and exposes a server-internal path.
Reproduction (model-agnostic)
- Configure any model/provider that returns a non-transient error on
image_url
input (any non-vision, gateway, or unknown model).
- Send a message containing an image.
- Observe: the retry replaces the image with
[image: <path>], and the model
describes a non-existent image and/or attempts to read_file the path.
Scope note (why this is its own fix)
This is the failure mode that fires when an image must be dropped. It is distinct
from, and complementary to:
Proposed fix
In the error-retry strip path only, replace the path-bearing placeholder with a
neutral, state-based marker that tells the model the image is gone and unviewable,
without asserting a cause or exposing a path:
[image omitted and cannot be viewed]
Leave image_placeholder_text and its history-serialization callers
(agent/loop.py, session/manager.py) unchanged — those represent images the model
did see in an earlier turn, where "cannot be viewed" would be inaccurate.
PR to follow.
Summary
nanobot has a provider-agnostic fallback (
LLMProvider._strip_image_content, added inc4628038): when a model returns a non-transient error onimage_urlinput, the imageblocks are stripped and the turn is retried as text. This fallback is correct and
desirable — but the text it substitutes is
[image: <path>](or[image omitted]),which reads like a live, available image.
So when an image is not actually delivered to the model, the model:
read_filethe leaked path, andThis is core behavior in
nanobot/providers/base.py, inherited by every provider — itis not specific to any one model or backend.
Observed behavior
A user sent the plain black-and-white line drawing below (people talking on top,
people doing crafts on the bottom — no color, no characters) and asked whether the bot
could see it. The transcript:
Nothing the bot described — the black background, the yellow character, the text — was
in the picture. The bot never received the image, but instead of saying so it fabricated
a completely different one.
The actual image that was sent:
Root cause
nanobot/providers/base.py,_strip_image_content/_strip_image_content_inplace:[image: <path>]is a neutral "an image was here" marker. It does not signal thatthe image is gone and unviewable, so a model that just lost the pixels treats the
marker as a live reference and fills in the blank. The embedded path additionally
invites
read_fileattempts and exposes a server-internal path.Reproduction (model-agnostic)
image_urlinput (any non-vision, gateway, or unknown model).
[image: <path>], and the modeldescribes a non-existent image and/or attempts to
read_filethe path.Scope note (why this is its own fix)
This is the failure mode that fires when an image must be dropped. It is distinct
from, and complementary to:
image; does not help when the image cannot be sent at all.
— let a separate vision model describe the image. Until such a path is configured (and
none is merged today), every text-only deployment hits this fallback, so it should at
least fail honestly instead of fabricating.
Proposed fix
In the error-retry strip path only, replace the path-bearing placeholder with a
neutral, state-based marker that tells the model the image is gone and unviewable,
without asserting a cause or exposing a path:
Leave
image_placeholder_textand its history-serialization callers(
agent/loop.py,session/manager.py) unchanged — those represent images the modeldid see in an earlier turn, where "cannot be viewed" would be inaccurate.
PR to follow.