Skip to content

fix(providers): use non-descriptive placeholder when stripping images - #4401

Merged
Re-bin merged 4 commits into
HKUDS:mainfrom
michaelxer:fix-image-strip-placeholder-4345
Jun 19, 2026
Merged

fix(providers): use non-descriptive placeholder when stripping images#4401
Re-bin merged 4 commits into
HKUDS:mainfrom
michaelxer:fix-image-strip-placeholder-4345

Conversation

@michaelxer

Copy link
Copy Markdown
Contributor

Summary

When a model returns a non-transient error on image_url input, the provider base strips image blocks and retries as text. The current fallback placeholder [image: <path>] (or [image omitted]) reads like a live, available image to the LLM, causing it to hallucinate about contents it never received, attempt read_file on the leaked server path, and expose internal file paths.

This PR replaces the placeholder with an explicit [Image not delivered to model — do not describe or reference it] message in both _strip_image_content and _strip_image_content_inplace.

Linked Issue

Fixes #4345

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Changes

  • nanobot/providers/base.py: Replace image_placeholder_text(path, empty="[image omitted]") with a fixed [Image not delivered to model — do not describe or reference it] placeholder in both _strip_image_content and _strip_image_content_inplace
  • Remove unused image_placeholder_text import from base.py
  • tests/providers/test_strip_image_content.py: 8 new tests covering both methods — verifies no path leakage, correct placeholder text, multi-image handling, no-image return, and in-place mutation

How to Test

pytest tests/providers/test_strip_image_content.py -v

All 8 tests verify that:

  1. The placeholder does NOT contain the file path
  2. The placeholder clearly indicates the image was not delivered
  3. Non-image content is preserved
  4. Multiple images are all replaced
  5. Images without _meta are handled
  6. In-place mutation works correctly

Checklist

  • I have tested these changes locally
  • My code follows the project's coding style (ruff passes)
  • I have added tests that prove my fix is effective

The image-strip fallback (triggered when a model errors on image input)
replaced image_url blocks with [image: <path>] or [image omitted]. Both
read like a live, available image to the LLM, causing it to:

1. hallucinate about image contents it never received
2. attempt read_file on the leaked server path
3. expose internal file paths to the model

Replace with an explicit '[Image not delivered to model — do not describe
or reference it]' placeholder that tells the LLM the image was stripped.

Fixes HKUDS#4345
…ve text

The _strip_image_content methods now use a fixed non-descriptive
placeholder instead of path-derived text. Update the 3 existing
test_provider_retry assertions to match the new placeholder format.

@chengyongru chengyongru left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated PR review by nanobot. This is not a human maintainer review or approval.

Summary

This PR replaces the file-path-leaking placeholder [image: <path>] / [image omitted] with a static, non-descriptive message [Image not delivered to model — do not describe or reference it] in both _strip_image_content and _strip_image_content_inplace.

What is done well

  • Correctly scoped: only touches the fallback path in base.py. The image_placeholder_text helper remains in use elsewhere (loop.py, session/manager.py) where path info is intentionally preserved.
  • Removes the now-unused import from base.py — no dead references.
  • Test coverage is solid: 8 new tests covering both methods, multi-image handling, no-image return, missing _meta, and in-place mutation semantics.
  • Existing test_provider_retry.py assertions updated to match the new placeholder text.

Observations

  • This is a genuine security improvement: the old placeholder leaked server-side file paths into LLM context, which could cause the model to hallucinate image contents or attempt to read_file on those paths.
  • No blocking concerns. The change is small, focused, and well-tested.

{
"role": "user",
"content": [
{"type": "text", "text": "Look at these:"},

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion [image not in placeholder.lower() or not delivered in placeholder.lower()is always True for the new placeholder, since the first clause is always satisfied. Consider simplifying to justassert "not delivered" in placeholder.lower()` for clarity. Non-blocking.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 3d2f422. I tightened the placeholder assertions to require not delivered directly, including the nearby equivalent checks, so the tests now state the intended contract without the always-satisfied alternative.

@Re-bin Re-bin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This fixes the image-strip fallback behavior in the right place: when image inputs are removed before retrying, the replacement text no longer leaks local paths or implies the model actually saw the image. The regression coverage is nicely focused around both strip paths.

I pushed two tiny follow-up commits: one to fix the ruff import-format issue in the new test file, and one to address chengyongru’s inline comment by tightening the placeholder assertions to require not delivered directly.

Validated locally after those fixes:

  • python -m pytest tests/providers/test_provider_retry.py tests/providers/test_strip_image_content.py -q
  • python -m ruff check nanobot/providers/base.py tests/providers/test_provider_retry.py tests/providers/test_strip_image_content.py

GitHub Actions is running again for the latest test-only follow-up commit; please wait for the matrix before merging.

@Re-bin
Re-bin merged commit 58a14d1 into HKUDS:main Jun 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image-strip fallback makes the model act as if it saw an image it never received (and leaks the file path)

3 participants