Skip to content

Commit 0272095

Browse files
committed
feat(#3996): fall back to an explicit user-prompt filename for one unnamed image
A live image-output model can ignore the [media-file:] marker instruction entirely, leaving a prompt like "Generate an image as sunshine.jpg" to land as generated-1.png. Add a deterministic fallback: when a turn returns exactly ONE media blob that marker pairing left unnamed, parse a single unambiguous explicit output filename from the triggering user message. The cue grammar is strict, deliberately not NLP: save (it) as / save to / write to / output to / name it / call it / filename:=, plus bare "as" only inside a narrow imperative output context (generation verb + optionally-articled media noun, e.g. "Generate an image as sunshine.jpg"), and a companion of-phrase form (generation verb + media noun + "of <subject>" + "as <filename>") so "Generate an image of a red panda as assets/red-panda.jpg" extracts the intended name. The of-phrase subject cannot cross quotes, clause punctuation, or CR/LF, and any subject containing "as", "with", or "in" is refused because those prepositions introduce open-ended attribute phrases whose trailing "as" compares — a false reject only costs the generic generated-N name, while a false capture could engage the escape-confirmation policy for a merely referenced file. Bare "called" and unanchored "as" are NOT cues, so comparative references to existing files never extract a name. Candidates are keyed by the filename capture's position: one occurrence matched by both grammars counts once; distinct occurrences stay ambiguous and extract nothing. Names may be quoted, backticked, or unquoted with a known image extension; zero or multiple candidates extract nothing. Precedence stays marker -> user-prompt filename -> provider display name -> generic generated-N, and the extracted path only fills MediaDelta.RequestedPath, so the existing untrusted-path pipeline (MIME/extension correction, collision suffixing, workspace containment and escape confirmation) applies unchanged.
1 parent 2af7bd6 commit 0272095

5 files changed

Lines changed: 637 additions & 12 deletions

File tree

pkg/chat/media.go

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -23,15 +23,17 @@ type MediaDelta struct {
2323
// runtime accumulator synthesizes one when needed.
2424
Name string `json:"name,omitempty"`
2525

26-
// RequestedPath is the prompt-directed target path the model asked for
27-
// (e.g. echoed from an "as sunshine.jpg" instruction), when one exists.
28-
// It is untrusted model input: the runtime routes it through
26+
// RequestedPath is the prompt-directed target path for this blob, when
27+
// one exists. It is untrusted input: the runtime routes it through
2928
// workspacemedia.ClassifyRequestedPath, and a path escaping the workspace
3029
// requires an explicit user confirmation before it is honored. The
3130
// runtime's response marker filter (the "[media-file: ...]" protocol,
3231
// pkg/runtime/generated_media_markers.go) populates it by pairing marker
33-
// paths with blobs in response order; blobs no marker names keep it empty
34-
// and materialization falls back to Name, then a generic name.
32+
// paths with blobs in response order; a single otherwise-unnamed blob may
33+
// instead get it from deterministic explicit-filename extraction on the
34+
// triggering user message (pkg/runtime/generated_media_prompt_filename.go).
35+
// Blobs neither source names keep it empty and materialization falls back
36+
// to Name, then a generic name.
3537
RequestedPath string `json:"requested_path,omitempty"`
3638

3739
// Size is the byte length of Data, cached because Data itself is
Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
package runtime
2+
3+
import (
4+
"path"
5+
"regexp"
6+
"strings"
7+
"unicode/utf8"
8+
9+
"github.com/docker/docker-agent/pkg/chat"
10+
)
11+
12+
// Deterministic user-prompt filename fallback: when a turn returns exactly
13+
// one media blob and no [media-file:] marker named it (a model may ignore
14+
// the marker instruction entirely), the triggering user message — never
15+
// model text or history — is scanned for a single unambiguous explicit
16+
// output filename ("Generate an image as sunshine.jpg", "Generate an image
17+
// of a red panda as assets/red-panda.jpg", "save it as `pics/cat.png`",
18+
// "filename: x.webp", ...). The naming precedence is
19+
// therefore: marker →
20+
// user-prompt explicit filename → provider display name → generic
21+
// generated-N. An extracted path is untrusted exactly like a marker path:
22+
// it only fills [chat.MediaDelta.RequestedPath] and flows through the same
23+
// workspacemedia classification, MIME/extension correction, collision
24+
// suffixing, and escape-confirmation pipeline. This is a strict grammar,
25+
// deliberately not generic NLP: zero or multiple candidates mean no
26+
// extraction.
27+
28+
// maxExplicitOutputFilenameBytes bounds an extracted filename; anything
29+
// longer is not treated as a candidate.
30+
const maxExplicitOutputFilenameBytes = 256
31+
32+
// explicitOutputFilenameRE matches an explicit output-naming cue followed by
33+
// a quoted, backticked, or unquoted filename. Bare "to" is deliberately not
34+
// a cue so input-file mentions ("add a border to photo.jpg") never match,
35+
// and bare "called" is not a cue because it usually references an existing
36+
// input file ("similar to the one called old-render.png"). Bare "as" only
37+
// counts inside a narrow imperative output context — a generation verb
38+
// directly followed by an optionally-articled media noun ("Generate an
39+
// image as sunshine.jpg"; explicitOutputFilenameOfPhraseRE below adds the
40+
// "of <subject>" variant of the same context) — because RE2 has no
41+
// lookbehind to exclude the comparative form ("in the same style as
42+
// sunshine.jpg", "the same background as assets/bg.png") any other way.
43+
// The unquoted form
44+
// additionally anchors on a known image extension so trailing punctuation
45+
// is excluded. Quoted/backticked contents may include spaces and are
46+
// validated separately by isExplicitImageFilename.
47+
var explicitOutputFilenameRE = regexp.MustCompile(
48+
`(?i)\b(?:(?:save\s+it\s+as|save\s+as|save\s+to|write\s+to|output\s+to|name\s+it|call\s+it)\s+|filename\s*[:=]\s*|` +
49+
`(?:re)?(?:generate|create|make|draw|render|produce)\s+(?:an?\s+|the\s+)?` +
50+
`(?:image|picture|photo|banner|logo|icon|graphic|drawing|illustration|thumbnail|sticker|avatar|gif)\s+as\s+)` +
51+
`(?:"([^"]+)"|'([^']+)'|` + "`([^`]+)`" + `|([^\s"'` + "`" + `]+\.(?:png|jpe?g|webp|gif))\b)`,
52+
)
53+
54+
// explicitOutputFilenameOfPhraseRE extends the imperative form above to a
55+
// media noun carrying an "of <subject>" description before the bare "as"
56+
// cue ("Generate an image of a red panda coding at a terminal as
57+
// assets/red-panda-terminal.jpg"). The subject cannot cross quotes,
58+
// clause punctuation, or CR/LF — so "Generate an image of a cat. Save it
59+
// as x.png" and the newline-separated equivalent stay a single
60+
// save-it-as candidate — and it is captured so extraction can reject
61+
// comparative phrasings inside it ("of a cat in the same style as
62+
// old.png", "of a beach with the exact palette as ref.png"; see
63+
// comparativeCueRE): RE2 has no lookbehind to express that exclusion in
64+
// the pattern itself. The subject MAY swallow a conjunction ahead of an
65+
// explicit cue ("of a wolf and save it as logo.png"); extraction
66+
// deduplicates that overlap with explicitOutputFilenameRE by capture
67+
// position.
68+
var explicitOutputFilenameOfPhraseRE = regexp.MustCompile(
69+
`(?i)\b(?:re)?(?:generate|create|make|draw|render|produce)\s+(?:an?\s+|the\s+)?` +
70+
`(?:image|picture|photo|banner|logo|icon|graphic|drawing|illustration|thumbnail|sticker|avatar|gif)\s+` +
71+
`(of\s+[^"'` + "`" + `.,;:!?\r\n]+?)\s+as\s+` +
72+
`(?:"([^"]+)"|'([^']+)'|` + "`([^`]+)`" + `|([^\s"'` + "`" + `]+\.(?:png|jpe?g|webp|gif))\b)`,
73+
)
74+
75+
// comparativeCueRE flags an of-phrase subject whose trailing "as" compares
76+
// against an existing file instead of naming the output ("of a cat in the
77+
// same style as old.png", "of something like ref.png"). Beyond explicit
78+
// comparative words, it conservatively rejects any subject containing
79+
// "as", "with", or "in": those introduce attribute phrases whose trailing
80+
// "as" compares ("with the exact palette as ref.png", "with identical
81+
// colors as ref.png", "as tall as tree.png") and comparative adjectives
82+
// are open-ended. The asymmetry justifies over-rejecting: a false reject
83+
// only costs the generic generated-N name, while a false capture can
84+
// engage the escape-confirmation policy for a merely referenced file.
85+
var comparativeCueRE = regexp.MustCompile(
86+
`(?i)\b(?:same|style|similar|like|such|as|with|in|exact|identical|matching|equivalent)\b`,
87+
)
88+
89+
// extractExplicitOutputFilename returns the single unambiguous explicit
90+
// output filename in the triggering user prompt, if there is exactly one.
91+
// Candidates that fail validation (unknown extension, control characters,
92+
// invalid UTF-8, over-long, extension-only, comparative of-phrase) are
93+
// ignored rather than counted. The two grammars overlap: an of-phrase
94+
// subject may swallow a conjunction ahead of an explicit cue ("make a
95+
// logo of a wolf and save it as logo.png"), so candidates are keyed by
96+
// the filename capture's position — the same occurrence matched by both
97+
// grammars counts once, while distinct occurrences (even of the same
98+
// name) stay ambiguous and yield no extraction.
99+
func extractExplicitOutputFilename(prompt string) (string, bool) {
100+
candidates := make(map[int]string)
101+
for _, m := range explicitOutputFilenameRE.FindAllStringSubmatchIndex(prompt, -1) {
102+
if offset, name := firstMatchedGroup(prompt, m, 1); isExplicitImageFilename(name) {
103+
candidates[offset] = name
104+
}
105+
}
106+
for _, m := range explicitOutputFilenameOfPhraseRE.FindAllStringSubmatchIndex(prompt, -1) {
107+
if comparativeCueRE.MatchString(prompt[m[2]:m[3]]) {
108+
continue
109+
}
110+
if offset, name := firstMatchedGroup(prompt, m, 2); isExplicitImageFilename(name) {
111+
candidates[offset] = name
112+
}
113+
}
114+
if len(candidates) != 1 {
115+
return "", false
116+
}
117+
for _, name := range candidates {
118+
return name, true
119+
}
120+
return "", false
121+
}
122+
123+
// firstMatchedGroup returns the start offset and text of the one capture
124+
// group the quoting alternation filled, scanning submatch index pairs from
125+
// firstGroup on; every group requires at least one character, so a filled
126+
// group has a non-negative start.
127+
func firstMatchedGroup(prompt string, m []int, firstGroup int) (int, string) {
128+
for g := firstGroup; 2*g+1 < len(m); g++ {
129+
if start, end := m[2*g], m[2*g+1]; start >= 0 {
130+
return start, prompt[start:end]
131+
}
132+
}
133+
return -1, ""
134+
}
135+
136+
// isExplicitImageFilename applies the same field constraints as the marker
137+
// grammar (valid UTF-8, no control characters, no edge whitespace, bounded)
138+
// plus a known image extension and a non-empty stem. Traversing or absolute
139+
// paths are valid candidates on purpose: the untrusted-path pipeline owns
140+
// containment and escape confirmation.
141+
func isExplicitImageFilename(name string) bool {
142+
if name == "" || len(name) > maxExplicitOutputFilenameBytes || !utf8.ValidString(name) {
143+
return false
144+
}
145+
if strings.TrimSpace(name) != name {
146+
return false
147+
}
148+
for _, r := range name {
149+
if r < 0x20 || r == 0x7f {
150+
return false
151+
}
152+
}
153+
switch strings.ToLower(path.Ext(name)) {
154+
case ".png", ".jpg", ".jpeg", ".webp", ".gif":
155+
default:
156+
return false
157+
}
158+
return len(path.Base(name)) > len(path.Ext(name))
159+
}
160+
161+
// applyUserPromptRequestedPath fills RequestedPath from the triggering user
162+
// prompt for a turn that returned exactly ONE media blob that marker pairing
163+
// left unnamed. Markers keep precedence (a non-empty RequestedPath is never
164+
// overwritten) and multi-blob turns are skipped entirely — a single prompt
165+
// filename cannot unambiguously name one blob among several.
166+
func applyUserPromptRequestedPath(media []chat.MediaDelta, prompt string) {
167+
if len(media) != 1 || media[0].RequestedPath != "" {
168+
return
169+
}
170+
if name, ok := extractExplicitOutputFilename(prompt); ok {
171+
media[0].RequestedPath = name
172+
}
173+
}

0 commit comments

Comments
 (0)