Skip to content

Commit 5e44994

Browse files
authored
feat(image_gen): multi-model FAL support with picker in hermes tools (NousResearch#11265)
* feat(image_gen): multi-model FAL support with picker in hermes tools Adds 8 FAL text-to-image models selectable via `hermes tools` → Image Generation → (FAL.ai | Nous Subscription) → model picker. Models supported: - fal-ai/flux-2/klein/9b (new default, <1s, $0.006/MP) - fal-ai/flux-2-pro (previous default, kept backward-compat upscaling) - fal-ai/z-image/turbo (Tongyi-MAI, bilingual EN/CN) - fal-ai/nano-banana (Gemini 2.5 Flash Image) - fal-ai/gpt-image-1.5 (with quality tier: low/medium/high) - fal-ai/ideogram/v3 (best typography) - fal-ai/recraft-v3 (vector, brand styles) - fal-ai/qwen-image (LLM-based) Architecture: - FAL_MODELS catalog declares per-model size family, defaults, supports whitelist, and upscale flag. Three size families handled uniformly: image_size_preset (flux family), aspect_ratio (nano-banana), and gpt_literal (gpt-image-1.5). - _build_fal_payload() translates unified inputs (prompt + aspect_ratio) into model-specific payloads, merges defaults, applies caller overrides, wires GPT quality_setting, then filters to the supports whitelist — so models never receive rejected keys. - IMAGEGEN_BACKENDS registry in tools_config prepares for future imagegen providers (Replicate, Stability, etc.); each provider entry tags itself with imagegen_backend: 'fal' to select the right catalog. - Upscaler (Clarity) defaults off for new models (preserves <1s value prop), on for flux-2-pro (backward-compat). Per-model via FAL_MODELS. Config: image_gen.model = fal-ai/flux-2/klein/9b (new) image_gen.quality_setting = medium (new, GPT only) image_gen.use_gateway = bool (existing) Agent-facing schema unchanged (prompt + aspect_ratio only) — model choice is a user-level config decision, not an agent-level arg. Picker uses curses_radiolist (arrow keys, auto numbered-fallback on non-TTY). Column-aligned: Model / Speed / Strengths / Price. Docs: image-generation.md rewritten with the model table and picker walkthrough. tools-reference, tool-gateway, overview updated to drop the stale "FLUX 2 Pro" wording. Tests: 42 new in tests/tools/test_image_generation.py covering catalog integrity, all 3 size families, supports filter, default merging, GPT quality wiring, model resolution fallback. 8 new in tests/hermes_cli/test_tools_config.py for picker wiring (registry, config writes, GPT quality follow-up prompt, corrupt-config repair). * feat(image_gen): translate managed-gateway 4xx to actionable error When the Nous Subscription managed FAL proxy rejects a model with 4xx (likely portal-side allowlist miss or billing gate), surface a clear message explaining: 1. The rejected model ID + HTTP status 2. Two remediation paths: set FAL_KEY for direct access, or pick a different model via `hermes tools` 5xx, connection errors, and direct-FAL errors pass through unchanged (those have different root causes and reasonable native messages). Motivation: new FAL models added to this release (flux-2-klein-9b, z-image-turbo, nano-banana, gpt-image-1.5, ideogram-v3, recraft-v3, qwen-image) are untested against the Nous Portal proxy. If the portal allowlists model IDs, users on Nous Subscription will hit cryptic 4xx errors without guidance on how to work around it. Tests: 8 new cases covering status extraction across httpx/fal error shapes and 4xx-vs-5xx-vs-ConnectionError translation policy. Docs: brief note in image-generation.md for Nous subscribers. Operator action (Nous Portal side): verify that fal-queue-gateway passes through these 7 new FAL model IDs. If the proxy has an allowlist, add them; otherwise Nous Subscription users will see the new translated error and fall back to direct FAL. * feat(image_gen): pin GPT-Image quality to medium (no user choice) Previously the tools picker asked a follow-up question for GPT-Image quality tier (low / medium / high) and persisted the answer to `image_gen.quality_setting`. This created two problems: 1. Nous Portal billing complexity — the 22x cost spread between tiers ($0.009 low / $0.20 high) forces the gateway to meter per-tier per user, which the portal team can't easily support at launch. 2. User footgun — anyone picking `high` by mistake burns through credit ~6x faster than `medium`. This commit pins quality at medium by baking it into FAL_MODELS defaults for gpt-image-1.5 and removes all user-facing override paths: - Removed `_resolve_gpt_quality()` runtime lookup - Removed `honors_quality_setting` flag on the model entry - Removed `_configure_gpt_quality_setting()` picker helper - Removed `_GPT_QUALITY_CHOICES` constant - Removed the follow-up prompt call in `_configure_imagegen_model()` - Even if a user manually edits `image_gen.quality_setting` in config.yaml, no code path reads it — always sends medium. Tests: - Replaced TestGptQualitySetting (6 tests) with TestGptQualityPinnedToMedium (5 tests) — proves medium is baked in, config is ignored, flag is removed, helper is removed, non-gpt models never get quality. - Replaced test_picker_with_gpt_image_also_prompts_quality with test_picker_with_gpt_image_does_not_prompt_quality — proves only 1 picker call fires when gpt-image is selected (no quality follow-up). Docs updated: image-generation.md replaces the quality-tier table with a short note explaining the pinning decision. * docs(image_gen): drop stale 'wires GPT quality tier' line from internals section Caught in a cleanup sweep after pinning quality to medium. The "How It Works Internally" walkthrough still described the removed quality-wiring step.
1 parent a55f2e7 commit 5e44994

8 files changed

Lines changed: 1251 additions & 474 deletions

File tree

hermes_cli/tools_config.py

Lines changed: 120 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -258,14 +258,16 @@ def _get_plugin_toolset_keys() -> set:
258258
"requires_nous_auth": True,
259259
"managed_nous_feature": "image_gen",
260260
"override_env_vars": ["FAL_KEY"],
261+
"imagegen_backend": "fal",
261262
},
262263
{
263264
"name": "FAL.ai",
264265
"badge": "paid",
265-
"tag": "FLUX 2 Pro with auto-upscaling",
266+
"tag": "Pick from flux-2-klein, flux-2-pro, gpt-image, nano-banana, etc.",
266267
"env_vars": [
267268
{"key": "FAL_KEY", "prompt": "FAL API key", "url": "https://fal.ai/dashboard/keys"},
268269
],
270+
"imagegen_backend": "fal",
269271
},
270272
],
271273
},
@@ -950,6 +952,106 @@ def _detect_active_provider_index(providers: list, config: dict) -> int:
950952
return 0
951953

952954

955+
# ─── Image Generation Model Pickers ───────────────────────────────────────────
956+
#
957+
# IMAGEGEN_BACKENDS is a per-backend catalog. Each entry exposes:
958+
# - config_key: top-level config.yaml key for this backend's settings
959+
# - model_catalog_fn: returns an OrderedDict-like {model_id: metadata}
960+
# - default_model: fallback when nothing is configured
961+
#
962+
# This prepares for future imagegen backends (Replicate, Stability, etc.):
963+
# each new backend registers its own entry; the FAL provider entry in
964+
# TOOL_CATEGORIES tags itself with `imagegen_backend: "fal"` to select the
965+
# right catalog at picker time.
966+
967+
968+
def _fal_model_catalog():
969+
"""Lazy-load the FAL model catalog from the tool module."""
970+
from tools.image_generation_tool import FAL_MODELS, DEFAULT_MODEL
971+
return FAL_MODELS, DEFAULT_MODEL
972+
973+
974+
IMAGEGEN_BACKENDS = {
975+
"fal": {
976+
"display": "FAL.ai",
977+
"config_key": "image_gen",
978+
"catalog_fn": _fal_model_catalog,
979+
},
980+
}
981+
982+
983+
def _format_imagegen_model_row(model_id: str, meta: dict, widths: dict) -> str:
984+
"""Format a single picker row with column-aligned speed / strengths / price."""
985+
return (
986+
f"{model_id:<{widths['model']}} "
987+
f"{meta.get('speed', ''):<{widths['speed']}} "
988+
f"{meta.get('strengths', ''):<{widths['strengths']}} "
989+
f"{meta.get('price', '')}"
990+
)
991+
992+
993+
def _configure_imagegen_model(backend_name: str, config: dict) -> None:
994+
"""Prompt the user to pick a model for the given imagegen backend.
995+
996+
Writes selection to ``config[backend_config_key]["model"]``. Safe to
997+
call even when stdin is not a TTY — curses_radiolist falls back to
998+
keeping the current selection.
999+
"""
1000+
backend = IMAGEGEN_BACKENDS.get(backend_name)
1001+
if not backend:
1002+
return
1003+
1004+
catalog, default_model = backend["catalog_fn"]()
1005+
if not catalog:
1006+
return
1007+
1008+
cfg_key = backend["config_key"]
1009+
cur_cfg = config.setdefault(cfg_key, {})
1010+
if not isinstance(cur_cfg, dict):
1011+
cur_cfg = {}
1012+
config[cfg_key] = cur_cfg
1013+
current_model = cur_cfg.get("model") or default_model
1014+
if current_model not in catalog:
1015+
current_model = default_model
1016+
1017+
model_ids = list(catalog.keys())
1018+
# Put current model at the top so the cursor lands on it by default.
1019+
ordered = [current_model] + [m for m in model_ids if m != current_model]
1020+
1021+
# Column widths
1022+
widths = {
1023+
"model": max(len(m) for m in model_ids),
1024+
"speed": max((len(catalog[m].get("speed", "")) for m in model_ids), default=6),
1025+
"strengths": max((len(catalog[m].get("strengths", "")) for m in model_ids), default=0),
1026+
}
1027+
1028+
print()
1029+
header = (
1030+
f" {'Model':<{widths['model']}} "
1031+
f"{'Speed':<{widths['speed']}} "
1032+
f"{'Strengths':<{widths['strengths']}} "
1033+
f"Price"
1034+
)
1035+
print(color(header, Colors.CYAN))
1036+
1037+
rows = []
1038+
for mid in ordered:
1039+
row = _format_imagegen_model_row(mid, catalog[mid], widths)
1040+
if mid == current_model:
1041+
row += " ← currently in use"
1042+
rows.append(row)
1043+
1044+
idx = _prompt_choice(
1045+
f" Choose {backend['display']} model:",
1046+
rows,
1047+
default=0,
1048+
)
1049+
1050+
chosen = ordered[idx]
1051+
cur_cfg["model"] = chosen
1052+
_print_success(f" Model set to: {chosen}")
1053+
1054+
9531055
def _configure_provider(provider: dict, config: dict):
9541056
"""Configure a single provider - prompt for API keys and set config."""
9551057
env_vars = provider.get("env_vars", [])
@@ -1006,6 +1108,10 @@ def _configure_provider(provider: dict, config: dict):
10061108
_print_success(f" {provider['name']} - no configuration needed!")
10071109
if managed_feature:
10081110
_print_info(" Requests for this tool will be billed to your Nous subscription.")
1111+
# Imagegen backends prompt for model selection after backend pick.
1112+
backend = provider.get("imagegen_backend")
1113+
if backend:
1114+
_configure_imagegen_model(backend, config)
10091115
return
10101116

10111117
# Prompt for each required env var
@@ -1040,6 +1146,10 @@ def _configure_provider(provider: dict, config: dict):
10401146

10411147
if all_configured:
10421148
_print_success(f" {provider['name']} configured!")
1149+
# Imagegen backends prompt for model selection after env vars are in.
1150+
backend = provider.get("imagegen_backend")
1151+
if backend:
1152+
_configure_imagegen_model(backend, config)
10431153

10441154

10451155
def _configure_simple_requirements(ts_key: str):
@@ -1211,6 +1321,10 @@ def _reconfigure_provider(provider: dict, config: dict):
12111321
_print_success(f" {provider['name']} - no configuration needed!")
12121322
if managed_feature:
12131323
_print_info(" Requests for this tool will be billed to your Nous subscription.")
1324+
# Imagegen backends prompt for model selection on reconfig too.
1325+
backend = provider.get("imagegen_backend")
1326+
if backend:
1327+
_configure_imagegen_model(backend, config)
12141328
return
12151329

12161330
for var in env_vars:
@@ -1228,6 +1342,11 @@ def _reconfigure_provider(provider: dict, config: dict):
12281342
else:
12291343
_print_info(" Kept current")
12301344

1345+
# Imagegen backends prompt for model selection on reconfig too.
1346+
backend = provider.get("imagegen_backend")
1347+
if backend:
1348+
_configure_imagegen_model(backend, config)
1349+
12311350

12321351
def _reconfigure_simple_requirements(ts_key: str):
12331352
"""Reconfigure simple env var requirements."""

tests/hermes_cli/test_tools_config.py

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -466,3 +466,90 @@ def test_numeric_mcp_server_name_does_not_crash_sorted():
466466

467467
# sorted() must not raise TypeError
468468
sorted(enabled)
469+
470+
471+
# ─── Imagegen Backend Picker Wiring ────────────────────────────────────────
472+
473+
class TestImagegenBackendRegistry:
474+
"""IMAGEGEN_BACKENDS tags drive the model picker flow in tools_config."""
475+
476+
def test_fal_backend_registered(self):
477+
from hermes_cli.tools_config import IMAGEGEN_BACKENDS
478+
assert "fal" in IMAGEGEN_BACKENDS
479+
480+
def test_fal_catalog_loads_lazily(self):
481+
"""catalog_fn should defer import to avoid import cycles."""
482+
from hermes_cli.tools_config import IMAGEGEN_BACKENDS
483+
catalog, default = IMAGEGEN_BACKENDS["fal"]["catalog_fn"]()
484+
assert default == "fal-ai/flux-2/klein/9b"
485+
assert "fal-ai/flux-2/klein/9b" in catalog
486+
assert "fal-ai/flux-2-pro" in catalog
487+
488+
def test_image_gen_providers_tagged_with_fal_backend(self):
489+
"""Both Nous Subscription and FAL.ai providers must carry the
490+
imagegen_backend tag so _configure_provider fires the picker."""
491+
from hermes_cli.tools_config import TOOL_CATEGORIES
492+
providers = TOOL_CATEGORIES["image_gen"]["providers"]
493+
for p in providers:
494+
assert p.get("imagegen_backend") == "fal", (
495+
f"{p['name']} missing imagegen_backend tag"
496+
)
497+
498+
499+
class TestImagegenModelPicker:
500+
"""_configure_imagegen_model writes selection to config and respects
501+
curses fallback semantics (returns default when stdin isn't a TTY)."""
502+
503+
def test_picker_writes_chosen_model_to_config(self):
504+
from hermes_cli.tools_config import _configure_imagegen_model
505+
config = {}
506+
# Force _prompt_choice to pick index 1 (second-in-ordered-list).
507+
with patch("hermes_cli.tools_config._prompt_choice", return_value=1):
508+
_configure_imagegen_model("fal", config)
509+
# ordered[0] == current (default klein), ordered[1] == first non-default
510+
assert config["image_gen"]["model"] != "fal-ai/flux-2/klein/9b"
511+
assert config["image_gen"]["model"].startswith("fal-ai/")
512+
513+
def test_picker_with_gpt_image_does_not_prompt_quality(self):
514+
"""GPT-Image quality is pinned to medium in the tool's defaults —
515+
no follow-up prompt, no config write for quality_setting."""
516+
from hermes_cli.tools_config import (
517+
_configure_imagegen_model,
518+
IMAGEGEN_BACKENDS,
519+
)
520+
catalog, default_model = IMAGEGEN_BACKENDS["fal"]["catalog_fn"]()
521+
model_ids = list(catalog.keys())
522+
ordered = [default_model] + [m for m in model_ids if m != default_model]
523+
gpt_idx = ordered.index("fal-ai/gpt-image-1.5")
524+
525+
# Only ONE picker call is expected (for model) — not two (model + quality).
526+
call_count = {"n": 0}
527+
def fake_prompt(*a, **kw):
528+
call_count["n"] += 1
529+
return gpt_idx
530+
531+
config = {}
532+
with patch("hermes_cli.tools_config._prompt_choice", side_effect=fake_prompt):
533+
_configure_imagegen_model("fal", config)
534+
535+
assert call_count["n"] == 1, (
536+
f"Expected 1 picker call (model only), got {call_count['n']}"
537+
)
538+
assert config["image_gen"]["model"] == "fal-ai/gpt-image-1.5"
539+
assert "quality_setting" not in config["image_gen"]
540+
541+
def test_picker_no_op_for_unknown_backend(self):
542+
from hermes_cli.tools_config import _configure_imagegen_model
543+
config = {}
544+
_configure_imagegen_model("nonexistent-backend", config)
545+
assert config == {} # untouched
546+
547+
def test_picker_repairs_corrupt_config_section(self):
548+
"""When image_gen is a non-dict (user-edit YAML), the picker should
549+
replace it with a fresh dict rather than crash."""
550+
from hermes_cli.tools_config import _configure_imagegen_model
551+
config = {"image_gen": "some-garbage-string"}
552+
with patch("hermes_cli.tools_config._prompt_choice", return_value=0):
553+
_configure_imagegen_model("fal", config)
554+
assert isinstance(config["image_gen"], dict)
555+
assert config["image_gen"]["model"] == "fal-ai/flux-2/klein/9b"

0 commit comments

Comments
 (0)