Skip to content

fix(dmr): detect and use locally-installed Docker Model Runner models#3206

Open
Sayt-0 wants to merge 1 commit into
mainfrom
fix/dmr-local-model-detection
Open

fix(dmr): detect and use locally-installed Docker Model Runner models#3206
Sayt-0 wants to merge 1 commit into
mainfrom
fix/dmr-local-model-detection

Conversation

@Sayt-0

@Sayt-0 Sayt-0 commented Jun 22, 2026

Copy link
Copy Markdown
Member

Summary

Fixes #2799. Docker Model Runner (DMR) models pulled locally were never enumerated, causing two reported problems:

Symptom Cause Handled by
Declining the ai/qwen3:latest pull showed "No model providers available." although DMR had other models Auto-selection hard-coded the DMR provider to DefaultModels["dmr"] (ai/qwen3:latest); declining the pull failed selection Auto-selection now prefers an already-pulled model
ctrl+m showed "No models available for selection" although DMR had models The picker only listed configured models plus the models.dev catalog; DMR locals are not in the catalog DMR models are now discovered and listed

Changes

Area File Change
DMR discovery pkg/model/provider/dmr/list.go (new) ListModels queries DMR's OpenAI /models endpoint, reusing the same endpoint/baseURL/httpClient resolution as the inference client (handles MODEL_RUNNER_HOST and the Docker Desktop unix-socket transport)
Auto-selection pkg/config/auto.go AutoModelConfig takes a DMRModelLister; pickDMRAutoModel prefers the configured default (including under a non-default tag), else the first non-embedding installed model, else the static default
Fallback error pkg/config/auto.go, pkg/teamloader/teamloader.go AutoModelFallbackError carries the cause (e.g. a declined pull), unwraps it, and suggests docker model pull
Model picker pkg/runtime/dmr_models.go (new), model_switcher.go buildDMRChoices mirrors the gateway discovery path (TTL cache + singleflight), filters embedding models, dedupes against configured models, and groups under "Other models"
Wiring runtime.go, teamloader.go, cmd/root/models.go Lister defaulted in NewLocalRuntime; the run path passes dmr.ListModels; docker agent models passes nil to stay side-effect-free

Behavior

  • DMR model IDs contain slashes (ai/qwen3:latest); the picker ref is dmr/<id> and round-trips through ParseModelRef (first-slash split) back to {provider: dmr, model: ai/qwen3:latest}, so selecting one does not re-trigger a pull.
  • Runtimes constructed directly (tests) get a nil lister and therefore no DMR entries, keeping existing picker assertions deterministic.
  • When no model is available at all, the message now names the underlying cause and points at docker model pull.

Tests

New coverage (35 cases): DMR /models parsing and the exported ListModels entry point (MODEL_RUNNER_HOST + unreachable), auto-selection across installed/default/tagged/embedding-only/error cases, the cause-carrying fallback error, and buildDMRChoices including catalog-family embedding filtering, dedup, and cache TTL/failure behavior.

go build ./..., go vet, gofmt, and golangci-lint are clean on the changed packages.

Not included

The enhancement request for LM Studio and other local endpoints is left as a separate follow-up; this PR is scoped to the two reported DMR defects.

Auto-selection hard-coded the DMR provider to ai/qwen3:latest, so a user
with other models pulled in Docker Model Runner was prompted to pull
qwen3 and, on declining, saw the misleading "No model providers
available." The ctrl+m model picker likewise showed nothing for DMR
because local models are not part of the models.dev catalog.

Add dmr.ListModels (querying DMR's OpenAI /models endpoint) and use it to:
  - prefer an already-pulled model during auto-selection (the configured
    default when present, including under a non-default tag, otherwise the
    first non-embedding installed model)
  - populate DMR entries in the model picker (cached, mirroring the gateway
    discovery path)

Also carry the underlying cause into AutoModelFallbackError and suggest
pulling a model, so the residual no-model case is actionable.

Fixes #2799
@Sayt-0 Sayt-0 requested a review from a team as a code owner June 22, 2026 20:06

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

Two issues found in the new DMR discovery code. The medium-severity singleflight/context bug can suppress DMR model listings for all users for up to one minute after a single cancelled request; the low-severity embedding filter gap is a latent defect in the repo-prefix auto-selection fallback.

Comment thread pkg/runtime/dmr_models.go
return ids, err
}

ids, err := r.dmrModelLister(ctx)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Singleflight closure captures caller's context — a cancelled picker open poisons the cache for ~1 minute

listDMRModels passes the first caller's ctx directly to r.dmrModelLister:

ids, err := r.dmrModelLister(ctx)   // ctx from the *first* sf.Do caller
c.mu.Lock()
c.ids, c.err, c.fetchedAt = ids, err, now()   // error is cached here
c.mu.Unlock()
return ids, err

Two compounding problems arise from this design:

  1. Spurious errors for concurrent waiters. singleflight.Do broadcasts the first caller's result to every goroutine waiting on the same key. If the first caller's context is cancelled (e.g. the user closes the model picker mid-request), dmrModelLister returns context.Canceled. All concurrent waiters receive that error even though their own contexts are still alive.

  2. Error is cached for the full TTL. The code sets c.fetchedAt = now() unconditionally, so the context.Canceled result is stored and replayed to every subsequent caller for up to one minute (the dmrModelsTTL). During that window DMR models disappear from the picker for all users — recoverable only by waiting out the TTL.

Suggested fix: Use a context that is not tied to any single caller's lifetime inside the singleflight body. The simplest approach is context.WithoutCancel(ctx) (Go 1.21+), which propagates values but detaches cancellation:

v, err, _ := c.sf.Do("models", func() (any, error) {
    if ids, ok, err := readFresh(); ok {
        return ids, err
    }
    ids, err := r.dmrModelLister(context.WithoutCancel(ctx))
    // ...
})

Additionally, consider not caching errors (or using a shorter error TTL) so a transient failure or cancellation doesn't block discovery for a full minute.

Comment thread pkg/config/auto.go
// still satisfies "prefer the default", so match on the repository.
defaultRepo := dmrModelRepo(defaultModel)
for _, m := range installed {
if dmrModelRepo(m) == defaultRepo {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[LOW] Repo-prefix fallback in pickDMRAutoModel skips the embedding-model filter

The general fallback loop (used when no exact tag match is found) correctly calls looksLikeEmbeddingModel:

for _, m := range installed {
    if !looksLikeEmbeddingModel(m) {
        return m   // safe: chat models only
    }
}

But the repo-prefix loop that runs first does not apply the same guard:

defaultRepo := dmrModelRepo(defaultModel)
for _, m := range installed {
    if dmrModelRepo(m) == defaultRepo {
        return m   // no embedding check here
    }
}

If an embedding-only variant that shares the same repository prefix as the default model is installed (e.g. a future ai/qwen3-embed:latest published under the ai/qwen3 namespace), auto-selection could return it as the chat model. The agent would then start and fail when it tries to use a text-embedding model for conversation.

The fix is a one-liner:

for _, m := range installed {
    if dmrModelRepo(m) == defaultRepo && !looksLikeEmbeddingModel(m) {
        return m
    }
}

@aheritier aheritier added area/providers/docker-model-runner Docker Model Runner (DMR) local inference kind/fix PR fixes a bug (maps to fix: commit prefix) labels Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/docker-model-runner Docker Model Runner (DMR) local inference kind/fix PR fixes a bug (maps to fix: commit prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No model providers found - but Docker Model Runner already setup with models

3 participants