Skip to content

Add KeyValueResponseCacheStore for distributed client response caching - #4479

Merged
jlowin merged 3 commits into
mainfrom
feature/client-cache-kv-store
Jul 17, 2026
Merged

Add KeyValueResponseCacheStore for distributed client response caching#4479
jlowin merged 3 commits into
mainfrom
feature/client-cache-kv-store

Conversation

@jlowin

@jlowin jlowin commented Jul 9, 2026

Copy link
Copy Markdown
Member

#4464 gave servers cache hints and #4450 taught the client to honor them — but the client's response cache is a per-process in-memory LRU, so a fleet of FastMCP proxy replicas can't share cache fills: every replica pays its own wire calls to warm the same listings. This adds KeyValueResponseCacheStore, a ResponseCacheStore adapter over the same AsyncKeyValue abstraction FastMCP already uses for its event store and OAuth state, so any py-key-value backend (Redis, memory, filetree) can back the SDK's client cache:

from fastmcp import Client
from fastmcp.client.caching import KeyValueResponseCacheStore
from mcp.client.caching import CacheConfig
from key_value.aio.stores.redis import RedisStore

store = KeyValueResponseCacheStore(storage=RedisStore(url="redis://localhost"))
config = CacheConfig(store=store, partition="tenant-a", target_id="weather-api")
client = Client("https://example.com/mcp", mode="auto", cache=config)

Two clients sharing one backend serve each other's fills: the included distributed test has client A populate tools/list and client B serve it with zero wire calls, while a distinct partition correctly misses. Stored values round-trip through a type-tagged envelope validated against an allowlist derived from the SDK's own cacheable-result registry — a forged or unknown tag in a shared store is a cache miss, never an import-by-name — and every CacheKey field (method, params, partition, negotiated protocol, server arm) is folded into the storage key so entries can't collide across tenants or protocol eras. clear() is scoped to the adapter's own collection namespace and never touches other tenants.

No Client signature changes — CacheConfig(store=...) already flows through from #4450. The dependency is already core.

Label: enhancements

@jlowin jlowin added the v4 label Jul 9, 2026
@marvin-context-protocol marvin-context-protocol Bot added enhancement Improvement to existing functionality. For issues and smaller PR improvements. client Related to the FastMCP client SDK or client-side functionality. labels Jul 9, 2026
@jlowin
jlowin merged commit 836ceac into main Jul 17, 2026
22 checks passed
@jlowin
jlowin deleted the feature/client-cache-kv-store branch July 17, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client Related to the FastMCP client SDK or client-side functionality. enhancement Improvement to existing functionality. For issues and smaller PR improvements. v4

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant