Skip to content

Commit b1d7eca

Browse files
authored
docs: add Xinference deployment route (#3117)
Add Xinference to the English and Chinese deployment matrix after xorbitsai/inference#5140 merged Fun-ASR-Nano model spec support.
1 parent 1b7c1b4 commit b1d7eca

2 files changed

Lines changed: 16 additions & 0 deletions

File tree

docs/deployment_matrix.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
99
| Colab notebook | Browser smoke tests, first evaluation, shareable demos | [Colab quickstart](../examples/colab/) | No local setup; first run downloads model files, GPU runtime is faster. |
1010
| Python API | Notebooks, offline jobs, first model evaluation | [README quick start](../README.md#quick-start) | Lowest ceremony; caller owns batching, retries, and files. |
1111
| OpenAI-compatible API | Private speech API, agents, Dify/LangChain/AutoGen-style clients | [OpenAI API example](../examples/openai_api/) | Easiest integration for apps that already support OpenAI audio APIs. |
12+
| Xinference | Teams that already operate Xinference model serving | [Xinference repository](https://github.com/xorbitsai/inference) | Use a Xinference version containing [xorbitsai/inference#5140](https://github.com/xorbitsai/inference/pull/5140) so Fun-ASR-Nano uses packaged `funasr~=1.3.0` instead of the old pinned git commit. |
1213
| Docker Compose API | Reproducible local smoke test or small internal service | [OpenAI API Docker docs](../examples/openai_api/#docker-deployment) | CPU by default; adapt the image before using CUDA in containers. |
1314
| Kubernetes API | Internal speech API for cluster services | [Kubernetes template](../examples/openai_api/kubernetes/) | Starts as private `ClusterIP`; add auth, TLS, network policy, and GPU scheduling before broader exposure. |
1415
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
@@ -28,6 +29,16 @@ Use the [Colab quickstart](../examples/colab/) when you want a browser-only smok
2829

2930
Use the OpenAI-compatible API. It exposes `/v1/audio/transcriptions`, `/v1/models`, `/health`, and Swagger docs. Start with `sensevoice`, run `examples/openai_api/smoke_test.sh` or `examples/openai_api/smoke_test.py`, then connect existing SDK or HTTP clients using [client recipes](../examples/openai_api/CLIENTS.md) and [JavaScript/TypeScript recipes](../examples/openai_api/JAVASCRIPT.md). For browser upload or microphone demos, use the [Gradio browser demo](../examples/openai_api/GRADIO.md). For Dify, n8n, HTTP nodes, or webhook workers, follow the [workflow recipes](../examples/openai_api/WORKFLOWS.md). For API gateways, developer portals, and schema-driven imports, use the [OpenAPI spec](../examples/openai_api/OPENAPI.md). Before sharing the service, review the [security and gateway guide](../examples/openai_api/SECURITY.md).
3031

32+
### I already run Xinference
33+
34+
Use Xinference when your stack already standardizes on its model registry,
35+
virtualenv isolation, and serving lifecycle. Make sure your Xinference build
36+
includes [xorbitsai/inference#5140](https://github.com/xorbitsai/inference/pull/5140);
37+
that update moved the Fun-ASR-Nano model specs from an old FunASR git SHA to the
38+
packaged `funasr~=1.3.0` dependency line. For first-time FunASR evaluation or
39+
agent-oriented OpenAI-compatible transcription, start with the native FunASR
40+
OpenAI API example above instead.
41+
3142
### I want a repeatable container demo
3243

3344
Use `examples/openai_api/docker-compose.yml` for a CPU-mode smoke test:

docs/deployment_matrix_zh.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
| Colab Notebook | 浏览器 smoke test、首次评估、可分享 demo | [Colab 快速体验](../examples/colab/README_zh.md) | 不需要本地环境;首次运行会下载模型,GPU runtime 更快。 |
1010
| Python API | Notebook、离线任务、首次模型评测 | [README 快速开始](../README_zh.md#快速开始) | 最简单;调用方自己负责批处理、重试和文件管理。 |
1111
| OpenAI 兼容 API | 私有语音 API、Agent、Dify/LangChain/AutoGen 风格客户端 | [OpenAI API 示例](../examples/openai_api/README_zh.md) | 已支持 OpenAI audio API 的应用最容易接入。 |
12+
| Xinference | 已经使用 Xinference 统一管理模型服务的团队 | [Xinference 仓库](https://github.com/xorbitsai/inference) | 使用包含 [xorbitsai/inference#5140](https://github.com/xorbitsai/inference/pull/5140) 的版本或 commit,确保 Fun-ASR-Nano 使用打包发布的 `funasr~=1.3.0`,而不是旧的 git commit pin。 |
1213
| Docker Compose API | 可复现本地 smoke test 或小型内部服务 | [OpenAI API Docker 文档](../examples/openai_api/README_zh.md) | 默认 CPU;容器里使用 CUDA 前需要先适配 CUDA-capable 镜像。 |
1314
| Kubernetes API | 集群内私有语音 API | [Kubernetes 模板](../examples/openai_api/kubernetes/README_zh.md) | 默认私有 `ClusterIP`;对外开放前补齐鉴权、TLS、网络策略和 GPU 调度。 |
1415
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
@@ -28,6 +29,10 @@
2829

2930
使用 OpenAI 兼容 API。它提供 `/v1/audio/transcriptions``/v1/models``/health` 和 Swagger docs。先用 `sensevoice` 跑通 `examples/openai_api/smoke_test.sh``examples/openai_api/smoke_test.py`,再根据 [客户端配方](../examples/openai_api/CLIENTS.md)[JavaScript/TypeScript 配方](../examples/openai_api/JAVASCRIPT_zh.md) 接入 SDK 或 HTTP 客户端。浏览器上传或麦克风 demo 可使用 [Gradio 浏览器 Demo](../examples/openai_api/GRADIO_zh.md)。Dify、n8n、HTTP 节点或 webhook worker 可参考 [工作流配方](../examples/openai_api/WORKFLOWS_zh.md)。API 网关、开发者门户或按 schema 导入时可使用 [OpenAPI 规范](../examples/openai_api/OPENAPI_zh.md)。跨团队共享服务前,请先阅读 [安全与网关指南](../examples/openai_api/SECURITY_zh.md)
3031

32+
### 我已经在使用 Xinference
33+
34+
如果你的系统已经用 Xinference 管理模型注册、virtualenv 隔离和服务生命周期,可以选择 Xinference 路径。请确认使用的 Xinference 版本或 commit 包含 [xorbitsai/inference#5140](https://github.com/xorbitsai/inference/pull/5140);该更新把 Fun-ASR-Nano model spec 从旧的 FunASR git SHA 改为打包发布的 `funasr~=1.3.0` 依赖。首次评估 FunASR,或需要面向 Agent 的 OpenAI 兼容转写接口时,仍建议先从上面的 FunASR 原生 OpenAI API 示例开始。
35+
3136
### 我想要可复现的容器 demo
3237

3338
使用 `examples/openai_api/docker-compose.yml` 跑 CPU smoke test:

0 commit comments

Comments
 (0)