Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 26 additions & 12 deletions docs/volcengine-setup.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,33 @@
# 火山引擎(volcengine)ASR 配置
# 火山引擎(Volcengine)配置

状态:canonical(2026-09-07 以源码为准重写);更新:2026-09-07。
## ASR 服务与鉴权

## 1. 代码中的定义
设置 → AI 服务 → 语音识别 → 添加渠道,选择火山引擎。服务选择与普通服务的鉴权模式独立,配置和密钥按渠道保存至系统凭据存储。

- Provider:`volcengine`(labelKey `asrVolcengine`),定义于 Core `provider_rules.rs`;`authRequirement = Volcengine`(专用鉴权形态),无内置默认端点/模型(`defaultEndpoint` / `defaultModel` 为空,按通道配置)。
- 验证探针:`asr_silence_allows_no_final`(静音段允许无 final 帧,验证以可取消的静音探测完成)。
- 凭据字段(`provider_rules.rs:300-302`):`volcengine_auth_mode`(鉴权模式,如 ApiKey/官方端点模式)、`volcengine_app_key`、`volcengine_access_key`(布尔项 + 模式选择;具体取值在设置界面录入,凭据走系统安全存储,不落明文)。
| 服务 | 鉴权 | WebSocket 端点 |
| --- | --- | --- |
| 普通服务 | APP ID + Access Token,或普通语音控制台 API Key | `wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async` |
| Agent Plan | Agent Plan 专属 API Key,不需要 APP ID | `wss://openspeech.bytedance.com/api/v3/plan/sauc/bigmodel_async` |

## 2. 在应用内配置
未包含服务字段的旧配置继续使用普通服务。Agent Plan 自动使用 API Key 鉴权,切回普通服务后保留原有鉴权模式;切换服务不会删除已保存的密钥。不同服务使用不同密钥,建议分别创建渠道。Coding Plan 没有 ASR 服务选项。

设置 → AI 服务 → 语音识别 → 添加渠道,选择火山引擎;按界面提示填入鉴权字段,保存后执行“验证”得到真实验证结果(成功/失败与时间会记录在渠道列表)
Resource ID 留空时使用 `volc.seedasr.sauc.duration`;Agent Plan 豆包流式 ASR 使用此资源。服务选择保存在 `volcengine.service`(`standard` / `agent_plan`),由 Core 的同一配置解析和连接路径用于验证、听写及其他 ASR 入口。未知服务值报错,不回退到普通计费端点

## 3. 端点与排错
“验证”发送可取消的静音探针,允许没有最终识别结果;连接验证成功后,还应通过实际录音检查转写及插入。连接日志包含端点、连接/请求 ID 和服务端 Log ID,不包含鉴权头。

- ApiKey 模式使用火山官方实时 ASR 端点(历史修复 #931 后的行为,以 `crates/openless-core/src/asr/volcengine.rs` 当前实现为准)。
- 弱网行为:连接超时与重试在 Host/Core 实现,失败信息展示在渠道验证结果中。
- 开通服务、创建应用与获取密钥属火山控制台操作,以[火山官方文档](https://www.volcengine.com/docs)为准;本仓库只维护代码行为。
官方依据:[Agent Plan 接入语音模型](https://docs.volcengine.com/docs/82379/2516286?lang=zh)、[普通流式语音识别](https://www.volcengine.com/docs/6561/1354869?lang=zh)。

## Ark 语言模型套餐

设置 → AI 服务 → 文本润色 → 添加火山方舟渠道,使用套餐专属 API Key 和对应 Endpoint:

- Agent Plan:`https://ark.cn-beijing.volces.com/api/plan/v3`
- Coding Plan:`https://ark.cn-beijing.volces.com/api/coding/v3`

填写控制台显示的**文本生成模型名称**,或使用 `ark-code-latest` 并在控制台选择其对应文本模型,再执行“验证”。不要将图片、视频或向量化模型用于文本润色;列表中的 ID 不代表该套餐均可调用,实际能力以控制台及连接验证为准。

模型列表与推理验证相互独立。目录请求返回 404 时,应用提示当前 Endpoint 无法提供列表,保留已有模型和手填入口,不将目录缺失判定为 API Key 无效,也不会替换 Endpoint。Coding Plan 的现有 `/models` 路径继续使用。

Agent Plan 的官方目录接口 [ListArkAgentPlanModel](https://api.volcengine.com/api-docs/view?action=ListArkAgentPlanModel&version=2024-01-01&serviceCode=ark) 使用独立签名鉴权,应用没有接入这一管控接口;它与推理 Key 的 `/models` 请求不同。手填模型配置见[官方快速开始](https://docs.volcengine.com/docs/82379/2373738?lang=zh)。

润色请求保留现有可选温度规则:发送配置值的十进制表示(例如默认 `0.3`),避免将 `f32` 扩展为长小数。自定义渠道未配置温度时仍省略该字段,OpenAI GPT-5 和各协议的既有省略规则保持不变。
115 changes: 101 additions & 14 deletions openless-all/app/crates/openless-core/src/asr/volcengine.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,17 @@ use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::time::{Duration, Instant};

use futures_util::{SinkExt, StreamExt};
use futures_util::{future::BoxFuture, SinkExt, StreamExt};
use parking_lot::Mutex as ParkingMutex;
use serde_json::{json, Value};
use tokio::net::TcpStream;
use tokio::sync::{mpsc, oneshot, Mutex as AsyncMutex, Notify};
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
use tokio_tungstenite::tungstenite::http::header::HeaderValue;
use tokio_tungstenite::tungstenite::Message;
use tokio_tungstenite::tungstenite::{
handshake::client::{Request as WebSocketRequest, Response as WebSocketResponse},
Error as WebSocketError, Message,
};
use tokio_tungstenite::{connect_async, MaybeTlsStream, WebSocketStream};
use uuid::Uuid;

Expand All @@ -31,6 +34,9 @@ use crate::ports::{TextStreamChunk, TextStreamSink};
/// 新旧两种鉴权模式共享同一端点,仅握手鉴权头不同。
const ENDPOINT_APP_ID_TOKEN: &str = "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async";
const ENDPOINT_API_KEY: &str = "wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async";
/// Agent Plan uses a dedicated subscription endpoint with API-key authentication.
/// https://docs.volcengine.com/docs/82379/2516286
const ENDPOINT_AGENT_PLAN: &str = "wss://openspeech.bytedance.com/api/v3/plan/sauc/bigmodel_async";
/// 200 ms of 16 kHz / 16-bit / mono PCM.
pub const TARGET_AUDIO_CHUNK_BYTES: usize = 6_400;
/// 16 kHz · 16-bit · mono = 32 000 bytes/sec → 32 bytes/ms.
Expand Down Expand Up @@ -90,8 +96,34 @@ impl VolcengineAuthMode {
}
}

/// Service selection is separate from the standard service's authentication mode.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub enum VolcengineService {
#[default]
Standard,
AgentPlan,
}

impl VolcengineService {
pub fn parse(value: &str) -> Result<Self, &'static str> {
match value.trim() {
"" | "standard" => Ok(Self::Standard),
"agent_plan" => Ok(Self::AgentPlan),
_ => Err("volcengineServiceInvalid"),
}
}

pub fn auth_mode(self, configured: VolcengineAuthMode) -> VolcengineAuthMode {
match self {
Self::Standard => configured,
Self::AgentPlan => VolcengineAuthMode::ApiKey,
}
}
}

#[derive(Clone, Debug)]
pub struct VolcengineCredentials {
pub service: VolcengineService,
pub auth_mode: VolcengineAuthMode,
/// App ID(AppIdToken 模式使用;ApiKey 模式下为空)。
pub app_id: String,
Expand All @@ -114,7 +146,9 @@ impl VolcengineCredentials {

/// 凭据是否满足当前鉴权模式的要求(统一 trim 语义,见 [`VolcengineAuthMode::auth_ok`])。
pub fn auth_ok(&self) -> bool {
self.auth_mode.auth_ok(&self.app_id, &self.access_token)
self.service
.auth_mode(self.auth_mode.clone())
.auth_ok(&self.app_id, &self.access_token)
}
}

Expand Down Expand Up @@ -144,8 +178,28 @@ pub enum VolcengineASRError {
DecodeFailed(String),
}

type WsStream = WebSocketStream<MaybeTlsStream<TcpStream>>;
type WsSink = futures_util::stream::SplitSink<WsStream, Message>;
pub type VolcengineWebSocket = WebSocketStream<MaybeTlsStream<TcpStream>>;

/// External connection boundary; the provider still owns the URL and auth headers.
pub trait VolcengineConnector: Send + Sync {
fn connect(
&self,
request: WebSocketRequest,
) -> BoxFuture<'static, Result<(VolcengineWebSocket, WebSocketResponse), WebSocketError>>;
}

struct DefaultVolcengineConnector;

impl VolcengineConnector for DefaultVolcengineConnector {
fn connect(
&self,
request: WebSocketRequest,
) -> BoxFuture<'static, Result<(VolcengineWebSocket, WebSocketResponse), WebSocketError>> {
Box::pin(connect_async(request))
}
}

type WsSink = futures_util::stream::SplitSink<VolcengineWebSocket, Message>;
type SharedWriter = Arc<AsyncMutex<Option<WsSink>>>;
type AudioFrameSender = mpsc::UnboundedSender<(i32, Vec<u8>)>;

Expand All @@ -167,6 +221,7 @@ struct SyncState {
}

pub struct VolcengineStreamingASR {
connector: Arc<dyn VolcengineConnector>,
credentials: VolcengineCredentials,
task_spawner: Arc<dyn TaskSpawner>,
hotwords: Vec<DictionaryHotword>,
Expand Down Expand Up @@ -200,6 +255,7 @@ impl VolcengineStreamingASR {
task_spawner: Arc<dyn TaskSpawner>,
) -> Self {
Self {
connector: Arc::new(DefaultVolcengineConnector),
credentials,
task_spawner,
hotwords,
Expand All @@ -217,6 +273,11 @@ impl VolcengineStreamingASR {
*self.partial_sink.lock() = Some(sink);
}

pub fn with_connector(mut self, connector: Arc<dyn VolcengineConnector>) -> Self {
self.connector = connector;
self
}

pub async fn open_session(self: &Arc<Self>) -> Result<(), VolcengineASRError> {
let creds = &self.credentials;
// 统一走 VolcengineCredentials::auth_ok(trim 语义),与概览页凭据状态检测、
Expand Down Expand Up @@ -336,11 +397,15 @@ impl VolcengineStreamingASR {
&self,
connect_id: &str,
request_id: &str,
) -> Result<tokio_tungstenite::tungstenite::handshake::client::Request, VolcengineASRError>
{
let endpoint = match &self.credentials.auth_mode {
VolcengineAuthMode::AppIdToken => ENDPOINT_APP_ID_TOKEN,
VolcengineAuthMode::ApiKey => ENDPOINT_API_KEY,
) -> Result<WebSocketRequest, VolcengineASRError> {
let auth_mode = self
.credentials
.service
.auth_mode(self.credentials.auth_mode.clone());
let endpoint = match (self.credentials.service, &auth_mode) {
(VolcengineService::AgentPlan, _) => ENDPOINT_AGENT_PLAN,
(_, VolcengineAuthMode::AppIdToken) => ENDPOINT_APP_ID_TOKEN,
(_, VolcengineAuthMode::ApiKey) => ENDPOINT_API_KEY,
};
let mut request = endpoint
.into_client_request()
Expand All @@ -350,7 +415,7 @@ impl VolcengineStreamingASR {
// 根据鉴权模式选择表头:
// - AppIdToken:X-Api-App-Key + X-Api-Access-Key(旧版语音控制台)
// - ApiKey:X-Api-Key(新版方舟语音模型,单头即可)
match &self.credentials.auth_mode {
match auth_mode {
VolcengineAuthMode::AppIdToken => {
headers.insert(
"X-Api-App-Key",
Expand Down Expand Up @@ -398,14 +463,34 @@ impl VolcengineStreamingASR {
/// (hung handshake or a transient blip) doesn't kill the whole dictation.
/// `AuthRejected` / `RateLimited` short-circuit — bad credentials never heal on
/// retry, and hammering a rate-limited account only makes the throttle worse.
async fn connect_with_retry(&self, connect_id: &str) -> Result<WsStream, VolcengineASRError> {
async fn connect_with_retry(
&self,
connect_id: &str,
) -> Result<VolcengineWebSocket, VolcengineASRError> {
let mut attempt = 0usize;
loop {
attempt += 1;
let request_id = Uuid::new_v4().to_string();
let request = self.build_connect_request(connect_id, &request_id)?;
match tokio::time::timeout(CONNECT_TIMEOUT, connect_async(request)).await {
Ok(Ok((ws, _resp))) => return Ok(ws),
log::info!(
"[asr] Volcengine connect endpoint={} connect_id={} request_id={}",
request.uri(),
connect_id,
request_id
);
match tokio::time::timeout(CONNECT_TIMEOUT, self.connector.connect(request)).await {
Ok(Ok((ws, response))) => {
log::info!(
"[asr] Volcengine connected connect_id={} log_id={}",
connect_id,
response
.headers()
.get("X-Tt-Logid")
.and_then(|value| value.to_str().ok())
.unwrap_or("-")
);
return Ok(ws);
}
Ok(Err(e)) => {
let classified = classify_connect_error(e);
if is_non_retryable(&classified) || attempt >= CONNECT_MAX_ATTEMPTS {
Expand Down Expand Up @@ -1062,6 +1147,7 @@ mod tests {
for (mode, endpoint, expects_app_headers, expects_api_key) in cases {
let asr = VolcengineStreamingASR::new(
VolcengineCredentials {
service: VolcengineService::Standard,
auth_mode: mode.clone(),
app_id: "app".into(),
access_token: "secret".into(),
Expand Down Expand Up @@ -1182,6 +1268,7 @@ mod tests {
async fn await_final_result_returns_error_when_final_frame_never_arrives() {
let asr = VolcengineStreamingASR::new(
VolcengineCredentials {
service: VolcengineService::Standard,
auth_mode: VolcengineAuthMode::AppIdToken,
app_id: "app".into(),
access_token: "token".into(),
Expand Down
13 changes: 13 additions & 0 deletions openless-all/app/crates/openless-core/src/cloud_providers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -559,6 +559,17 @@ async fn build_cloud_transcription_session(
)
}
ActiveAsrProviderKind::Volcengine => {
let service = read_channel_credential(
credentials,
CredentialNamespace::Asr,
channel_id,
crate::credentials::VOLCENGINE_SERVICE_ACCOUNT,
)
.await?;
let service = crate::asr::volcengine::VolcengineService::parse(
service.as_deref().unwrap_or_default(),
)
.map_err(|message| BackendError::new(BackendErrorCode::InvalidArgument, message))?;
let auth_mode = read_channel_credential(
credentials,
CredentialNamespace::Asr,
Expand All @@ -568,6 +579,7 @@ async fn build_cloud_transcription_session(
.await?
.map(|value| VolcengineAuthMode::parse(&value))
.unwrap_or(VolcengineAuthMode::AppIdToken);
let auth_mode = service.auth_mode(auth_mode);
let app_id = read_channel_credential(
credentials,
CredentialNamespace::Asr,
Expand Down Expand Up @@ -599,6 +611,7 @@ async fn build_cloud_transcription_session(
)
.await?;
let credentials = VolcengineCredentials {
service,
auth_mode,
app_id,
access_token,
Expand Down
1 change: 1 addition & 0 deletions openless-all/app/crates/openless-core/src/credentials.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ pub const ASR_ADVANCED_CONFIG_ACCOUNT: &str = "asr.advanced_config";
pub const VOLCENGINE_APP_KEY_ACCOUNT: &str = "volcengine.app_key";
pub const VOLCENGINE_ACCESS_KEY_ACCOUNT: &str = "volcengine.access_key";
pub const VOLCENGINE_RESOURCE_ID_ACCOUNT: &str = "volcengine.resource_id";
pub const VOLCENGINE_SERVICE_ACCOUNT: &str = "volcengine.service";
pub const VOLCENGINE_AUTH_MODE_ACCOUNT: &str = "volcengine.auth_mode";
pub const VOLCENGINE_API_KEY_ACCOUNT: &str = "volcengine.api_key";
pub const XFYUN_APP_ID_ACCOUNT: &str = "xfyun.app_id";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ struct LegacyEntry {
app_key: Option<String>,
access_key: Option<String>,
resource_id: Option<String>,
volcengine_service: Option<String>,
auth_mode: Option<String>,
volcengine_api_key: Option<String>,
vocabulary_id: Option<String>,
Expand Down Expand Up @@ -124,6 +125,7 @@ impl Default for LegacyEntry {
app_key: None,
access_key: None,
resource_id: None,
volcengine_service: None,
auth_mode: None,
volcengine_api_key: None,
vocabulary_id: None,
Expand Down Expand Up @@ -153,6 +155,7 @@ impl LegacyEntry {
&self.app_key,
&self.access_key,
&self.resource_id,
&self.volcengine_service,
&self.auth_mode,
&self.volcengine_api_key,
&self.vocabulary_id,
Expand Down Expand Up @@ -382,6 +385,7 @@ fn decode_entry(
(VOLCENGINE_APP_KEY_ACCOUNT, entry.app_key),
(VOLCENGINE_ACCESS_KEY_ACCOUNT, entry.access_key),
(VOLCENGINE_RESOURCE_ID_ACCOUNT, entry.resource_id),
(VOLCENGINE_SERVICE_ACCOUNT, entry.volcengine_service),
(VOLCENGINE_AUTH_MODE_ACCOUNT, entry.auth_mode),
(VOLCENGINE_API_KEY_ACCOUNT, entry.volcengine_api_key),
(ASR_VOCABULARY_ID_ACCOUNT, entry.vocabulary_id),
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/crates/openless-core/src/llm_protocol.rs
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ pub(crate) fn request_body(
&& !(config.protocol.format == LlmRequestFormat::Messages && config.thinking_enabled)
{
if let Some(temperature) = config.temperature {
body["temperature"] = json!(temperature);
body["temperature"] = crate::polish::temperature_json(temperature);
}
}
body
Expand Down
Loading