feat: Add VisionTool and TranscriptionTool with configurable providers

[Yaphet-Chen]

Summary

• VisionTool: Generic vision tool supporting any OpenAI-compatible VLM API
• TranscriptionTool: Configurable transcription provider (SiliconFlow/Groq/OpenAI/custom)
• Config schema: Added and
• No hardcoded secrets: All API keys read from config file

Features

VisionTool ()

• Supports any OpenAI-compatible VLM (GPT-4V, Qwen-VL, Gemini Vision, etc.)
• Configurable via :
  • : API key for the vision provider
  • : Base URL for the API
  • : Model name to use
• File type filtering: only processes image files (.jpg, .jpeg, .png, .gif, .webp, .bmp)

Transcription ()

• Configurable provider: , , usage: openai [-h] [-v] [-b API_BASE] [-k API_KEY] [-p PROXY [PROXY ...]]
  [-o ORGANIZATION] [-t {openai,azure}]
  [--api-version API_VERSION] [--azure-endpoint AZURE_ENDPOINT]
  [--azure-ad-token AZURE_AD_TOKEN] [-V]
  {api,tools,migrate,grit} ...

positional arguments:
{api,tools,migrate,grit}
api Direct API calls
tools Client side tools for convenience

options:
-h, --help show this help message and exit
-v, --verbose Set verbosity.
-b API_BASE, --api-base API_BASE
What API base url to use.
-k API_KEY, --api-key API_KEY
What API key to use.
-p PROXY [PROXY ...], --proxy PROXY [PROXY ...]
What proxy to use.
-o ORGANIZATION, --organization ORGANIZATION
Which organization to run as (will use your default
organization if not specified)
-t {openai,azure}, --api-type {openai,azure}
The backend API to call, must be openai or azure
--api-version API_VERSION
The Azure API version, e.g.
'https://learn.microsoft.com/en-us/azure/ai-
services/openai/reference#rest-api-versioning'
--azure-endpoint AZURE_ENDPOINT
The Azure endpoint, e.g.
'https://endpoint.openai.azure.com'
--azure-ad-token AZURE_AD_TOKEN
A token from Azure Active Directory,
https://www.microsoft.com/en-
us/security/business/identity-access/microsoft-entra-
id
-V, --version show program's version number and exit, or

• Configured via
• Supports multiple transcription backends

Configuration Example

{
  "providers": {
    "vision": {
      "api_key": "sk-xxx",
      "api_base": "https://api.siliconflow.cn/v1",
      "model": "Qwen/Qwen2.5-VL-72B-Instruct"
    },
    "transcription": {
      "enabled": true,
      "provider": "siliconflow",
      "api_key": "sk-xxx",
      "api_base": "https://api.siliconflow.cn/v1",
      "model": "FunAudioLLM/SenseVoiceSmall"
    }
  }
}

Testing

• ✅ Voice transcription: Working
• ✅ Image understanding: Working with Qwen2.5-VL-72B

Co-authored-by: Vega vega@nanobot