|
| 1 | +# GenAI and ARMS field alignment |
| 2 | + |
| 3 | +## Scope and standards baseline |
| 4 | + |
| 5 | +This design aligns the first set of Qwen Code span attributes whose names, |
| 6 | +types, and meanings agree between OpenTelemetry GenAI semantic conventions and |
| 7 | +Alibaba Cloud ARMS LLM Trace. It does not change span names, span kinds, |
| 8 | +parenting, retry topology, or sensitive payload collection. |
| 9 | + |
| 10 | +The OpenTelemetry GenAI convention is still Development status. This change is |
| 11 | +pinned to commit |
| 12 | +[`2e994c6d59a93bb4fc1752c5378eedb9b8e14d6b`](https://github.com/open-telemetry/semantic-conventions-genai/tree/2e994c6d59a93bb4fc1752c5378eedb9b8e14d6b): |
| 13 | + |
| 14 | +- [Inference spans](https://raw.githubusercontent.com/open-telemetry/semantic-conventions-genai/2e994c6d59a93bb4fc1752c5378eedb9b8e14d6b/docs/gen-ai/gen-ai-spans.md) |
| 15 | +- [Agent spans](https://raw.githubusercontent.com/open-telemetry/semantic-conventions-genai/2e994c6d59a93bb4fc1752c5378eedb9b8e14d6b/docs/gen-ai/gen-ai-agent-spans.md) |
| 16 | +- [GenAI registry](https://raw.githubusercontent.com/open-telemetry/semantic-conventions-genai/2e994c6d59a93bb4fc1752c5378eedb9b8e14d6b/model/gen-ai/registry.yaml) |
| 17 | + |
| 18 | +The ARMS baseline is [LLM Trace field definitions](https://help.aliyun.com/zh/arms/application-monitoring/developer-reference/llm-trace-field-definition-description). |
| 19 | +An upgrade to either baseline requires regenerating and reviewing this matrix. |
| 20 | + |
| 21 | +## Field contract |
| 22 | + |
| 23 | +| Span | Standard attributes emitted in this phase | Source and omission rule | |
| 24 | +| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | |
| 25 | +| LLM | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.conversation.id`, `gen_ai.request.model` | Written at span creation. Conversation ID is the existing session ID. | |
| 26 | +| LLM response | `gen_ai.response.id`, `gen_ai.response.model`, `gen_ai.response.finish_reasons` | Provider response data only. Missing response model is omitted rather than replaced with the request model. All candidate finish reasons are ordered by candidate index. | |
| 27 | +| LLM output | `gen_ai.output.type` | Gemini and Vertex AI only, and only when an explicit response MIME type or one unambiguous response modality is sent on the wire. | |
| 28 | +| LLM usage | `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, `gen_ai.usage.cache_read.input_tokens`, `gen_ai.usage.cache_creation.input_tokens` | Only provider-reported non-negative safe integers. Explicit zero is retained. When only a total is reported, input/output are omitted instead of estimated. | |
| 29 | +| Tool | `gen_ai.operation.name=execute_tool`, `gen_ai.tool.name`, `gen_ai.tool.type=function`, `gen_ai.tool.call.id` | Tool call ID prefers the provider/model ID and falls back to Qwen Code's internal ID. | |
| 30 | +| Agent | `gen_ai.operation.name=invoke_agent`, `gen_ai.agent.name`, `gen_ai.agent.description`, `gen_ai.conversation.id`, optional `gen_ai.request.model` | Description uses the existing 1024-UTF-16-code-unit truncation threshold and never splits surrogate pairs. Internal invocation IDs remain private. | |
| 31 | + |
| 32 | +Private attributes without an exact standard equivalent remain available for |
| 33 | +compatibility. Exact-equivalent private aliases and invalid GenAI aliases are |
| 34 | +removed without a dual-write period: |
| 35 | + |
| 36 | +| Removed attribute | Replacement | |
| 37 | +| ----------------------------------- | --------------------------------------------------------------------------------------------------------------------- | |
| 38 | +| LLM `qwen-code.model` | `gen_ai.request.model`; interaction spans continue using `qwen-code.model` because they are not GenAI inference spans | |
| 39 | +| LLM `response_id` | `gen_ai.response.id`; API response/error logs retain their existing `response_id` schema | |
| 40 | +| LLM `input_tokens` | `gen_ai.usage.input_tokens` when the provider reports an input breakdown | |
| 41 | +| LLM `output_tokens` | `gen_ai.usage.output_tokens` when the provider reports an output breakdown | |
| 42 | +| LLM `cached_input_tokens` | `gen_ai.usage.cache_read.input_tokens` when the provider reports cache reads | |
| 43 | +| `qwen-code.tool` Span `tool.name` | `gen_ai.tool.name`; blocked-on-user and hook spans continue using `tool.name` | |
| 44 | +| `gen_ai.usage.cached_tokens` | `gen_ai.usage.cache_read.input_tokens` when the provider reports cache reads | |
| 45 | +| `gen_ai.server.time_to_first_token` | No common attribute; continue querying private `ttft_ms` | |
| 46 | +| `gen_ai.usage.reasoning_tokens` | No ARMS/GenAI common attribute in this baseline; continue querying private `thoughts_token_count` | |
| 47 | + |
| 48 | +## Provider and operation resolution |
| 49 | + |
| 50 | +Resolution is a pure function over the effective content-generator config. It |
| 51 | +never returns a URL, credential, arbitrary proxy hostname, or a value inferred |
| 52 | +from the model name. |
| 53 | + |
| 54 | +1. Qwen OAuth and an exact `DASHSCOPE_PROXY_BASE_URL` match resolve to |
| 55 | + `dashscope`. |
| 56 | +2. A boundary-safe hostname match recognizes Alibaba Model Studio endpoints and |
| 57 | + internal Alibaba gateways, Azure OpenAI, and the supported third-party |
| 58 | + endpoints (DeepSeek, xAI, Mistral, MiniMax, Z.AI, ModelScope, MiMo, |
| 59 | + OpenRouter, and Requesty). |
| 60 | +3. If the host is unknown, a known `apiKeyEnvKey` identifies the configured |
| 61 | + provider. Host identity wins on conflict. |
| 62 | +4. Unknown endpoints fall back to the protocol provider: `openai`, |
| 63 | + `anthropic`, `gcp.gemini`, or `gcp.vertex_ai`. |
| 64 | + |
| 65 | +OpenAI-compatible, Anthropic, and Qwen OAuth requests use operation `chat`. |
| 66 | +Gemini and Vertex AI requests use `generate_content`. |
| 67 | + |
| 68 | +## Response and usage provenance |
| 69 | + |
| 70 | +Provider converters attach internal provenance to normalized Gemini usage |
| 71 | +objects with a `WeakMap`. It records whether a cache-read field was actually |
| 72 | +present and Anthropic cache-creation tokens. This preserves the public response |
| 73 | +JSON shape and lets garbage collection follow the normalized usage object. |
| 74 | + |
| 75 | +When an OpenAI-compatible provider reports only `total_tokens`, the normalized |
| 76 | +total remains available to existing internal consumers, but no input/output |
| 77 | +split is synthesized and neither standard usage attribute is emitted. |
| 78 | + |
| 79 | +OpenAI `response.model`/`chunk.model` and Anthropic message model are preserved |
| 80 | +as `modelVersion`. A missing provider model remains missing for tracing; |
| 81 | +request-model fallback remains limited to existing API logs and UI behavior. |
| 82 | +Stream merging carries the last known provider model and usage provenance into |
| 83 | +the terminal response. Anthropic `message_start` input and cache usage is |
| 84 | +attached to the first subsequent yielded chunk so partial stream failures retain |
| 85 | +provider-reported usage without synthesizing an output count. |
| 86 | + |
| 87 | +## ARMS configuration |
| 88 | + |
| 89 | +ARMS automatic GenAI application recognition requires this resource attribute: |
| 90 | + |
| 91 | +```json |
| 92 | +{ |
| 93 | + "telemetry": { |
| 94 | + "resourceAttributes": { |
| 95 | + "acs.arms.service.feature": "genai_app" |
| 96 | + } |
| 97 | + } |
| 98 | +} |
| 99 | +``` |
| 100 | + |
| 101 | +Qwen Code does not inject that vendor-specific resource attribute or |
| 102 | +`gen_ai.span.kind`. ARMS can infer LLM, Tool, and Agent roles from |
| 103 | +`gen_ai.operation.name`. |
| 104 | + |
| 105 | +## Deferred work |
| 106 | + |
| 107 | +- Request sampling, choice, maximum-output, and stop fields need hooks after |
| 108 | + provider adapters finalize the wire request. |
| 109 | +- `seed` and `top_k` have incompatible ARMS and GenAI types in the baselines. |
| 110 | +- Messages, instructions, tool definitions, arguments, and results require a |
| 111 | + standard JSON schema, privacy controls, and payload caps. |
| 112 | +- Embedding needs a correct requested-model lifecycle before tracing. |
| 113 | +- ARMS time-to-first-token and OpenTelemetry time-to-first-chunk differ in name, |
| 114 | + unit, and meaning, so private `ttft_ms` remains authoritative. |
| 115 | +- Full GenAI span naming, CLIENT span kind, and logical retry topology are a |
| 116 | + separate compliance project. |
0 commit comments