Update vendored ConfluentKafka instrumentation to 0.2.0-alpha.2 - #19319
Update vendored ConfluentKafka instrumentation to 0.2.0-alpha.2#19319Sébastien Ros (sebastienros) wants to merge 3 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sébastien Ros (sebastienros)
added
the
breaking-change
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 19319Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 19319" |
github-actions
Bot
added
the
area-integrations
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Updates vendored Confluent Kafka instrumentation to 0.2.0-alpha.2 and OpenTelemetry messaging conventions v1.43.
Changes:
- Updates Kafka spans, metrics, attributes, and exception telemetry.
- Aligns Aspire’s legacy Kafka meter with renamed signals.
- Updates tests, telemetry documentation, and vendoring instructions.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
tests/Aspire.Confluent.Kafka.Tests/OtelTracesTests.cs |
Verifies renamed spans and kinds. |
tests/Aspire.Confluent.Kafka.Tests/OtelMetricsTests.cs |
Verifies new metric names. |
tests/Aspire.Confluent.Kafka.Tests/Aspire8MetricsTests.cs |
Updates legacy metric assertions. |
src/Vendoring/README.md |
Updates the vendored version and adaptations. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/TracerProviderBuilderExtensions.Producer.cs |
Registers the producer activity source. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/TracerProviderBuilderExtensions.Consumer.cs |
Registers the consumer activity source. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/Shared/SemanticConventions.cs |
Updates semantic-convention constants. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/Shared/Guard.cs |
Synchronizes validation helpers. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/OpenTelemetryConsumeResultExtensions.cs |
Updates process-span behavior. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/MeterProviderBuilderExtensions.Producer.cs |
Registers the producer meter. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/MeterProviderBuilderExtensions.Consumer.cs |
Registers the consumer meter. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/InstrumentedProducer.cs |
Updates producer telemetry. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/InstrumentedConsumerBuilder.cs |
Synchronizes consumer construction. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/InstrumentedConsumer.cs |
Updates consumer and exception telemetry. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/ConfluentKafkaInstrumentedProducerBuilderOptions.cs |
Adds producer builder options. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/ConfluentKafkaInstrumentedConsumerBuilderOptions.cs |
Adds consumer builder options. |
src/Vendoring/OpenTelemetry.Instrumentation.ConfluentKafka/ConfluentKafkaCommon.cs |
Defines instrumentation identity and instruments. |
src/Components/Telemetry.md |
Documents renamed Kafka metrics. |
src/Components/Aspire.Confluent.Kafka/ConfluentKafkaMetrics.cs |
Renames legacy metrics and client tag. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 908e2f89-e34c-451e-bb4e-d239dd7b566c
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.
Suppressed comments (2)
src/Components/Aspire.Confluent.Kafka/ConfluentKafkaMetrics.cs:59
- These names do not match the underlying librdkafka statistics.
txmsgs/rxmsgscount messages transmitted to or consumed from brokers, while the v1.43 metrics represent producer send attempts and deliveries to the application. Queued or failed sends and prefetched-but-undelivered records therefore make this legacy meter disagree with the vendored meter for the same metric names. Keep the statistics under custom names, or emit these semantic-convention counters from the producer/consumer operation paths instead.
public const string TxMessages = "messaging.client.sent.messages";
public const string TxMessageBytes = "messaging.kafka.message.transmitted";
public const string RxMessages = "messaging.client.consumed.messages";
tests/Aspire.Confluent.Kafka.Tests/OtelInstrumentationTests.cs:15
- This new class can run in parallel with
OtelTracesTestsandOtelMetricsTests, although all three attach exporters to the same static KafkaActivitySource/Meter. Telemetry emitted here can consequently enter another test's exporter (and its unsynchronizedList), causing count assertions such asAssert.Equal(5, activities.Count)to fail nondeterministically. Put all Kafka telemetry tests in one serialized collection or otherwise disable parallel execution between them.
public class OtelInstrumentationTests
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 908e2f89-e34c-451e-bb4e-d239dd7b566c
Tests selector (audit mode)The full test matrix and all jobs still run in audit mode. The tests and jobs below are what selective CI would run under enforcement. 10 / 102 test projects · 3 jobs, from 22 changed files. Selected test projects (10 / 102)
Selected jobs (3)
How these were chosen — grouped by what changed
🔧 🔧 📦 affected project 🔧 🔧 🧪 🧪 🧪 🧪 🧪 🧪 🧪 Job reasons
Selection computed for commit |
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
Adam Ratzman (adamint)
left a comment
Adam Ratzman (adamint)
left a comment
There was a problem hiding this comment.
Could you take a look at these? I think they need to be fixed before merging.
| } | ||
| => this.consumer.Close(); | ||
|
|
||
| private static bool ShouldInstrument(ConsumeResult<TKey, TValue>? result, string? errorType) => |
There was a problem hiding this comment.
Could we record messaging.client.operation.duration for every completed poll here? This still skips timeout/null and partition-EOF results, so the new operation metric only measures polls that returned a message (or threw ConsumeException). I think the message counter can stay gated, but the operation duration should include empty polls too.
| } | ||
|
|
||
| Assert.Equal(5, activities.Where(x => x.OperationName == $"{topic} publish").Count()); | ||
| Assert.Equal(5, activities.Count); |
There was a problem hiding this comment.
Could we filter these activities to this test's unique topic before asserting? OtelInstrumentationTests subscribes to the same process-wide ActivitySource outside the Kafka collection, so it can run concurrently and add activities to this exporter. The old operation-name filter avoided that contamination; this exact global count will be flaky.
| var metricNames = metrics | ||
| .Where(x => x.MeterName == "OpenTelemetry.Instrumentation.ConfluentKafka") | ||
| .Select(x => x.Name) | ||
| .Distinct() |
There was a problem hiding this comment.
Can we assert the send and poll duration points separately instead of reducing this to distinct metric names? Both operations use messaging.client.operation.duration now, so producer emission alone makes this pass if consumer duration recording disappears.
|
FYI we shipped 0.3.0-alpha.1 last week. |
Description
Updates the vendored
OpenTelemetry.Instrumentation.ConfluentKafkasource from0.1.0-alpha.2to0.2.0-alpha.2. This brings Kafka telemetry onto the OpenTelemetry messaging semantic conventions v1.43, includes the0.1.0-alpha.7fix that records consumer metrics whenConsumeExceptionis thrown, and incorporates the remaining 0.2 instrumentation changes.The vendored fork keeps Aspire's internal visibility, explicit instrumentation name/version, AOT-compatible property fetcher, and analyzer exemptions. The vendoring instructions now document the updated pin and local adaptations.
Upstream changelog since 0.1.0-alpha.2
OpenTelemetry.Instrumentation.ConfluentKafka 0.1.0-alpha.3 through 0.2.0-alpha.2
0.2.0-alpha.2 (2026-Jul-17)
1.17.0. (#4773)0.2.0-alpha.1 (2026-Jul-08)
1.16.0. (#4487)sendspan (previouslypublish) and consumerpollspan (previouslyreceive, and now emitted withActivityKind.Client) now set:messaging.operation.nameandmessaging.operation.type(replacingmessaging.operation)messaging.client.id(replacingmessaging.client_id)messaging.consumer.group.name(replacingmessaging.kafka.consumer.group)messaging.destination.partition.idas a string (replacingmessaging.kafka.destination.partition)messaging.kafka.offset(replacingmessaging.kafka.message.offset)messaging.kafka.message.keyis emitted as a string when the key has an unambiguous canonical representation; otherwise, the attribute is omitted.messaging.client.operation.duration,messaging.client.sent.messages, andmessaging.client.consumed.messages.error.typenow reports the language-agnostic Kafka error code for Kafka errors (ProduceException/ConsumeException), or the exception type name for other failures. (#4636)0.1.0-alpha.7 (2026-May-29)
InstrumentedConsumerwheremessaging.receive.durationandmessaging.receive.messagesmetrics were not emitted whenConsumeExceptionis thrown by theConsumemethod. (#4433)0.1.0-alpha.6 (2026-Apr-21)
1.15.3. (#4166)0.1.0-alpha.5 (2026-Jan-21)
1.15.0. (#3721)0.1.0-alpha.4 (2025-Nov-13)
10.0.0-rc.2.25502.107to10.0.0. (#3403)1.14.0. (#3403)0.1.0-alpha.3 (2025-Oct-23)
Activity.SetStatusinstead of the deprecated OpenTelemetry API package extension when setting span status. (#2358)messaging.receive.durationandmessaging.publish.durationhistograms use the Advice API to set default explicit buckets following the OpenTelemetry specification. (#2430)1.13.1. (#3218)Breaking changes
Kafka telemetry consumers must update dashboards, alerts, and queries for these signal changes:
<topic> publishsend <topic><topic> receive(Consumerkind)poll <topic>(Clientkind)messaging.publish.duration/messaging.receive.durationmessaging.client.operation.durationmessaging.publish.messagesmessaging.client.sent.messagesmessaging.receive.messagesmessaging.client.consumed.messagesmessaging.client_idmessaging.client.idmessaging.kafka.consumer.groupmessaging.consumer.group.namemessaging.kafka.destination.partitionmessaging.destination.partition.idmessaging.kafka.message.offsetmessaging.kafka.offsetAspire's opt-in legacy
Aspire.Confluent.Kafkastatistics meter now uses the same sent/consumed message metric names and client ID attribute as the vendored instrumentation.Validation
MSBUILDTERMINALLOGGER=false dotnet build src/Components/Aspire.Confluent.Kafka/Aspire.Confluent.Kafka.csproj --no-restore --nologoMSBUILDTERMINALLOGGER=false dotnet test --project tests/Aspire.Confluent.Kafka.Tests/Aspire.Confluent.Kafka.Tests.csproj --no-launch-profile -- --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"(501 passed, 18 skipped across net8.0, net9.0, and net10.0)Fixes # (issue)
Checklist
<remarks />and<code />elements on your triple slash comments?