Python: Add OpenTelemetry instrumentation to ClaudeAgent (#4278)#4326
Open
amitmukh wants to merge 5 commits intomicrosoft:mainfrom
Open
Python: Add OpenTelemetry instrumentation to ClaudeAgent (#4278)#4326amitmukh wants to merge 5 commits intomicrosoft:mainfrom
amitmukh wants to merge 5 commits intomicrosoft:mainfrom
Conversation
) Add inline telemetry to ClaudeAgent.run() so that enable_instrumentation() emits invoke_agent spans and metrics. Covers both streaming and non-streaming paths using the same observability helpers as AgentTelemetryLayer. Adds 5 unit tests for telemetry behavior. Co-Authored-By: amitmukh <amimukherjee@microsoft.com>
Member
Python Test Coverage Report •
Python Unit Test Overview
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds OpenTelemetry span emission to ClaudeAgent.run() so enable_instrumentation() affects Claude agents the same way it affects core Agent implementations, covering both streaming and non-streaming execution paths.
Changes:
- Wrap
ClaudeAgent.run()with OTEL span creation, duration tracking, response capture, and exception capture. - Add streaming-specific span finalization via
ResponseStreamcleanup hooks andweakref.finalize. - Add new Claude telemetry unit tests for enabled/disabled instrumentation, streaming, exceptions, and provider name.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| python/packages/claude/agent_framework_claude/_agent.py | Adds inline OTEL instrumentation to ClaudeAgent.run() via new _run_with_telemetry* helpers. |
| python/packages/claude/tests/test_claude_agent.py | Adds unit tests validating ClaudeAgent telemetry behavior across key run paths. |
- Add justification comment for private observability API imports - Pass system_instructions to capture_messages for system prompt capture - Use monkeypatch instead of try/finally for test global state isolation Co-Authored-By: amitmukh <amitmukh@users.noreply.github.com> Co-Authored-By: Claude <noreply@anthropic.com>
Member
|
I would prefer this to adopt the agent telemetry layer instead is reimplementing a bunch, if that means we need to do some updates to it thats fine, but otel is a changing spec so we don't want to do double work |
Restructure ClaudeAgent to inherit from AgentTelemetryLayer via a _ClaudeAgentRunImpl mixin, eliminating duplicated telemetry code and private API imports. MRO: ClaudeAgent → AgentTelemetryLayer → _ClaudeAgentRunImpl → BaseAgent - Remove inline _run_with_telemetry / _run_with_telemetry_stream methods - Remove private observability helper imports (_capture_messages, etc.) - Add default_options property mapping system_prompt → instructions - Net -105 lines by reusing core telemetry layer Co-Authored-By: amitmukh <amitmukh@users.noreply.github.com> Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
Author
Restructured to adopt Changes in latest commit:
|
…ryLayer.run() Remove explicit `options` parameter from mixin's run() signature and extract it from **kwargs to match AgentTelemetryLayer's signature. Also align overload return types (ResponseStream, Awaitable) to match. Co-Authored-By: Claude <noreply@anthropic.com>
dmytrostruk
reviewed
Feb 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BaseAgentdirectly, bypassingAgentTelemetryLayer, soenable_instrumentation()has no effectClaudeAgent.run()using the same observability helpers asAgentTelemetryLayerinvoke_agentspans, duration tracking, exception capture, and sensitive data supportApproach
Instead of changing the inheritance chain (which would cause MRO conflicts since
ClaudeAgentdefines its ownrun()and doesn't use a standard chat client), this PR integrates telemetry directly intoClaudeAgent.run()using the framework's existing observability helpers (_get_span,_get_span_attributes,_capture_messages,_capture_response,capture_exception). This is the safest approach with zero risk of breaking existing behavior.Changes
python/packages/claude/agent_framework_claude/_agent.py: Add telemetry wrapping inrun(), with two new private methods_run_with_telemetry()and_run_with_telemetry_stream()python/packages/claude/tests/test_claude_agent.py: Add 5 new tests covering span emission, disabled instrumentation, streaming spans, exception capture, and provider name verificationTest plan
invoke_agentspan appears in App Insights whenenable_instrumentation()is calledFuture Work: Tool-level telemetry granularity
This PR provides the outer
invoke_agentspan for ClaudeAgent. However, visibility intoindividual tool calls (Read, Write, Bash, etc.) and per-LLM-call spans that happen inside
the Claude CLI subprocess is not possible at the MAF layer — these are opaque to the framework.
A related issue has been raised upstream: anthropics/claude-agent-sdk-python#611
requesting OpenTelemetry instrumentation in the Claude Agent SDK itself.
Once Anthropic ships SDK-level telemetry, a follow-up PR can wire those events as child spans
under the
invoke_agentspan created here. The trace context parent is already in place —the integration should be a small, targeted change once the Anthropic SDK defines its
telemetry surface.
Note: Custom
@tool-decorated functions already get tool-level spans today viaFunctionTool.invoke()in core. Only Claude's built-in CLI tools require the upstream fix.