Skip to content

fix: spans built from non-SpanContext parents crash or corrupt trace continuity#514

Open
reevejd wants to merge 2 commits into
instana:masterfrom
reevejd:fix-sampler-context-type-confusion
Open

fix: spans built from non-SpanContext parents crash or corrupt trace continuity#514
reevejd wants to merge 2 commits into
instana:masterfrom
reevejd:fix-sampler-context-type-confusion

Conversation

@reevejd

@reevejd reevejd commented Jul 3, 2026

Copy link
Copy Markdown

fixes #513

Problem

non_recording_span wraps any object and exposes it as the span's #context, which the
next span start reads back as its parent. Three paths hand it something other than a
SpanContext, and should_sample? then crashes the request with
NoMethodError: undefined method 'tracestate':

  • start_span's agent-not-ready path passes the raw OpenTelemetry::Context (the
    agent-warmup crash from the issue)
  • the gRPC server instrumentation passes {} for any request without trace headers
  • ActionCable stores {} for connections processed outside a trace

non_recording_span(nil) also fabricates a random but valid
SpanContext, so a warmup request with no parent at all records later spans under a trace
id that was never reported.

Fix

I considered just adding a guard like my monkey-patch in the issue, which is enough to
stop the 500s, but it only changes the symptom: malformed parents keep flowing and requests
silently produce broken traces (dropped continuity, fabricated trace ids) instead of
crashing. So this PR fixes the producers and keeps the guard as another layer of defense:

  • tracer.rb: the not-ready path extracts the current span's SpanContext from
    with_parent (defaulting to the current context, like the recording path), yielding
    SpanContext::INVALID when there is no parent
  • samplers.rb: reads tracestate only when the parent supports it, falling back to
    Tracestate::DEFAULT instead of nil (the SpanContext contract requires a Tracestate)
  • grpc.rb: headerless requests use nil instead of {} and start a root span
  • action_cable.rb: stores SpanContext::INVALID instead of {}

Tests

Five of the six new tests fail against master, reproducing each defect: the raw-Context
parent, the fabricated trace ids, the sampler NoMethodError, and a headerless gRPC call
that errors the RPC. The sixth pins tracestate propagation (with a tracestate distinct
from the default) so the new fallback can't silently drop real incoming tracestate.

Signed-off-by: James Reeve <james.reeve@ibm.com>
@arjun-rajappa

arjun-rajappa commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Hi @reevejd
Thanks for tracking this down — identifying the root cause and the fix for something that's been reported by so many people is great work, and this looks very promising!

A couple of minor suggestions before we merge:

Everything else looks good. I want to spend a bit more time verifying the fix to make sure we're not missing any other scenarios or code paths where this could surface — I'll post an update once I'm done.

FYI - I closed and reopened the PR to trigger the test pipeline.

Signed-off-by: James Reeve <james.reeve@ibm.com>
@reevejd

reevejd commented Jul 6, 2026

Copy link
Copy Markdown
Author

Thanks @arjun-rajappa! I've made the changes you suggested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Intermittent empty-body HTTP 500s -- "NoMethodError: undefined method 'tracestate' for an instance of OpenTelemetry::Context"

2 participants