Add option to ignore invalid parent spans

[cdavis-joy]

What was changed

Adds an option to ignore errors when interpreting parent spans from headers.

Why?

When migrating from one tracing library to another, the new library will not find the parent span headers it expects, and will throw an errors, thus failing any workflows/activities that were running at the time of the worker deploy.

Checklist

1. Closes Allow ignoring invalid SpanContext in OpenTelemetry TracingInterceptor #1174

2. How was this tested:

Forced an error to return from t.readSpanFromHeader, and saw that error was not returned.

3. Any docs updates needed?

Comment added to option.

[Quinn-With-Two-Ns]

LGTM, will let @cretz take a look

[cretz]

@cdavis-joy - I think the better approach to supporting multiple tracer interceptors is to set the SpanContextKey and HeaderKey different for each. Then multiple tracing interceptors can be provided until you're ready to remove one.

But since this is opt-in, this addition is harmless. Will leave open to hear your thoughts on just setting different keys, then will merge.

[cdavis-joy]

@cretz that's a good call. I would agree with you, but I'm failing to understand why were encountering an issue in the first place if the opentracing interceptor and the opentelemetry interceptor are using the same defaultHeaderKey of _tracer-data. After our worker using the opentelemtry tracer was deployed, we had a workflow fail with:

"failed extracting OpenTelemetry span from map",

That said, I appreciate the merge, and don't feel we need to dwell on this unless you know off the top of your head what we might be doing wrong to get that failure. 🙏

[cretz]

I'm failing to understand why were encountering an issue in the first place if the opentracing interceptor and the opentelemetry interceptor are using the same defaultHeaderKey of _tracer-data.

Because _tracer-data has OpenTracing-format serialized span, so it can't deserialize into an OpenTelemetry span.

[cdavis-joy]

Missed adding this to the otel interceptors Options func. Updated.

[cdavis-joy]

@cretz I misunderstood what you initially said. By changing the HeaderKey and SpanContextKey, the newer interceptor would not even attempt to interpret spans from the old interceptor, thus avoiding these errors. Thanks for that, I think we're gonna give that a shot.

[cdavis-joy]

This is something I'd still like to see as an option. It looks like these integration test failures are unrelated though?

[cretz]

Let me do a re-run and see. If/when it passes, we'll merge.