Remove dependencies on unstable OpenTelemetry packages in Aspire Components

[eerhardt]

As our plan is to stabilize .NET Aspire for v8.0.0 we need to remove unstable dependencies.

We have the following dependencies on unstable OpenTelemetry instrumentation packages in our Aspire Components:

• OpenTelemetry.Instrumentation.EventCounters
• OpenTelemetry.Instrumentation.SqlClient
• OpenTelemetry.Instrumentation.StackExchangeRedis

None of these packages has a plan to ship a stable/GA version by the time we ship v8.0.0 of .NET Aspire. We need to formulate a plan for how to handle these dependencies.

OpenTelemetry.Instrumentation.EventCounters

Used by all 5 EntityFramework components and Aspire.Microsoft.Data.SqlClient to turn the existing EventCounters into OTel metrics / Meters.

I think the following 2 options are possible here:

1. Remove these Metrics for now, and ask the lower level libraries (like efcore, sqlclient, etc) to produce metrics natively using the Meter APIs.
2. "Vendor"/copy the OpenTelemetry.Instrumentation.EventCounters into our Aspire Components for now until either:
   a. The EventCounters library ships stable.
   b. The underlying libraries get native metrics using the Meter APIs.

OpenTelemetry.Instrumentation.SqlClient

This is used by Aspire.Microsoft.Data.SqlClient (and after #2814 Aspire.Microsoft.EntityFrameworkCore.SqlServer) to produce traces / Activities for the SqlClient ADO.NET provider.

Tracing is more critical than Metrics, so I don't think it is a valid option to remove this instrumentation from the component. I think the only viable option here is to "vendor"/copy the instrumentation library code into the Aspire components.

OpenTelemetry.Instrumentation.StackExchangeRedis

This is used by the 3 Redis components to produce traces / Activities when communicating with a Redis server.

For the same reason above as SqlClient, I don't think it is a valid option to remove this instrumentation from the component. I think the only viable option here is to "vendor"/copy the instrumentation library code into the Aspire components.

cc @joperezr @DamianEdwards @AndriySvyryd @roji @mgravell

[roji]

For reference, the issue tracking emitting metrics natively from SqlClient is dotnet/SqlClient#2211 (the one for tracing is dotnet/SqlClient#2210).

[samsp-msft]

Another possible option for EventCounters is to use the DiagnosticSource hooks that is being used for tracing to collect statistics and fabricate metrics based on those. The metrics can then be designed to match the OTel semantic conventions with dimensions etc.
What are the most important metrics for database scenarios from the client (as the server should probably be emitting its own).

[roji]

What are the most important metrics for database scenarios from the client (as the server should probably be emitting its own).

Here's the OTel experimental semantic conventions for database client metrics. Notably missing from there is command metrics (executions, failures...), which I'd assume would need to be in any finalized spec... For reference, here's the Npgsql built-in implementation which emits both the (experimentally) standardized connection metrics and non-standardized command metrics.

[eerhardt]

With

• Remove OpenTelemetry.Instrumentation.EventCounters usages #3110
• Remove dependency from OpenTelemetry.Instrumentation.SqlClient #3059
• [release/8.0-preview5] Remove dependency from OpenTelemetry.Instrumentation.StackExchangeRedis #3198

This work is now complete.