diff --git a/docs/messaging/kafka.md b/docs/messaging/kafka.md index 47d36bb22f..052bebacb8 100644 --- a/docs/messaging/kafka.md +++ b/docs/messaging/kafka.md @@ -11,10 +11,6 @@ linkTitle: Kafka - [Span attributes](#span-attributes) - [Examples](#examples) * [Apache Kafka with Quarkus or Spring Boot Example](#apache-kafka-with-quarkus-or-spring-boot-example) -- [Metrics](#metrics) - * [General Metrics](#general-metrics) - * [Producer Metrics](#producer-metrics) - * [Consumer Metrics](#consumer-metrics) @@ -88,75 +84,4 @@ Process CB: | Span Rcv2 | | `messaging.kafka.partition` | `"1"` | `"1"` | `"1"` | `"3"` | `"3"` | | `messaging.kafka.message.offset` | `"12"` | `"12"` | `"12"` | `"32"` | `"32"` | -## Metrics - -This section defines how to apply semantic conventions when collecting Kafka metrics. - -### General Metrics - -**Description:** General Kafka metrics. - -| Name | Instrument | Value type | Unit | Unit ([UCUM](/docs/general/metrics.md#instrument-units)) | Description | Attribute Key | Attribute Values | -| ---------------------------------------------| ------------- | ---------- | ------ | -------------------------------------------- | -------------- | ------------- | ---------------- | -| messaging.kafka.messages | Counter | Int64 | messages | `{message}` | The number of messages received by the broker. | | | -| messaging.kafka.requests.failed | Counter | Int64 | requests | `{request}` | The number of requests to the broker resulting in a failure. | `type` | `produce`, `fetch` | -| messaging.kafka.requests.queue | UpDownCounter | Int64 | requests | `{request}` | The number of requests in the request queue. | | | -| messaging.kafka.network.io | Counter | Int64 | bytes | `By` | The bytes received or sent by the broker. | `state` | `in`, `out` | -| messaging.kafka.purgatory.size | UpDownCounter | Int64 | requests | `{request}` | The number of requests waiting in the purgatory. | `type` | `produce`, `fetch` | -| messaging.kafka.partitions.all | UpDownCounter | Int64 | partitions | `{partition}` | The number of partitions in the broker. | | | -| messaging.kafka.partitions.offline | UpDownCounter | Int64 | partitions | `{partition}` | The number of offline partitions. | | | -| messaging.kafka.partitions.under-replicated | UpDownCounter | Int64 | partition | `{partition}` | The number of under replicated partitions. | | | -| messaging.kafka.isr.operations | Counter | Int64 | operations | `{operation}` | The number of in-sync replica shrink and expand operations. | `operation` | `shrink`, `expand` | -| messaging.kafka.lag_max | Gauge | Int64 | lag max | `{message}` | Max lag in messages between follower and leader replicas. | | | -| messaging.kafka.controllers.active | UpDownCounter | Int64 | controllers | `{controller}` | The number of active controllers in the broker. | | | -| messaging.kafka.leader.elections | Counter | Int64 | elections | `{election}` | Leader election rate (increasing values indicates broker failures). | | | -| messaging.kafka.leader.unclean-elections | Counter | Int64 | elections | `{election}` | Unclean leader election rate (increasing values indicates broker failures). | | | -| messaging.kafka.brokers | UpDownCounter | Int64 | brokers | `{broker}` | Number of brokers in the cluster. | | | -| messaging.kafka.topic.partitions | UpDownCounter | Int64 | partitions | `{partition}` | Number of partitions in topic. | `topic` | The ID (integer) of a topic | -| messaging.kafka.partition.current_offset | Gauge | Int64 | partition offset | `{partition offset}` | Current offset of partition of topic. | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition | -| messaging.kafka.partition.oldest_offset | Gauge | Int64 | partition offset | `{partition offset}` | Oldest offset of partition of topic | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition | -| messaging.kafka.partition.replicas.all | UpDownCounter | Int64 | replicas | `{replica}` | Number of replicas for partition of topic | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition | -| messaging.kafka.partition.replicas.in_sync | UpDownCounter | Int64 | replicas | `{replica}` | Number of synchronized replicas of partition | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition| - -### Producer Metrics - -**Description:** Kafka Producer level metrics. - -| Name | Instrument | Value type | Unit | Unit ([UCUM](/docs/general/metrics.md#instrument-units)) | Description | Attribute Key | Attribute Values | -| --------------------------------------------- | ------------- | ---------- | ------ | -------------------------------------------- | -------------- | ------------- | ---------------- | -| messaging.kafka.producer.outgoing-bytes.rate | Gauge | Double | bytes per second | `By/s` | The average number of outgoing bytes sent per second to all servers. | `client-id` | `client-id` value | -| messaging.kafka.producer.responses.rate | Gauge | Double | responses per second | `{response}/s` | The average number of responses received per second. | `client-id` | `client-id` value | -| messaging.kafka.producer.bytes.rate | Gauge | Double | bytes per second | `By/s` | The average number of bytes sent per second for a specific topic. | `client-id` | `client-id` value | -| | | | | | | `topic` | topic name | -| messaging.kafka.producer.compression-ratio | Gauge | Double | compression ratio | `{compression}` | The average compression ratio of record batches for a specific topic. | `client-id` | `client-id` value | -| | | | | | | `topic` | topic name | -| messaging.kafka.producer.record-error.rate | Gauge | Double | error rate | `{error}/s` | The average per-second number of record sends that resulted in errors for a specific topic. | `client-id` | `client-id` value | -| | | | | | | `topic` | topic name | -| messaging.kafka.producer.record-retry.rate | Gauge | Double | retry rate | `{retry}/s` | The average per-second number of retried record sends for a specific topic. | `client-id` | `client-id` value | -| | | | | | | `topic` | topic name | -| messaging.kafka.producer.record-sent.rate | Gauge | Double | records sent rate | `{record_sent}/s` | The average number of records sent per second for a specific topic. | `client-id` | `client-id` value | -| | | | | | | `topic` | topic name | - -### Consumer Metrics - -**Description:** Kafka Consumer level metrics. - -| Name | Instrument | Value type | Unit | Unit ([UCUM](/docs/general/metrics.md#instrument-units)) | Description | Attribute Key | Attribute Values | -| --------------------------------------------- | ------------- | ---------- | ------ | -------------------------------------------- | -------------- | ------------- | ---------------- | -| messaging.kafka.consumer.members | UpDownCounter | Int64 | members | `{member}` | Count of members in the consumer group | `group` | The ID (string) of a consumer group | -| messaging.kafka.consumer.offset | Gauge | Int64 | offset | `{offset}` | Current offset of the consumer group at partition of topic | `group` | The ID (string) of a consumer group | -| | | | | | | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition | -| messaging.kafka.consumer.offset_sum | Gauge | Int64 | offset sum | `{offset sum}` | Sum of consumer group offset across partitions of topic | `group` | The ID (string) of a consumer group | -| | | | | | | `topic` | The ID (integer) of a topic | -| messaging.kafka.consumer.lag | Gauge | Int64 | lag | `{lag}` | Current approximate lag of consumer group at partition of topic | `group` | The ID (string) of a consumer group | -| | | | | | | `topic` | The ID (integer) of a topic | -| | | | | | | `partition` | The number (integer) of the partition | -| messaging.kafka.consumer.lag_sum | Gauge | Int64 | lag sum | `{lag sum}` | Current approximate sum of consumer group lag across all partitions of topic | `group` | The ID (string) of a consumer group | -| | | | | | | `topic` | The ID (integer) of a topic | - [DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/tree/v1.22.0/specification/document-status.md