Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose librdkafka stats to clients #28

Open
agis opened this issue Jul 20, 2017 · 1 comment
Open

Expose librdkafka stats to clients #28

agis opened this issue Jul 20, 2017 · 1 comment
Labels
enhancement

Comments

@agis
Copy link
Contributor

agis commented Jul 20, 2017 •
edited
Loading

The Go driver is able to report stats events that we could expose to clients (see confluentinc/confluent-kafka-go@18b5a55). This will be useful for monitoring Rafka (eg. monitor lagging consumers).

To begin with, the server, we could implement 2 commands for consumer-specific monitoring:

  1. one to list all the active consumers
  2. one to fetch stats for a particular consumer

We need to decide which available Redis commands serve as good candidates for such a command. Possibly CLIENT LIST is good for (1).
@agis agis added this to the 1.0 milestone Sep 5, 2017
@agis
Copy link
Contributor Author

agis commented Sep 5, 2017

For now, what we want is a simple count of errors in the producers & consumers.

@agis agis mentioned this issue Sep 6, 2017
Merged
agis added a commit that referenced this issue Sep 7, 2017
@agis
Implement stats command
04ac92c 
This allows users to monitor Rafka for errors that are not propagated to
the clients. Until now such errors were just being logged. It allows for
simple alerting rules (eg. alert when producer errors are more than 10).

We used the HGETALL Redis command, since the stats structure can be
considered a hash with stat names as keys and counters as values.

Part of #28
@agis agis removed this from the 1.0 milestone May 23, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@agis