kms: add support for fetching server logs

[aead]

This commit adds support for fetching server logs
via the /v1/log API.

The log API returns a stream of log records. Each
record is a protobuf (or JSON) encoded message
containing the log leve, message, timestamp and
other information about a log event.

Clients can use a LogRequest to filter based on
log level, message, IP address etc.

[ramondeklein]

I guess this Go client is meant for minkms, but I couldn't find any matching protobuf specifications in minkms.

I wonder why we are using custom serialization formats, but that's probably a choice that was made when the server-side was created.

[aead]

I guess this Go client is meant for minkms, but I couldn't find any matching protobuf specifications in minkms.
I wonder why we are using custom serialization formats, but that's probably a choice that was made when the server-side > was created.

The protobuf specification is in github.com/minio/kms-go/kms/protobuf. The server imports this directly. There is no need to specify this twice (and potentially get out-of-sync).

Are you referring to the lenght-prefixed protobuf messages?

If you want to write multiple messages to a single file or stream, it is up to you to keep track of where one message ends and the next begins. The Protocol Buffer wire format is not self-delimiting, so protocol buffer parsers cannot determine where a message ends on their own. The easiest way to solve this problem is to write the size of each message before you write the message itself. When you read the messages back in, you read the size, then read the bytes into a separate buffer, then parse from that buffer.

Ref: https://protobuf.dev/programming-guides/techniques/

[aead]

For some context @ramondeklein
The KMS server supports log streaming in the sense that a client can subscribe to the KMS logging system and receive a continous stream of logs. It's not an API that receives "just" the last N log messages. It's an API that allows direct monitoring.

[ramondeklein]

The protobuf specification is in github.com/minio/kms-go/kms/protobuf. The server imports this directly. There is no need to specify this twice (and potentially get out-of-sync).

I guess I'm used to specify the specification in the server and let the client reference that one. Agree that you shouldn't specify it twice. Fortunately, Go makes it rather easy to reference code from another repo. Try that with Java or .NET and not reach Git submodule hell...