Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor fixes to Kafka Source #3174

Merged
merged 2 commits into from
Aug 16, 2023
Merged

Conversation

kkondaka
Copy link
Collaborator

Description

This change contains the following fixes

  1. Change Message Format types to lowercase "JSON" -> "json", "AVRO" -> "avro", "PLAINTEXT" -> "plaintext"
  2. Fix Max poll interval from 300000 seconds to 300000 milliseconds.
  3. Fix "kafka_partition" attribute value stored in the event metadata to be string.
  4. Introduce 10 second delay when an exception is encountered during consumer poll

Issues Resolved

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • [ X] New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • [ X] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Krishna Kondaka <[email protected]>
@@ -33,7 +33,7 @@ public class TopicConfig {
static final Duration DEFAULT_RETRY_BACKOFF = Duration.ofSeconds(10);
static final Duration DEFAULT_RECONNECT_BACKOFF = Duration.ofSeconds(10);
static final Integer DEFAULT_MAX_PARTITION_FETCH_BYTES = 1048576;
static final Duration DEFAULT_MAX_POLL_INTERVAL = Duration.ofSeconds(300000);
static final Duration DEFAULT_MAX_POLL_INTERVAL = Duration.ofSeconds(300);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set a maximum value on maxRetryDelay? A user could set it back to 300000

Signed-off-by: Krishna Kondaka <[email protected]>
public static class SslAuthConfig {
// TODO Add Support for SSL authentication types like
// one-way or two-way authentication
/*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take it or leave it: I would recommend deleting this dead code and leave a single like comment instead of commenting it out a large code block with no description for the TODO.

@kkondaka kkondaka merged commit 1f0ad76 into opensearch-project:main Aug 16, 2023
23 of 24 checks passed
@kkondaka kkondaka deleted the kafka-fixes1 branch May 13, 2024 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants