Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Net.ReadTimeout is not a network-level property #2820

Open
leonidboykov opened this issue Feb 29, 2024 · 4 comments
Open

Net.ReadTimeout is not a network-level property #2820

leonidboykov opened this issue Feb 29, 2024 · 4 comments
Labels
needs-investigation Issues that require followup from maintainers stale/exempt Issues and pull requests that should never be closed as stale

Comments

@leonidboykov
Copy link

Description

The description of the Net struct says that inner properties correspond to network-level configuration. However, Net.ReadTimeout is affected by some other timeouts, i.e. if Consumer.Group.Rebalance.Timeout is greater than Net.ReadTimeout the rebalance fails with i/o timeout error. Perhaps, Kafka does not reply immediately to the client.

At the moment, the only way to make other timeouts work is to set Net.ReadTimeout as the longest timeout. However, this may lead to problems with actual network issues because there is no actual network-level timeout.

Versions
Sarama Kafka Go
v1.41.3 3.1.0 1.21
Configuration
conf := sarama.NewConfig()
conf.Version = sarama.V3_3_1_0
conf.Consumer.Return.Errors = true
conf.Consumer.Offsets.Initial = sarama.OffsetOldest
conf.Consumer.MaxProcessingTime = 1 * time.Minute
conf.Consumer.Group.Rebalance.Timeout = 10 * time.Minute
conf.Consumer.Group.Session.Timeout = 10 * time.Minute
conf.Consumer.Group.Heartbeat.Interval = 5 * time.Second
conf.Net.ReadTimeout = 11 * time.Minute // Consumer.Group.Rebalance.Timeout with an extra 1 minute.
}
Additional Context

I've done some research, and it seems, other Sarama users are dealing with the same problem: #1422 (comment).

I'm not sure, but perhaps, this parameter wasn't tested properly. While default settings for sarama.Config set Consumer.Group.Rebalance.Timeout (60s) greater than Net.ReadTimeout (30s), the value Consumer.Group.Rebalance.Timeout is set to only 10 seconds in the corresponding test:

func defaultConfig(clientID string) *Config {
config := NewFunctionalTestConfig()
config.ClientID = clientID
config.Consumer.Return.Errors = true
config.Consumer.Offsets.Initial = OffsetOldest
config.Consumer.Group.Rebalance.Timeout = 10 * time.Second
config.Metadata.Full = false
config.Metadata.RefreshFrequency = 5 * time.Second
return config
}

@dnwe dnwe added the needs-investigation Issues that require followup from maintainers label Mar 4, 2024

This comment was marked as outdated.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Jun 2, 2024
@leonidboykov

This comment was marked as outdated.

@github-actions github-actions bot removed the stale Issues and pull requests without any recent activity label Jun 2, 2024

This comment was marked as outdated.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Aug 31, 2024
@leonidboykov

This comment was marked as outdated.

@github-actions github-actions bot removed the stale Issues and pull requests without any recent activity label Aug 31, 2024
@dnwe dnwe added the stale/exempt Issues and pull requests that should never be closed as stale label Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-investigation Issues that require followup from maintainers stale/exempt Issues and pull requests that should never be closed as stale
Projects
None yet
Development

No branches or pull requests

2 participants