feat: Support setting health check interval for model service #2825
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @jopemachine and the rest of your teammates on |
github-actions
bot
added
comp:agent
kyujin-cho
reviewed
Sep 25, 2024
jopemachine
force-pushed
the
topic/09-10-feat_support_setting_health_check_interval_for_model_service
branch
from
70ef470
to
71bbc7a
Compare
|
Remaining tasks: We need to decide on the default value for the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #2817.
The existing code does not allow adjusting the
intervalbetween health check requests, and themax_wait_timeis always used as both the interval and the response timeout value.This PR keeps the
max_wait_timeas the value that triggers a timeout and adds a separate setting that specifies the health checkinterval.Detail
For example, if a health check request is successful, the next health check request will be sent after the
interval.If the health check request fails, meaning the server responds but with an incorrect response code, the next health check request will be sent after waiting for the
interval.If the server does not respond, the health check request will be sent after waiting for the
max_wait_time.Checklist: (if applicable)
interval