🐛 wait for lb dns name to propagate before resolving #5033
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of trying to resolve the primary LB DNS name right after its creation, wait for it to propagate so the resolution is most likely to succeed. This fixes an issue where the first "no such host" cached dns response with high TTL would make CAPA spin for minutes (as high as 15!) waiting for the DNS name to resolve even though it had already propagated a few minutes after the first attempt.
This should help speed things up a bit while we are waiting for the primary LB DNS name to propagate.
k8s-ci-robot
added
release-note
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
needs-ok-to-test
|
Hi @r4f4. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
r4f4
added a commit
to r4f4/installer
that referenced
this pull request
Jun 22, 2024
* Fix for DNS name resolution too early: kubernetes-sigs/cluster-api-provider-aws#5033
|
Openshift e2e tests show no regressions for cases with low TTL. For example in this run the DNS name is resolved right after the wait is done: and we can see that the secondary LB is reconciling while we wait for the dns name: |
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
|
/test pull-cluster-api-provider-aws-e2e-blocking |
5 tasks
|
The Kubernetes project currently lacks enough contributors to adequately respond to all PRs. This bot triages PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
k8s-ci-robot
added
the
lifecycle/stale
|
/lgtm Can you please reword the changelog entry – "a possible issue" is too vague. |
k8s-ci-robot
added
the
lgtm
Done. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind bug
What this PR does / why we need it:
Instead of trying to resolve the primary LB DNS name right after its creation, wait for it to propagate so the resolution is most likely to succeed.
This fixes an issue where the first "no such host" cached dns response with high TTL would make CAPA spin for minutes (as high as 15!) waiting for the DNS name to resolve even though it had already propagated a few minutes after the first attempt.
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #5032
Special notes for your reviewer:
I couldn't find a more elegant way to solve this other than a sleep after the LB is created. I wanted to add a
retryAfterDurationhere right after the DNS name is set and before the name resolution is attempted but it would involve somehow saving state of the timestamp in between reconcile loops.Checklist:
Release note: