Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not gracefully drain the proxy on SIGTERM #2266

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

olix0r
Copy link
Member

@olix0r olix0r commented Feb 24, 2023

Per the discussion in linkerd/linkerd2#10379, the proxy's shutdown behavior is overly aggressive and does not properly honor a pod's terminationGracePeriodSeconds configuration: while an application container may continue to run with a server for some period after the pod's delete process begins, the proxy will not permit any new connections in or out of the pod. This may also interfere with administrative probes during the shutdown process.

To address this, the proxy is updated to only change its readiness probe state when it receives a SIGTERM. The proxy continues to run-- without closing any connections or rejecting any work--until it is terminated (i.e., by kubelet) with a SIGKILL.

It is expected that, in most cases, clients will voluntarily move traffic as they process discovery updates. We need not do anything to interfere with the application's graceful termination behavior.

The proxy continues its former graceful shutdown behavior when shutdown is initiated via the admin server /shutdown endpoint (e.g., by linkerd-await).

Per the discussion in linkerd/linkerd2#10379, the proxy's shutdown
behavior is overly aggressive and does not properly honor a pod's
`terminationGracePeriodSeconds` configuration: while an application
container may continue to run with a server for some period after the
pod's delete process begins, the proxy will not permit any new
connections in or out of the pod. This may also interfere with
administrative probes during the shutdown process.

To address this, the proxy is updated to only change its readiness
probe state when it receives a SIGTERM. The proxy continues to run--
without closing any connections or rejecting any work--until it is
terminated (i.e., by kubelet) with a SIGKILL.

It is expected that, in most cases, clients will voluntarily move
traffic as they process discovery updates. We need not do anything to
interfere with the application's graceful termination behavior.

The proxy continues its former graceful shutdown behavior when shutdown
is initiated via the admin server `/shutdown` endpoint (e.g., by
linkerd-await).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant