diff --git a/docs/content/enterprise-addons/index.md b/docs/content/enterprise-addons/index.md new file mode 100644 index 00000000..0308c8d4 --- /dev/null +++ b/docs/content/enterprise-addons/index.md @@ -0,0 +1,20 @@ +# Enterprise Addons + +This document contains the documentation of the available Kamaji addons. + +## What is a Kamaji Addon + +A Kamaji Addon is a separate component installed in the same Kamaji cluster. +It offers an additional set of features required for enterprise-grade usage. + +The developed Kamaji addons are closed-source available and work with the upstream Kamaji edition. + +## Distribution of Addons + +The Kamaji Addons are available behind an active [subscription license](https://clastix.io/support/). + +Once a subscription is activated, the [CLASTIX](https://clastix.io) team will automate the push of required OCI (container images) and Helm Chart artefacts to the customer's OCI-compatible repository. + +## Available addons + +- [Ingress Addon](/enterprise-addons/ingress): expose Tenant Control Planes behind an Ingress Controller to reduce the amount of required LoadBalancer services \ No newline at end of file diff --git a/docs/content/enterprise-addons/ingress.md b/docs/content/enterprise-addons/ingress.md new file mode 100644 index 00000000..323c2620 --- /dev/null +++ b/docs/content/enterprise-addons/ingress.md @@ -0,0 +1,176 @@ +# Ingress Addon + +A Kubernetes API Server could be announced to users in several ways. +The most preferred way is leveraging on Load Balancers with their dedicated IP. + +However, IPv4 addresses could be limited and scarce in availability, as well as expensive for public ones when running in the Cloud. +A possible optimisation could be implementing an Ingress Controller which routes traffic to Kubernetes API Servers on a host-routing basis. + +Despite this solution sounding optimal for end users, it brings some challenges from the worker nodes' standpoint. + +## Challenges + +Internally deployed applications that need to interact with the Kubernetes API Server will leverage on the `kubernetes` endpoint in the `default` namespace: +every request sent to the `https://kubernetes.default.svc` endpoint will be forwarded to the Kubernetes API Server. + +The routing put in place by the Kubernetes CNI is based on the L4, meaning that all the requests will be forwarded to the Ingress Controller with no `Host` header, +making impossible a routing based on the FQDN. + +## Solution + +The `kamaji-addon-ingress` is an addon that will expose the Tenant Control Plane behind an Ingress Controller. +It's responsible for creating an `Ingress` object with the required HTTP rules, as well as the annotations needed for the TLS/SSL passthrough. + +Following is the list of supported Ingress Controllers: + +- [HAProxy Technologies Kubernetes Ingress](https://github.com/haproxytech/kubernetes-ingress) + +> Active subscribers can request additional Ingress Controller flavours + +## How to enable the Addon + +Annotate the Tenant Control Plane instances with the key `kamaji.clastix.io/ingress.domain` and the domain suffix domain value: + +```shell +kubectl annotate tenantcontrolplane $NAME kamaji.clastix.io/ingress.domain=$SUFFIX_DOMAIN +``` + +The value must be the expected suffix domain of generated resources. + +```yaml +apiVersion: kamaji.clastix.io/v1alpha1 +kind: TenantControlPlane +metadata: + annotations: + kamaji.clastix.io/ingress.domain: clastix.cloud # the expected kamaji-addon-ingress label + name: tenant-00 + namespace: apezzuto +``` + +Once a Tenant Control Plane has been annotated with this key, the addon will generate the following `Ingress` object. + +```yaml +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + annotations: + haproxy.org/ssl-passthrough: "true" + name: 592ee7b8-cd07-48cf-b754-76f370c3f87c + namespace: apezzuto +spec: + ingressClassName: haproxy + rules: + - host: apezzuto-tenant-00.k8s.clastix.cloud + http: + paths: + - backend: + service: + name: tenant-00 + port: + number: 6443 + path: / + pathType: Prefix + - host: apezzuto-tenant-00.konnectivity.clastix.cloud + http: + paths: + - backend: + service: + name: tenant-00 + port: + number: 8132 + path: / + pathType: Prefix +``` + +The pattern for the generated hosts is the following: +`${tcp.namespace}-${tcp.name}.{k8s|konnectivity}.${ADDON_ANNOTATION_VALUE}` + +> Please, notice the `konnectivity` rule will be created only if the `konnectivity` addon has been enabled. + +## Infrastructure requirements + +For Tenant Control Plane objects leveraging on this addon, the following changes must be implemented. + +### Ingress Controller + +The Ingress Controller must be deployed to listen for `https` connection on the default port `443`: +if you have different requirements, please, engage with the CLASTIX team. + +### DNS resolution + +The following zones must be configured properly according to your DNS provider: + +``` +*.konnectivity.clastix.cloud A +*.k8s.clastix.cloud A +``` + +### Certificate SANs + +```yaml + networkProfile: + certSANs: + - apezzuto-tenant-00.k8s.clastix.cloud + - apezzuto-tenant-00.konnectivity.clastix.cloud + dnsServiceIPs: + - 10.96.0.10 + podCidr: 10.244.0.0/16 + port: 6443 + serviceCidr: 10.96.0.0/16 +``` + +### Service type and Ingress + +The Kubernetes API Server can be exposed using a `ClusterIP`, rather than a Load Balancer. + +```yaml +spec: + controlPlane: + service: + serviceType: ClusterIP + ingress: + hostname: apezzuto-tenant-00.k8s.clastix.cloud:443 + ingressClassName: unhandled +``` + +The `ingressClassName` value must match a non-handled `IngressClass` object, +the addon will take care of generating the correct object. + +> Nota Bene: the `hostname` must absolutely point to the 443 port + +### Kubernetes components extra Arguments + +The Kubernetes API Server must start with the following flag: + +```yaml +spec: + controlPlane: + deployment: + extraArgs: + apiServer: + - --endpoint-reconciler-type=none +``` + +The `kamaji-addon-ingress` will be responsible for populating the `kubernetes` EndpointSlice object in the Tenant cluster. + +If you're running with `konnectivity`, also this extra argument must be enforced: + +```yaml +spec: + addons: + konnectivity: + agent: + extraArgs: + - --proxy-server-host=apezzuto-tenant-00.konnectivity.clastix.cloud + - --proxy-server-port=443 +``` + +## Air-gapped environments + +The `kamaji-addon-ingress` works with a deployed component in the Tenant Cluster based on the container image `docker.io/clastix/tcp-proxy:latest`. + +The same image can be replaced by customising the Addon Helm value upon installation: + +``` +--set options.tcpProxyImage=private.repository.tld/tcp-proxy:latest +``` diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 66aa8769..30d20220 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -77,4 +77,7 @@ nav: - reference/versioning.md - reference/api.md - 'Telemetry': telemetry.md +- 'Enterprise Addons': + - enterprise-addons/index.md + - enterprise-addons/ingress.md - 'Contribute': contribute.md