diff --git a/CNAME b/CNAME
index 2715751..c584db5 100644
--- a/CNAME
+++ b/CNAME
@@ -1 +1 @@
-blog.ogenki.io
\ No newline at end of file
+blog.ogenki.io.
diff --git a/fr/about/index.html b/fr/about/index.html
index 59761ea..6db6bb0 100644
--- a/fr/about/index.html
+++ b/fr/about/index.html
@@ -6,9 +6,7 @@
 <a href=https://blog.ogenki.io/fr/ class="nav_child nav_item">Français</a></div></div><div class=follow><a href=https://github.com/Smana><svg class="icon"><title>github</title><use xlink:href="#github"/></svg></a><a href=https://twitter.com/_smana_><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href=https://www.linkedin.com/in/sma%C3%AFne-kahlouch-44374110/><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><div class=color_mode><input type=checkbox class=color_choice id=mode></div></div></div></nav></header><main><div class="grid-inverse wrap content"><article class=post_content><h1 class=post_title>About</h1><div class=post_meta><span><svg class="icon"><title>calendar</title><use xlink:href="#calendar"/></svg></span><span class=post_date>18-07-2022</span>
 <span class=post_time>· 1 mins de lecture</span>
 <span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=About&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fabout%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fabout%2f&t=About" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/about/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_body><h2 id=qui-suis-je>Qui suis-je?</h2><p>I'm a senior Site Reliability Engineer with a particular interest in Linux containers and cloud technologies.
-I worked in different companies (small startups and large scale) and I've been working in different areas in order to improve the reliability, availability of the platform as well as the developer experience.
-I helped several companies in their transition to the Cloud. I was leading SRE/DevOps teams (diverse profiles with developers and SREs) and I really enjoy seeing them engaged in the same direction.</p><p>As side activities, I am an organizer of the Cloud Native Computing meetup in Paris and the <a href=https://www.kcdfrance.fr/>Kubernetes Community Days France</a>.</p><p>Hobbies: Reading SF books, Kick Boxing, Surfing/Skating/Inline Roller</p><center><img src=about.png width=250></center></div><div class=post_comments></div><div class=post_i18n><h3 class=mb-0>Traductions:</h3><nav class=tags_nav><a href=https://blog.ogenki.io/about/ class="post_tag button button_translucent">English</a></nav></div></article><aside class=sidebar><section class=sidebar_inner><br><div class=search><input type=search class="search_field form_field" placeholder=Rechercher... id=find autocomplete=off data-scope>
+<a href="https://twitter.com/intent/tweet?text=About&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fabout%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fabout%2f&t=About" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/about/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_body><h2 id=qui-suis-je>Qui suis-je?</h2><p>Je suis un <strong>ingénieur système / SRE senior</strong>. Je porte un intérêt particulier aux conteneurs Linux et les technologies dîtes "Cloud Native". Au fil de ma carrière, j'ai eu l'occasion de collaborer avec une variété d'entreprises, des startups dynamiques aux grandes structures, où j'ai contribué à optimiser la fiabilité et la disponibilité des plateformes, tout en améliorant l'expérience des développeurs. Mon parcours inclut l'accompagnement de plusieurs sociétés dans leur migration vers des solutions Cloud. J'ai également pu <strong>diriger des équipes SRE/DevOps</strong> composées de profils variés, incluant développeurs et ingénieurs SRE, avec pour objectif un engagement collectif vers des objectifs communs.</p><p>En parallèle de mon parcours professionnel, je m'investis dans l'organisation du meetup Cloud Native Computing à Paris ainsi que les <a href=https://www.kcdfrance.fr/>Kubernetes Community Days France</a> , reflétant mon engagement continu dans l'écosystème du Cloud.</p><p>Autres centre d'intéret : Lecture de romans de science-fiction, Kickboxing, Surf, Longboard et Rollers..</p><center><img src=about.png width=250></center></div><div class=post_comments></div><div class=post_i18n><h3 class=mb-0>Traductions:</h3><nav class=tags_nav><a href=https://blog.ogenki.io/about/ class="post_tag button button_translucent">English</a></nav></div></article><aside class=sidebar><section class=sidebar_inner><br><div class=search><input type=search class="search_field form_field" placeholder=Rechercher... id=find autocomplete=off data-scope>
 <label for=find class=search_label><svg class="icon"><title>search</title><use xlink:href="#search"/></svg></label><div class="search_results results"></div></div><div class=author_header><img src=https://blog.ogenki.io/images/smana.jpg alt="Smaine Kahlouch photo"><h2>Smaine Kahlouch</h2></div><div class=author_bio>Lead SRE/DevOps, Team leader, Open Source enthusiast.</div><a href=https://blog.ogenki.io/fr/about/ class="button mt-1" role=button title='Lire la suite…'>Lire la suite…</a><h2 class=mt-4>Billets en vedette</h2><ul><li><a href=https://blog.ogenki.io/fr/post/tailscale/ class=nav-link title="Sécuriser le Cloud avec `Tailscale` : Mise en œuvre d'un VPN simplifiée">Sécuriser le Cloud avec <code>Tailscale</code> : Mise en œuvre d'un VPN simplifiée</a></li><li><a href=https://blog.ogenki.io/fr/post/cilium-gateway-api/ class=nav-link title="`Gateway API`: Remplacer mon Ingress Controller avec `Cilium`?"><code>Gateway API</code>: Remplacer mon Ingress Controller avec <code>Cilium</code>?</a></li><li><a href=https://blog.ogenki.io/fr/post/terraform-controller/ class=nav-link title="Appliquer les principes de GitOps à l'infrastructure: Introduction à `tf-controller`">Appliquer les principes de GitOps à l'infrastructure: Introduction à <code>tf-controller</code></a></li></ul><h2 class=mt-4>Billets récents</h2><ul class=flex-column><li><a href=https://blog.ogenki.io/fr/post/cnpg/ class=nav-link title="`CloudNativePG`: et PostgreSQL devient facile sur Kubernetes"><code>CloudNativePG</code>: et PostgreSQL devient facile sur Kubernetes</a></li><li><a href=https://blog.ogenki.io/fr/post/devflux/ class=nav-link title="100% `GitOps` using Flux">100% <code>GitOps</code> using Flux</a></li><li><a href=https://blog.ogenki.io/fr/post/crossplane_k3d/ class=nav-link title="Mon cluster Kubernetes (GKE) avec `Crossplane`">Mon cluster Kubernetes (GKE) avec <code>Crossplane</code></a></li><li><a href=https://blog.ogenki.io/fr/post/asdf/asdf/ class=nav-link title="Gérer les versions d'outils avec `asdf`">Gérer les versions d'outils avec <code>asdf</code></a></li></ul><div><h2 class="mt-4 taxonomy" id=tags-section>Tags</h2><nav class=tags_nav><a href=https://blog.ogenki.io/fr/tags/infrastructure/ class="post_tag button button_translucent" title=infrastructure>INFRASTRUCTURE
 <span class=button_tally>3</span>
 </a><a href=https://blog.ogenki.io/fr/tags/kubernetes/ class="post_tag button button_translucent" title=kubernetes>KUBERNETES
diff --git a/fr/index.json b/fr/index.json
index b22de54..626b036 100644
--- a/fr/index.json
+++ b/fr/index.json
@@ -1 +1 @@
-[{"body":"","link":"https://blog.ogenki.io/fr/","section":"","tags":null,"title":""},{"body":"","link":"https://blog.ogenki.io/fr/tags/network/","section":"tags","tags":null,"title":"network"},{"body":"","link":"https://blog.ogenki.io/fr/post/","section":"post","tags":["index"],"title":"Posts"},{"body":"Lorsqu'on parle de sécurisation de l'accès aux ressources Cloud, l'une des règles d'or est d'éviter les expositions directes à Internet. La question qui se pose alors pour les Devs/Ops est : comment, par exemple, accéder à une base de données, un cluster Kubernetes ou un serveur via SSH sans compromettre la sécurité? Les réseaux privés virtuels (VPN) offrent une réponse en établissant un lien sécurisé entre différents éléments d'un réseau, indépendamment de leur localisation géographique. De nombreuses solutions existent, allant de modèles en SaaS aux solutions que l'on peut héberger soi-même, utilisant divers protocoles et étant soit open source, soit propriétaires.\nParmi ces options, je souhaitais vous parler de Tailscale. Cette solution utilise le protocole WireGuard, réputé pour sa simplicité et sa performance. Avec Tailscale, il est possible de connecter des appareils ou serveurs de manière sécurisée, comme s'ils étaient sur un même réseau local, bien qu'ils soient répartis à travers le monde.\n:bullseye: Nos objectifs Comprendre comment fonctionne Tailscale Mise en oeuvre d'une connexion sécurisée avec AWS en quelques minutes Interragir avec l'API d'un cluster EKS via un réseau privé Accéder à des services hébergés sur Kubernetes en utilisant le réseau privé Pour le reste de cet article il faudra évidemment créer un compte Tailscale. A noter que l'authentification est déléguée à des fournisseurs d'identité tiers (ex: Okta, Onelogin, Google ...).\nLorsque le compte est crée, on a directement accès à la console de gestion ci-dessus. Elle permet notamment de lister les appareils connectés, de consulter les logs, de modifier la plupart des paramètres...\n💡 Sous le capot Terminologie Mesh VPN: Un mesh VPN est un type de réseau VPN où chaque nœud (c'est-à-dire chaque appareil ou machine) est connecté à tous les autres nœuds du réseau, formant ainsi un maillage. À distinguer des configurations VPN traditionnelles qui sont conçues généralement \u0026quot;en étoile\u0026quot;, où plusieurs clients se connectent à un serveur central.\nZero trust: Signifie que chaque demande d'accès à un réseau est traitée comme si elle venait d'une source non fiable. Une application ou utilisateur doit prouver son identité et être autorisée avant d'accéder à une ressource. On ne fait pas confiance simplement parce qu'une machine ou un utilisateur provient d'un réseau interne ou d'une certaine zone géographique.\nTailnet: Dès la première utilisation de Tailscale, un Tailnet est crée pour vous et correspond à votre propre réseau privé. Chaque appareil dans un tailnet reçoit une IP Tailscale unique, permettant une communication directe entre eux.\nL'architecture de Tailscale est conçue de telle sorte que le Control plane et le Data plane sont clairement séparés:\nD'une part, il y a le serveur de coordination. Son rôle est d'échanger des métadonnées et des clés publiques entre tous les participants d'un Tailnet (La clé privée étant gardée en toute sécurité son nœud d'origine).\nD'autre part, les nœuds du Tailnet s'organisent en un réseau maillé (Mesh). Au lieu de passer par le serveur de coordination pour échanger des données, ces nœuds communiquent directement les uns avec les autres en mode point à point. Chaque nœud dispose d'une identité unique pour s'authentifier et rejoindre le Tailnet.\n\u0026#x1f4e5; Installation du client La majorité des plateformes sont supportées et les procédures d'installation sont listées ici. En ce qui me concerne je suis sur Archlinux:\n1sudo pacman -S tailscale Il est possible de démarrer le service automatiquement au démarrage de la machine.\n1sudo systemctl enable --now tailscaled Pour enregistrer son ordinateur perso, lancer la commande suivante:\n1sudo tailscale up --accept-routes 2 3To authenticate, visit: 4 5 https://login.tailscale.com/a/f50... ℹ️ l'option --accept-routes est nécessaire sur Linux et permettra d'accepter les routes annoncées par les Subnet routers. On verra cela dans la suite de l'article\nVérifier que vous avez bien obtenu une IP du réseau Tailscale:\n1tailscale ip -4 2100.118.83.67 3 4tailscale status 5100.118.83.67 ogenki smainklh@ linux - ℹ️ Pour les utilisateurs de Linux, vérifier que Tailscale fonctionne bien avec votre configuration DNS: Suivre cette documentation.\nLes sources Toutes les étapes réalisées dans cet article proviennent de ce dépôt git\nIl va permettre de créer l'ensemble des composants qui ont pour objectif d'obtenir un cluster EKS de Lab et font suite à un précédent article sur Cilium et Gateway API.\n☁️ Accéder à AWS en privé Afin de pouvoir accéder de manière sécurisée à l'ensemble des ressources disponibles sur AWS, il est possible de déployer un Subnet router.\nUn Subnet router est une instance Tailscale qui permet d'accéder à des sous-réseaux qui ne sont pas directement liés à Tailscale. Il fait office de pont entre le réseau privé virtuel de Tailscale (Tailnet) et d'autres réseaux locaux.\nNous pouvons alors router des sous réseaux du Clouder à travers le VPN de Tailscale.\n⚠️ Pour ce faire, sur AWS, il faudra bien entendu configurer les security groups correctement pour autoriser les Subnet routers.\n🚀 Déployer un Subnet router Entrons dans le vif du sujet et deployons un Subnet router sur un réseau AWS! Tout est fait en utilisant le code Terraform présent dans le répertoire terraform/network. Nous allons analyser la configuration spécifique à Tailscale qui est présente dans le fichier tailscale.tf avant de procéder au déploiement.\nLe provider Terraform Il est possible de configurer certains paramètres au travers de l'API Tailscale grâce au provider Terraform. Pour cela il faut au préalable génerer une clé d'API 🔑 sur la console d'admin:\nIl faudra conserver cette clé dans un endroit sécurisé car elle est utilisée pour déployer le Subnet router\n1provider \u0026#34;tailscale\u0026#34; { 2 api_key = var.tailscale.api_key 3 tailnet = var.tailscale.tailnet 4} les ACL's\nLes ACL's permettent de définir qui est autorisé à communiquer avec qui (utilisateur ou appareil). À la création d'un compte, celle-cis sont très permissives et il n'y a aucune restriction (tout le monde peut parler avec tout le monde).\n1resource \u0026#34;tailscale_acl\u0026#34; \u0026#34;this\u0026#34; { 2 acl = jsonencode({ 3 acls = [ 4 { 5 action = \u0026#34;accept\u0026#34; 6 src = [\u0026#34;*\u0026#34;] 7 dst = [\u0026#34;*:*\u0026#34;] 8 } 9 ] 10... 11} Note Pour mon environnement de Lab, j'ai conservé cette configuration par défault car je suis la seule personne à y accéder. De plus les seuls appareils connectés à mon Tailnet sont mon laptop et le Subnet router. En revanche dans un cadre d'entreprise, il faudra bien y réfléchir. Il est alors possible de définir une politique basée sur des groupes d'utilisitateurs ou sur les tags des noeuds.\nConsulter cette doc pour plus d'info.\nLes noms de domaines (DNS)\nIl y a différentes façons possibles de gérer les noms de domaines avec Tailscale:\nMagic DNS: Lorsqu'un appareil rejoint le Tailnet, il s'enregistre avec un nom et celui-ci peut-être utilisé directement pour communiquer avec l'appareil.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.115.31.152 ip-10-0-43-98 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 3044 rx 2588 4 5ping ip-10-0-43-98 6PING ip-10-0-43-98.tail9c382.ts.net (100.115.31.152) 56(84) bytes of data. 764 bytes from ip-10-0-43-98.tail9c382.ts.net (100.115.31.152): icmp_seq=1 ttl=64 time=11.4 ms AWS: Pour utiliser les noms de domaines internes à AWS il est possible d'utiliser la deuxième IP du VPC qui correspond toujours au serveur DNS. Cela permet d'utiliser les éventuelles zones privées sur route53 ou de se connecter aux ressources en utilisant les noms de domaines.\nLa configuration la plus simple est donc de déclarer la liste des serveurs DNS à utiliser et d'y ajouter celui de AWS. Ici un exemple avec le DNS publique de Cloudflare.\n1resource \u0026#34;tailscale_dns_nameservers\u0026#34; \u0026#34;this\u0026#34; { 2 nameservers = [ 3 \u0026#34;1.1.1.1\u0026#34;, 4 cidrhost(module.vpc.vpc_cidr_block, 2) 5 ] 6} La clé d'authentification (\u0026quot;auth key\u0026quot;)\nPour qu'un appareil puisse rejoindre le Tailnet au démarrage il faut que Tailscale soit démarré en utilisant une clé d'authentification. Celle-ci est générée comme suit\n1resource \u0026#34;tailscale_tailnet_key\u0026#34; \u0026#34;this\u0026#34; { 2 reusable = true 3 ephemeral = false 4 preauthorized = true 5} reusable: S'agissant d'un autoscaling group, il faut que cette même clé puisse être utilisée plusieurs fois. ephemeral: Pour cette démo nous créons une clé qui n'expire pas. En production il serait préférable d'activer l'expiration. preauthorized: Il faut que cette clé soit déjà valide et autorisée pour que l'instance rejoigne automatiquement le Tailscale. La clé ainsi générée est utilisée pour lancer tailscale avec le paramètre --auth-key\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; Annoncer les routes pour les réseaux AWS\nEnfin il faut annoncer le réseau que l'on souhaite faire passer par le Subnet router. Dans notre exemple, nous décidons de router tout le réseau du VPC qui a pour CIDR 10.0.0.0/16.\nAfin que cela soit possible de façon automatique, il y a une règle autoApprovers à ajouter. Cela permet d'indiquer que les routes annoncées par l'utilisateur smainklh@gmail.com sont autorisées sans que cela requiert une étape d'approbation.\n1 autoApprovers = { 2 routes = { 3 \u0026#34;10.0.0.0/16\u0026#34; = [\u0026#34;smainklh@gmail.com\u0026#34;] 4 } 5 } La commande lancée au démarrage de l'instance Subnet router est la suivante:\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; --advertise-routes=\u0026#34;10.0.0.0/16\u0026#34; Le module Terraform J'ai créé un module très simple qui permet de déployer un autoscaling group sur AWS et de configurer Tailscale. Au démarrage de l'instance, elle s'authentifiera en utilisant une auth_key et annoncera les réseaux indiqués. Dans l'exemple ci-dessous l'instance annonce le CIDR du VPC sur AWS.\n1module \u0026#34;tailscale_subnet_router\u0026#34; { 2 source = \u0026#34;Smana/tailscale-subnet-router/aws\u0026#34; 3 version = \u0026#34;1.0.4\u0026#34; 4 5 region = var.region 6 env = var.env 7 8 name = var.tailscale.subnet_router_name 9 auth_key = tailscale_tailnet_key.this.key 10 11 vpc_id = module.vpc.vpc_id 12 subnet_ids = module.vpc.private_subnets 13 advertise_routes = [module.vpc.vpc_cidr_block] 14... 15} Maintenant que nous avons analysé les différents paramètres, il est temps de démarrer notre Subnet router 🚀 !! Il faut au préalable créer un fichier variable.tfvars dans le répertoire terraform/network.\n1env = \u0026#34;dev\u0026#34; 2region = \u0026#34;eu-west-3\u0026#34; 3private_domain_name = \u0026#34;priv.cloud.ogenki.io\u0026#34; 4 5tailscale = { 6 subnet_router_name = \u0026#34;ogenki\u0026#34; 7 tailnet = \u0026#34;smainklh@gmail.com\u0026#34; 8 api_key = \u0026#34;tskey-api-...\u0026#34; 9} 10 11tags = { 12 project = \u0026#34;demo-secured-eks\u0026#34; 13 owner = \u0026#34;Smana\u0026#34; 14} Puis lancer la commande suivante:\n1tofu plan --var-file variables.tfvars Après vérification du plan, appliquer les changements\n1tofu apply --var-file variables.tfvars Quand l'instance est démarrée, elle apparaitra dans la liste des appareils du Tailnet.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.68.109.138 ip-10-0-26-99 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 33868 rx 32292 Nous pouvons aussi vérifier que la route est bien annoncée comme suit:\n1tailscale status --json|jq \u0026#39;.Peer[] | select(.HostName == \u0026#34;ip-10-0-26-99\u0026#34;) .PrimaryRoutes\u0026#39; 2[ 3 \u0026#34;10.0.0.0/16\u0026#34; 4] ⚠️ Pour des raisons de sécurité, pensez à supprimer le fichier variables.tfvars car il contient la clé d'API.\n👏 Et voilà ! Nous sommes maintenant en mesure d'accéder au réseau sur AWS, à condition d'avoir également configuré les règles de filtrage, comme les ACL et les security groups. Nous pouvons par exemple accéder à une base de données depuis le poste de travail\n1psql -h demo-tailscale.cymnaynfchjt.eu-west-3.rds.amazonaws.com -U postgres 2Password for user postgres: 3psql (15.4, server 15.3) 4SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off) 5Type \u0026#34;help\u0026#34; for help. 6 7postgres=\u0026gt; 💻 Une autre façon de faire du SSH Traditionnellement, nous devons parfois nous connecter à des serveurs en utilisant le protocole SSH. Pour ce faire, il faut générer une clé privée et distribuer la clé publique correspondante sur les serveurs distants.\nContrairement à l'utilisation des clés SSH classiques, étant donné que Tailscale utilise Wireguard pour l'authentification et le chiffrement des connexions il n'est pas nécessaire de ré-authentifier le client. De plus, Tailscale gère également la distribution des clés SSH d'hôtes. Les règles ACL permettent de révoquer l'accès des utilisateurs sans avoir à supprimer les clés SSH. De plus, il est possible d'activer un mode de vérification qui renforce la sécurité en exigeant une ré-authentification périodique. On peut donc affirmer que l'utilisation de Tailscale SSH simplifie l'authentification, la gestion des connexions SSH et améliore le niveau de sécurité.\nLes autorisations pour utiliser SSH sont aussi gérées au niveau des ACL's\n1... 2 ssh = [ 3 { 4 action = \u0026#34;check\u0026#34; 5 src = [\u0026#34;autogroup:member\u0026#34;] 6 dst = [\u0026#34;autogroup:self\u0026#34;] 7 users = [\u0026#34;autogroup:nonroot\u0026#34;] 8 } 9 ] 10... La règle ci-dessus autorise tous les utilisateurs à accéder à leurs propres appareils en utilisant SSH. Lorsqu'ils essaient de se connecter, ils doivent utiliser un compte utilisateur autre que root. Pour chaque tentative de connexion, une authentification supplémentaire est nécessaire (action=check). Cette authentification se fait en visitant un lien web spécifique\n1ssh ubuntu@ip-10-0-26-99 2... 3# Tailscale SSH requires an additional check. 4# To authenticate, visit: https://login.tailscale.com/a/f1f09a548cc6 5... 6ubuntu@ip-10-0-26-99:~$ Pour que cela soit possible il faut aussi démarrer Tailscale avec l'option --ssh\nLes logs d'accès à la machine peuvent être consultés en utilisant journalctl\n1ubuntu@ip-10-0-26-99:~$ journalctl -aeu tailscaled|grep ssh 2Oct 15 15:51:34 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155130-00ede660b8: handling conn: 100.118.83.67:55098-\u0026gt;ubuntu@100.68.109.138:22 3Oct 15 15:51:56 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: handling conn: 100.118.83.67:44560-\u0026gt;ubuntu@100.68.109.138:22 4Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: starting session: sess-20231015T155252-5b2acc170e 5Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): handling new SSH connection from smainklh@gmail.com (100.118.83.67) to ssh-user \u0026#34;ubuntu\u0026#34; 6Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): access granted to smainklh@gmail.com as ssh-user \u0026#34;ubuntu\u0026#34; 7Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): starting pty command: [/usr/sbin/tailscaled be-child ssh --uid=1000 --gid=1000 --groups=1000,4,20,24,25,27,29,30,44,46,115,116 --local-user=ubuntu --remote-user=smainklh@gmail.com --remote-ip=100.118.83.67 --has-tty=true --tty-name=pts/0 --shell --login-cmd=/usr/bin/login --cmd=/bin/bash -- -l] ℹ️ Avec Tailscale SSH il est possible de se connecter en SSH peu importe où est situé l'appareil. En revanche dans un contexte 100% AWS, on préferera probablement utiliser AWS SSM.\nLogs 💾 En sécurité il est primordial de pouvoir conserver les logs pour un usage ultérieur. Il existe différents types de logs:\nLogs d'audit: Ils sont essentiels pour savoir qui a fait quoi. Ils sont accessibles sur la console d'admin et peuvent aussi être envoyés vers un SIEM.\nLogs sur les appareils: Ceux-cis peuvent être consultés en utilisant les commandes appropriées à l'appareil. (journalctl -u tailscaled sur Linux)\nLogs réseau: Utiles pour visualiser quels appareils sont connectés les uns aux autres.\n☸ Qu'en est-il de Kubernetes? Sur Kubernetes il existe plusieurs options pour accéder à un Service:\nProxy: Il s'agit d'un pod supplémentaire qui transfert les appels à un Service existant. Sidecar: Permet de connecter le pod au Tailnet. Donc la connectivité se fait de bout en bout et il est même possible de communiquer dans les 2 sens. (du pod vers les noeuds du Tailnet). Operator: Permet d'exposer les services et l'API Kubernetes (ingress) ainsi que de permettre aux pods d'accéder aux noeuds du Tailnet (egress). La configuration se fait en configurant les ressources existantes: Services et Ingresses Dans notre cas, nous disposons déjà d'un Subnet router qui route tout le réseau du VPC. Il suffit donc que notre service soit exposé sur une IP privée.\nL'API Kubernetes Pour accéder à l'API Kubernetes il est nécessaire d'autoriser le Subnet router. Cela se fait en définissant la règle suivante pour le security group source.\n1module \u0026#34;eks\u0026#34; { 2... 3 cluster_security_group_additional_rules = { 4 ingress_source_security_group_id = { 5 description = \u0026#34;Ingress from the Tailscale security group to the API server\u0026#34; 6 protocol = \u0026#34;tcp\u0026#34; 7 from_port = 443 8 to_port = 443 9 type = \u0026#34;ingress\u0026#34; 10 source_security_group_id = data.aws_security_group.tailscale.id 11 } 12 } 13... 14} Nous allons vérifier que l'API est bien accessible sur une IP privée.\n1CLUSTER_URL=$(TERM=dumb kubectl cluster-info | grep \u0026#34;Kubernetes control plane\u0026#34; | awk \u0026#39;{print $NF}\u0026#39;) 2 3curl -s -o /dev/null -w \u0026#39;%{remote_ip}\\n\u0026#39; ${CLUSTER_URL} 410.228.244.167 5 6kubectl get ns 7NAME STATUS AGE 8cilium-secrets Active 5m46s 9crossplane-system Active 4m1s 10default Active 23m 11flux-system Active 5m29s 12infrastructure Active 4m1s 13... Accéder aux services en privé Un Service Kubernetes exposé est une resource AWS comme une autre 😉. Il faut juste s'assurer que ce service utilise bien une IP privée. Dans mon exemple j'utilise Gateway API pour configurer la répartition de charge du Clouder et je vous invite à lire mon précédent article sur le sujet.\nIl suffirait donc de créer un NLB interne en s'assurant que le Service ait bien l'annotation service.beta.kubernetes.io/aws-load-balancer-scheme ayant pour valeur internal. Dans le cas de Gateway API, cela se fait via la clusterPolicy Kyverno.\n1 metadata: 2 annotations: 3 external-dns.alpha.kubernetes.io/hostname: gitops-${cluster_name}.priv.${domain_name},grafana-${cluster_name}.priv.${domain_name} 4 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internal\u0026#34; 5 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 6 spec: 7 loadBalancerClass: service.k8s.aws/nlb Il y a cependant un prérequis supplémentaire car nous ne pouvons pas utiliser Let's Encrypt pour les certificats internes. J'ai donc généré une PKI interne qui génère des certificates auto-signés avec Cert-manager.\nIci je ne détaillerai pas le déploiement du cluster EKS, ni la configuration de Flux. Lorsque le cluster est créé et que toutes les ressources Kubernetes ont été réconcilié, nous avons un service qui est exposé via un LoadBalancer interne AWS.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2dig +short ${NLB_DOMAIN} 310.0.33.5 410.0.26.228 510.0.9.183 Une entrée DNS est également créée automatiquement pour les services exposés et nous pouvons donc accéder en privé grâce à Tailscale.\n1dig +short gitops-mycluster-0.priv.cloud.ogenki.io 210.0.9.183 310.0.26.228 410.0.33.5 💭 Dernières remarques Il y a quelques temps, dans le cadre professionnel, j'ai mis en place Cloudflare Zero Trust. Je découvre ici que Tailscale présente de nombreuses similitudes avec cette solution. La décision entre les deux est loin d'être triviale et dépend grandement du contexte. Pour ma part, j'ai été particulièrement convaincu par la simplicité de mise en œuvre de Tailscale, répondant parfaitement à mon besoin d'accéder au réseau du Clouder. Bien entendu il existe d'autres solutions comme Teleport, qui offre une approche différente pour accéder à des ressources internes.\nCela dit, focalisons-nous sur Tailscale.\nUne partie du code de Tailscale est open source, notamment le client qui est sous license BSD 3-Clause. La partie propriétaire concerne éssentiellement la plateforme de coordination. À noter qu'il existe une alternative open source nommée Headscale. Celle-ci est une initiative distincte qui n'a aucun lien avec la société Tailscale.\nPour un usage personnel, Tailscale est vraiment généreux, offrant un accès gratuit pour jusqu'à 100 appareils et 3 utilisateurs. Ceci-dit Tailscale est une option sérieuse à considérer en entreprise et il est important, selon moi, d'encourager ce type d'entreprises qui ont une politique open source claire et un produit de qualité.\n","link":"https://blog.ogenki.io/fr/post/tailscale/","section":"post","tags":["security","network"],"title":"Sécuriser le Cloud avec `Tailscale` : Mise en œuvre d'un VPN simplifiée"},{"body":"","link":"https://blog.ogenki.io/fr/tags/security/","section":"tags","tags":null,"title":"security"},{"body":"","link":"https://blog.ogenki.io/fr/tags/","section":"tags","tags":null,"title":"Tags"},{"body":"Lorsque l'on déploie une application sur Kubernetes, l'étape suivante consiste généralement à l'exposer aux utilisateurs. On utilise habituellement des \u0026quot;Ingress controllers\u0026quot;, comme Nginx, Haproxy, Traefik ou encore ceux des différents Clouders afin de diriger le trafic entrant vers l'application, gérer l'équilibrage de charge, la terminaison TLS et j'en passe.\nIl faut alors choisir parmi la pléthore d'options disponibles 🤯 la solution qui sera en charge de tous ces aspects et Cilium est, depuis relativement récemment, l'une d'entre elles.\nCilium est une solution Open-Source de connectivité réseau et de sécurité basée sur eBPF dont l'adoption est grandissante. Il s'agit probablement du plugin réseau qui fournit le plus de fonctionnalités. Nous n'allons pas toutes les parcourir mais l'une d'entre elles consiste à gérer le trafic entrant en utilisant le standard Gateway API (GAPI).\n:bullseye: Notre objectif Comprendre ce qu'est exactement Gateway API et en quoi il s'agit d'une évolution par rapport à l'API Ingress. Démonstrations de cas concrets à la sauce GitOps. Les limitations actuelles et les évolutions à venir. Tip Toutes les étapes réalisées dans cet article proviennent de ce dépôt git\nJe vous invite à le parcourir car, il va bien au delà du contexte de cet article:\nInstallation d'un cluster EKS avec Cilium configuré en mode sans kube-proxy et un Daemonset dédié à Envoy Proposition de structure de configuration Flux avec une gestion des dépendances et une factorisation que je trouve efficaces. Crossplane et composition IRSA qui simplifie la gestion des permissions IAM pour les composants plateforme Gestion des noms de domaine ainsi que des certificats automatisés avec External-DNS et Let's Encrypt L'idée étant d'avoir l'ensemble configuré au bout de quelques minutes, en une seule ligne de commande 🤩.\n☸ Introduction à Gateway API Comme évoqué précédemment, il y a de nombreuses options qui font office d' Ingress controller et chacune a ses propres spécificités et des fonctionnalités particulières, rendant leur utilisation parfois complexe. Par ailleurs, l'API Ingress, utilisée historiquement dans Kubernetes possède très peu d'options. Certaines solutions ont d'ailleurs créé des CRDs (Ressources personalisées) quand d'autres font usage des annotations pour lever ces limites.\nC'est dans ce contexte que Gateway API fait son apparition. Il s'agit d'un standard qui permet de définir des fonctionnalités réseau avancées sans nécessiter d'extensions spécifiques au contrôleur sous-jacent. De plus étant donné que tous les contrôleurs utilisent la même API , il est possible de passer d'une solution à une autre sans changer de configuration (les ressources qui gèrent le trafic entrant restent les mêmes).\nParmi les concepts que nous allons explorer la GAPI introduit un schéma de répartition des responsabilités. Elle définit des roles explicites avec des permissions bien distinctes. (Plus d'informations sur le modèle de sécurité GAPI ici).\nEnfin il est important de noter que ce projet est dirigé par le groupe de travail sig-network-kubernetes et un canal slack vous permettra de les solliciter si nécessaire.\nVoyons comment cela s'utilise concrètement 🚀!\n\u0026#x2611;\u0026#xfe0f; Prérequis Pour le reste de cet article nous considérons qu'un cluster EKS a été déployé. Si vous n'utilisez pas la méthode proposée dans le repo de démo servant de socle à cet article, il y a certains points à valider pour que GAPI puisse être utilisé.\nℹ️ La méthode d'installation decrite ici se base sur Helm, l'ensemble des values peuvent être consultées ici.\nInstaller les CRDs (resources personnalisés) disponibles dans le repository Gateway API Note Si Cilium est configuré avec le support GAPI (voir ci-dessous) et que les CRDs sont absentes, il ne démarrera pas.\nDans le repo de demo les CRDs GAPI sont installées une première fois lors de la création du cluster afin que Cilium puisse démarrer puis elles sont ensuite gérées par Flux.\nRemplacer kube-proxy par les fonctionnalités de transfert réseau apportées par Cilium et eBPF.\n1kubeProxyReplacement: true Activer le support de Gateway API 1gatewayAPI: 2 enabled: true Vérifier L'installation Il faut pour cela installer le client en ligne de commande cilium. J'utilise personnellement asdf pour cela:\n1asdf plugin-add cilium-cli 2asdf install cilium-cli 0.15.7 3asdf global cilium 0.15.7 La commande suivante permet de s'assurer que tous les composants sont démarrés et opérationnels\n1cilium status --wait 2 /¯¯\\ 3/¯¯\\__/¯¯\\ Cilium: OK 4\\__/¯¯\\__/ Operator: OK 5/¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6\\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled 8 9Deployment cilium-operator Desired: 2, Ready: 2/2, Available: 2/2 10DaemonSet cilium Desired: 2, Ready: 2/2, Available: 2/2 11DaemonSet cilium-envoy Desired: 2, Ready: 2/2, Available: 2/2 12Containers: cilium Running: 2 13 cilium-operator Running: 2 14 cilium-envoy Running: 2 15Cluster Pods: 33/33 managed by Cilium 16Helm chart version: 1.14.2 17Image versions cilium quay.io/cilium/cilium:v1.14.2@sha256:6263f3a3d5d63b267b538298dbeb5ae87da3efacf09a2c620446c873ba807d35: 2 18 cilium-operator quay.io/cilium/operator-aws:v1.14.2@sha256:8d514a9eaa06b7a704d1ccead8c7e663334975e6584a815efe2b8c15244493f1: 2 19 cilium-envoy quay.io/cilium/cilium-envoy:v1.25.9-e198a2824d309024cb91fb6a984445e73033291d@sha256:52541e1726041b050c5d475b3c527ca4b8da487a0bbb0309f72247e8127af0ec: 2 Enfin le support de GAPI peut être vérifié comme suit\n1cilium config view | grep -w \u0026#34;enable-gateway-api\u0026#34; 2enable-gateway-api true 3enable-gateway-api-secrets-sync true Il est aussi possible de lancer des tests de connectivité pour s'assurer qu'il n'y a pas de problèmes avec la configuration réseau du cluster:\n1cilium connectivity test ⚠️ Cette commande (connectivity test) provoque actuellement des erreurs lors de l'activation d'Envoy en tant que DaemonSet. (Issue Github).\nInfo as DaemonSet\nPar défaut l'agent cilium intégre Envoy et lui délegue les opérations réseau de niveau 7. Depuis la version v1.14, il est possible de déployer Envoy séparément ce qui apporte certains avantages:\nSi l'on modifie/redémarre un composant (que ce soit Cilium ou Envoy), cela n'affecte pas l'autre. Mieux attribuer les ressources à chacun des composants afin d'optimiser les perfs. Limite la surface d'attaque en cas de compromission d'un des pods. Les logs Envoy et de l'agent Cilium ne sont pas mélangés Il est possible d'utiliser la commande suivante pour vérifier que cette fonctionnalité est bien active:\n1cilium status 2 /¯¯\\ 3 /¯¯\\__/¯¯\\ Cilium: OK 4 \\__/¯¯\\__/ Operator: OK 5 /¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6 \\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled Plus d'information.\n🚪 La porte d'entrée: GatewayClass et Gateway Une fois les conditions nécessaires remplies, nous avons accès à plusieurs éléments. Nous pouvons notamment utiliser les ressources de la Gateway API grâce aux CRDs. D'ailleurs, dès l'installation de Cilium, une GatewayClass est directement disponible.\n1kubectl get gatewayclasses.gateway.networking.k8s.io 2NAME CONTROLLER ACCEPTED AGE 3cilium io.cilium/gateway-controller True 7m59s Sur un cluster il est possible de configurer plusieurs GatewayClass et donc d'avoir la possibilité de faire usage de différentes implémentations. Nous pouvons par exemple utiliser Linkerd en référencant la GatewayClass dans la configuration de la Gateway.\nLa Gateway est la ressource qui permet déclencher la création de composants de répartition de charge chez le Clouder.\nVoici un exemple simple: apps/base/echo/gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo-gateway 5 namespace: echo 6spec: 7 gatewayClassName: cilium 8 listeners: 9 - protocol: HTTP 10 port: 80 11 name: echo-1-echo-server 12 allowedRoutes: 13 namespaces: 14 from: Same Sur AWS (EKS), quand on configure une Gateway, Cilium crée un Service de type LoadBalancer. Ce service est alors interprété par un autre contrôleur, l'(AWS Load Balancer Controler), qui produit un NLB.\n1kubectl get svc -n echo cilium-gateway-echo 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3cilium-gateway-echo LoadBalancer 172.20.19.82 k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com 80:30395/TCP 2m58s Il est intéressant de noter que l'adresse en question est aussi associée à la resource Gateway.\n1kubectl get gateway -n echo echo 2NAME CLASS ADDRESS PROGRAMMED AGE 3echo cilium k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com True 16m \u0026#x21aa;\u0026#xfe0f; Les règles de routage: HTTPRoute Un routage simple Pour résumer le schéma ci-dessus en quelques mots: Une ressource HTTPRoute permet de configurer le routage vers le service en référencant la gateway et en définissant le les paramètres de routage souhaités.\nNote workaround\nÀ ce jour, il n'est pas possible de configurer les annotations des services générés par les Gateways (Issue Github). Une solution de contournement a été proposé afin de modifier le service généré par la Gateway dès lors qu'il est créé.\nKyverno est un outil qui permet de garantir la conformité des configurations par rapport aux bonnes pratiques et aux exigences de sécurité. Nous utilisons ici uniquement sa capacité à décrire une règle de mutation facilement.\nsecurity/mycluster-0/echo-gw-clusterpolicy.yaml\n1spec: 2 rules: 3 - name: mutate-svc-annotations 4 match: 5 any: 6 - resources: 7 kinds: 8 - Service 9 namespaces: 10 - echo 11 name: cilium-gateway-echo 12 mutate: 13 patchStrategicMerge: 14 metadata: 15 annotations: 16 external-dns.alpha.kubernetes.io/hostname: echo.${domain_name} 17 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internet-facing\u0026#34; 18 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 19 spec: 20 loadBalancerClass: service.k8s.aws/nlb Le service cilium-gateway-echo se verra donc ajouter les annotations du contrôleur AWS ainsi qu'une annotation permettant de configurer une entrée DNS automatiquement.\napps/base/echo/httproute.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 rules: 11 - matches: 12 - path: 13 type: PathPrefix 14 value: / 15 backendRefs: 16 - name: echo-1-echo-server 17 port: 80 L'exemple utilisé ci-dessus est simpliste: toutes les requêtes sont transférées au service echo-1-echo-server.\nparentRefs permet d'indiquer la Gateway à utiliser puis les règles de routage sont définies dans rules.\nLes règles de routages pourraient aussi être basées sur le path.\n1... 2spec: 3 hostnames: 4 - foo.bar.com 5 rules: 6 - matches: 7 - path: 8 type: PathPrefix 9 value: /login Ou une entête HTTP\n1... 2spec: 3 rules: 4 - matches: 5 headers: 6 - name: \u0026#34;version\u0026#34; 7 value: \u0026#34;2\u0026#34; 8... Vérifions que le service est joignable:\n1curl -s http://echo.cloud.ogenki.io | jq -rc \u0026#39;.environment.HOSTNAME\u0026#39; 2echo-1-echo-server-fd88497d-w6sgn Comme vous pouvez le voir le service est exposé en HTTP sans certificat. Essayons de corriger cela 😉\nExposer un service en utilisant un certificat TLS Il existe plusieurs méthodes pour configurer du TLS avec GAPI. Ici nous allons utiliser le cas le plus commun: protocole HTTPS et terminaison TLS sur la Gateway.\nSupposons que nous souhaitons configurer le nom de domaine echo.cloud.ogenki.io utilisé précédemment. La configuration se fait principalement au niveau de la Gateway\napps/base/echo/tls-gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo 5 namespace: echo 6 annotations: 7 cert-manager.io/cluster-issuer: letsencrypt-prod 8spec: 9 gatewayClassName: cilium 10 listeners: 11 - name: http 12 hostname: \u0026#34;echo.${domain_name}\u0026#34; 13 port: 443 14 protocol: HTTPS 15 allowedRoutes: 16 namespaces: 17 from: Same 18 tls: 19 mode: Terminate 20 certificateRefs: 21 - name: echo-tls Le point essentiel ici est la référence à un secret contenant le certificat echo-tls. Ce certificat peut être créé manuellement mais j'ai décidé pour cet article d'automatiser cela avec Let's Encrypt et cert-manager.\nInfo cert-manager\nAvec cert-manager il est très simple d'automatiser la création et la mise à jour des certificats exposés par la Gateway. Pour cela, il faut permettre au contrôlleur d'accéder à route53 afin de résoudre un challenge DNS01 (Mécanisme qui permet de s'assurer que les clients peuvent seulement demander des certificats pour des domaines qu'ils possèdent).\nUne ressource ClusterIssuer décrit la configuration nécessaire pour générer des certificats grâce à cert-manager.\nEnsuite il suffit d'ajouter une annotation cert-manager.io/cluster-issuer et indiquer le secret Kubernetes où sera stocké le certificat.\nℹ️ Dans le repo de demo les permissions sont attribuées en utilisant Crossplane qui se charge de configurer cela au niveau du Cloud AWS.\nPlus d'informations\nPour que le routage se fasse correctement il faut aussi bien entendu référencer la bonne Gateway mais aussi indiquer le nom de domaine dans la ressource HTTPRoute.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 hostnames: 11 - \u0026#34;echo.${domain_name}\u0026#34; 12... Il faut patienter quelques minutes le temps que le certificat soit créé.\n1kubectl get cert -n echo 2NAME READY SECRET AGE 3echo-tls True echo-tls 43m Nous pouvons enfin vérifier que le certificat est bien issue de Let's Encrypt comme suit:\n1curl https://echo.cloud.ogenki.io -v 2\u0026gt;\u0026amp;1 | grep -A 6 \u0026#39;Server certificate\u0026#39; 2* Server certificate: 3* subject: CN=echo.cloud.ogenki.io 4* start date: Sep 15 14:43:00 2023 GMT 5* expire date: Dec 14 14:42:59 2023 GMT 6* subjectAltName: host \u0026#34;echo.cloud.ogenki.io\u0026#34; matched cert\u0026#39;s \u0026#34;echo.cloud.ogenki.io\u0026#34; 7* issuer: C=US; O=Let\u0026#39;s Encrypt; CN=R3 8* SSL certificate verify ok. Info GAPI permet aussi de configurer le TLS de bout en bout, jusqu'au conteneur. Cela se fait en configurant la Gateway en Passthrough et en utilisant une ressource TLSRoute. Il faut aussi que le certificat soit porté par le pod qui fait terminaison TLS.\nUne Gateway partagée par plusieurs namespaces Avec GAPI il est possible de router le trafic à travers les Namespaces. Cela est rendu possible grâce à des ressources distinctes pour chaque fonction: Une Gateway qui permet de configurer l'infrastructure et notamment de provisionner une adresse IP, et les *Routes. Ces routes peuvent référencer une Gateway située dans un autre namespace. Il est ainsi possible pour différent(e)s équipes/projets de partager les mêmes éléments d'infrastructure.\nIl est cependant requis de spécifier quelle route est autorisée à référencer la Gateway. Ici nous supposons que nous avons une Gateway dédiée aux outils internes qui s'appelle platform. En utilisant le paramètre allowedRoutes, nous spécifions explicitement quelles sont les namespaces autorisés à référencer cette Gateway.\ninfrastructure/base/gapi/platform-gateway.yaml\n1... 2 allowedRoutes: 3 namespaces: 4 from: Selector 5 selector: 6 matchExpressions: 7 - key: kubernetes.io/metadata.name 8 operator: In 9 values: 10 - observability 11 - flux-system 12 tls: 13 mode: Terminate 14 certificateRefs: 15 - name: platform-tls Les HTTPRoutes situées dans les namespaces observability et flux-system réferéncent cette même Gateway.\n1... 2spec: 3 parentRefs: 4 - name: platform 5 namespace: infrastructure Et utilisent le même répartiteur de charge du Clouder\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2 3dig +short ${NLB_DOMAIN} 413.36.89.108 5 6dig +short grafana-mycluster-0.cloud.ogenki.io 713.36.89.108 8 9dig +short gitops-mycluster-0.cloud.ogenki.io 1013.36.89.108 Note 🔒 Ces outils internes ne devraient pas être exposés sur Internet mais vous comprendrez qu'il s'agit là d'une démo 🙏. On pourrait par exemple, utiliser et une Gateway interne (IP privée) en jouant sur les annotations et un moyen de connexion privé (VPN, tunnels ...)\nTraffic splitting Il est souvent utile de tester une application sur une portion du trafic lorsqu'une nouvelle version est disponible (A/B testing ou Canary deployment). GAPI permet cela de façon très simple en utilisant des poids.\nVoici un exemple permettant de tester sur 5% du trafic vers le service echo-2-echo-server\napps/base/echo/httproute-split.yaml\n1... 2 hostnames: 3 - \u0026#34;split-echo.${domain_name}\u0026#34; 4 rules: 5 - matches: 6 - path: 7 type: PathPrefix 8 value: / 9 backendRefs: 10 - name: echo-1-echo-server 11 port: 80 12 weight: 95 13 - name: echo-2-echo-server 14 port: 80 15 weight: 5 Vérifions que la répartition se fait bien comme attendu:\nscripts/check-split.sh\n1./scripts/check-split.sh https://split-echo.cloud.ogenki.io 2Number of requests for echo-1: 95 3Number of requests for echo-2: 5 Manipulation des entêtes HTTP (Headers) Il est aussi possible de jouer avec les entêtes HTTP (Headers): en ajouter, modifier ou supprimer. Ces modifications peuvent se faire sur les Headers de requête ou de réponse par le biais de filtres ajoutés à la ressource HTTPRoute.\nNous allons par exemple ajouter un Header à la requête\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7... 8 rules: 9 - matches: 10 - path: 11 type: PathPrefix 12 value: /req-header-add 13 filters: 14 - type: RequestHeaderModifier 15 requestHeaderModifier: 16 add: 17 - name: foo 18 value: bar 19 backendRefs: 20 - name: echo-1-echo-server 21 port: 80 22... La commande suivante permet de vérifier que le header est bien présent.\n1curl -s https://echo.cloud.ogenki.io/req-header-add -sk | jq \u0026#39;.request.headers\u0026#39; 2{ 3 \u0026#34;host\u0026#34;: \u0026#34;echo.cloud.ogenki.io\u0026#34;, 4 \u0026#34;user-agent\u0026#34;: \u0026#34;curl/8.2.1\u0026#34;, 5 \u0026#34;accept\u0026#34;: \u0026#34;*/*\u0026#34;, 6 \u0026#34;x-forwarded-for\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 7 \u0026#34;x-forwarded-proto\u0026#34;: \u0026#34;https\u0026#34;, 8 \u0026#34;x-envoy-external-address\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 9 \u0026#34;x-request-id\u0026#34;: \u0026#34;320ba4d2-3bd6-4c2f-8a97-74296a9f3f26\u0026#34;, 10 \u0026#34;foo\u0026#34;: \u0026#34;bar\u0026#34; 11} 🪪 Les roles et permissions GAPI offre un modèle de partage des permissions claire entre l'infrastructure de routage du trafic (gérée par les administrateurs de cluster) et les applications (gérées par les développeurs).\nLe fait de disposer de plusieurs ressources nous permet d'utiliser les ressources RBAC dans Kubernetes pour attribuer les droits de façon déclarative. J'ai ajouté quelques exemples qui n'ont aucun effet dans mon cluster de démo mais qui peuvent vous permettre de vous faire une idée.\nLa configuration suivante permet aux membres du groupe developers de gérer les HTTPRoutes dans le namespace echo. En revanche ils ne possédent que des droits en lecture sur les Gateways.\n1--- 2apiVersion: rbac.authorization.k8s.io/v1 3kind: Role 4metadata: 5 namespace: echo 6 name: gapi-developer 7rules: 8 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 9 resources: [\u0026#34;httproutes\u0026#34;] 10 verbs: [\u0026#34;*\u0026#34;] 11 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 12 resources: [\u0026#34;gateways\u0026#34;] 13 verbs: [\u0026#34;get\u0026#34;, \u0026#34;list\u0026#34;] 14--- 15apiVersion: rbac.authorization.k8s.io/v1 16kind: RoleBinding 17metadata: 18 name: gapi-developer 19 namespace: echo 20subjects: 21 - kind: Group 22 name: \u0026#34;developers\u0026#34; 23 apiGroup: rbac.authorization.k8s.io 24roleRef: 25 kind: Role 26 name: gapi-developer 27 apiGroup: rbac.authorization.k8s.io 🤔 Un périmètre pas évident Il ne faut pas confondre GAPI avec ce que l'on nomme couramment une API Gateway. Une section de la FAQ a d'ailleurs été créé pour éclaircir ce point. Bien que GAPI offre des fonctionnalités typiquement présentes dans une API Gateway, il s'agit avant tout d'une implémentation spécifique pour Kubernetes. Cependant, ce choix de dénomination peut prêter à confusion.\nIl est essentiel de mentionner que cet article se concentre uniquement sur le trafic entrant, appelé north-south, traditionnellement géré par les Ingress Controllers. Ce trafic représente le périmètre initial de GAPI. Une initiative récente nommée GAMMA vise à également gérer le routage east-west, ce qui permettra de standardiser certaines fonctionnalités des solutions de Service Mesh à l'avenir. (voir cet article pour plus d'informations).\n💭 Dernières remarques Pour être honnête, j'ai entendu parlé de Gateway API depuis un petit moment. J'ai lu quelques articles mais jusqu'ici je n'avais pas pris le temps d'approfondir le sujet. Je me disais \u0026quot;Pourquoi? J'arrive à faire ce que je veux avec mon Ingress Controller ? et puis il faut apprendre à utiliser de nouvelles ressources\u0026quot;.\nGAPI gagne en maturité et nous sommes proche d'une version GA. De nombreux projets l'ont déjà adopté, Istio et Linkerd par exemple sont totalement compatibles avec la version 0.8.0 et cette façon de gérer le trafic au sein de Kubernetes deviendra rapidement la norme.\nToujours est-il que j'ai beaucoup aimé la déclaration des différentes configurations que je trouve très intuitive et explicite ❤️. D'autre part le modèle de sécurité permet de donner le pouvoir aux developpeurs sans sacrifier la sécurité. Enfin la gestion de l'infrastructure se fait de façon transparente, nous pouvons rapidement passer d'une implémentation (contrôleur sous-jacent) à une autre sans toucher aux *Routes.\nAlors suis-je prêt à changer mon Ingress Controller pour Cilium aujourd'hui? La réponse courte est Non mais bientôt!.\nTout d'abord j'aimerais mettre en évidence sur l'étendue des possiblités offertes par Cilium: De nombreuses personnes se sentent noyées sous les nombreux outils qui gravitent autour de Kubernetes. Cilium permettrait de remplir les fonctionnalités de nombre d'entre eux (metrics, tracing, service-mesh, sécurité et ... Ingress Controller avec GAPI).\nCependant, bien que nous puissions faire du routage HTTP de base, il y à certains points d'attention:\nLe support de TCP et UDP Le support de GRPC Devoir passer par une règle de mutation pour pouvoir configurer les composants cloud. (Issue Github) De nombreuses fonctionnalités explorées sont toujours au stade expérimental. On peut citer les fonctions étendues qui supportés depuis quelques jours: J'ai par exemple tenté de configurer une redirection HTTP\u0026gt;HTTPS simple mais je suis tombé sur ce problème. Je m'attends donc à ce qu'il y ait des changements dans l'API très prochainement. Je n'ai pas abordé toutes les fonctionnalités de l'implémentation Cilium de GAPI (Honnêtement, cet article est déjà bien fourni 😜). Néanmoins, je suis vraiment convaincu de son potentiel. J'ai bon espoir qu'on pourra bientôt envisager son utilisation en production. Si vous n'avez pas encore envisagé cette transition, c'est le moment de s'y pencher 😉 ! Toutefois, compte tenu des aspects évoqués précédemment, je conseillerais de patienter un peu.\n🔖 References https://gateway-api.sigs.k8s.io/ https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#gs-gateway-api https://isovalent.com/blog/post/cilium-gateway-api/ https://isovalent.com/blog/post/tutorial-getting-started-with-the-cilium-gateway-api/ Les labs d'Isovalent permettent de rapidement de tester GAPI et vous pourrez ajouter des badges à votre collection 😄 ","link":"https://blog.ogenki.io/fr/post/cilium-gateway-api/","section":"post","tags":["kubernetes","infrastructure","network"],"title":"`Gateway API`: Remplacer mon Ingress Controller avec `Cilium`?"},{"body":"","link":"https://blog.ogenki.io/fr/tags/infrastructure/","section":"tags","tags":null,"title":"infrastructure"},{"body":"","link":"https://blog.ogenki.io/fr/tags/kubernetes/","section":"tags","tags":null,"title":"kubernetes"},{"body":"Terraform est probablement l'outil \u0026quot;Infrastructure As Code\u0026quot; le plus utilisé pour construire, modifier et versionner les changements d'infrastructure Cloud. Il s'agit d'un projet Open Source développé par Hashicorp et qui utilise le langage HCL pour déclarer l'état souhaité de ressources Cloud. L'état des ressources créées est stocké dans un fichier d'état (terraform state).\nOn peut considérer que Terraform est un outil \u0026quot;semi-déclaratif\u0026quot; car il n'y a pas de fonctionnalité de réconciliation automatique intégrée. Il existe différentes approches pour répondre à cette problématique, mais en règle générale, une modification sera appliquée en utilisant terraform apply. Le code est bien décrit dans des fichiers de configuration HCL (déclaratif) mais l'exécution est faite de manière impérative. De ce fait, il peut y avoir de la dérive entre l'état déclaré et le réel (par exemple, un collègue qui serait passé sur la console pour changer un paramètre 😉).\n❓❓ Alors, comment m'assurer que ce qui est commit dans mon repo git est vraiment appliqué. Comment être alerté s'il y a un changement par rapport à l'état désiré et comment appliquer automatiquement ce qui est dans mon code (GitOps) ?\nC'est la promesse de tf-controller, un operateur Kubernetes Open Source de Weaveworks, étroitement lié à Flux (un moteur GitOps de la même société). Flux est l'une des solutions que je plébiscite, et je vous invite donc à lire un précédent article.\nInfo L'ensemble des étapes décrites ci-dessous sont faites avec ce repo Git\n:bullseye: Notre objectif En suivant les étapes de cet article nous visons les objectifs suivant:\nDéployer un cluster Kubernetes qui servira de \u0026quot;Control plane\u0026quot;. Pour résumer il hébergera le controlleur Terraform qui nous permettra de déclarer tous les éléments d'infrastructure souhaités. Utiliser Flux comme moteur GitOps pour toutes les ressources Kubernetes. Concernant le controleur Terraform, nous allons voir:\nQuelle est le moyen de définir des dépendances entre modules Création de plusieurs ressources AWS: Zone route53, Certificat ACM, réseau, cluster EKS. Les différentes options de reconciliation (automatique, nécessitant une confirmation) Comment sauvegarder et restaurer un fichier d'état (tfstate) 🛠️ Installer le controleur Terraform ☸ Le cluster \u0026quot;Control Plane\u0026quot; Afin de pouvoir utiliser le controleur Kubernetes tf-controller, il nous faut d'abord un cluster Kubernetes 😆. Nous allons donc créer un cluster control plane en utilisant la ligne de commande terraform et les bonnes pratiques EKS.\nWarning Il est primordial que ce cluster soit résiliant, sécurisé et supervisé car il sera responsable de la gestion de l'ensemble des ressources AWS créées par la suite.\nSans entrer dans le détail, le cluster \u0026quot;control plane\u0026quot; a été créé un utilisant ce code. Celà-dit, il est important de noter que toutes les opérations de déploiement d'application se font en utilisant Flux.\nInfo En suivant les instructions du README, un cluster EKS sera créé mais pas uniquement! Il faut en effet donner les permissions au controlleur Terraform pour appliquer les changements d'infrastructure. De plus, Flux doit être installé et configuré afin d'appliquer la configuration définie ici.\nAu final on se retrouve donc avec plusieurs éléments installés et configurés:\nles addons quasi indispensables que sont aws-loadbalancer-controller et external-dns les roles IRSA pour ces mêmes composants sont installés en utilisant tf-controller La stack de supervision Prometheus / Grafana. external-secrets pour pouvoir récupérer des éléments sensibles depuis AWS secretsmanager. Afin de démontrer tout cela au bout de quelques minutes l'interface web pour Flux est accessible via l'URL gitops-\u0026lt;cluster_name\u0026gt;.\u0026lt;domain_name\u0026gt; Vérifier toute de même que le cluster est accessible et que Flux fonctionne correctement\n1aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0 2Updated context controlplane-0 in /home/smana/.kube/config 1flux check 2... 3✔ all checks passed 4 5flux get kustomizations 6NAME REVISION SUSPENDED READY MESSAGE 7flux-config main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 8flux-system main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 9infrastructure main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 10security main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 11tf-controller main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 12... 📦 Le chart Helm et Flux Maintenant que notre cluster \u0026quot;controlplane\u0026quot; est opérationnel, l'ajout le contrôleur Terraform consiste à utiliser le chart Helm.\nIl faut tout d'abord déclarer la source:\nsource.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: tf-controller 5spec: 6 interval: 30m 7 url: https://weaveworks.github.io/tf-controller Et définir la HelmRelease:\nrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: tf-controller 5spec: 6 releaseName: tf-controller 7 chart: 8 spec: 9 chart: tf-controller 10 sourceRef: 11 kind: HelmRepository 12 name: tf-controller 13 namespace: flux-system 14 version: \u0026#34;0.12.0\u0026#34; 15 interval: 10m0s 16 install: 17 remediation: 18 retries: 3 19 values: 20 resources: 21 limits: 22 memory: 1Gi 23 requests: 24 cpu: 200m 25 memory: 500Mi 26 runner: 27 serviceAccount: 28 annotations: 29 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/tfcontroller_${cluster_name}\u0026#34; Lorsque ce changement est écrit dans le repo Git, la HelmRelease sera déployée et le contrôlleur tf-controller démarera\n1kubectl get hr -n flux-system 2NAME AGE READY STATUS 3tf-controller 67m True Release reconciliation succeeded 4 5kubectl get po -n flux-system -l app.kubernetes.io/instance=tf-controller 6NAME READY STATUS RESTARTS AGE 7tf-controller-7ffdc69b54-c2brg 1/1 Running 0 2m6s Dans le repo de demo il y a déjà un certain nombre de ressources AWS déclarées. Par conséquent, au bout de quelques minutes, le cluster se charge de la création de celles-cis: Info Bien que la majorité des tâches puisse être réalisée de manière déclarative ou via les utilitaires de ligne de commande tels que kubectl et flux, un autre outil existe qui offre la possibilité d'interagir avec les ressources terraform : tfctl\n🚀 Appliquer un changement Parmis les bonnes pratiques avec Terraform, il y a l'usage de modules. Un module est un ensemble de ressources Terraform liées logigement afin d'obtenir une seule unité réutilisable. Cela permet d'abstraire la complexité, de prendre des entrées, effectuer des actions spécifiques et produire des sorties.\nIl est possible de créer ses propres modules et de les mettre à disposition dans des Sources ou d'utiliser les nombreux modules partagés et maintenus par les communautés. Il suffit alors d'indiquer quelques variables afin de l'adapter au contexte.\nAvec tf-controller, la première étape consiste donc à indiquer la Source du module. Ici nous allons configurer le socle réseau sur AWS (vpc, subnets...) avec le module terraform-aws-vpc.\nsources/terraform-aws-vpc.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1 2kind: GitRepository 3metadata: 4 name: terraform-aws-vpc 5 namespace: flux-system 6spec: 7 interval: 30s 8 ref: 9 tag: v5.0.0 10 url: https://github.com/terraform-aws-modules/terraform-aws-vpc Nous pouvons ensuite créer la ressource Terraform qui en fait usage:\nvpc/dev.yaml\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6 interval: 8m 7 path: . 8 destroyResourcesOnDeletion: true # You wouldn\u0026#39;t do that on a prod env ;) 9 storeReadablePlan: human 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-vpc 13 namespace: flux-system 14 vars: 15 - name: name 16 value: vpc-dev 17 - name: cidr 18 value: \u0026#34;10.42.0.0/16\u0026#34; 19 - name: azs 20 value: 21 - \u0026#34;eu-west-3a\u0026#34; 22 - \u0026#34;eu-west-3b\u0026#34; 23 - \u0026#34;eu-west-3c\u0026#34; 24 - name: private_subnets 25 value: 26 - \u0026#34;10.42.0.0/19\u0026#34; 27 - \u0026#34;10.42.32.0/19\u0026#34; 28 - \u0026#34;10.42.64.0/19\u0026#34; 29 - name: public_subnets 30 value: 31 - \u0026#34;10.42.96.0/24\u0026#34; 32 - \u0026#34;10.42.97.0/24\u0026#34; 33 - \u0026#34;10.42.98.0/24\u0026#34; 34 - name: enable_nat_gateway 35 value: true 36 - name: single_nat_gateway 37 value: true 38 - name: private_subnet_tags 39 value: 40 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 41 \u0026#34;karpenter.sh/discovery\u0026#34;: dev 42 - name: public_subnet_tags 43 value: 44 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 45 writeOutputsToSecret: 46 name: vpc-dev Si l'on devait résumer grossièrement: le code terraform provenant de la source terraform-aws-vpc est utilisé avec les variables vars.\nIl y a ensuite plusieurs paramètres qui influent sur le fonctionnement de tf-controller. Les principaux paramètres qui permettent de contrôler la façon dont sont appliquées les modifications sont .spec.approvePlan et .spec.autoApprove\n🚨 Détection de la dérive Définir spec.approvePlan avec une valeur à disable permet uniquement de notifier que l'état actuel des ressources a dérivé par rapport au code Terraform. Cela permet notamment de choisir le moment et la manière dont l'application des changements sera effectuée.\nNote De mon point de vue il manque une section sur les notifications: La dérive, les plans en attentes, les problèmese de réconcilation. J'essaye d'identifier les méthodes possibles (de préférence avec Prometheus) et de mettre à jour cet article dès que possible.\n🔧 Application manuelle L'exemple donné précédemment (vpc-dev) ne contient pas le paramètre .spec.approvePlan et hérite donc de la valeur par défaut qui est false. Par conséquent, l'application concrète des modifications (apply), n'est pas faite automatiquement.\nUn plan est exécuté et sera en attente d'une validation:\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system vpc-dev Unknown Plan generated: set approvePlan: \u0026#34;plan-v5.0.0-26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. true 2 minutes Je conseille d'ailleurs de configurer le paramètre storeReadablePlan à human. Cela permet de visualiser simplement les modifications en attente en utilisant tfctl:\n1tfctl show plan vpc-dev 2 3Terraform used the selected providers to generate the following execution 4plan. ressource actions are indicated with the following symbols: 5 + create 6 7Terraform will perform the following actions: 8 9 # aws_default_network_acl.this[0] will be created 10 + ressource \u0026#34;aws_default_network_acl\u0026#34; \u0026#34;this\u0026#34; { 11 + arn = (known after apply) 12 + default_network_acl_id = (known after apply) 13 + id = (known after apply) 14 + owner_id = (known after apply) 15 + tags = { 16 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 17 } 18 + tags_all = { 19 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 20 } 21 + vpc_id = (known after apply) 22 23 + egress { 24 + action = \u0026#34;allow\u0026#34; 25 + from_port = 0 26 + ipv6_cidr_block = \u0026#34;::/0\u0026#34; 27 + protocol = \u0026#34;-1\u0026#34; 28 + rule_no = 101 29 + to_port = 0 30 } 31 + egress { 32... 33Plan generated: set approvePlan: \u0026#34;plan-v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. 34To set the field, you can also run: 35 36 tfctl approve vpc-dev -f filename.yaml Après revue des modifications ci-dessus, il suffit donc d'ajouter l'identifiant du plan à valider et de pousser le changement sur git comme suit:\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6... 7 approvePlan: plan-v5.0.0-26c38a66f1 8... En quelques instants un runner sera lancé qui se chargera d'appliquer les changements:\n1kubectl logs -f -n flux-system vpc-dev-tf-runner 22023/07/01 15:33:36 Starting the runner... version sha 3... 4aws_vpc.this[0]: Creating... 5aws_vpc.this[0]: Still creating... [10s elapsed] 6... 7aws_route_table_association.private[1]: Creation complete after 0s [id=rtbassoc-01b7347a7e9960a13] 8aws_nat_gateway.this[0]: Still creating... [10s elapsed] La réconciliation éffectuée, la ressource passe à l'état READY: True\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m 🤖 Application automatique Nous pouvons aussi activer la réconciliation automatique. Pour ce faire il faut déclarer le paramètre .spec.autoApprove à true.\nToutes les ressources IRSA sont configurées de la sorte:\nexternal-secrets.yaml\n1piVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: irsa-external-secrets 5spec: 6 approvePlan: auto 7 destroyResourcesOnDeletion: true 8 interval: 8m 9 path: ./modules/iam-role-for-service-accounts-eks 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-iam 13 namespace: flux-system 14 vars: 15 - name: role_name 16 value: ${cluster_name}-external-secrets 17 - name: attach_external_secrets_policy 18 value: true 19 - name: oidc_providers 20 value: 21 main: 22 provider_arn: ${oidc_provider_arn} 23 namespace_service_accounts: [\u0026#34;security:external-secrets\u0026#34;] Donc si je fais le moindre changement sur la console AWS par exemple, celui-ci sera rapidement écrasé par celui géré par tf-controller.\nInfo La politique de suppression d'une ressource Terraform est définie par le paramètre destroyResourcesOnDeletion. Par défaut elles sont conservées et il faut donc que ce paramètre ait pour valeur true afin de détruire les éléments crées lorsque l'objet Kubernetes est supprimé.\nIci nous voulons la possibilité de supprimer les rôles IRSA. Ils sont en effet étroitement liés aux clusters.\n🔄 Entrées et sorties: dépendances entre modules Lorsque qu'on utilise Terraform, on a souvent besoin de passer des données d'un module à l'autre. Généralement ce sont les outputs du module qui exportent ces informations. Il faut donc un moyen de les importer dans un autre module.\nReprenons encore l'exemple donné ci-dessus (vpc-dev). Nous notons en bas du YAML la directive suivante:\n1... 2 writeOutputsToSecret: 3 name: vpc-dev Lorsque cette ressource est appliquée nous aurons un message qui confirme que les outputs sont disponibles (\u0026quot;Outputs written\u0026quot;):\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m En effet ce module exporte de nombreuses informations (126):\n1kubectl get secrets -n flux-system vpc-dev 2NAME TYPE DATA AGE 3vpc-dev Opaque 126 15s 4 5kubectl get secret -n flux-system vpc-dev --template=\u0026#39;{{.data.vpc_id}}\u0026#39; | base64 -d 6vpc-0c06a6d153b8cc4db Certains de ces éléments d'informations sont ensuite utilisés pour créer un cluster EKS de dev:\nvpc/dev.yaml\n1... 2 varsFrom: 3 - kind: Secret 4 name: vpc-dev 5 varsKeys: 6 - vpc_id 7 - private_subnets 8... 💾 Sauvegarder et restaurer un tfstate Dans mon cas je ne souhaite pas recréer la zone et le certificat à chaque destruction du controlplane. Voici un exemple des étapes à mener pour que je puisse restaurer l'état de ces ressources lorsque j'utilise cette demo.\nNote Il s'agit là d'une procédure manuelle afin de démontrer le comportement de tf-controller par rapport aux fichiers d'état. Par défaut ces tfstates sont stockés dans des secrets mais on préferera configurer un backend GCS ou S3\nLa création initiale de l'environnement de démo m'a permis de sauvegarder les fichiers d'état (tfstate) de cette façon.\n1WORKSPACE=\u0026#34;default\u0026#34; 2STACK=\u0026#34;route53-cloud-hostedzone\u0026#34; 3BACKUPDIR=\u0026#34;${HOME}/tf-controller-backup\u0026#34; 4 5mkdir -p ${BACKUPDIR} 6 7kubectl get secrets -n flux-system tfstate-${WORKSPACE}-${STACK} -o jsonpath=\u0026#39;{.data.tfstate}\u0026#39; | \\ 8base64 -d | gzip -d \u0026gt; ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate Lorsque le cluster est créé à nouveau, tf-controller essaye de créer la zone car le fichier d'état est vide.\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system route53-cloud-hostedzone Unknown Plan generated: set approvePlan: \u0026#34;plan-main@sha1:345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. true 16 minutes 5 6tfctl show plan route53-cloud-hostedzone 7 8Terraform used the selected providers to generate the following execution 9plan. resource actions are indicated with the following symbols: 10 + create 11 12Terraform will perform the following actions: 13 14 # aws_route53_zone.this will be created 15 + resource \u0026#34;aws_route53_zone\u0026#34; \u0026#34;this\u0026#34; { 16 + arn = (known after apply) 17 + comment = \u0026#34;Experimentations for blog.ogenki.io\u0026#34; 18 + force_destroy = false 19 + id = (known after apply) 20 + name = \u0026#34;cloud.ogenki.io\u0026#34; 21 + name_servers = (known after apply) 22 + primary_name_server = (known after apply) 23 + tags = { 24 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 25 } 26 + tags_all = { 27 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 28 } 29 + zone_id = (known after apply) 30 } 31 32Plan: 1 to add, 0 to change, 0 to destroy. 33 34Changes to Outputs: 35 + domain_name = \u0026#34;cloud.ogenki.io\u0026#34; 36 + nameservers = (known after apply) 37 + zone_arn = (known after apply) 38 + zone_id = (known after apply) 39 40Plan generated: set approvePlan: \u0026#34;plan-main@345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. 41To set the field, you can also run: 42 43 tfctl approve route53-cloud-hostedzone -f filename.yaml La procédure de restauration consiste donc à créer le secret à nouveau:\n1gzip ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate 2 3cat \u0026lt;\u0026lt;EOF | kubectl apply -f - 4apiVersion: v1 5kind: Secret 6metadata: 7 name: tfstate-${WORKSPACE}-${STACK} 8 namespace: flux-system 9 annotations: 10 encoding: gzip 11type: Opaque 12data: 13 tfstate: $(cat ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz | base64 -w 0) 14EOF Il faudra aussi relancer un plan de façon explicite pour mettre à jour l'état de la ressource en question\n1tfctl replan route53-cloud-hostedzone 2 Replan requested for flux-system/route53-cloud-hostedzone 3Error: timed out waiting for the condition Nous pouvons alors vérifier que le fichier d'état a bien été mis à jour\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3flux-system route53-cloud-hostedzone True Outputs written: main@sha1:d0934f979d832feb870a8741ec01a927e9ee6644 false 19 minutes 🔍 Focus sur certaines fonctionnalités de Flux Oui j'ai un peu menti sur l'agenda 😝. Il me semblait nécessaire de mettre en lumière 2 fonctionnalités que je n'avais pas exploité jusque là et qui sont fort utiles!\nSubstition de variables Lorsque Flux est initiliasé un certain nombre de Kustomization spécifique à ce cluster sont créés. Il est possible d'y indiquer des variables de substitution qui pourront être utilisées dans l'ensemble des ressources déployées par cette Kustomization. Cela permet d'éviter un maximum la déduplication de code.\nJ'ai découvert l'efficacité de cette fonctionnalité très récemment. Je vais décrire ici la façon dont je l'utilise:\nLe code terraform qui crée un cluster EKS, génère aussi une ConfigMap qui contient les variables propres au cluster. On y retrouvera, bien sûr, le nom du cluster, mais aussi tous les paramètres qui varient entre les clusters et qui sont utilisés dans les manifests Kubernetes.\nflux.tf\n1resource \u0026#34;kubernetes_config_map\u0026#34; \u0026#34;flux_clusters_vars\u0026#34; { 2 metadata { 3 name = \u0026#34;eks-${var.cluster_name}-vars\u0026#34; 4 namespace = \u0026#34;flux-system\u0026#34; 5 } 6 7 data = { 8 cluster_name = var.cluster_name 9 oidc_provider_arn = module.eks.oidc_provider_arn 10 aws_account_id = data.aws_caller_identity.this.account_id 11 region = var.region 12 environment = var.env 13 vpc_id = module.vpc.vpc_id 14 } 15 depends_on = [flux_bootstrap_git.this] 16} Comme spécifié précedemment, les variables de substition sont définies dans les Kustomization. Prenons un exemple concret. Ci-dessous on définie la Kustomization qui déploie toutes les ressources qui sont consommées par tf-controller On déclare ici la ConfigMap eks-controlplane-0-vars qui avait été généré à la création du cluster EKS.\ninfrastructure.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1 2kind: Kustomization 3metadata: 4 name: tf-custom-resources 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 path: ./infrastructure/controlplane-0/terraform/custom-resources 10 postBuild: 11 substitute: 12 domain_name: \u0026#34;cloud.ogenki.io\u0026#34; 13 substituteFrom: 14 - kind: ConfigMap 15 name: eks-controlplane-0-vars 16 - kind: Secret 17 name: eks-controlplane-0-vars 18 optional: true 19 sourceRef: 20 kind: GitRepository 21 name: flux-system 22 dependsOn: 23 - name: tf-controller Enfin voici un exemple de ressource Kubernetes qui en fait usage. Cet unique manifest peut être utilisé par tous les clusters!.\nexternal-dns/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: external-dns 5spec: 6... 7 values: 8 global: 9 imageRegistry: public.ecr.aws 10 fullnameOverride: external-dns 11 aws: 12 region: ${region} 13 zoneType: \u0026#34;public\u0026#34; 14 batchChangeSize: 1000 15 domainFilters: [\u0026#34;${domain_name}\u0026#34;] 16 logFormat: json 17 txtOwnerId: \u0026#34;${cluster_name}\u0026#34; 18 serviceAccount: 19 annotations: 20 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/${cluster_name}-external-dns\u0026#34; Cela élimine totalement les overlays qui consistaient à ajouter les paramètres spécifiques au cluster.\nWeb UI (Weave GitOps) Dans mon précédent article sur Flux, je mentionnais le fait que l'un des inconvénients (si l'on compare avec son principale concurrent: ArgoCD) est le manque d'une interface Web. Bien que je sois un adepte de la ligne de commande, c'est parfois bien utile d'avoir une vue synthétique et de pouvoir effectuer certaines opération en quelques clicks \u0026#x1f5b1;\u0026#xfe0f;\nC'est désormais possible avec Weave Gitops! Bien entendu ce n'est pas comparable avec l'UI d'ArgoCD, mais l'essentiel est là: Mettre en pause la réconcilation, visualiser les manifests, les dépendances, les événements...\nIl existe aussi le plugin VSCode comme alternative.\n💭 Remarques Et voilà, nous arrivons au bout de notre exploration de cet autre outil de gestion d'infrastructure sur Kubernetes. Malgré quelques petits soucis rencontrés en cours de route, que j'ai partagé sur le repo Git du projet, l'expérience m'a beaucoup plu. tf-controller offre une réponse concrète à une question fréquente : comment gérer notre infra comme on gère notre code ?\nJ'aime beaucoup l'approche GitOps appliquée à l'infrastructure, j'avais d'ailleurs écrit un article sur Crossplane. tf-controller aborde la problématique sous un angle différent: utiliser du Terraform directement. Cela signifie qu'on peut utiliser nos connaissances actuelles et notre code existant. Pas besoin d'apprendre une nouvelle façon de déclarer nos ressources. C'est un critère à prendre en compte car migrer vers un nouvel outil lorsque l'on a un existant représente un éffort non négligeable. Cependant j'ajouterais aussi que tf-controller s'adresse aux utilisateurs de Flux uniquement et, de ce fait, restreint le publique cible.\nAujourd'hui, j'utilise une combinaison de Terraform, Terragrunt et RunAtlantis. tf-controller pourrait devenir une alternative viable: Nous avons en effet évoqué l'intérêt de Kustomize associé aux substitions de variables pour la factorisation de code. Dans la roadmap du projet il y a aussi l'objectif d'afficher les plans dans les pull-requests. Autre problématique fréquente: la nécessité de passer des éléments sensibles aux modules. En utilisant une ressource Terraform, on peut injecter des variables depuis des secrets Kubernetes. Ce qui permet d'utiliser certains outils, tels que external-secrets, sealed-secrets ...\nJe vous encourage donc à essayer tf-controller vous-même, et peut-être même d'y apporter votre contribution 🙂.\nNote La démo que j'ai faite ici utilise pas mal de ressources, dont certaines assez cruciales (comme le réseau). Donc, gardez en tête que c'est juste pour la démo ! Je suggère une approche progressive si vous envisagez de le mettre en ouvre: commencez par utiliser la détection de dérives, puis créez des ressources simples. J'ai aussi pris quelques raccourcis en terme de sécurité à éviter absolument, notamment le fait de donner les droits admin au contrôleur. ","link":"https://blog.ogenki.io/fr/post/terraform-controller/","section":"post","tags":["infrastructure"],"title":"Appliquer les principes de GitOps à l'infrastructure: Introduction à `tf-controller`"},{"body":"Kubernetes est désormais la plate-forme privilégiée pour orchestrer les applications \u0026quot;sans état\u0026quot; aussi appelé \u0026quot;stateless\u0026quot;. Les conteneurs qui ne stockent pas de données peuvent être détruits et recréés ailleurs sans impact. En revanche, la gestion d'applications \u0026quot;stateful\u0026quot; dans un environnement dynamique tel que Kubernetes peut être un véritable défi. Malgré le fait qu'il existe un nombre croissant de solutions de base de données \u0026quot;Cloud Native\u0026quot; (comme CockroachDB, TiDB, K8ssandra, Strimzi ...) et il y a de nombreux éléments à considérer lors de leur évaluation:\nQuelle est la maturité de l'opérateur? (Dynamisme et contributeurs, gouvernance du projet) Quels sont les resources personalisées disponibles (\u0026quot;custom resources\u0026quot;), quelles opérations permettent t-elles de réaliser? Quels sont les type de stockage disponibles: HDD / SSD, stockage local / distant? Que se passe-t-il lorsque quelque chose se passe mal: Quelle est le niveau de résilience de la solution? Sauvegarde et restauration: est-il facile d'effectuer et de planifier des sauvegardes? Quelles options de réplication et de mise à l'échelle sont disponibles? Qu'en est-il des limites de connexion et de concurrence, les pools de connexion? A propos de la supervision, quelles sont les métriques exposées et comment les exploiter? J'étais à la recherche d'une solution permettant de gérer un serveur PostgreSQL. La base de données qui y serait hébergée est nécessaire pour un logiciel de réservation de billets nommé Alf.io. Nous sommes en effet en train d'organiser les Kubernetes Community Days France vous êtes tous conviés! 👐.\nJe cherchais spécifiquement une solution indépendante d'un clouder (cloud agnostic) et l'un des principaux critères était la simplicité d'utilisation. Je connaissais déjà plusieurs opérateurs Kubernetes, et j'ai fini par évaluer une solution relativement récente: CloudNativePG.\nCloudNativepg est l'opérateur de Kubernetes qui couvre le cycle de vie complet d'un cluster de base de données PostgreSQL hautement disponible avec une architecture de réplication native en streaming.\nCe projet été créé par l'entreprise EnterpriseDB et a été soumis à la CNCF afin de rejoindre les projets Sandbox.\n:bullseye: Notre objectif Je vais donner ici une introduction aux principales fonctionnalités de CloudNativePG.\nL'objectif est de:\nCréer une base de données PostgreSQL sur un cluster GKE, Ajouter une instance secondaire (réplication) Exécuter quelques tests de résilience. Nous verrons également comment tout cela se comporte en terme de performances et quels sont les outils de supervision disponibles. Enfin, nous allons jeter un œil aux méthodes de sauvegarde/restauration.\nInfo Dans cet article, nous allons tout créer et tout mettre à jour manuellement. Mais dans un environnement de production, il est conseillé d'utiliser un moteur GitOps, par exemple Flux (sujet couvert dans un article précédent).\nSi vous souhaitez voir un exemple complet, vous pouvez consulter le dépôt git KCD France infrastructure.\nToutes les resources de cet article sont dans ce dépôt.\n\u0026#x2611;\u0026#xfe0f; Prérequis \u0026#x1f4e5; Outils gcloud SDK: Nous allons déployer sur Google Cloud (en particulier sur GKE) et, pour ce faire, nous devrons créer quelques ressources dans notre projet GCP. Nous aurons donc besoin du SDK et de la CLI Google Cloud. Il est donc nécessaire de l'installer en suivant cette documentation.\nkubectl plugin: Pour faciliter la gestion des clusters, il existe un plugin kubectl qui donne des informations synthétiques sur l'instance PostgreSQL et permet aussi d'effectuer certaines opérations. Ce plugin peut être installé en utilisant krew:\n1kubectl krew install cnpg ☁️ Créer les resources Google Cloud Avant de créer notre instance PostgreSQL, nous devons configurer certaines choses:\nNous avons besoin d'un cluster Kubernetes. (Cet article suppose que vous avez déjà pris soin de provisionner un cluster GKE) Nous allons créer un bucket (Google Cloud Storage) pour stocker les sauvegardes et Fichiers WAL. Nous configurerons les permissions pour nos pods afin qu'ils puissent écrire dans ce bucket. Créer le bucket à l'aide de CLI gcloud\n1gcloud storage buckets create --location=eu --default-storage-class=coldline gs://cnpg-ogenki 2Creating gs://cnpg-ogenki/... 3 4gcloud storage buckets describe gs://cnpg-ogenki 5[...] 6name: cnpg-ogenki 7owner: 8 entity: project-owners-xxxx0008 9projectNumber: \u0026#39;xxx00008\u0026#39; 10rpo: DEFAULT 11selfLink: https://www.googleapis.com/storage/v1/b/cnpg-ogenki 12storageClass: STANDARD 13timeCreated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; 14updated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; Nous allons maintenant configurer les permissions afin que les pods (PostgreSQL Server) puissent permettant écrire/lire à partir du bucket grâce à Workload Identity.\nNote Workload Identity doit être activé au niveau du cluster GKE. Afin de vérifier que le cluster est bien configuré, vous pouvez lancer la commande suivante:\n1gcloud container clusters describe \u0026lt;cluster_name\u0026gt; --format json --zone \u0026lt;zone\u0026gt; | jq .workloadIdentityConfig 2{ 3 \u0026#34;workloadPool\u0026#34;: \u0026#34;{{ gcp_project }}.svc.id.goog\u0026#34; 4} Créer un compte de service Google Cloud\n1gcloud iam service-accounts create cloudnative-pg --project={{ gcp_project }} 2Created service account [cloudnative-pg]. Attribuer au compte de service la permission storage.admin\n1gcloud projects add-iam-policy-binding {{ gcp_project }} \\ 2--member \u0026#34;serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com\u0026#34; \\ 3--role \u0026#34;roles/storage.admin\u0026#34; 4[...] 5- members: 6 - serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 7 role: roles/storage.admin 8etag: BwXrGA_VRd4= 9version: 1 Autoriser le compte de service (Attention il s'agit là du compte de service au niveau Kubernetes) afin d'usurper le compte de service IAM. \u0026#x2139;\u0026#xfe0f; Assurez-vous d'utiliser le format approprié serviceAccount:{{ gcp_project }}.svc.id.goog[{{ kubernetes_namespace }}/{{ kubernetes_serviceaccount }}]\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 7 role: roles/iam.workloadIdentityUser 8etag: BwXrGBjt5kQ= 9version: 1 Nous sommes prêts à créer les ressources Kubernetes \u0026#x1f4aa;\n\u0026#x1f511; Créer les secrets pour les utilisateurs PostgreSQL Nous devons créer les paramètres d'authentification des utilisateurs qui seront créés pendant la phase de \u0026quot;bootstrap\u0026quot; (nous y reviendrons par la suite): le superutilisateur et le propriétaire de base de données nouvellement créé.\n1kubectl create secret generic cnpg-mydb-superuser --from-literal=username=postgres --from-literal=password=foobar --namespace demo 2secret/cnpg-mydb-superuser created 1kubectl create secret generic cnpg-mydb-user --from-literal=username=smana --from-literal=password=barbaz --namespace demo 2secret/cnpg-mydb-user created \u0026#x1f6e0;\u0026#xfe0f; Déployer l'opérateur CloudNativePG avec Helm Ici nous utiliserons le chart Helm pour déployer CloudNativePG:\n1helm repo add cnpg https://cloudnative-pg.github.io/charts 2 3helm upgrade --install cnpg --namespace cnpg-system \\ 4--create-namespace charts/cloudnative-pg 5 6kubectl get po -n cnpg-system 7NAME READY STATUS RESTARTS AGE 8cnpg-74488f5849-8lhjr 1/1 Running 0 6h17m Cela installe aussi quelques resources personnalisées (Custom Resources Definitions)\n1kubectl get crds | grep cnpg.io 2backups.postgresql.cnpg.io 2022-10-08T16:15:14Z 3clusters.postgresql.cnpg.io 2022-10-08T16:15:14Z 4poolers.postgresql.cnpg.io 2022-10-08T16:15:14Z 5scheduledbackups.postgresql.cnpg.io 2022-10-08T16:15:14Z Pour une liste complète des paramètres possibles, veuillez vous référer à la doc de l'API.\n\u0026#x1f680; Créer un serveur PostgreSQL Nous pouvons désormais créer notre première instance en utilisant une resource personnalisée Cluster. La définition suivante est assez simple: Nous souhaitons démarrer un serveur PostgreSQL, créer automatiquement une base de données nommée mydb et configurer les informations d'authentification en utilisant les secrets créés précédemment.\n1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki 5 namespace: demo 6spec: 7 description: \u0026#34;PostgreSQL Demo Ogenki\u0026#34; 8 imageName: ghcr.io/cloudnative-pg/postgresql:14.5 9 instances: 1 10 11 bootstrap: 12 initdb: 13 database: mydb 14 owner: smana 15 secret: 16 name: cnpg-mydb-user 17 18 serviceAccountTemplate: 19 metadata: 20 annotations: 21 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 22 23 superuserSecret: 24 name: cnpg-mydb-superuser 25 26 storage: 27 storageClass: standard 28 size: 10Gi 29 30 backup: 31 barmanObjectStore: 32 destinationPath: \u0026#34;gs://cnpg-ogenki\u0026#34; 33 googleCredentials: 34 gkeEnvironment: true 35 retentionPolicy: \u0026#34;30d\u0026#34; 36 37 resources: 38 requests: 39 memory: \u0026#34;1Gi\u0026#34; 40 cpu: \u0026#34;500m\u0026#34; 41 limits: 42 memory: \u0026#34;1Gi\u0026#34; Créer le namespace où notre instance postgresql sera déployée\n1kubectl create ns demo 2namespace/demo created Adapdez le fichier YAML ci-dessus vos besoins et appliquez comme suit:\n1kubectl apply -f cluster.yaml 2cluster.postgresql.cnpg.io/ogenki created Vous remarquerez que le cluster sera en phase Initializing. Nous allons utiliser le plugin CNPG pour la première fois afin de vérifier son état. Cet outil deviendra par la suite notre meilleur ami pour afficher une vue synthétique de l'état du cluster.\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Primary server is initializing 4Name: ogenki 5Namespace: demo 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: (switching to ogenki-1) 8Status: Setting up primary Creating primary instance ogenki-1 9Instances: 1 10Ready instances: 0 11 12Certificates Status 13Certificate Name Expiration Date Days Left Until Expiration 14---------------- --------------- -------------------------- 15ogenki-ca 2023-01-13 20:02:40 +0000 UTC 90.00 16ogenki-replication 2023-01-13 20:02:40 +0000 UTC 90.00 17ogenki-server 2023-01-13 20:02:40 +0000 UTC 90.00 18 19Continuous Backup status 20First Point of Recoverability: Not Available 21No Primary instance found 22Streaming Replication status 23Not configured 24 25Instances status 26Name Database Size Current LSN Replication role Status QoS Manager Version Node 27---- ------------- ----------- ---------------- ------ --- --------------- ---- immédiatement après la déclaration de notre nouveau Cluster, une action de bootstrap est lancée. Dans notre exemple, nous créons une toute nouvelle base de données nommée mydb avec un propriétaire smana dont les informations d'authentification viennent du secret créé précédemment.\n1[...] 2 bootstrap: 3 initdb: 4 database: mydb 5 owner: smana 6 secret: 7 name: cnpg-mydb-user 8[...] 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 0/1 Running 0 55s 4ogenki-1-initdb-q75cz 0/1 Completed 0 2m32s Après quelques secondes, le cluster change de statut et devient Ready (configuré et prêt à l'usage) \u0026#x1f44f;\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7154833472216277012 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 1 10Ready instances: 1 11 12[...] 13 14Instances status 15Name Database Size Current LSN Replication role Status QoS Manager Version Node 16---- ------------- ----------- ---------------- ------ --- --------------- ---- 17ogenki-1 33 MB 0/17079F8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xczh Info Il existe de nombreuses façons de bootstrap un cluster. Par exemple, la restauration d'une sauvegarde dans une toute nouvelle instance ou en exécutant du code SQL ... Plus d'infos ici.\n🩹 Instance de secours et résilience Info Dans les architectures postgresql traditionnelles, nous trouvons généralement un composant supplémentaire pour gérer la haute disponibilité (ex: Patroni). Un particularité de l'opérateur CloudNativePG est qu'il bénéficie des fonctionnalités de base de Kubernetes et s'appuie sur un composant nommé Postgres instance manager.\nAjoutez une instance de secours (\u0026quot;standby\u0026quot;) en définissant le nombre de répliques sur 2.\n1kubectl edit cluster -n demo ogenki 2cluster.postgresql.cnpg.io/ogenki edited 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3[...] 4spec: 5 instances: 2 6[...] L'opérateur remarque immédiatement le changement, ajoute une instance de secours et démarre le processus de réplication.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Creating a new replica Creating replica ogenki-2-join 9Instances: 2 10Ready instances: 1 11Current Write LSN: 0/1707A30 (Timeline: 1 - WAL File: 000000010000000000000001) 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 0 3m16s 4ogenki-2-join-xxrwx 0/1 Pending 0 82s Après un certain temps (qui dépend de la quantité de données à répliquer), l'instance de secours devient opérationnelle et nous pouvons voir les statistiques de réplication.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 2 10Ready instances: 2 11Current Write LSN: 0/3000060 (Timeline: 1 - WAL File: 000000010000000000000003) 12 13[...] 14 15Streaming Replication status 16Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 17---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 18ogenki-2 0/3000060 0/3000060 0/3000060 0/3000060 00:00:00 00:00:00 00:00:00 streaming async 0 19 20Instances status 21Name Database Size Current LSN Replication role Status QoS Manager Version Node 22---- ------------- ----------- ---------------- ------ --- --------------- ---- 23ogenki-1 33 MB 0/3000060 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 24ogenki-2 33 MB 0/3000060 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc Nous allons désormais éffectuer ce que l'on appelle un \u0026quot;Switchover\u0026quot;: Nous allons promouvoir l'instance de secours en instance primaire.\nLe plugin cnpg permet de le faire de façon impérative, en utilisant la ligne de commande suivante:\n1kubectl cnpg promote ogenki ogenki-2 -n demo 2Node ogenki-2 in cluster ogenki will be promoted Dans mon cas, le basculement était vraiment rapide. Nous pouvons vérifier que l'instance ogenki-2 est devenu primaire et que la réplication est effectuée dans l'autre sens.\n1kubectl cnpg status -n demo ogenki 2[...] 3Status: Switchover in progress Switching over to ogenki-2 4Instances: 2 5Ready instances: 1 6[...] 7Streaming Replication status 8Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 9---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 10ogenki-1 0/4004CA0 0/4004CA0 0/4004CA0 0/4004CA0 00:00:00 00:00:00 00:00:00 streaming async 0 11 12Instances status 13Name Database Size Current LSN Replication role Status QoS Manager Version Node 14---- ------------- ----------- ---------------- ------ --- --------------- ---- 15ogenki-2 33 MB 0/4004CA0 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc 16ogenki-1 33 MB 0/4004CA0 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 Maintenant, provoquons un Failover en supprimant le pod principal\n1kubectl delete po -n demo --grace-period 0 --force ogenki-2 2Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 3pod \u0026#34;ogenki-2\u0026#34; force deleted 1Cluster Summary 2Name: ogenki 3Namespace: demo 4System ID: 7155095145869606932 5PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 6Primary instance: ogenki-1 7Status: Failing over Failing over from ogenki-2 to ogenki-1 8Instances: 2 9Ready instances: 1 10Current Write LSN: 0/4005D98 (Timeline: 3 - WAL File: 000000030000000000000004) 11 12[...] 13Instances status 14Name Database Size Current LSN Replication role Status QoS Manager Version Node 15---- ------------- ----------- ---------------- ------ --- --------------- ---- 16ogenki-1 33 MB 0/40078D8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 17ogenki-2 - - - pod not available Burstable - gke-kcdfrance-main-np-0e87115b-xszc Quelques secondes plus tard le cluster devient opérationnel à nouveau.\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 13m 2 2 Cluster in healthy state ogenki-1 Jusqu'ici tout va bien, nous avons pu faire quelques tests de la haute disponibilité et c'était assez probant 😎.\n👁️ Supervision Nous allons utiliser la Stack Prometheus. Nous ne couvrirons pas son installation dans cet article. Si vous voulez voir comment l'installer avec Flux, vous pouvez jeter un oeil à cet exemple.\nPour récupérer les métriques de notre instance, nous devons créer un PodMonitor.\n1apiVersion: monitoring.coreos.com/v1 2kind: PodMonitor 3metadata: 4 labels: 5 prometheus-instance: main 6 name: cnpg-ogenki 7 namespace: demo 8spec: 9 namespaceSelector: 10 matchNames: 11 - demo 12 podMetricsEndpoints: 13 - port: metrics 14 selector: 15 matchLabels: 16 postgresql: ogenki Nous pouvons ensuite ajouter le tableau de bord Grafana disponible ici.\nEnfin, vous souhaiterez peut-être configurer des alertes et vous pouvez créer un PrometheusRule en utilisant ces règles.\n\u0026#x1f525; Performances and benchmark Info Mise à jour: Il est désormais possible de faire un test de performance avec le plugin cnpg\nAfin de connaitre les limites de votre serveur, vous devriez faire un test de performances et de conserver une base de référence pour de futures améliorations.\nNote Au sujet des performances, il existe de nombreux domaines d'amélioration sur lesquels nous pouvons travailler.Cela dépend principalement de l'objectif que nous voulons atteindre. En effet, nous ne voulons pas perdre du temps et de l'argent pour les performances dont nous n'aurons probablement jamais besoin.\nVoici les principaux éléments à analyser:\nTuning de la configuration PostgreSQL Resources systèmes (cpu et mémoire) Types de Disque : IOPS, stockage locale (local-volume-provisioner), Disques dédiées pour les WAL et les données PG_DATA \u0026quot;Pooling\u0026quot; de connexions PGBouncer. CloudNativePG fourni une resource personnalisée Pooler qui permet de configurer cela facilement. Optimisation de la base de données, analyser les plans d'exécution grâce à explain, utiliser l'extension pg_stat_statement ... Tout d'abord, nous ajouterons des \u0026quot;labels\u0026quot; aux nœuds afin d'exécuter la commande pgbench sur différentes machines de celles hébergeant la base de données.\n1PG_NODE=$(kubectl get po -n demo -l postgresql=ogenki,role=primary -o jsonpath={.items[0].spec.nodeName}) 2kubectl label node ${PG_NODE} workload=postgresql 3node/gke-kcdfrance-main-np-0e87115b-vlzm labeled 4 5 6# Choose any other node different than the ${PG_NODE} 7kubectl label node gke-kcdfrance-main-np-0e87115b-p5d7 workload=pgbench 8node/gke-kcdfrance-main-np-0e87115b-p5d7 labeled Et nous déploierons le chart Helm comme suit\n1git clone git@github.com:EnterpriseDB/cnp-bench.git 2cd cnp-bench 3 4cat \u0026gt; pgbench-benchmark/myvalues.yaml \u0026lt;\u0026lt;EOF 5cnp: 6 existingCluster: true 7 existingHost: ogenki-rw 8 existingCredentials: cnpg-mydb-superuser 9 existingDatabase: mydb 10 11pgbench: 12 # Node where to run pgbench 13 nodeSelector: 14 workload: pgbench 15 initialize: true 16 scaleFactor: 1 17 time: 600 18 clients: 10 19 jobs: 1 20 skipVacuum: false 21 reportLatencies: false 22EOF 23 24helm upgrade --install -n demo pgbench -f pgbench-benchmark/myvalues.yaml pgbench-benchmark/ Info Il existe différents services selon que vous souhaitez lire et écrire ou de la lecture seule.\n1kubectl get ep -n demo 2NAME ENDPOINTS AGE 3ogenki-any 10.64.1.136:5432,10.64.1.3:5432 15d 4ogenki-r 10.64.1.136:5432,10.64.1.3:5432 15d 5ogenki-ro 10.64.1.136:5432 15d 6ogenki-rw 10.64.1.3:5432 15d 1kubectl logs -n demo job/pgbench-pgbench-benchmark -f 2Defaulted container \u0026#34;pgbench\u0026#34; out of: pgbench, wait-for-cnp (init), pgbench-init (init) 3pgbench (14.1, server 14.5 (Debian 14.5-2.pgdg110+2)) 4starting vacuum...end. 5transaction type: \u0026lt;builtin: TPC-B (sort of)\u0026gt; 6scaling factor: 1 7query mode: simple 8number of clients: 10 9number of threads: 1 10duration: 600 s 11number of transactions actually processed: 545187 12latency average = 11.004 ms 13initial connection time = 111.585 ms 14tps = 908.782896 (without initial connection time) 💽 Sauvegarde and restaurations Note Le fait de pouvoir stocker des sauvegarde et fichiers WAL dans le bucket GCP est possible car nous avons attribué les autorisations en utilisant une annotation présente dans le ServiceAccount utilisé par le cluster\n1serviceAccountTemplate: 2 metadata: 3 annotations: 4 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com Nous pouvons d'abord déclencher une sauvegarde on demand à l'aide de la ressource personnalisée Backup\n1apiVersion: postgresql.cnpg.io/v1 2kind: Backup 3metadata: 4 name: ogenki-now 5 namespace: demo 6spec: 7 cluster: 8 name: ogenki 1kubectl apply -f backup.yaml 2backup.postgresql.cnpg.io/ogenki-now created 3 4kubectl get backup -n demo 5NAME AGE CLUSTER PHASE ERROR 6ogenki-now 36s ogenki completed Si vous jetez un œil au contenu du bucket GCS, vous verrez un nouveau répertoire qui stocke les sauvegardes de base (\u0026quot;base backups\u0026quot;).\n1gcloud storage ls gs://cnpg-ogenki/ogenki/base 2gs://cnpg-ogenki/ogenki/base/20221023T130327/ Mais la plupart du temps, nous préfererons configurer une sauvegarde planifiée (\u0026quot;scheduled\u0026quot;). Ci-dessous un exemple pour une sauvegarde quotidienne:\n1apiVersion: postgresql.cnpg.io/v1 2kind: ScheduledBackup 3metadata: 4 name: ogenki-daily 5 namespace: demo 6spec: 7 backupOwnerReference: self 8 cluster: 9 name: ogenki 10 schedule: 0 0 0 * * * Les restaurations ne peuvent être effectuées que sur de nouvelles instances. Ici, nous utiliserons la sauvegarde que nous avions créée précédemment afin d'initialiser une toute nouvelle instance.\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore] 7 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 8 role: roles/iam.workloadIdentityUser 9etag: BwXrs755FPA= 10version: 1 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki-restore 5 namespace: demo 6spec: 7 instances: 1 8 9 serviceAccountTemplate: 10 metadata: 11 annotations: 12 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 13 14 storage: 15 storageClass: standard 16 size: 10Gi 17 18 resources: 19 requests: 20 memory: \u0026#34;1Gi\u0026#34; 21 cpu: \u0026#34;500m\u0026#34; 22 limits: 23 memory: \u0026#34;1Gi\u0026#34; 24 25 superuserSecret: 26 name: cnpg-mydb-superuser 27 28 bootstrap: 29 recovery: 30 backup: 31 name: ogenki-now Nous notons qu'un pod se charge immédiatement de la restauration complète (\u0026quot;full recovery\u0026quot;).\n1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 1 (18h ago) 18h 4ogenki-2 1/1 Running 0 18h 5ogenki-restore-1 0/1 Init:0/1 0 0s 6ogenki-restore-1-full-recovery-5p4ct 0/1 Completed 0 51s Le nouveau cluster devient alors opérationnel (\u0026quot;Ready\u0026quot;).\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 18h 2 2 Cluster in healthy state ogenki-1 4ogenki-restore 80s 1 1 Cluster in healthy state ogenki-restore-1 \u0026#x1f9f9; Nettoyage Suppression du cluster\n1kubectl delete cluster -n demo ogenki ogenki-restore 2cluster.postgresql.cnpg.io \u0026#34;ogenki\u0026#34; deleted 3cluster.postgresql.cnpg.io \u0026#34;ogenki-restore\u0026#34; deleted Supprimer le service IAM\n1gcloud iam service-accounts delete cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 2You are about to delete service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 3 4Do you want to continue (Y/n)? y 5 6deleted service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com] 💭 Conclusion Je viens tout juste de découvrir CloudNativePG et je n'ai fait qu'en percevoir la surface, mais une chose est sûre: la gestion d'une instance PostgreSQL est vraiment facilitée. Cependant, le choix d'une solution de base de données est une décision complexe. Il faut prendre en compte le cas d'usage, les contraintes de l'entreprise, la criticité de l'application et les compétences des équipes opérationnelles. Il existe de nombreuses options: bases de données gérées par le fournisseur Cloud, installation traditionnelle sur serveur baremetal, solutions distribuées ...\nNous pouvons également envisager d'utiliser Crossplane et une Composition pour fournir un niveau d'abstraction supplémentaire afin de déclarer des bases de données des fournisseurs Cloud, mais cela nécessite plus de configuration.\nCloudNativePG sort du lot par sa simplicité: Super facile à exécuter et à comprendre. De plus, la documentation est excellente (l'une des meilleures que j'aie jamais vues!), Surtout pour un si jeune projet open source (cela aidera peut être pour être accepté en tant que projet \u0026quot;Sandbox\u0026quot; CNCF 🤞).\nSi vous voulez en savoir plus, il y avait une présentation à ce sujet à KubeCon NA 2022.\n","link":"https://blog.ogenki.io/fr/post/cnpg/","section":"post","tags":["data"],"title":"`CloudNativePG`: et PostgreSQL devient facile sur Kubernetes"},{"body":"","link":"https://blog.ogenki.io/fr/tags/data/","section":"tags","tags":null,"title":"data"},{"body":"In a previous article, we've seen how to use Crossplane so that we can manage cloud resources the same way as our applications. \u0026#x2764;\u0026#xfe0f; Declarative approach! There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.\nHere we'll achieve exactly the same thing but we'll do that in the GitOps way. According to the OpenGitOps working group there are 4 GitOps principles:\nThe desired state of our system must be expressed declaratively. This state must be stored in a versioning system. Changes are pulled and applied automatically in the target platform whenever the desired state changes. If, for any reason, the current state is modified, it will be automatically reconciled with the desired state. There are several GitOps engine options. The most famous ones are ArgoCD and Flux. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).\nLearn more about GitOps toolkit components here.\n:bullseye: Our target Here we want to declare our desired infrastructure components only by adding git changes. By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance. We'll discover Flux basics and how to use it in order to build a complete GitOps CD workflow.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Install required tools First of all we need to install a few tools using asdf\nCreate a local file .tool-versions\n1cd ~/sources/devflux/ 2 3cat \u0026gt; .tool-versions \u0026lt;\u0026lt;EOF 4flux2 0.31.3 5kubectl 1.24.3 6kubeseal 0.18.1 7kustomize 4.5.5 8EOF 1for PLUGIN in $(cat .tool-versions | awk \u0026#39;{print $1}\u0026#39;); do asdf plugin-add $PLUGIN; done 2 3asdf install 4Downloading ... 100.0% 5Copying Binary 6... Check that all the required tools are actually installed.\n1asdf current 2flux2 0.31.3 /home/smana/sources/devflux/.tool-versions 3kubectl 1.24.3 /home/smana/sources/devflux/.tool-versions 4kubeseal 0.18.1 /home/smana/sources/devflux/.tool-versions 5kustomize 4.5.5 /home/smana/sources/devflux/.tool-versions \u0026#x1f511; Create a Github personal access token In this article the git repository is hosted in Github. In order to be able to use the flux bootstrap a personnal access token is required.\nPlease follow this procedure.\nWarning Store the Github token in a safe place for later use\n\u0026#x1f9d1;\u0026zwj;\u0026#x1f4bb; Clone the devflux repository All the files used for the upcoming steps can be retrieved from this repository. You should clone it, that will be easier to copy them into your own repository.\n1git clone https://github.com/Smana/devflux.git \u0026#x1f680; Bootstrap flux in the Crossplane cluster As we will often be using the flux CLI you may want to configure the bash|zsh completion\n1source \u0026lt;(flux completion bash) Warning Here we consider that you already have a local k3d instance. If not you may want to either go through the whole previous article or just run the local cluster creation.\nEnsure that you're working in the right context\n1kubectl config current-context 2k3d-crossplane Run the bootstrap command that will basically deploy all Flux's components in the namespace flux-system. Here I'll create a repository named devflux using my personal Github account.\n1export GITHUB_USER=\u0026lt;YOUR_ACCOUNT\u0026gt; 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/k3d-crossplane 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/k3d-crossplane\u0026#34; for \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux\u0026#34; 9... 10✔ all components are healthy Check that all the pods are running properly and that the kustomization flux-system has been successfully reconciled.\n1kubectl get po -n flux-system 2NAME READY STATUS RESTARTS AGE 3helm-controller-5985c795f8-gs2pc 1/1 Running 0 86s 4notification-controller-6b7d7485fc-lzlpg 1/1 Running 0 86s 5kustomize-controller-6d4669f847-9x844 1/1 Running 0 86s 6source-controller-5fb4888d8f-wgcqv 1/1 Running 0 86s 7 8flux get kustomizations 9NAME REVISION SUSPENDED READY MESSAGE 10flux-system main/33ebef1 False True Applied revision: main/33ebef1 Running the bootstap command actually creates a github repository if it doesn't exist yet. Clone it now for our upcoming changes. You'll notice that the first commit has been made by Flux.\n1git clone https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git 2Cloning into \u0026#39;devflux\u0026#39;... 3 4cd devflux 5 6git log -1 7commit 2beb6aafea67f3386b50cbc706fb34575844040d (HEAD -\u0026gt; main, origin/main, origin/HEAD) 8Author: Flux \u0026lt;\u0026gt; 9Date: Thu Jul 14 17:13:27 2022 +0200 10 11 Add Flux sync manifests 12 13ls clusters/k3d-crossplane/flux-system/ 14gotk-components.yaml gotk-sync.yaml kustomization.yaml \u0026#x1f4c2; Flux repository structure There are several options for organizing your resources in the Flux configuration repository. Here is a proposition for the sake of this article.\n1tree -d -L 2 2. 3├── apps 4│ ├── base 5│ └── dev-cluster 6├── clusters 7│ ├── dev-cluster 8│ └── k3d-crossplane 9├── infrastructure 10│ ├── base 11│ ├── dev-cluster 12│ └── k3d-crossplane 13├── observability 14│ ├── base 15│ ├── dev-cluster 16│ └── k3d-crossplane 17└── security 18 ├── base 19 ├── dev-cluster 20 └── k3d-crossplane Directory Description Example /apps our applications Here we'll deploy a demo application \u0026quot;online-boutique\u0026quot; /infrastructure base infrastructure/network components Crossplane as it will be used to provision cloud resources but we can also find CSI/CNI/EBS drivers... /observability All metrics/apm/logging tools Prometheus of course, Opentelemetry ... /security Any component that enhance our security level SealedSecrets (see below) Info For the upcoming steps please refer to the demo repository here\nLet's use this structure and begin to deploy applications \u0026#x1f680;.\n\u0026#x1f510; SealedSecrets There are plenty of alternatives when it comes to secrets management in Kubernetes. In order to securely store secrets in a git repository the GitOps way we'll make use of SealedSecrets. It uses a custom resource definition named SealedSecrets in order to encrypt the Kubernetes secret at the client side then the controller is in charge of decrypting and generating the expected secret in the cluster.\n\u0026#x1f6e0;\u0026#xfe0f; Deploy the controller using Helm The first thing to do is to declare the kustomization that handles all the security tools.\nclusters/k3d-crossplane/security.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 2kind: Kustomization 3metadata: 4 name: security 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 sourceRef: 10 kind: GitRepository 11 name: flux-system 12 path: ./security/k3d-crossplane 13 healthChecks: 14 - apiVersion: helm.toolkit.fluxcd.io/v1beta1 15 kind: HelmRelease 16 name: sealed-secrets 17 namespace: kube-system Info A Kustomization is a custom resource that comes with Flux. It basically points to a set of Kubernetes resources managed with kustomize The above security kustomization points to a local directory where the kustomize resources are.\n1... 2spec: 3 path: ./security/k3d-crossplane 4... Note This is worth noting that there are two types on kustomizations. That can be confusing when you start playing with Flux.\nOne managed by flux's kustomize controller. Its API is kustomization.kustomize.toolkit.fluxcd.io The other kustomization.kustomize.config.k8s.io is for the kustomize overlay The kustomization.yaml file is always used for the kustomize overlay. Flux itself doesn't need this overlay in all cases, but if you want to use features of a Kustomize overlay you will occasionally need to create it in order to access them. It provides instructions to the Kustomize CLI.\nWe will deploy SealedSecrets using the Helm chart. So we need to declare the source of this chart. Using the kustomize overlay system, we'll first create the base files that will be inherited at the cluster level.\nsecurity/base/sealed-secrets/source.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: sealed-secrets 5 namespace: flux-system 6spec: 7 interval: 30m 8 url: https://bitnami-labs.github.io/sealed-secrets Then we'll define the HelmRelease which references the above source. Put the values you want to apply to the Helm chart under spec.values\nsecurity/base/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 releaseName: sealed-secrets 8 chart: 9 spec: 10 chart: sealed-secrets 11 sourceRef: 12 kind: HelmRepository 13 name: sealed-secrets 14 namespace: flux-system 15 version: \u0026#34;2.4.0\u0026#34; 16 interval: 10m0s 17 install: 18 remediation: 19 retries: 3 20 values: 21 fullnameOverride: sealed-secrets-controller 22 resources: 23 requests: 24 cpu: 80m 25 memory: 100Mi If you're starting your repository from scratch you'll need to generate the kustomization.yaml file (kustomize overlay).\n1kustomize create --autodetect security/base/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4- helmrelease.yaml 5- source.yaml Now we declare the sealed-secret kustomization at the cluster level. Just for the example we'll overwrite a value at the cluster level using kustomize's overlay system.\nsecurity/k3d-crossplane/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 values: 8 resources: 9 requests: 10 cpu: 100m security/k3d-crossplane/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3bases: 4 - ../../base 5patches: 6 - helmrelease.yaml Pushing our changes is the only thing to do in order to get sealed-secrets deployed in the target cluster.\n1git commit -m \u0026#34;security: deploy sealed-secrets in k3d-crossplane\u0026#34; 2[security/sealed-secrets 283648e] security: deploy sealed-secrets in k3d-crossplane 3 6 files changed, 66 insertions(+) 4 create mode 100644 clusters/k3d-crossplane/security.yaml 5 create mode 100644 security/base/sealed-secrets/helmrelease.yaml 6 create mode 100644 security/base/sealed-secrets/kustomization.yaml 7 create mode 100644 security/base/sealed-secrets/source.yaml 8 create mode 100644 security/k3d-crossplane/sealed-secrets/helmrelease.yaml 9 create mode 100644 security/k3d-crossplane/sealed-secrets/kustomization.yaml After a few seconds (1 minutes by default) a new kustomization will appear.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3flux-system main/d36a33c False True Applied revision: main/d36a33c 4security main/d36a33c False True Applied revision: main/d36a33c And all the resources that we declared in the flux repository should be available and READY.\n1flux get sources helm 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee False True stored artifact for revision \u0026#39;4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee\u0026#39; 1flux get helmrelease -n kube-system 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 2.2.0 False True Release reconciliation succeeded 🧪 A first test SealedSecret Let's use the CLI kubeseal to test it out. We'll create a SealedSecret that will be decrypted by the sealed-secrets controller in the cluster and create the expected secret foobar\n1kubectl create secret generic foobar -n default --dry-run=client -o yaml --from-literal=foo=bar \\ 2| kubeseal --namespace default --format yaml | kubectl apply -f - 3sealedsecret.bitnami.com/foobar created 4 5kubectl get secret -n default foobar 6NAME TYPE DATA AGE 7foobar Opaque 1 3m13s 8 9kubectl delete sealedsecrets.bitnami.com foobar 10sealedsecret.bitnami.com \u0026#34;foobar\u0026#34; deleted \u0026#x2601;\u0026#xfe0f; Deploy and configure Crossplane \u0026#x1f511; Create the Google service account secret The first thing we need to do in order to get Crossplane working is to create the GCP serviceaccount. The steps have been covered here in the previous article. We'll create a SealedSecret gcp-creds that contains the serviceaccount file crossplane.json.\ninfrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml\n1kubectl create secret generic gcp-creds --context k3d-crossplane -n crossplane-system --from-file=creds=./crossplane.json --dry-run=client -o yaml \\ 2| kubeseal --format yaml --namespace crossplane-system - \u0026gt; infrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml 🔄 Crossplane dependencies Now we will deploy Crossplane with Flux. I won't put the manifests here you'll find all of them in this repository. However it's important to understand that, in order to deploy and configure Crossplane properly we need to do that in a specific order. Indeed several CRD's (custom resource definitions) are required:\nFirst of all we'll install the crossplane controller. Then we'll configure the provider because the custom resource is now available thanks to the crossplane controller installation. Finally a provider installation deploys several CRDs that can be used to configure the provider itself and cloud resources. The dependencies between kustomizations can be controlled using the parameters dependsOn. Looking at the file clusters/k3d-crossplane/infrastructure.yaml, we can see for example that the kustomization infrastructure-custom-resources depends on the kustomization crossplane_provider which itself depends on crossplane-configuration....\n1--- 2apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 3kind: Kustomization 4metadata: 5 name: crossplane-provider 6spec: 7... 8 dependsOn: 9 - name: crossplane-core 10--- 11apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 12kind: Kustomization 13metadata: 14 name: crossplane-configuration 15spec: 16... 17 dependsOn: 18 - name: crossplane-provider 19--- 20apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 21kind: Kustomization 22metadata: 23 name: infrastructure-custom-resources 24spec: 25... 26 dependsOn: 27 - name: crossplane-configuration Commit and push the changes for the kustomisations to appear. Note that they'll be reconciled in the defined order.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3infrastructure-custom-resources False False dependency \u0026#39;flux-system/crossplane-configuration\u0026#39; is not ready 4crossplane-configuration False False dependency \u0026#39;flux-system/crossplane-provider\u0026#39; is not ready 5security main/666f85a False True Applied revision: main/666f85a 6flux-system main/666f85a False True Applied revision: main/666f85a 7crossplane-core main/666f85a False True Applied revision: main/666f85a 8crossplane-provider main/666f85a False True Applied revision: main/666f85a Then all Crossplane components will be deployed, we can have a look to the HelmRelease status for instance.\n1kubectl describe helmrelease -n crossplane-system crossplane 2... 3Status: 4 Conditions: 5 Last Transition Time: 2022-07-15T19:12:04Z 6 Message: Release reconciliation succeeded 7 Reason: ReconciliationSucceeded 8 Status: True 9 Type: Ready 10 Last Transition Time: 2022-07-15T19:12:04Z 11 Message: Helm upgrade succeeded 12 Reason: UpgradeSucceeded 13 Status: True 14 Type: Released 15 Helm Chart: crossplane-system/crossplane-system-crossplane 16 Last Applied Revision: 1.9.0 17 Last Attempted Revision: 1.9.0 18 Last Attempted Values Checksum: 056dc1c6029b3a644adc7d6a69a93620afd25b65 19 Last Release Revision: 2 20 Observed Generation: 1 21Events: 22 Type Reason Age From Message 23 ---- ------ ---- ---- ------- 24 Normal info 20m helm-controller HelmChart \u0026#39;crossplane-system/crossplane-system-crossplane\u0026#39; is not ready 25 Normal info 20m helm-controller Helm upgrade has started 26 Normal info 19m helm-controller Helm upgrade succeeded And our GKE cluster should also be created because we defined a bunch of crossplane custom resources in infrastructure/k3d-crossplane/custom-resources/crossplane\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RUNNING 34.x.x.190 europe-west9-a 22m \u0026#x1f680; Bootstrap flux in the dev cluster Our local Crossplane cluster is now ready and it created our dev cluster and we also want it to be managed with Flux. So let's configure Flux for this dev cluster using the same bootstrap command.\nAuthenticate to the newly created cluster. The following command will automatically change your current context.\n1gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project \u0026lt;your_project\u0026gt; 2Fetching cluster endpoint and auth data. 3kubeconfig entry generated for dev-cluster. 4 5kubectl config current-context 6gke_\u0026lt;your_project\u0026gt;_europe-west9-a_dev-cluster Run the bootstrap command for the dev-cluster.\n1export GITHUB_USER=Smana 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/dev-cluster 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/Smana/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/dev-cluster\u0026#34; for \u0026#34;https://github.com/Smana/devflux\u0026#34; 9... 10✔ all components are healthy Note It's worth noting that each Kubernetes cluster generates its own sealing keys. That means that if you recreate the dev-cluster, you must regenerate all the sealedsecrets. In our example we declared a secret in order to set the Grafana credentials. Here's the command you need to run in order to create a new version of the sealedsecret and don't forget to use the proper context \u0026#x1f609;.\n1kubectl create secret generic kube-prometheus-stack-grafana \\ 2--from-literal=admin-user=admin --from-literal=admin-password=\u0026lt;yourpassword\u0026gt; --namespace observability --dry-run=client -o yaml \\ 3| kubeseal --namespace observability --format yaml \u0026gt; observability/dev-cluster/kube-prometheus-stack/sealedsecrets.yaml After a few seconds we'll get the following kustomizations deployed.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3apps main/1380eaa False True Applied revision: main/1380eaa 4flux-system main/1380eaa False True Applied revision: main/1380eaa 5observability main/1380eaa False True Applied revision: main/1380eaa 6security main/1380eaa False True Applied revision: main/1380eaa Here we configured the prometheus stack and deployed a demo microservices stack named \u0026quot;online-boutique\u0026quot; This demo application exposes the frontend through a service of type LoadBalancer.\n1kubectl get svc -n demo frontend-external 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3frontend-external LoadBalancer 10.140.174.201 34.155.121.2 80:31943/TCP 7m44s Use the EXTERNAL_IP\n\u0026#x1f575;\u0026#xfe0f; Troubleshooting The cheatsheet in Flux's documentation contains many ways for troubleshooting when something goes wrong. Here I'll just give a sample of my favorite command lines.\nObjects that aren't ready\n1flux get all -A --status-selector ready=false Checking the logs of a given kustomization\n1flux logs --kind kustomization --name infrastructure-custom-resources 22022-07-15T19:38:52.996Z info Kustomization/infrastructure-custom-resources.flux-system - server-side apply completed 32022-07-15T19:38:53.016Z info Kustomization/infrastructure-custom-resources.flux-system - Reconciliation finished in 66.12266ms, next run in 4m0s 42022-07-15T19:11:34.697Z info Kustomization/infrastructure-custom-resources.flux-system - Discarding event, no alerts found for the involved object Show how a given pod is managed by Flux.\n1flux trace -n crossplane-system pod/crossplane-5dc8d888d7-g95qx 2 3Object: Pod/crossplane-5dc8d888d7-g95qx 4Namespace: crossplane-system 5Status: Managed by Flux 6--- 7HelmRelease: crossplane 8Namespace: crossplane-system 9Revision: 1.9.0 10Status: Last reconciled at 2022-07-15 21:12:04 +0200 CEST 11Message: Release reconciliation succeeded 12--- 13HelmChart: crossplane-system-crossplane 14Namespace: crossplane-system 15Chart: crossplane 16Version: 1.9.0 17Revision: 1.9.0 18Status: Last reconciled at 2022-07-15 21:11:36 +0200 CEST 19Message: pulled \u0026#39;crossplane\u0026#39; chart with version \u0026#39;1.9.0\u0026#39; 20--- 21HelmRepository: crossplane 22Namespace: crossplane-system 23URL: https://charts.crossplane.io/stable 24Revision: 362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d 25Status: Last reconciled at 2022-07-15 21:11:35 +0200 CEST 26Message: stored artifact for revision \u0026#39;362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d\u0026#39; If you want to check what would be the changes before pushing your commit. In thi given example I just increased the cpu requests for the sealed-secrets controller.\n1flux diff kustomization security --path security/k3d-crossplane 2✓ Kustomization diffing... 3► HelmRelease/kube-system/sealed-secrets drifted 4 5metadata.generation 6 ± value change 7 - 6 8 + 7 9 10spec.values.resources.requests.cpu 11 ± value change 12 - 100m 13 + 120m 14 15⚠️ identified at least one change, exiting with non-zero exit code \u0026#x1f9f9; Cleanup Don't forget to delete the Cloud resources if you don't want to have a bad suprise \u0026#x1f4b5;! Just comment the file infrastructure/k3d-crossplane/custom-resources/crossplane/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4 # - cluster.yaml 5 - network.yaml \u0026#x1f44f; Achievements With our current setup everything is configured using the GitOps approach:\nWe can manage infrastructure resources using Crossplane. Our secrets are securely stored in our git repository. We have a dev-cluster that we can enable or disable just but commenting a yaml file. Our demo application can be deployed from scratch in seconds. 💭 final thoughts Flux is probably the tool I'm using the most on a daily basis. It's really amazing!\nWhen you get familiar with its concepts and the command line it becomes really easy to use and troubleshoot. You can use either Helm when a chart is available or Kustomize.\nHowever we faced a few issues:\nIt's not straightforward to find an efficient structure depending on the company needs. Especially when you have several Kubernetes controllers that depend on other CRDs. The Helm controller doesn't maintain a state of the Kubernetes resources deployed by the Helm chart. That means that if you delete a resource which has been deployed through a Helm chart, it won't be reconciled (It will change soon. Being discussed here) Flux doesn't provide itself a web UI and switching between CLIs (kubectl, flux ...) can be annoying from a developer perspective. (I'm going to test weave-gitops ) I've been using Flux in production for more than a year and we configured it with the image automation so that the only thing a developer has to do is to merge a pull request and the new version of the application is automatically deployed in the target cluster.\nI should probably give another try to ArgoCD in order to be able to compare these precisely 🤔.\n","link":"https://blog.ogenki.io/fr/post/devflux/","section":"post","tags":["gitops","devxp"],"title":"100% `GitOps` using Flux"},{"body":"Qui suis-je? I'm a senior Site Reliability Engineer with a particular interest in Linux containers and cloud technologies. I worked in different companies (small startups and large scale) and I've been working in different areas in order to improve the reliability, availability of the platform as well as the developer experience. I helped several companies in their transition to the Cloud. I was leading SRE/DevOps teams (diverse profiles with developers and SREs) and I really enjoy seeing them engaged in the same direction.\nAs side activities, I am an organizer of the Cloud Native Computing meetup in Paris and the Kubernetes Community Days France.\nHobbies: Reading SF books, Kick Boxing, Surfing/Skating/Inline Roller\n","link":"https://blog.ogenki.io/fr/about/","section":"","tags":null,"title":"About"},{"body":"","link":"https://blog.ogenki.io/fr/tags/devxp/","section":"tags","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/fr/tags/gitops/","section":"tags","tags":null,"title":"gitops"},{"body":"La cible de cette documentation est de pouvoir créer et gérer un cluster GKE en utilisant Crossplane.\nCrossplane exploite les principes de base de Kubernetes afin de fournir des ressources cloud et bien plus encore: une ** approche déclarative ** avec ** Drift Detections ** et ** réconciliations ** Utilisation de boucles de contrôle: Exploseding_head:.En d'autres termes, nous déclarons les ressources cloud que nous voulons et Crossplane garantit que l'état cible correspond à celui appliqué via l'API Kubernetes.\nVoici les étapes que nous suivrons afin d'obtenir un cluster Kubernetes pour le développement et les cas d'utilisation des expérimentations.\n\u0026#x1f433; Create the local k3d cluster for Crossplane's control plane k3d est un cluster Kubernetes léger qui exploite K3S qui s'exécute dans notre ordinateur portable local. Il existe plusieurs modèles de déploiement pour Crossplane, nous pourrions par exemple déployer le plan de contrôle sur un cluster de gestion sur Kubernetes ou un plan de contrôle par cluster Kubernetes. Ici, j'ai choisi une méthode simple qui est bien pour un cas d'utilisation personnelle: une instance Kubernetes locale ** dans laquelle je vais déployer Crossplane.\nLet's install k3d using asdf.\n1asdf plugin-add k3d 2 3asdf install k3d $(asdf latest k3d) 4* Downloading k3d release 5.4.1... 5k3d 5.4.1 installation was successful! Create a single node Kubernetes cluster.\n1k3d cluster create crossplane 2... 3INFO[0043] You can now use it like this: 4kubectl cluster-info 5 6k3d cluster list 7crossplane 1/1 0/0 true Check that the cluster is reachable using the kubectl CLI.\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:40643 3CoreDNS is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy We only need a single node for our Crossplane use case.\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-crossplane-server-0 Ready control-plane,master 26h v1.22.7+k3s1 \u0026#x2601;\u0026#xfe0f; Generate the Google Cloud service account Warning Store the downloaded crossplane.json credentials file in a safe place.\nCreate a service account\n1GCP_PROJECT=\u0026lt;your_project\u0026gt; 2gcloud iam service-accounts create crossplane --display-name \u0026#34;Crossplane\u0026#34; --project=${GCP_PROJECT} 3Created service account [crossplane]. Assign the proper permissions to the service account.\nCompute Network Admin Kubernetes Engine Admin Service Account User 1SA_EMAIL=$(gcloud iam service-accounts list --filter=\u0026#34;email ~ ^crossplane\u0026#34; --format=\u0026#39;value(email)\u0026#39;) 2 3gcloud projects add-iam-policy-binding \u0026#34;${GCP_PROJECT}\u0026#34; --member=serviceAccount:\u0026#34;${SA_EMAIL}\u0026#34; \\ 4--role=roles/container.admin --role=roles/compute.networkAdmin --role=roles/iam.serviceAccountUser 5Updated IAM policy for project [\u0026lt;project\u0026gt;]. 6bindings: 7- members: 8 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 9 role: roles/compute.networkAdmin 10- members: 11 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 12... 13version: 1 Download the service account key (json format)\n1gcloud iam service-accounts keys create crossplane.json --iam-account ${SA_EMAIL} 2created key [ea2eb9ce2939127xxxxxxxxxx] of type [json] as [crossplane.json] for [crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com] \u0026#x1f6a7; Deploy and configure Crossplane Now that we have a credentials file for Google Cloud, we can deploy the Crossplane operator and configure the provider-gcp provider.\nInfo Most of the following steps are issued from the official documentation\nWe'll first use Helm in order to install the operator\n1helm repo add crossplane-master https://charts.crossplane.io/master/ 2\u0026#34;crossplane-master\u0026#34; has been added to your repositories 3 4helm repo update 5...Successfully got an update from the \u0026#34;crossplane-master\u0026#34; chart repository 6 7helm install crossplane --namespace crossplane-system --create-namespace \\ 8--version 1.18.1 crossplane-stable/crossplane 9 10NAME: crossplane 11LAST DEPLOYED: Mon Jun 6 22:00:02 2022 12NAMESPACE: crossplane-system 13STATUS: deployed 14REVISION: 1 15TEST SUITE: None 16NOTES: 17Release: crossplane 18... Check that the operator is running properly.\n1kubectl get po -n crossplane-system 2NAME READY STATUS RESTARTS AGE 3crossplane-rbac-manager-54d96cd559-222hc 1/1 Running 0 3m37s 4crossplane-688c575476-lgklq 1/1 Running 0 3m37s Info All the files used for the upcoming steps are stored within this blog repository. So you should clone and change the current directory:\n1git clone https://github.com/Smana/smana.github.io.git 2 3cd smana.github.io/content/resources/crossplane_k3d Now we'll configure Crossplane so that it will be able to create and manage GCP resources. This is done by configuring the provider provider-gcp as follows.\nprovider.yaml\n1apiVersion: pkg.crossplane.io/v1 2kind: Provider 3metadata: 4 name: crossplane-provider-gcp 5spec: 6 package: crossplane/provider-gcp:v0.21.0 1kubectl apply -f provider.yaml 2provider.pkg.crossplane.io/crossplane-provider-gcp created 3 4kubectl get providers 5NAME INSTALLED HEALTHY PACKAGE AGE 6crossplane-provider-gcp True True crossplane/provider-gcp:v0.21.0 10s Create the Kubernetes secret that holds the GCP credentials file created above\n1kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./crossplane.json 2secret/gcp-creds created Then we need to create a resource named ProviderConfig and reference the newly created secret.\nprovider-config.yaml\n1apiVersion: gcp.crossplane.io/v1beta1 2kind: ProviderConfig 3metadata: 4 name: default 5spec: 6 projectID: ${GCP_PROJECT} 7 credentials: 8 source: Secret 9 secretRef: 10 namespace: crossplane-system 11 name: gcp-creds 12 key: creds 1kubectl apply -f provider-config.yaml 2providerconfig.gcp.crossplane.io/default created Info If the serviceaccount has the proper permissions we can create resources in GCP. In order to learn about all the available resources and parameters we can have a look to the provider's API reference.\nThe first resource we'll create is the network that will host our Kubernetes cluster.\nnetwork.yaml\n1apiVersion: compute.gcp.crossplane.io/v1beta1 2kind: Network 3metadata: 4 name: dev-network 5 labels: 6 service: vpc 7 creation: crossplane 8spec: 9 forProvider: 10 autoCreateSubnetworks: false 11 description: \u0026#34;Network used for experimentations and POCs\u0026#34; 12 routingConfig: 13 routingMode: REGIONAL 1kubectl get network 2NAME READY SYNCED 3dev-network True True You can even get more details by describing this resource. For instance if something fails you would see the message returned by the Cloud provider in the events.\n1kubectl describe network dev-network | grep -A 20 \u0026#39;^Status:\u0026#39; 2Status: 3 At Provider: 4 Creation Timestamp: 2022-06-28T09:45:30.703-07:00 5 Id: 3005424280727359173 6 Self Link: https://www.googleapis.com/compute/v1/projects/${GCP_PROJECT}/global/networks/dev-network 7 Conditions: 8 Last Transition Time: 2022-06-28T16:45:31Z 9 Reason: Available 10 Status: True 11 Type: Ready 12 Last Transition Time: 2022-06-30T16:36:59Z 13 Reason: ReconcileSuccess 14 Status: True 15 Type: Synced \u0026#x1f680; Create a GKE cluster Everything is ready so that we can create our GKE cluster. Applying the file cluster.yaml will create a cluster and attach a node group to it.\ncluster.yaml\n1--- 2apiVersion: container.gcp.crossplane.io/v1beta2 3kind: Cluster 4metadata: 5 name: dev-cluster 6spec: 7 forProvider: 8 description: \u0026#34;Kubernetes cluster for experimentations and POCs\u0026#34; 9 initialClusterVersion: \u0026#34;1.24\u0026#34; 10 releaseChannel: 11 channel: \u0026#34;RAPID\u0026#34; 12 location: europe-west9-a 13 addonsConfig: 14 gcePersistentDiskCsiDriverConfig: 15 enabled: true 16 networkPolicyConfig: 17 disabled: false 18 networkRef: 19 name: dev-network 20 ipAllocationPolicy: 21 createSubnetwork: true 22 useIpAliases: true 23 defaultMaxPodsConstraint: 24 maxPodsPerNode: 110 25 networkPolicy: 26 enabled: false 27 writeConnectionSecretToRef: 28 namespace: default 29 name: gke-conn 30--- 31apiVersion: container.gcp.crossplane.io/v1beta1 32kind: NodePool 33metadata: 34 name: main-np 35spec: 36 forProvider: 37 initialNodeCount: 1 38 autoscaling: 39 autoprovisioned: false 40 enabled: true 41 maxNodeCount: 4 42 minNodeCount: 1 43 clusterRef: 44 name: dev-cluster 45 config: 46 machineType: n2-standard-2 47 diskSizeGb: 120 48 diskType: pd-standard 49 imageType: cos_containerd 50 preemptible: true 51 labels: 52 environment: dev 53 managed-by: crossplane 54 oauthScopes: 55 - \u0026#34;https://www.googleapis.com/auth/devstorage.read_only\u0026#34; 56 - \u0026#34;https://www.googleapis.com/auth/logging.write\u0026#34; 57 - \u0026#34;https://www.googleapis.com/auth/monitoring\u0026#34; 58 - \u0026#34;https://www.googleapis.com/auth/servicecontrol\u0026#34; 59 - \u0026#34;https://www.googleapis.com/auth/service.management.readonly\u0026#34; 60 - \u0026#34;https://www.googleapis.com/auth/trace.append\u0026#34; 61 metadata: 62 disable-legacy-endpoints: \u0026#34;true\u0026#34; 63 shieldedInstanceConfig: 64 enableIntegrityMonitoring: true 65 enableSecureBoot: true 66 management: 67 autoRepair: true 68 autoUpgrade: true 69 maxPodsConstraint: 70 maxPodsPerNode: 60 71 locations: 72 - \u0026#34;europe-west9-a\u0026#34; 1kubectl apply -f cluster.yaml 2cluster.container.gcp.crossplane.io/dev-cluster created 3nodepool.container.gcp.crossplane.io/main-np created Note that it takes around 10 minutes for the Kubernetes API and the nodes to be available. The STATE will transition from PROVISIONING to RUNNING and when a change is being applied the cluster status is RECONCILING\n1watch \u0026#39;kubectl get cluster,nodepool\u0026#39; 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3cluster.container.gcp.crossplane.io/dev-cluster False True PROVISIONING 34.155.122.6 europe-west9-a 3m15s 4 5NAME READY SYNCED STATE CLUSTER-REF AGE 6nodepool.container.gcp.crossplane.io/main-np False False dev-cluster 3m15s When the column READY switches to True you can download the cluster's credentials.\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RECONCILING 34.42.42.42 europe-west9-a 6m23s 4 5gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project ${GCP_PROJECT} 6Fetching cluster endpoint and auth data. 7kubeconfig entry generated for dev-cluster. For better readability you may want to rename the context id for the newly created cluster\n1kubectl config rename-context gke_${GCP_PROJECT}_europe-west9-a_dev-cluster dev-cluster 2Context \u0026#34;gke_${GCP_PROJECT}_europe-west9-a_dev-cluster\u0026#34; renamed to \u0026#34;dev-cluster\u0026#34;. 3 4kubectl config get-contexts 5CURRENT NAME CLUSTER AUTHINFO NAMESPACE 6* dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster 7 k3d-crossplane k3d-crossplane admin@k3d-crossplane Check that you can call our brand new GKE API\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3gke-dev-cluster-main-np-d0d978f9-5fc0 Ready \u0026lt;none\u0026gt; 10m v1.24.1-gke.1400 That's great \u0026#x1f389; we know have a GKE cluster up and running.\n💭 final thoughts I've been using Crossplane for a few months now in a production environment.\nEven if I'm conviced about the declarative approach using the Kubernetes API, we decided to move with caution with it. It clearly doesn't have Terraform's community and maturity. We're still declaring our resources using the deletionPolicy: Orphan so that even if something goes wrong on the controller side the resource won't be deleted.\nFurthermore we limited to a specific list of usual AWS resources requested by our developers. Nevertheless our target has always been to empower developers and we had really positive feedback from them. That's the best indicator for us. As the project matures, we'll move more and more resources from Terraform to Crossplane.\nIMHO the key success of Crossplane depends on the providers maintenance and evolution. The Cloud providers interest and involvement is really important.\nIn our next article we'll see how to use a GitOps engine to run all the above steps.\n","link":"https://blog.ogenki.io/fr/post/crossplane_k3d/","section":"post","tags":["kubernetes","infrastructure"],"title":"Mon cluster Kubernetes (GKE) avec `Crossplane`"},{"body":"Afin d'installer des binaires et de pouvoir passer d'une version à une autre, j'aime utiliser asdf.\n\u0026#x1f4e5; Installation L'installation recommandée consiste à utiliser Git comme suit\n1git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.0 Il y a quelques étapes supplémentaires qui dépendent de votre shell. Voici celles que j'utilise pour bash:\n1. $HOME/.asdf/asdf.sh Vous voudrez probablement configurer la completion du shell comme suit\n1. $HOME/.asdf/completions/asdf.bash \u0026#x1f680; Prenons un exemple Listons tous les plugins disponibles pour trouver k3d\n1asdf plugin-list-all | grep k3d 2k3d https://github.com/spencergilbert/asdf-k3d.git Installons k3d\n1asdf plugin-add k3d Vérifier les versions disponibles\n1asdf list-all k3d| tail -n 3 25.4.0-dev.3 35.4.0 45.4.1 Nous installerons la dernière version\n1asdf install k3d latest 2* Downloading k3d release 5.4.1... 3k3d 5.4.1 installation was successful! Enfin, nous pouvons passer d'une version à une autre. Nous pouvons définir une version \u0026quot;globale\u0026quot; qui serait utilisée sur tous les répertoires.\n1asdf global k3d 5.4.1 ou utilisez une version locale en fonction du répertoire actuel\n1cd /tmp 2asdf local k3d 5.4.1 3 4asdf current k3d 5k3d 5.4.1 /tmp/.tool-versions \u0026#x1f9f9; Faire le ménage Désinstaller une version donnée\n1asdf uninstall k3d 5.4.1 Retirer un plugin\n1asdf plugin remove k3d ","link":"https://blog.ogenki.io/fr/post/asdf/asdf/","section":"post","tags":["tooling","local"],"title":"Gérer les versions d'outils avec `asdf`"},{"body":"","link":"https://blog.ogenki.io/fr/tags/local/","section":"tags","tags":null,"title":"local"},{"body":"","link":"https://blog.ogenki.io/fr/tags/tooling/","section":"tags","tags":null,"title":"tooling"},{"body":"","link":"https://blog.ogenki.io/fr/categories/","section":"categories","tags":null,"title":"Categories"},{"body":"","link":"https://blog.ogenki.io/fr/tags/index/","section":"tags","tags":null,"title":"index"},{"body":"","link":"https://blog.ogenki.io/fr/series/","section":"series","tags":null,"title":"Series"}]
\ No newline at end of file
+[{"body":"","link":"https://blog.ogenki.io/fr/","section":"","tags":null,"title":""},{"body":"","link":"https://blog.ogenki.io/fr/tags/network/","section":"tags","tags":null,"title":"network"},{"body":"","link":"https://blog.ogenki.io/fr/post/","section":"post","tags":["index"],"title":"Posts"},{"body":"Lorsqu'on parle de sécurisation de l'accès aux ressources Cloud, l'une des règles d'or est d'éviter les expositions directes à Internet. La question qui se pose alors pour les Devs/Ops est : comment, par exemple, accéder à une base de données, un cluster Kubernetes ou un serveur via SSH sans compromettre la sécurité? Les réseaux privés virtuels (VPN) offrent une réponse en établissant un lien sécurisé entre différents éléments d'un réseau, indépendamment de leur localisation géographique. De nombreuses solutions existent, allant de modèles en SaaS aux solutions que l'on peut héberger soi-même, utilisant divers protocoles et étant soit open source, soit propriétaires.\nParmi ces options, je souhaitais vous parler de Tailscale. Cette solution utilise le protocole WireGuard, réputé pour sa simplicité et sa performance. Avec Tailscale, il est possible de connecter des appareils ou serveurs de manière sécurisée, comme s'ils étaient sur un même réseau local, bien qu'ils soient répartis à travers le monde.\n🎯 Nos objectifs Comprendre comment fonctionne Tailscale Mise en oeuvre d'une connexion sécurisée avec AWS en quelques minutes Interragir avec l'API d'un cluster EKS via un réseau privé Accéder à des services hébergés sur Kubernetes en utilisant le réseau privé Pour le reste de cet article il faudra évidemment créer un compte Tailscale. A noter que l'authentification est déléguée à des fournisseurs d'identité tiers (ex: Okta, Onelogin, Google ...).\nLorsque le compte est crée, on a directement accès à la console de gestion ci-dessus. Elle permet notamment de lister les appareils connectés, de consulter les logs, de modifier la plupart des paramètres...\n💡 Sous le capot Terminologie Mesh VPN: Un mesh VPN est un type de réseau VPN où chaque nœud (c'est-à-dire chaque appareil ou machine) est connecté à tous les autres nœuds du réseau, formant ainsi un maillage. À distinguer des configurations VPN traditionnelles qui sont conçues généralement \u0026quot;en étoile\u0026quot;, où plusieurs clients se connectent à un serveur central.\nZero trust: Signifie que chaque demande d'accès à un réseau est traitée comme si elle venait d'une source non fiable. Une application ou utilisateur doit prouver son identité et être autorisée avant d'accéder à une ressource. On ne fait pas confiance simplement parce qu'une machine ou un utilisateur provient d'un réseau interne ou d'une certaine zone géographique.\nTailnet: Dès la première utilisation de Tailscale, un Tailnet est crée pour vous et correspond à votre propre réseau privé. Chaque appareil dans un tailnet reçoit une IP Tailscale unique, permettant une communication directe entre eux.\nL'architecture de Tailscale est conçue de telle sorte que le Control plane et le Data plane sont clairement séparés:\nD'une part, il y a le serveur de coordination. Son rôle est d'échanger des métadonnées et des clés publiques entre tous les participants d'un Tailnet (La clé privée étant gardée en toute sécurité son nœud d'origine).\nD'autre part, les nœuds du Tailnet s'organisent en un réseau maillé (Mesh). Au lieu de passer par le serveur de coordination pour échanger des données, ces nœuds communiquent directement les uns avec les autres en mode point à point. Chaque nœud dispose d'une identité unique pour s'authentifier et rejoindre le Tailnet.\n\u0026#x1f4e5; Installation du client La majorité des plateformes sont supportées et les procédures d'installation sont listées ici. En ce qui me concerne je suis sur Archlinux:\n1sudo pacman -S tailscale Il est possible de démarrer le service automatiquement au démarrage de la machine.\n1sudo systemctl enable --now tailscaled Pour enregistrer son ordinateur perso, lancer la commande suivante:\n1sudo tailscale up --accept-routes 2 3To authenticate, visit: 4 5 https://login.tailscale.com/a/f50... ℹ️ l'option --accept-routes est nécessaire sur Linux et permettra d'accepter les routes annoncées par les Subnet routers. On verra cela dans la suite de l'article\nVérifier que vous avez bien obtenu une IP du réseau Tailscale:\n1tailscale ip -4 2100.118.83.67 3 4tailscale status 5100.118.83.67 ogenki smainklh@ linux - ℹ️ Pour les utilisateurs de Linux, vérifier que Tailscale fonctionne bien avec votre configuration DNS: Suivre cette documentation.\nLes sources Toutes les étapes réalisées dans cet article proviennent de ce dépôt git\nIl va permettre de créer l'ensemble des composants qui ont pour objectif d'obtenir un cluster EKS de Lab et font suite à un précédent article sur Cilium et Gateway API.\n☁️ Accéder à AWS en privé Afin de pouvoir accéder de manière sécurisée à l'ensemble des ressources disponibles sur AWS, il est possible de déployer un Subnet router.\nUn Subnet router est une instance Tailscale qui permet d'accéder à des sous-réseaux qui ne sont pas directement liés à Tailscale. Il fait office de pont entre le réseau privé virtuel de Tailscale (Tailnet) et d'autres réseaux locaux.\nNous pouvons alors router des sous réseaux du Clouder à travers le VPN de Tailscale.\n⚠️ Pour ce faire, sur AWS, il faudra bien entendu configurer les security groups correctement pour autoriser les Subnet routers.\n🚀 Déployer un Subnet router Entrons dans le vif du sujet et deployons un Subnet router sur un réseau AWS! Tout est fait en utilisant le code Terraform présent dans le répertoire terraform/network. Nous allons analyser la configuration spécifique à Tailscale qui est présente dans le fichier tailscale.tf avant de procéder au déploiement.\nLe provider Terraform Il est possible de configurer certains paramètres au travers de l'API Tailscale grâce au provider Terraform. Pour cela il faut au préalable génerer une clé d'API 🔑 sur la console d'admin:\nIl faudra conserver cette clé dans un endroit sécurisé car elle est utilisée pour déployer le Subnet router\n1provider \u0026#34;tailscale\u0026#34; { 2 api_key = var.tailscale.api_key 3 tailnet = var.tailscale.tailnet 4} les ACL's\nLes ACL's permettent de définir qui est autorisé à communiquer avec qui (utilisateur ou appareil). À la création d'un compte, celle-cis sont très permissives et il n'y a aucune restriction (tout le monde peut parler avec tout le monde).\n1resource \u0026#34;tailscale_acl\u0026#34; \u0026#34;this\u0026#34; { 2 acl = jsonencode({ 3 acls = [ 4 { 5 action = \u0026#34;accept\u0026#34; 6 src = [\u0026#34;*\u0026#34;] 7 dst = [\u0026#34;*:*\u0026#34;] 8 } 9 ] 10... 11} Note Pour mon environnement de Lab, j'ai conservé cette configuration par défault car je suis la seule personne à y accéder. De plus les seuls appareils connectés à mon Tailnet sont mon laptop et le Subnet router. En revanche dans un cadre d'entreprise, il faudra bien y réfléchir. Il est alors possible de définir une politique basée sur des groupes d'utilisitateurs ou sur les tags des noeuds.\nConsulter cette doc pour plus d'info.\nLes noms de domaines (DNS)\nIl y a différentes façons possibles de gérer les noms de domaines avec Tailscale:\nMagic DNS: Lorsqu'un appareil rejoint le Tailnet, il s'enregistre avec un nom et celui-ci peut-être utilisé directement pour communiquer avec l'appareil.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.115.31.152 ip-10-0-43-98 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 3044 rx 2588 4 5ping ip-10-0-43-98 6PING ip-10-0-43-98.tail9c382.ts.net (100.115.31.152) 56(84) bytes of data. 764 bytes from ip-10-0-43-98.tail9c382.ts.net (100.115.31.152): icmp_seq=1 ttl=64 time=11.4 ms AWS: Pour utiliser les noms de domaines internes à AWS il est possible d'utiliser la deuxième IP du VPC qui correspond toujours au serveur DNS. Cela permet d'utiliser les éventuelles zones privées sur route53 ou de se connecter aux ressources en utilisant les noms de domaines.\nLa configuration la plus simple est donc de déclarer la liste des serveurs DNS à utiliser et d'y ajouter celui de AWS. Ici un exemple avec le DNS publique de Cloudflare.\n1resource \u0026#34;tailscale_dns_nameservers\u0026#34; \u0026#34;this\u0026#34; { 2 nameservers = [ 3 \u0026#34;1.1.1.1\u0026#34;, 4 cidrhost(module.vpc.vpc_cidr_block, 2) 5 ] 6} La clé d'authentification (\u0026quot;auth key\u0026quot;)\nPour qu'un appareil puisse rejoindre le Tailnet au démarrage il faut que Tailscale soit démarré en utilisant une clé d'authentification. Celle-ci est générée comme suit\n1resource \u0026#34;tailscale_tailnet_key\u0026#34; \u0026#34;this\u0026#34; { 2 reusable = true 3 ephemeral = false 4 preauthorized = true 5} reusable: S'agissant d'un autoscaling group, il faut que cette même clé puisse être utilisée plusieurs fois. ephemeral: Pour cette démo nous créons une clé qui n'expire pas. En production il serait préférable d'activer l'expiration. preauthorized: Il faut que cette clé soit déjà valide et autorisée pour que l'instance rejoigne automatiquement le Tailscale. La clé ainsi générée est utilisée pour lancer tailscale avec le paramètre --auth-key\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; Annoncer les routes pour les réseaux AWS\nEnfin il faut annoncer le réseau que l'on souhaite faire passer par le Subnet router. Dans notre exemple, nous décidons de router tout le réseau du VPC qui a pour CIDR 10.0.0.0/16.\nAfin que cela soit possible de façon automatique, il y a une règle autoApprovers à ajouter. Cela permet d'indiquer que les routes annoncées par l'utilisateur smainklh@gmail.com sont autorisées sans que cela requiert une étape d'approbation.\n1 autoApprovers = { 2 routes = { 3 \u0026#34;10.0.0.0/16\u0026#34; = [\u0026#34;smainklh@gmail.com\u0026#34;] 4 } 5 } La commande lancée au démarrage de l'instance Subnet router est la suivante:\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; --advertise-routes=\u0026#34;10.0.0.0/16\u0026#34; Le module Terraform J'ai créé un module très simple qui permet de déployer un autoscaling group sur AWS et de configurer Tailscale. Au démarrage de l'instance, elle s'authentifiera en utilisant une auth_key et annoncera les réseaux indiqués. Dans l'exemple ci-dessous l'instance annonce le CIDR du VPC sur AWS.\n1module \u0026#34;tailscale_subnet_router\u0026#34; { 2 source = \u0026#34;Smana/tailscale-subnet-router/aws\u0026#34; 3 version = \u0026#34;1.0.4\u0026#34; 4 5 region = var.region 6 env = var.env 7 8 name = var.tailscale.subnet_router_name 9 auth_key = tailscale_tailnet_key.this.key 10 11 vpc_id = module.vpc.vpc_id 12 subnet_ids = module.vpc.private_subnets 13 advertise_routes = [module.vpc.vpc_cidr_block] 14... 15} Maintenant que nous avons analysé les différents paramètres, il est temps de démarrer notre Subnet router 🚀 !! Il faut au préalable créer un fichier variable.tfvars dans le répertoire terraform/network.\n1env = \u0026#34;dev\u0026#34; 2region = \u0026#34;eu-west-3\u0026#34; 3private_domain_name = \u0026#34;priv.cloud.ogenki.io\u0026#34; 4 5tailscale = { 6 subnet_router_name = \u0026#34;ogenki\u0026#34; 7 tailnet = \u0026#34;smainklh@gmail.com\u0026#34; 8 api_key = \u0026#34;tskey-api-...\u0026#34; 9} 10 11tags = { 12 project = \u0026#34;demo-secured-eks\u0026#34; 13 owner = \u0026#34;Smana\u0026#34; 14} Puis lancer la commande suivante:\n1tofu plan --var-file variables.tfvars Après vérification du plan, appliquer les changements\n1tofu apply --var-file variables.tfvars Quand l'instance est démarrée, elle apparaitra dans la liste des appareils du Tailnet.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.68.109.138 ip-10-0-26-99 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 33868 rx 32292 Nous pouvons aussi vérifier que la route est bien annoncée comme suit:\n1tailscale status --json|jq \u0026#39;.Peer[] | select(.HostName == \u0026#34;ip-10-0-26-99\u0026#34;) .PrimaryRoutes\u0026#39; 2[ 3 \u0026#34;10.0.0.0/16\u0026#34; 4] ⚠️ Pour des raisons de sécurité, pensez à supprimer le fichier variables.tfvars car il contient la clé d'API.\n👏 Et voilà ! Nous sommes maintenant en mesure d'accéder au réseau sur AWS, à condition d'avoir également configuré les règles de filtrage, comme les ACL et les security groups. Nous pouvons par exemple accéder à une base de données depuis le poste de travail\n1psql -h demo-tailscale.cymnaynfchjt.eu-west-3.rds.amazonaws.com -U postgres 2Password for user postgres: 3psql (15.4, server 15.3) 4SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off) 5Type \u0026#34;help\u0026#34; for help. 6 7postgres=\u0026gt; 💻 Une autre façon de faire du SSH Traditionnellement, nous devons parfois nous connecter à des serveurs en utilisant le protocole SSH. Pour ce faire, il faut générer une clé privée et distribuer la clé publique correspondante sur les serveurs distants.\nContrairement à l'utilisation des clés SSH classiques, étant donné que Tailscale utilise Wireguard pour l'authentification et le chiffrement des connexions il n'est pas nécessaire de ré-authentifier le client. De plus, Tailscale gère également la distribution des clés SSH d'hôtes. Les règles ACL permettent de révoquer l'accès des utilisateurs sans avoir à supprimer les clés SSH. De plus, il est possible d'activer un mode de vérification qui renforce la sécurité en exigeant une ré-authentification périodique. On peut donc affirmer que l'utilisation de Tailscale SSH simplifie l'authentification, la gestion des connexions SSH et améliore le niveau de sécurité.\nLes autorisations pour utiliser SSH sont aussi gérées au niveau des ACL's\n1... 2 ssh = [ 3 { 4 action = \u0026#34;check\u0026#34; 5 src = [\u0026#34;autogroup:member\u0026#34;] 6 dst = [\u0026#34;autogroup:self\u0026#34;] 7 users = [\u0026#34;autogroup:nonroot\u0026#34;] 8 } 9 ] 10... La règle ci-dessus autorise tous les utilisateurs à accéder à leurs propres appareils en utilisant SSH. Lorsqu'ils essaient de se connecter, ils doivent utiliser un compte utilisateur autre que root. Pour chaque tentative de connexion, une authentification supplémentaire est nécessaire (action=check). Cette authentification se fait en visitant un lien web spécifique\n1ssh ubuntu@ip-10-0-26-99 2... 3# Tailscale SSH requires an additional check. 4# To authenticate, visit: https://login.tailscale.com/a/f1f09a548cc6 5... 6ubuntu@ip-10-0-26-99:~$ Pour que cela soit possible il faut aussi démarrer Tailscale avec l'option --ssh\nLes logs d'accès à la machine peuvent être consultés en utilisant journalctl\n1ubuntu@ip-10-0-26-99:~$ journalctl -aeu tailscaled|grep ssh 2Oct 15 15:51:34 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155130-00ede660b8: handling conn: 100.118.83.67:55098-\u0026gt;ubuntu@100.68.109.138:22 3Oct 15 15:51:56 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: handling conn: 100.118.83.67:44560-\u0026gt;ubuntu@100.68.109.138:22 4Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: starting session: sess-20231015T155252-5b2acc170e 5Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): handling new SSH connection from smainklh@gmail.com (100.118.83.67) to ssh-user \u0026#34;ubuntu\u0026#34; 6Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): access granted to smainklh@gmail.com as ssh-user \u0026#34;ubuntu\u0026#34; 7Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): starting pty command: [/usr/sbin/tailscaled be-child ssh --uid=1000 --gid=1000 --groups=1000,4,20,24,25,27,29,30,44,46,115,116 --local-user=ubuntu --remote-user=smainklh@gmail.com --remote-ip=100.118.83.67 --has-tty=true --tty-name=pts/0 --shell --login-cmd=/usr/bin/login --cmd=/bin/bash -- -l] ℹ️ Avec Tailscale SSH il est possible de se connecter en SSH peu importe où est situé l'appareil. En revanche dans un contexte 100% AWS, on préferera probablement utiliser AWS SSM.\nLogs 💾 En sécurité il est primordial de pouvoir conserver les logs pour un usage ultérieur. Il existe différents types de logs:\nLogs d'audit: Ils sont essentiels pour savoir qui a fait quoi. Ils sont accessibles sur la console d'admin et peuvent aussi être envoyés vers un SIEM.\nLogs sur les appareils: Ceux-cis peuvent être consultés en utilisant les commandes appropriées à l'appareil. (journalctl -u tailscaled sur Linux)\nLogs réseau: Utiles pour visualiser quels appareils sont connectés les uns aux autres.\n☸ Qu'en est-il de Kubernetes? Sur Kubernetes il existe plusieurs options pour accéder à un Service:\nProxy: Il s'agit d'un pod supplémentaire qui transfert les appels à un Service existant. Sidecar: Permet de connecter le pod au Tailnet. Donc la connectivité se fait de bout en bout et il est même possible de communiquer dans les 2 sens. (du pod vers les noeuds du Tailnet). Operator: Permet d'exposer les services et l'API Kubernetes (ingress) ainsi que de permettre aux pods d'accéder aux noeuds du Tailnet (egress). La configuration se fait en configurant les ressources existantes: Services et Ingresses Dans notre cas, nous disposons déjà d'un Subnet router qui route tout le réseau du VPC. Il suffit donc que notre service soit exposé sur une IP privée.\nL'API Kubernetes Pour accéder à l'API Kubernetes il est nécessaire d'autoriser le Subnet router. Cela se fait en définissant la règle suivante pour le security group source.\n1module \u0026#34;eks\u0026#34; { 2... 3 cluster_security_group_additional_rules = { 4 ingress_source_security_group_id = { 5 description = \u0026#34;Ingress from the Tailscale security group to the API server\u0026#34; 6 protocol = \u0026#34;tcp\u0026#34; 7 from_port = 443 8 to_port = 443 9 type = \u0026#34;ingress\u0026#34; 10 source_security_group_id = data.aws_security_group.tailscale.id 11 } 12 } 13... 14} Nous allons vérifier que l'API est bien accessible sur une IP privée.\n1CLUSTER_URL=$(TERM=dumb kubectl cluster-info | grep \u0026#34;Kubernetes control plane\u0026#34; | awk \u0026#39;{print $NF}\u0026#39;) 2 3curl -s -o /dev/null -w \u0026#39;%{remote_ip}\\n\u0026#39; ${CLUSTER_URL} 410.228.244.167 5 6kubectl get ns 7NAME STATUS AGE 8cilium-secrets Active 5m46s 9crossplane-system Active 4m1s 10default Active 23m 11flux-system Active 5m29s 12infrastructure Active 4m1s 13... Accéder aux services en privé Un Service Kubernetes exposé est une resource AWS comme une autre 😉. Il faut juste s'assurer que ce service utilise bien une IP privée. Dans mon exemple j'utilise Gateway API pour configurer la répartition de charge du Clouder et je vous invite à lire mon précédent article sur le sujet.\nIl suffirait donc de créer un NLB interne en s'assurant que le Service ait bien l'annotation service.beta.kubernetes.io/aws-load-balancer-scheme ayant pour valeur internal. Dans le cas de Gateway API, cela se fait via la clusterPolicy Kyverno.\n1 metadata: 2 annotations: 3 external-dns.alpha.kubernetes.io/hostname: gitops-${cluster_name}.priv.${domain_name},grafana-${cluster_name}.priv.${domain_name} 4 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internal\u0026#34; 5 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 6 spec: 7 loadBalancerClass: service.k8s.aws/nlb Il y a cependant un prérequis supplémentaire car nous ne pouvons pas utiliser Let's Encrypt pour les certificats internes. J'ai donc généré une PKI interne qui génère des certificates auto-signés avec Cert-manager.\nIci je ne détaillerai pas le déploiement du cluster EKS, ni la configuration de Flux. Lorsque le cluster est créé et que toutes les ressources Kubernetes ont été réconcilié, nous avons un service qui est exposé via un LoadBalancer interne AWS.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2dig +short ${NLB_DOMAIN} 310.0.33.5 410.0.26.228 510.0.9.183 Une entrée DNS est également créée automatiquement pour les services exposés et nous pouvons donc accéder en privé grâce à Tailscale.\n1dig +short gitops-mycluster-0.priv.cloud.ogenki.io 210.0.9.183 310.0.26.228 410.0.33.5 💭 Dernières remarques Il y a quelques temps, dans le cadre professionnel, j'ai mis en place Cloudflare Zero Trust. Je découvre ici que Tailscale présente de nombreuses similitudes avec cette solution. La décision entre les deux est loin d'être triviale et dépend grandement du contexte. Pour ma part, j'ai été particulièrement convaincu par la simplicité de mise en œuvre de Tailscale, répondant parfaitement à mon besoin d'accéder au réseau du Clouder. Bien entendu il existe d'autres solutions comme Teleport, qui offre une approche différente pour accéder à des ressources internes.\nCela dit, focalisons-nous sur Tailscale.\nUne partie du code de Tailscale est open source, notamment le client qui est sous license BSD 3-Clause. La partie propriétaire concerne éssentiellement la plateforme de coordination. À noter qu'il existe une alternative open source nommée Headscale. Celle-ci est une initiative distincte qui n'a aucun lien avec la société Tailscale.\nPour un usage personnel, Tailscale est vraiment généreux, offrant un accès gratuit pour jusqu'à 100 appareils et 3 utilisateurs. Ceci-dit Tailscale est une option sérieuse à considérer en entreprise et il est important, selon moi, d'encourager ce type d'entreprises qui ont une politique open source claire et un produit de qualité.\n","link":"https://blog.ogenki.io/fr/post/tailscale/","section":"post","tags":["security","network"],"title":"Sécuriser le Cloud avec `Tailscale` : Mise en œuvre d'un VPN simplifiée"},{"body":"","link":"https://blog.ogenki.io/fr/tags/security/","section":"tags","tags":null,"title":"security"},{"body":"","link":"https://blog.ogenki.io/fr/tags/","section":"tags","tags":null,"title":"Tags"},{"body":"Lorsque l'on déploie une application sur Kubernetes, l'étape suivante consiste généralement à l'exposer aux utilisateurs. On utilise habituellement des \u0026quot;Ingress controllers\u0026quot;, comme Nginx, Haproxy, Traefik ou encore ceux des différents Clouders afin de diriger le trafic entrant vers l'application, gérer l'équilibrage de charge, la terminaison TLS et j'en passe.\nIl faut alors choisir parmi la pléthore d'options disponibles 🤯 la solution qui sera en charge de tous ces aspects et Cilium est, depuis relativement récemment, l'une d'entre elles.\nCilium est une solution Open-Source de connectivité réseau et de sécurité basée sur eBPF dont l'adoption est grandissante. Il s'agit probablement du plugin réseau qui fournit le plus de fonctionnalités. Nous n'allons pas toutes les parcourir mais l'une d'entre elles consiste à gérer le trafic entrant en utilisant le standard Gateway API (GAPI).\n🎯 Notre objectif Comprendre ce qu'est exactement Gateway API et en quoi il s'agit d'une évolution par rapport à l'API Ingress. Démonstrations de cas concrets à la sauce GitOps. Les limitations actuelles et les évolutions à venir. Tip Toutes les étapes réalisées dans cet article proviennent de ce dépôt git\nJe vous invite à le parcourir car, il va bien au delà du contexte de cet article:\nInstallation d'un cluster EKS avec Cilium configuré en mode sans kube-proxy et un Daemonset dédié à Envoy Proposition de structure de configuration Flux avec une gestion des dépendances et une factorisation que je trouve efficaces. Crossplane et composition IRSA qui simplifie la gestion des permissions IAM pour les composants plateforme Gestion des noms de domaine ainsi que des certificats automatisés avec External-DNS et Let's Encrypt L'idée étant d'avoir l'ensemble configuré au bout de quelques minutes, en une seule ligne de commande 🤩.\n☸ Introduction à Gateway API Comme évoqué précédemment, il y a de nombreuses options qui font office d' Ingress controller et chacune a ses propres spécificités et des fonctionnalités particulières, rendant leur utilisation parfois complexe. Par ailleurs, l'API Ingress, utilisée historiquement dans Kubernetes possède très peu d'options. Certaines solutions ont d'ailleurs créé des CRDs (Ressources personalisées) quand d'autres font usage des annotations pour lever ces limites.\nC'est dans ce contexte que Gateway API fait son apparition. Il s'agit d'un standard qui permet de définir des fonctionnalités réseau avancées sans nécessiter d'extensions spécifiques au contrôleur sous-jacent. De plus étant donné que tous les contrôleurs utilisent la même API , il est possible de passer d'une solution à une autre sans changer de configuration (les ressources qui gèrent le trafic entrant restent les mêmes).\nParmi les concepts que nous allons explorer la GAPI introduit un schéma de répartition des responsabilités. Elle définit des roles explicites avec des permissions bien distinctes. (Plus d'informations sur le modèle de sécurité GAPI ici).\nEnfin il est important de noter que ce projet est dirigé par le groupe de travail sig-network-kubernetes et un canal slack vous permettra de les solliciter si nécessaire.\nVoyons comment cela s'utilise concrètement 🚀!\n\u0026#x2611;\u0026#xfe0f; Prérequis Pour le reste de cet article nous considérons qu'un cluster EKS a été déployé. Si vous n'utilisez pas la méthode proposée dans le repo de démo servant de socle à cet article, il y a certains points à valider pour que GAPI puisse être utilisé.\nℹ️ La méthode d'installation decrite ici se base sur Helm, l'ensemble des values peuvent être consultées ici.\nInstaller les CRDs (resources personnalisés) disponibles dans le repository Gateway API Note Si Cilium est configuré avec le support GAPI (voir ci-dessous) et que les CRDs sont absentes, il ne démarrera pas.\nDans le repo de demo les CRDs GAPI sont installées une première fois lors de la création du cluster afin que Cilium puisse démarrer puis elles sont ensuite gérées par Flux.\nRemplacer kube-proxy par les fonctionnalités de transfert réseau apportées par Cilium et eBPF.\n1kubeProxyReplacement: true Activer le support de Gateway API 1gatewayAPI: 2 enabled: true Vérifier L'installation Il faut pour cela installer le client en ligne de commande cilium. J'utilise personnellement asdf pour cela:\n1asdf plugin-add cilium-cli 2asdf install cilium-cli 0.15.7 3asdf global cilium 0.15.7 La commande suivante permet de s'assurer que tous les composants sont démarrés et opérationnels\n1cilium status --wait 2 /¯¯\\ 3/¯¯\\__/¯¯\\ Cilium: OK 4\\__/¯¯\\__/ Operator: OK 5/¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6\\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled 8 9Deployment cilium-operator Desired: 2, Ready: 2/2, Available: 2/2 10DaemonSet cilium Desired: 2, Ready: 2/2, Available: 2/2 11DaemonSet cilium-envoy Desired: 2, Ready: 2/2, Available: 2/2 12Containers: cilium Running: 2 13 cilium-operator Running: 2 14 cilium-envoy Running: 2 15Cluster Pods: 33/33 managed by Cilium 16Helm chart version: 1.14.2 17Image versions cilium quay.io/cilium/cilium:v1.14.2@sha256:6263f3a3d5d63b267b538298dbeb5ae87da3efacf09a2c620446c873ba807d35: 2 18 cilium-operator quay.io/cilium/operator-aws:v1.14.2@sha256:8d514a9eaa06b7a704d1ccead8c7e663334975e6584a815efe2b8c15244493f1: 2 19 cilium-envoy quay.io/cilium/cilium-envoy:v1.25.9-e198a2824d309024cb91fb6a984445e73033291d@sha256:52541e1726041b050c5d475b3c527ca4b8da487a0bbb0309f72247e8127af0ec: 2 Enfin le support de GAPI peut être vérifié comme suit\n1cilium config view | grep -w \u0026#34;enable-gateway-api\u0026#34; 2enable-gateway-api true 3enable-gateway-api-secrets-sync true Il est aussi possible de lancer des tests de connectivité pour s'assurer qu'il n'y a pas de problèmes avec la configuration réseau du cluster:\n1cilium connectivity test ⚠️ Cette commande (connectivity test) provoque actuellement des erreurs lors de l'activation d'Envoy en tant que DaemonSet. (Issue Github).\nInfo as DaemonSet\nPar défaut l'agent cilium intégre Envoy et lui délegue les opérations réseau de niveau 7. Depuis la version v1.14, il est possible de déployer Envoy séparément ce qui apporte certains avantages:\nSi l'on modifie/redémarre un composant (que ce soit Cilium ou Envoy), cela n'affecte pas l'autre. Mieux attribuer les ressources à chacun des composants afin d'optimiser les perfs. Limite la surface d'attaque en cas de compromission d'un des pods. Les logs Envoy et de l'agent Cilium ne sont pas mélangés Il est possible d'utiliser la commande suivante pour vérifier que cette fonctionnalité est bien active:\n1cilium status 2 /¯¯\\ 3 /¯¯\\__/¯¯\\ Cilium: OK 4 \\__/¯¯\\__/ Operator: OK 5 /¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6 \\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled Plus d'information.\n🚪 La porte d'entrée: GatewayClass et Gateway Une fois les conditions nécessaires remplies, nous avons accès à plusieurs éléments. Nous pouvons notamment utiliser les ressources de la Gateway API grâce aux CRDs. D'ailleurs, dès l'installation de Cilium, une GatewayClass est directement disponible.\n1kubectl get gatewayclasses.gateway.networking.k8s.io 2NAME CONTROLLER ACCEPTED AGE 3cilium io.cilium/gateway-controller True 7m59s Sur un cluster il est possible de configurer plusieurs GatewayClass et donc d'avoir la possibilité de faire usage de différentes implémentations. Nous pouvons par exemple utiliser Linkerd en référencant la GatewayClass dans la configuration de la Gateway.\nLa Gateway est la ressource qui permet déclencher la création de composants de répartition de charge chez le Clouder.\nVoici un exemple simple: apps/base/echo/gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo-gateway 5 namespace: echo 6spec: 7 gatewayClassName: cilium 8 listeners: 9 - protocol: HTTP 10 port: 80 11 name: echo-1-echo-server 12 allowedRoutes: 13 namespaces: 14 from: Same Sur AWS (EKS), quand on configure une Gateway, Cilium crée un Service de type LoadBalancer. Ce service est alors interprété par un autre contrôleur, l'(AWS Load Balancer Controler), qui produit un NLB.\n1kubectl get svc -n echo cilium-gateway-echo 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3cilium-gateway-echo LoadBalancer 172.20.19.82 k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com 80:30395/TCP 2m58s Il est intéressant de noter que l'adresse en question est aussi associée à la resource Gateway.\n1kubectl get gateway -n echo echo 2NAME CLASS ADDRESS PROGRAMMED AGE 3echo cilium k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com True 16m \u0026#x21aa;\u0026#xfe0f; Les règles de routage: HTTPRoute Un routage simple Pour résumer le schéma ci-dessus en quelques mots: Une ressource HTTPRoute permet de configurer le routage vers le service en référencant la gateway et en définissant le les paramètres de routage souhaités.\nNote workaround\nÀ ce jour, il n'est pas possible de configurer les annotations des services générés par les Gateways (Issue Github). Une solution de contournement a été proposé afin de modifier le service généré par la Gateway dès lors qu'il est créé.\nKyverno est un outil qui permet de garantir la conformité des configurations par rapport aux bonnes pratiques et aux exigences de sécurité. Nous utilisons ici uniquement sa capacité à décrire une règle de mutation facilement.\nsecurity/mycluster-0/echo-gw-clusterpolicy.yaml\n1spec: 2 rules: 3 - name: mutate-svc-annotations 4 match: 5 any: 6 - resources: 7 kinds: 8 - Service 9 namespaces: 10 - echo 11 name: cilium-gateway-echo 12 mutate: 13 patchStrategicMerge: 14 metadata: 15 annotations: 16 external-dns.alpha.kubernetes.io/hostname: echo.${domain_name} 17 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internet-facing\u0026#34; 18 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 19 spec: 20 loadBalancerClass: service.k8s.aws/nlb Le service cilium-gateway-echo se verra donc ajouter les annotations du contrôleur AWS ainsi qu'une annotation permettant de configurer une entrée DNS automatiquement.\napps/base/echo/httproute.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 rules: 11 - matches: 12 - path: 13 type: PathPrefix 14 value: / 15 backendRefs: 16 - name: echo-1-echo-server 17 port: 80 L'exemple utilisé ci-dessus est simpliste: toutes les requêtes sont transférées au service echo-1-echo-server.\nparentRefs permet d'indiquer la Gateway à utiliser puis les règles de routage sont définies dans rules.\nLes règles de routages pourraient aussi être basées sur le path.\n1... 2spec: 3 hostnames: 4 - foo.bar.com 5 rules: 6 - matches: 7 - path: 8 type: PathPrefix 9 value: /login Ou une entête HTTP\n1... 2spec: 3 rules: 4 - matches: 5 headers: 6 - name: \u0026#34;version\u0026#34; 7 value: \u0026#34;2\u0026#34; 8... Vérifions que le service est joignable:\n1curl -s http://echo.cloud.ogenki.io | jq -rc \u0026#39;.environment.HOSTNAME\u0026#39; 2echo-1-echo-server-fd88497d-w6sgn Comme vous pouvez le voir le service est exposé en HTTP sans certificat. Essayons de corriger cela 😉\nExposer un service en utilisant un certificat TLS Il existe plusieurs méthodes pour configurer du TLS avec GAPI. Ici nous allons utiliser le cas le plus commun: protocole HTTPS et terminaison TLS sur la Gateway.\nSupposons que nous souhaitons configurer le nom de domaine echo.cloud.ogenki.io utilisé précédemment. La configuration se fait principalement au niveau de la Gateway\napps/base/echo/tls-gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo 5 namespace: echo 6 annotations: 7 cert-manager.io/cluster-issuer: letsencrypt-prod 8spec: 9 gatewayClassName: cilium 10 listeners: 11 - name: http 12 hostname: \u0026#34;echo.${domain_name}\u0026#34; 13 port: 443 14 protocol: HTTPS 15 allowedRoutes: 16 namespaces: 17 from: Same 18 tls: 19 mode: Terminate 20 certificateRefs: 21 - name: echo-tls Le point essentiel ici est la référence à un secret contenant le certificat echo-tls. Ce certificat peut être créé manuellement mais j'ai décidé pour cet article d'automatiser cela avec Let's Encrypt et cert-manager.\nInfo cert-manager\nAvec cert-manager il est très simple d'automatiser la création et la mise à jour des certificats exposés par la Gateway. Pour cela, il faut permettre au contrôlleur d'accéder à route53 afin de résoudre un challenge DNS01 (Mécanisme qui permet de s'assurer que les clients peuvent seulement demander des certificats pour des domaines qu'ils possèdent).\nUne ressource ClusterIssuer décrit la configuration nécessaire pour générer des certificats grâce à cert-manager.\nEnsuite il suffit d'ajouter une annotation cert-manager.io/cluster-issuer et indiquer le secret Kubernetes où sera stocké le certificat.\nℹ️ Dans le repo de demo les permissions sont attribuées en utilisant Crossplane qui se charge de configurer cela au niveau du Cloud AWS.\nPlus d'informations\nPour que le routage se fasse correctement il faut aussi bien entendu référencer la bonne Gateway mais aussi indiquer le nom de domaine dans la ressource HTTPRoute.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 hostnames: 11 - \u0026#34;echo.${domain_name}\u0026#34; 12... Il faut patienter quelques minutes le temps que le certificat soit créé.\n1kubectl get cert -n echo 2NAME READY SECRET AGE 3echo-tls True echo-tls 43m Nous pouvons enfin vérifier que le certificat est bien issue de Let's Encrypt comme suit:\n1curl https://echo.cloud.ogenki.io -v 2\u0026gt;\u0026amp;1 | grep -A 6 \u0026#39;Server certificate\u0026#39; 2* Server certificate: 3* subject: CN=echo.cloud.ogenki.io 4* start date: Sep 15 14:43:00 2023 GMT 5* expire date: Dec 14 14:42:59 2023 GMT 6* subjectAltName: host \u0026#34;echo.cloud.ogenki.io\u0026#34; matched cert\u0026#39;s \u0026#34;echo.cloud.ogenki.io\u0026#34; 7* issuer: C=US; O=Let\u0026#39;s Encrypt; CN=R3 8* SSL certificate verify ok. Info GAPI permet aussi de configurer le TLS de bout en bout, jusqu'au conteneur. Cela se fait en configurant la Gateway en Passthrough et en utilisant une ressource TLSRoute. Il faut aussi que le certificat soit porté par le pod qui fait terminaison TLS.\nUne Gateway partagée par plusieurs namespaces Avec GAPI il est possible de router le trafic à travers les Namespaces. Cela est rendu possible grâce à des ressources distinctes pour chaque fonction: Une Gateway qui permet de configurer l'infrastructure et notamment de provisionner une adresse IP, et les *Routes. Ces routes peuvent référencer une Gateway située dans un autre namespace. Il est ainsi possible pour différent(e)s équipes/projets de partager les mêmes éléments d'infrastructure.\nIl est cependant requis de spécifier quelle route est autorisée à référencer la Gateway. Ici nous supposons que nous avons une Gateway dédiée aux outils internes qui s'appelle platform. En utilisant le paramètre allowedRoutes, nous spécifions explicitement quelles sont les namespaces autorisés à référencer cette Gateway.\ninfrastructure/base/gapi/platform-gateway.yaml\n1... 2 allowedRoutes: 3 namespaces: 4 from: Selector 5 selector: 6 matchExpressions: 7 - key: kubernetes.io/metadata.name 8 operator: In 9 values: 10 - observability 11 - flux-system 12 tls: 13 mode: Terminate 14 certificateRefs: 15 - name: platform-tls Les HTTPRoutes situées dans les namespaces observability et flux-system réferéncent cette même Gateway.\n1... 2spec: 3 parentRefs: 4 - name: platform 5 namespace: infrastructure Et utilisent le même répartiteur de charge du Clouder\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2 3dig +short ${NLB_DOMAIN} 413.36.89.108 5 6dig +short grafana-mycluster-0.cloud.ogenki.io 713.36.89.108 8 9dig +short gitops-mycluster-0.cloud.ogenki.io 1013.36.89.108 Note 🔒 Ces outils internes ne devraient pas être exposés sur Internet mais vous comprendrez qu'il s'agit là d'une démo 🙏. On pourrait par exemple, utiliser et une Gateway interne (IP privée) en jouant sur les annotations et un moyen de connexion privé (VPN, tunnels ...)\nTraffic splitting Il est souvent utile de tester une application sur une portion du trafic lorsqu'une nouvelle version est disponible (A/B testing ou Canary deployment). GAPI permet cela de façon très simple en utilisant des poids.\nVoici un exemple permettant de tester sur 5% du trafic vers le service echo-2-echo-server\napps/base/echo/httproute-split.yaml\n1... 2 hostnames: 3 - \u0026#34;split-echo.${domain_name}\u0026#34; 4 rules: 5 - matches: 6 - path: 7 type: PathPrefix 8 value: / 9 backendRefs: 10 - name: echo-1-echo-server 11 port: 80 12 weight: 95 13 - name: echo-2-echo-server 14 port: 80 15 weight: 5 Vérifions que la répartition se fait bien comme attendu:\nscripts/check-split.sh\n1./scripts/check-split.sh https://split-echo.cloud.ogenki.io 2Number of requests for echo-1: 95 3Number of requests for echo-2: 5 Manipulation des entêtes HTTP (Headers) Il est aussi possible de jouer avec les entêtes HTTP (Headers): en ajouter, modifier ou supprimer. Ces modifications peuvent se faire sur les Headers de requête ou de réponse par le biais de filtres ajoutés à la ressource HTTPRoute.\nNous allons par exemple ajouter un Header à la requête\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7... 8 rules: 9 - matches: 10 - path: 11 type: PathPrefix 12 value: /req-header-add 13 filters: 14 - type: RequestHeaderModifier 15 requestHeaderModifier: 16 add: 17 - name: foo 18 value: bar 19 backendRefs: 20 - name: echo-1-echo-server 21 port: 80 22... La commande suivante permet de vérifier que le header est bien présent.\n1curl -s https://echo.cloud.ogenki.io/req-header-add -sk | jq \u0026#39;.request.headers\u0026#39; 2{ 3 \u0026#34;host\u0026#34;: \u0026#34;echo.cloud.ogenki.io\u0026#34;, 4 \u0026#34;user-agent\u0026#34;: \u0026#34;curl/8.2.1\u0026#34;, 5 \u0026#34;accept\u0026#34;: \u0026#34;*/*\u0026#34;, 6 \u0026#34;x-forwarded-for\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 7 \u0026#34;x-forwarded-proto\u0026#34;: \u0026#34;https\u0026#34;, 8 \u0026#34;x-envoy-external-address\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 9 \u0026#34;x-request-id\u0026#34;: \u0026#34;320ba4d2-3bd6-4c2f-8a97-74296a9f3f26\u0026#34;, 10 \u0026#34;foo\u0026#34;: \u0026#34;bar\u0026#34; 11} 🪪 Les roles et permissions GAPI offre un modèle de partage des permissions claire entre l'infrastructure de routage du trafic (gérée par les administrateurs de cluster) et les applications (gérées par les développeurs).\nLe fait de disposer de plusieurs ressources nous permet d'utiliser les ressources RBAC dans Kubernetes pour attribuer les droits de façon déclarative. J'ai ajouté quelques exemples qui n'ont aucun effet dans mon cluster de démo mais qui peuvent vous permettre de vous faire une idée.\nLa configuration suivante permet aux membres du groupe developers de gérer les HTTPRoutes dans le namespace echo. En revanche ils ne possédent que des droits en lecture sur les Gateways.\n1--- 2apiVersion: rbac.authorization.k8s.io/v1 3kind: Role 4metadata: 5 namespace: echo 6 name: gapi-developer 7rules: 8 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 9 resources: [\u0026#34;httproutes\u0026#34;] 10 verbs: [\u0026#34;*\u0026#34;] 11 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 12 resources: [\u0026#34;gateways\u0026#34;] 13 verbs: [\u0026#34;get\u0026#34;, \u0026#34;list\u0026#34;] 14--- 15apiVersion: rbac.authorization.k8s.io/v1 16kind: RoleBinding 17metadata: 18 name: gapi-developer 19 namespace: echo 20subjects: 21 - kind: Group 22 name: \u0026#34;developers\u0026#34; 23 apiGroup: rbac.authorization.k8s.io 24roleRef: 25 kind: Role 26 name: gapi-developer 27 apiGroup: rbac.authorization.k8s.io 🤔 Un périmètre pas évident Il ne faut pas confondre GAPI avec ce que l'on nomme couramment une API Gateway. Une section de la FAQ a d'ailleurs été créé pour éclaircir ce point. Bien que GAPI offre des fonctionnalités typiquement présentes dans une API Gateway, il s'agit avant tout d'une implémentation spécifique pour Kubernetes. Cependant, ce choix de dénomination peut prêter à confusion.\nIl est essentiel de mentionner que cet article se concentre uniquement sur le trafic entrant, appelé north-south, traditionnellement géré par les Ingress Controllers. Ce trafic représente le périmètre initial de GAPI. Une initiative récente nommée GAMMA vise à également gérer le routage east-west, ce qui permettra de standardiser certaines fonctionnalités des solutions de Service Mesh à l'avenir. (voir cet article pour plus d'informations).\n💭 Dernières remarques Pour être honnête, j'ai entendu parlé de Gateway API depuis un petit moment. J'ai lu quelques articles mais jusqu'ici je n'avais pas pris le temps d'approfondir le sujet. Je me disais \u0026quot;Pourquoi? J'arrive à faire ce que je veux avec mon Ingress Controller ? et puis il faut apprendre à utiliser de nouvelles ressources\u0026quot;.\nGAPI gagne en maturité et nous sommes proche d'une version GA. De nombreux projets l'ont déjà adopté, Istio et Linkerd par exemple sont totalement compatibles avec la version 0.8.0 et cette façon de gérer le trafic au sein de Kubernetes deviendra rapidement la norme.\nToujours est-il que j'ai beaucoup aimé la déclaration des différentes configurations que je trouve très intuitive et explicite ❤️. D'autre part le modèle de sécurité permet de donner le pouvoir aux developpeurs sans sacrifier la sécurité. Enfin la gestion de l'infrastructure se fait de façon transparente, nous pouvons rapidement passer d'une implémentation (contrôleur sous-jacent) à une autre sans toucher aux *Routes.\nAlors suis-je prêt à changer mon Ingress Controller pour Cilium aujourd'hui? La réponse courte est Non mais bientôt!.\nTout d'abord j'aimerais mettre en évidence sur l'étendue des possiblités offertes par Cilium: De nombreuses personnes se sentent noyées sous les nombreux outils qui gravitent autour de Kubernetes. Cilium permettrait de remplir les fonctionnalités de nombre d'entre eux (metrics, tracing, service-mesh, sécurité et ... Ingress Controller avec GAPI).\nCependant, bien que nous puissions faire du routage HTTP de base, il y à certains points d'attention:\nLe support de TCP et UDP Le support de GRPC Devoir passer par une règle de mutation pour pouvoir configurer les composants cloud. (Issue Github) De nombreuses fonctionnalités explorées sont toujours au stade expérimental. On peut citer les fonctions étendues qui supportés depuis quelques jours: J'ai par exemple tenté de configurer une redirection HTTP\u0026gt;HTTPS simple mais je suis tombé sur ce problème. Je m'attends donc à ce qu'il y ait des changements dans l'API très prochainement. Je n'ai pas abordé toutes les fonctionnalités de l'implémentation Cilium de GAPI (Honnêtement, cet article est déjà bien fourni 😜). Néanmoins, je suis vraiment convaincu de son potentiel. J'ai bon espoir qu'on pourra bientôt envisager son utilisation en production. Si vous n'avez pas encore envisagé cette transition, c'est le moment de s'y pencher 😉 ! Toutefois, compte tenu des aspects évoqués précédemment, je conseillerais de patienter un peu.\n🔖 References https://gateway-api.sigs.k8s.io/ https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#gs-gateway-api https://isovalent.com/blog/post/cilium-gateway-api/ https://isovalent.com/blog/post/tutorial-getting-started-with-the-cilium-gateway-api/ Les labs d'Isovalent permettent de rapidement de tester GAPI et vous pourrez ajouter des badges à votre collection 😄 ","link":"https://blog.ogenki.io/fr/post/cilium-gateway-api/","section":"post","tags":["kubernetes","infrastructure","network"],"title":"`Gateway API`: Remplacer mon Ingress Controller avec `Cilium`?"},{"body":"","link":"https://blog.ogenki.io/fr/tags/infrastructure/","section":"tags","tags":null,"title":"infrastructure"},{"body":"","link":"https://blog.ogenki.io/fr/tags/kubernetes/","section":"tags","tags":null,"title":"kubernetes"},{"body":"Terraform est probablement l'outil \u0026quot;Infrastructure As Code\u0026quot; le plus utilisé pour construire, modifier et versionner les changements d'infrastructure Cloud. Il s'agit d'un projet Open Source développé par Hashicorp et qui utilise le langage HCL pour déclarer l'état souhaité de ressources Cloud. L'état des ressources créées est stocké dans un fichier d'état (terraform state).\nOn peut considérer que Terraform est un outil \u0026quot;semi-déclaratif\u0026quot; car il n'y a pas de fonctionnalité de réconciliation automatique intégrée. Il existe différentes approches pour répondre à cette problématique, mais en règle générale, une modification sera appliquée en utilisant terraform apply. Le code est bien décrit dans des fichiers de configuration HCL (déclaratif) mais l'exécution est faite de manière impérative. De ce fait, il peut y avoir de la dérive entre l'état déclaré et le réel (par exemple, un collègue qui serait passé sur la console pour changer un paramètre 😉).\n❓❓ Alors, comment m'assurer que ce qui est commit dans mon repo git est vraiment appliqué. Comment être alerté s'il y a un changement par rapport à l'état désiré et comment appliquer automatiquement ce qui est dans mon code (GitOps) ?\nC'est la promesse de tf-controller, un operateur Kubernetes Open Source de Weaveworks, étroitement lié à Flux (un moteur GitOps de la même société). Flux est l'une des solutions que je plébiscite, et je vous invite donc à lire un précédent article.\nInfo L'ensemble des étapes décrites ci-dessous sont faites avec ce repo Git\n🎯 Notre objectif En suivant les étapes de cet article nous visons les objectifs suivant:\nDéployer un cluster Kubernetes qui servira de \u0026quot;Control plane\u0026quot;. Pour résumer il hébergera le controlleur Terraform qui nous permettra de déclarer tous les éléments d'infrastructure souhaités. Utiliser Flux comme moteur GitOps pour toutes les ressources Kubernetes. Concernant le controleur Terraform, nous allons voir:\nQuelle est le moyen de définir des dépendances entre modules Création de plusieurs ressources AWS: Zone route53, Certificat ACM, réseau, cluster EKS. Les différentes options de reconciliation (automatique, nécessitant une confirmation) Comment sauvegarder et restaurer un fichier d'état (tfstate) 🛠️ Installer le controleur Terraform ☸ Le cluster \u0026quot;Control Plane\u0026quot; Afin de pouvoir utiliser le controleur Kubernetes tf-controller, il nous faut d'abord un cluster Kubernetes 😆. Nous allons donc créer un cluster control plane en utilisant la ligne de commande terraform et les bonnes pratiques EKS.\nWarning Il est primordial que ce cluster soit résiliant, sécurisé et supervisé car il sera responsable de la gestion de l'ensemble des ressources AWS créées par la suite.\nSans entrer dans le détail, le cluster \u0026quot;control plane\u0026quot; a été créé un utilisant ce code. Celà-dit, il est important de noter que toutes les opérations de déploiement d'application se font en utilisant Flux.\nInfo En suivant les instructions du README, un cluster EKS sera créé mais pas uniquement! Il faut en effet donner les permissions au controlleur Terraform pour appliquer les changements d'infrastructure. De plus, Flux doit être installé et configuré afin d'appliquer la configuration définie ici.\nAu final on se retrouve donc avec plusieurs éléments installés et configurés:\nles addons quasi indispensables que sont aws-loadbalancer-controller et external-dns les roles IRSA pour ces mêmes composants sont installés en utilisant tf-controller La stack de supervision Prometheus / Grafana. external-secrets pour pouvoir récupérer des éléments sensibles depuis AWS secretsmanager. Afin de démontrer tout cela au bout de quelques minutes l'interface web pour Flux est accessible via l'URL gitops-\u0026lt;cluster_name\u0026gt;.\u0026lt;domain_name\u0026gt; Vérifier toute de même que le cluster est accessible et que Flux fonctionne correctement\n1aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0 2Updated context controlplane-0 in /home/smana/.kube/config 1flux check 2... 3✔ all checks passed 4 5flux get kustomizations 6NAME REVISION SUSPENDED READY MESSAGE 7flux-config main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 8flux-system main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 9infrastructure main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 10security main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 11tf-controller main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 12... 📦 Le chart Helm et Flux Maintenant que notre cluster \u0026quot;controlplane\u0026quot; est opérationnel, l'ajout le contrôleur Terraform consiste à utiliser le chart Helm.\nIl faut tout d'abord déclarer la source:\nsource.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: tf-controller 5spec: 6 interval: 30m 7 url: https://weaveworks.github.io/tf-controller Et définir la HelmRelease:\nrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: tf-controller 5spec: 6 releaseName: tf-controller 7 chart: 8 spec: 9 chart: tf-controller 10 sourceRef: 11 kind: HelmRepository 12 name: tf-controller 13 namespace: flux-system 14 version: \u0026#34;0.12.0\u0026#34; 15 interval: 10m0s 16 install: 17 remediation: 18 retries: 3 19 values: 20 resources: 21 limits: 22 memory: 1Gi 23 requests: 24 cpu: 200m 25 memory: 500Mi 26 runner: 27 serviceAccount: 28 annotations: 29 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/tfcontroller_${cluster_name}\u0026#34; Lorsque ce changement est écrit dans le repo Git, la HelmRelease sera déployée et le contrôlleur tf-controller démarera\n1kubectl get hr -n flux-system 2NAME AGE READY STATUS 3tf-controller 67m True Release reconciliation succeeded 4 5kubectl get po -n flux-system -l app.kubernetes.io/instance=tf-controller 6NAME READY STATUS RESTARTS AGE 7tf-controller-7ffdc69b54-c2brg 1/1 Running 0 2m6s Dans le repo de demo il y a déjà un certain nombre de ressources AWS déclarées. Par conséquent, au bout de quelques minutes, le cluster se charge de la création de celles-cis: Info Bien que la majorité des tâches puisse être réalisée de manière déclarative ou via les utilitaires de ligne de commande tels que kubectl et flux, un autre outil existe qui offre la possibilité d'interagir avec les ressources terraform : tfctl\n🚀 Appliquer un changement Parmis les bonnes pratiques avec Terraform, il y a l'usage de modules. Un module est un ensemble de ressources Terraform liées logigement afin d'obtenir une seule unité réutilisable. Cela permet d'abstraire la complexité, de prendre des entrées, effectuer des actions spécifiques et produire des sorties.\nIl est possible de créer ses propres modules et de les mettre à disposition dans des Sources ou d'utiliser les nombreux modules partagés et maintenus par les communautés. Il suffit alors d'indiquer quelques variables afin de l'adapter au contexte.\nAvec tf-controller, la première étape consiste donc à indiquer la Source du module. Ici nous allons configurer le socle réseau sur AWS (vpc, subnets...) avec le module terraform-aws-vpc.\nsources/terraform-aws-vpc.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1 2kind: GitRepository 3metadata: 4 name: terraform-aws-vpc 5 namespace: flux-system 6spec: 7 interval: 30s 8 ref: 9 tag: v5.0.0 10 url: https://github.com/terraform-aws-modules/terraform-aws-vpc Nous pouvons ensuite créer la ressource Terraform qui en fait usage:\nvpc/dev.yaml\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6 interval: 8m 7 path: . 8 destroyResourcesOnDeletion: true # You wouldn\u0026#39;t do that on a prod env ;) 9 storeReadablePlan: human 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-vpc 13 namespace: flux-system 14 vars: 15 - name: name 16 value: vpc-dev 17 - name: cidr 18 value: \u0026#34;10.42.0.0/16\u0026#34; 19 - name: azs 20 value: 21 - \u0026#34;eu-west-3a\u0026#34; 22 - \u0026#34;eu-west-3b\u0026#34; 23 - \u0026#34;eu-west-3c\u0026#34; 24 - name: private_subnets 25 value: 26 - \u0026#34;10.42.0.0/19\u0026#34; 27 - \u0026#34;10.42.32.0/19\u0026#34; 28 - \u0026#34;10.42.64.0/19\u0026#34; 29 - name: public_subnets 30 value: 31 - \u0026#34;10.42.96.0/24\u0026#34; 32 - \u0026#34;10.42.97.0/24\u0026#34; 33 - \u0026#34;10.42.98.0/24\u0026#34; 34 - name: enable_nat_gateway 35 value: true 36 - name: single_nat_gateway 37 value: true 38 - name: private_subnet_tags 39 value: 40 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 41 \u0026#34;karpenter.sh/discovery\u0026#34;: dev 42 - name: public_subnet_tags 43 value: 44 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 45 writeOutputsToSecret: 46 name: vpc-dev Si l'on devait résumer grossièrement: le code terraform provenant de la source terraform-aws-vpc est utilisé avec les variables vars.\nIl y a ensuite plusieurs paramètres qui influent sur le fonctionnement de tf-controller. Les principaux paramètres qui permettent de contrôler la façon dont sont appliquées les modifications sont .spec.approvePlan et .spec.autoApprove\n🚨 Détection de la dérive Définir spec.approvePlan avec une valeur à disable permet uniquement de notifier que l'état actuel des ressources a dérivé par rapport au code Terraform. Cela permet notamment de choisir le moment et la manière dont l'application des changements sera effectuée.\nNote De mon point de vue il manque une section sur les notifications: La dérive, les plans en attentes, les problèmese de réconcilation. J'essaye d'identifier les méthodes possibles (de préférence avec Prometheus) et de mettre à jour cet article dès que possible.\n🔧 Application manuelle L'exemple donné précédemment (vpc-dev) ne contient pas le paramètre .spec.approvePlan et hérite donc de la valeur par défaut qui est false. Par conséquent, l'application concrète des modifications (apply), n'est pas faite automatiquement.\nUn plan est exécuté et sera en attente d'une validation:\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system vpc-dev Unknown Plan generated: set approvePlan: \u0026#34;plan-v5.0.0-26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. true 2 minutes Je conseille d'ailleurs de configurer le paramètre storeReadablePlan à human. Cela permet de visualiser simplement les modifications en attente en utilisant tfctl:\n1tfctl show plan vpc-dev 2 3Terraform used the selected providers to generate the following execution 4plan. ressource actions are indicated with the following symbols: 5 + create 6 7Terraform will perform the following actions: 8 9 # aws_default_network_acl.this[0] will be created 10 + ressource \u0026#34;aws_default_network_acl\u0026#34; \u0026#34;this\u0026#34; { 11 + arn = (known after apply) 12 + default_network_acl_id = (known after apply) 13 + id = (known after apply) 14 + owner_id = (known after apply) 15 + tags = { 16 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 17 } 18 + tags_all = { 19 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 20 } 21 + vpc_id = (known after apply) 22 23 + egress { 24 + action = \u0026#34;allow\u0026#34; 25 + from_port = 0 26 + ipv6_cidr_block = \u0026#34;::/0\u0026#34; 27 + protocol = \u0026#34;-1\u0026#34; 28 + rule_no = 101 29 + to_port = 0 30 } 31 + egress { 32... 33Plan generated: set approvePlan: \u0026#34;plan-v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. 34To set the field, you can also run: 35 36 tfctl approve vpc-dev -f filename.yaml Après revue des modifications ci-dessus, il suffit donc d'ajouter l'identifiant du plan à valider et de pousser le changement sur git comme suit:\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6... 7 approvePlan: plan-v5.0.0-26c38a66f1 8... En quelques instants un runner sera lancé qui se chargera d'appliquer les changements:\n1kubectl logs -f -n flux-system vpc-dev-tf-runner 22023/07/01 15:33:36 Starting the runner... version sha 3... 4aws_vpc.this[0]: Creating... 5aws_vpc.this[0]: Still creating... [10s elapsed] 6... 7aws_route_table_association.private[1]: Creation complete after 0s [id=rtbassoc-01b7347a7e9960a13] 8aws_nat_gateway.this[0]: Still creating... [10s elapsed] La réconciliation éffectuée, la ressource passe à l'état READY: True\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m 🤖 Application automatique Nous pouvons aussi activer la réconciliation automatique. Pour ce faire il faut déclarer le paramètre .spec.autoApprove à true.\nToutes les ressources IRSA sont configurées de la sorte:\nexternal-secrets.yaml\n1piVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: irsa-external-secrets 5spec: 6 approvePlan: auto 7 destroyResourcesOnDeletion: true 8 interval: 8m 9 path: ./modules/iam-role-for-service-accounts-eks 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-iam 13 namespace: flux-system 14 vars: 15 - name: role_name 16 value: ${cluster_name}-external-secrets 17 - name: attach_external_secrets_policy 18 value: true 19 - name: oidc_providers 20 value: 21 main: 22 provider_arn: ${oidc_provider_arn} 23 namespace_service_accounts: [\u0026#34;security:external-secrets\u0026#34;] Donc si je fais le moindre changement sur la console AWS par exemple, celui-ci sera rapidement écrasé par celui géré par tf-controller.\nInfo La politique de suppression d'une ressource Terraform est définie par le paramètre destroyResourcesOnDeletion. Par défaut elles sont conservées et il faut donc que ce paramètre ait pour valeur true afin de détruire les éléments crées lorsque l'objet Kubernetes est supprimé.\nIci nous voulons la possibilité de supprimer les rôles IRSA. Ils sont en effet étroitement liés aux clusters.\n🔄 Entrées et sorties: dépendances entre modules Lorsque qu'on utilise Terraform, on a souvent besoin de passer des données d'un module à l'autre. Généralement ce sont les outputs du module qui exportent ces informations. Il faut donc un moyen de les importer dans un autre module.\nReprenons encore l'exemple donné ci-dessus (vpc-dev). Nous notons en bas du YAML la directive suivante:\n1... 2 writeOutputsToSecret: 3 name: vpc-dev Lorsque cette ressource est appliquée nous aurons un message qui confirme que les outputs sont disponibles (\u0026quot;Outputs written\u0026quot;):\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m En effet ce module exporte de nombreuses informations (126):\n1kubectl get secrets -n flux-system vpc-dev 2NAME TYPE DATA AGE 3vpc-dev Opaque 126 15s 4 5kubectl get secret -n flux-system vpc-dev --template=\u0026#39;{{.data.vpc_id}}\u0026#39; | base64 -d 6vpc-0c06a6d153b8cc4db Certains de ces éléments d'informations sont ensuite utilisés pour créer un cluster EKS de dev:\nvpc/dev.yaml\n1... 2 varsFrom: 3 - kind: Secret 4 name: vpc-dev 5 varsKeys: 6 - vpc_id 7 - private_subnets 8... 💾 Sauvegarder et restaurer un tfstate Dans mon cas je ne souhaite pas recréer la zone et le certificat à chaque destruction du controlplane. Voici un exemple des étapes à mener pour que je puisse restaurer l'état de ces ressources lorsque j'utilise cette demo.\nNote Il s'agit là d'une procédure manuelle afin de démontrer le comportement de tf-controller par rapport aux fichiers d'état. Par défaut ces tfstates sont stockés dans des secrets mais on préferera configurer un backend GCS ou S3\nLa création initiale de l'environnement de démo m'a permis de sauvegarder les fichiers d'état (tfstate) de cette façon.\n1WORKSPACE=\u0026#34;default\u0026#34; 2STACK=\u0026#34;route53-cloud-hostedzone\u0026#34; 3BACKUPDIR=\u0026#34;${HOME}/tf-controller-backup\u0026#34; 4 5mkdir -p ${BACKUPDIR} 6 7kubectl get secrets -n flux-system tfstate-${WORKSPACE}-${STACK} -o jsonpath=\u0026#39;{.data.tfstate}\u0026#39; | \\ 8base64 -d | gzip -d \u0026gt; ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate Lorsque le cluster est créé à nouveau, tf-controller essaye de créer la zone car le fichier d'état est vide.\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system route53-cloud-hostedzone Unknown Plan generated: set approvePlan: \u0026#34;plan-main@sha1:345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. true 16 minutes 5 6tfctl show plan route53-cloud-hostedzone 7 8Terraform used the selected providers to generate the following execution 9plan. resource actions are indicated with the following symbols: 10 + create 11 12Terraform will perform the following actions: 13 14 # aws_route53_zone.this will be created 15 + resource \u0026#34;aws_route53_zone\u0026#34; \u0026#34;this\u0026#34; { 16 + arn = (known after apply) 17 + comment = \u0026#34;Experimentations for blog.ogenki.io\u0026#34; 18 + force_destroy = false 19 + id = (known after apply) 20 + name = \u0026#34;cloud.ogenki.io\u0026#34; 21 + name_servers = (known after apply) 22 + primary_name_server = (known after apply) 23 + tags = { 24 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 25 } 26 + tags_all = { 27 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 28 } 29 + zone_id = (known after apply) 30 } 31 32Plan: 1 to add, 0 to change, 0 to destroy. 33 34Changes to Outputs: 35 + domain_name = \u0026#34;cloud.ogenki.io\u0026#34; 36 + nameservers = (known after apply) 37 + zone_arn = (known after apply) 38 + zone_id = (known after apply) 39 40Plan generated: set approvePlan: \u0026#34;plan-main@345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. 41To set the field, you can also run: 42 43 tfctl approve route53-cloud-hostedzone -f filename.yaml La procédure de restauration consiste donc à créer le secret à nouveau:\n1gzip ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate 2 3cat \u0026lt;\u0026lt;EOF | kubectl apply -f - 4apiVersion: v1 5kind: Secret 6metadata: 7 name: tfstate-${WORKSPACE}-${STACK} 8 namespace: flux-system 9 annotations: 10 encoding: gzip 11type: Opaque 12data: 13 tfstate: $(cat ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz | base64 -w 0) 14EOF Il faudra aussi relancer un plan de façon explicite pour mettre à jour l'état de la ressource en question\n1tfctl replan route53-cloud-hostedzone 2 Replan requested for flux-system/route53-cloud-hostedzone 3Error: timed out waiting for the condition Nous pouvons alors vérifier que le fichier d'état a bien été mis à jour\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3flux-system route53-cloud-hostedzone True Outputs written: main@sha1:d0934f979d832feb870a8741ec01a927e9ee6644 false 19 minutes 🔍 Focus sur certaines fonctionnalités de Flux Oui j'ai un peu menti sur l'agenda 😝. Il me semblait nécessaire de mettre en lumière 2 fonctionnalités que je n'avais pas exploité jusque là et qui sont fort utiles!\nSubstition de variables Lorsque Flux est initiliasé un certain nombre de Kustomization spécifique à ce cluster sont créés. Il est possible d'y indiquer des variables de substitution qui pourront être utilisées dans l'ensemble des ressources déployées par cette Kustomization. Cela permet d'éviter un maximum la déduplication de code.\nJ'ai découvert l'efficacité de cette fonctionnalité très récemment. Je vais décrire ici la façon dont je l'utilise:\nLe code terraform qui crée un cluster EKS, génère aussi une ConfigMap qui contient les variables propres au cluster. On y retrouvera, bien sûr, le nom du cluster, mais aussi tous les paramètres qui varient entre les clusters et qui sont utilisés dans les manifests Kubernetes.\nflux.tf\n1resource \u0026#34;kubernetes_config_map\u0026#34; \u0026#34;flux_clusters_vars\u0026#34; { 2 metadata { 3 name = \u0026#34;eks-${var.cluster_name}-vars\u0026#34; 4 namespace = \u0026#34;flux-system\u0026#34; 5 } 6 7 data = { 8 cluster_name = var.cluster_name 9 oidc_provider_arn = module.eks.oidc_provider_arn 10 aws_account_id = data.aws_caller_identity.this.account_id 11 region = var.region 12 environment = var.env 13 vpc_id = module.vpc.vpc_id 14 } 15 depends_on = [flux_bootstrap_git.this] 16} Comme spécifié précedemment, les variables de substition sont définies dans les Kustomization. Prenons un exemple concret. Ci-dessous on définie la Kustomization qui déploie toutes les ressources qui sont consommées par tf-controller On déclare ici la ConfigMap eks-controlplane-0-vars qui avait été généré à la création du cluster EKS.\ninfrastructure.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1 2kind: Kustomization 3metadata: 4 name: tf-custom-resources 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 path: ./infrastructure/controlplane-0/terraform/custom-resources 10 postBuild: 11 substitute: 12 domain_name: \u0026#34;cloud.ogenki.io\u0026#34; 13 substituteFrom: 14 - kind: ConfigMap 15 name: eks-controlplane-0-vars 16 - kind: Secret 17 name: eks-controlplane-0-vars 18 optional: true 19 sourceRef: 20 kind: GitRepository 21 name: flux-system 22 dependsOn: 23 - name: tf-controller Enfin voici un exemple de ressource Kubernetes qui en fait usage. Cet unique manifest peut être utilisé par tous les clusters!.\nexternal-dns/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: external-dns 5spec: 6... 7 values: 8 global: 9 imageRegistry: public.ecr.aws 10 fullnameOverride: external-dns 11 aws: 12 region: ${region} 13 zoneType: \u0026#34;public\u0026#34; 14 batchChangeSize: 1000 15 domainFilters: [\u0026#34;${domain_name}\u0026#34;] 16 logFormat: json 17 txtOwnerId: \u0026#34;${cluster_name}\u0026#34; 18 serviceAccount: 19 annotations: 20 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/${cluster_name}-external-dns\u0026#34; Cela élimine totalement les overlays qui consistaient à ajouter les paramètres spécifiques au cluster.\nWeb UI (Weave GitOps) Dans mon précédent article sur Flux, je mentionnais le fait que l'un des inconvénients (si l'on compare avec son principale concurrent: ArgoCD) est le manque d'une interface Web. Bien que je sois un adepte de la ligne de commande, c'est parfois bien utile d'avoir une vue synthétique et de pouvoir effectuer certaines opération en quelques clicks \u0026#x1f5b1;\u0026#xfe0f;\nC'est désormais possible avec Weave Gitops! Bien entendu ce n'est pas comparable avec l'UI d'ArgoCD, mais l'essentiel est là: Mettre en pause la réconcilation, visualiser les manifests, les dépendances, les événements...\nIl existe aussi le plugin VSCode comme alternative.\n💭 Remarques Et voilà, nous arrivons au bout de notre exploration de cet autre outil de gestion d'infrastructure sur Kubernetes. Malgré quelques petits soucis rencontrés en cours de route, que j'ai partagé sur le repo Git du projet, l'expérience m'a beaucoup plu. tf-controller offre une réponse concrète à une question fréquente : comment gérer notre infra comme on gère notre code ?\nJ'aime beaucoup l'approche GitOps appliquée à l'infrastructure, j'avais d'ailleurs écrit un article sur Crossplane. tf-controller aborde la problématique sous un angle différent: utiliser du Terraform directement. Cela signifie qu'on peut utiliser nos connaissances actuelles et notre code existant. Pas besoin d'apprendre une nouvelle façon de déclarer nos ressources. C'est un critère à prendre en compte car migrer vers un nouvel outil lorsque l'on a un existant représente un éffort non négligeable. Cependant j'ajouterais aussi que tf-controller s'adresse aux utilisateurs de Flux uniquement et, de ce fait, restreint le publique cible.\nAujourd'hui, j'utilise une combinaison de Terraform, Terragrunt et RunAtlantis. tf-controller pourrait devenir une alternative viable: Nous avons en effet évoqué l'intérêt de Kustomize associé aux substitions de variables pour la factorisation de code. Dans la roadmap du projet il y a aussi l'objectif d'afficher les plans dans les pull-requests. Autre problématique fréquente: la nécessité de passer des éléments sensibles aux modules. En utilisant une ressource Terraform, on peut injecter des variables depuis des secrets Kubernetes. Ce qui permet d'utiliser certains outils, tels que external-secrets, sealed-secrets ...\nJe vous encourage donc à essayer tf-controller vous-même, et peut-être même d'y apporter votre contribution 🙂.\nNote La démo que j'ai faite ici utilise pas mal de ressources, dont certaines assez cruciales (comme le réseau). Donc, gardez en tête que c'est juste pour la démo ! Je suggère une approche progressive si vous envisagez de le mettre en ouvre: commencez par utiliser la détection de dérives, puis créez des ressources simples. J'ai aussi pris quelques raccourcis en terme de sécurité à éviter absolument, notamment le fait de donner les droits admin au contrôleur. ","link":"https://blog.ogenki.io/fr/post/terraform-controller/","section":"post","tags":["infrastructure"],"title":"Appliquer les principes de GitOps à l'infrastructure: Introduction à `tf-controller`"},{"body":"Kubernetes est désormais la plate-forme privilégiée pour orchestrer les applications \u0026quot;sans état\u0026quot; aussi appelé \u0026quot;stateless\u0026quot;. Les conteneurs qui ne stockent pas de données peuvent être détruits et recréés ailleurs sans impact. En revanche, la gestion d'applications \u0026quot;stateful\u0026quot; dans un environnement dynamique tel que Kubernetes peut être un véritable défi. Malgré le fait qu'il existe un nombre croissant de solutions de base de données \u0026quot;Cloud Native\u0026quot; (comme CockroachDB, TiDB, K8ssandra, Strimzi ...) et il y a de nombreux éléments à considérer lors de leur évaluation:\nQuelle est la maturité de l'opérateur? (Dynamisme et contributeurs, gouvernance du projet) Quels sont les resources personalisées disponibles (\u0026quot;custom resources\u0026quot;), quelles opérations permettent t-elles de réaliser? Quels sont les type de stockage disponibles: HDD / SSD, stockage local / distant? Que se passe-t-il lorsque quelque chose se passe mal: Quelle est le niveau de résilience de la solution? Sauvegarde et restauration: est-il facile d'effectuer et de planifier des sauvegardes? Quelles options de réplication et de mise à l'échelle sont disponibles? Qu'en est-il des limites de connexion et de concurrence, les pools de connexion? A propos de la supervision, quelles sont les métriques exposées et comment les exploiter? J'étais à la recherche d'une solution permettant de gérer un serveur PostgreSQL. La base de données qui y serait hébergée est nécessaire pour un logiciel de réservation de billets nommé Alf.io. Nous sommes en effet en train d'organiser les Kubernetes Community Days France vous êtes tous conviés! 👐.\nJe cherchais spécifiquement une solution indépendante d'un clouder (cloud agnostic) et l'un des principaux critères était la simplicité d'utilisation. Je connaissais déjà plusieurs opérateurs Kubernetes, et j'ai fini par évaluer une solution relativement récente: CloudNativePG.\nCloudNativepg est l'opérateur de Kubernetes qui couvre le cycle de vie complet d'un cluster de base de données PostgreSQL hautement disponible avec une architecture de réplication native en streaming.\nCe projet été créé par l'entreprise EnterpriseDB et a été soumis à la CNCF afin de rejoindre les projets Sandbox.\n🎯 Notre objectif Je vais donner ici une introduction aux principales fonctionnalités de CloudNativePG.\nL'objectif est de:\nCréer une base de données PostgreSQL sur un cluster GKE, Ajouter une instance secondaire (réplication) Exécuter quelques tests de résilience. Nous verrons également comment tout cela se comporte en terme de performances et quels sont les outils de supervision disponibles. Enfin, nous allons jeter un œil aux méthodes de sauvegarde/restauration.\nInfo Dans cet article, nous allons tout créer et tout mettre à jour manuellement. Mais dans un environnement de production, il est conseillé d'utiliser un moteur GitOps, par exemple Flux (sujet couvert dans un article précédent).\nSi vous souhaitez voir un exemple complet, vous pouvez consulter le dépôt git KCD France infrastructure.\nToutes les resources de cet article sont dans ce dépôt.\n\u0026#x2611;\u0026#xfe0f; Prérequis \u0026#x1f4e5; Outils gcloud SDK: Nous allons déployer sur Google Cloud (en particulier sur GKE) et, pour ce faire, nous devrons créer quelques ressources dans notre projet GCP. Nous aurons donc besoin du SDK et de la CLI Google Cloud. Il est donc nécessaire de l'installer en suivant cette documentation.\nkubectl plugin: Pour faciliter la gestion des clusters, il existe un plugin kubectl qui donne des informations synthétiques sur l'instance PostgreSQL et permet aussi d'effectuer certaines opérations. Ce plugin peut être installé en utilisant krew:\n1kubectl krew install cnpg ☁️ Créer les resources Google Cloud Avant de créer notre instance PostgreSQL, nous devons configurer certaines choses:\nNous avons besoin d'un cluster Kubernetes. (Cet article suppose que vous avez déjà pris soin de provisionner un cluster GKE) Nous allons créer un bucket (Google Cloud Storage) pour stocker les sauvegardes et Fichiers WAL. Nous configurerons les permissions pour nos pods afin qu'ils puissent écrire dans ce bucket. Créer le bucket à l'aide de CLI gcloud\n1gcloud storage buckets create --location=eu --default-storage-class=coldline gs://cnpg-ogenki 2Creating gs://cnpg-ogenki/... 3 4gcloud storage buckets describe gs://cnpg-ogenki 5[...] 6name: cnpg-ogenki 7owner: 8 entity: project-owners-xxxx0008 9projectNumber: \u0026#39;xxx00008\u0026#39; 10rpo: DEFAULT 11selfLink: https://www.googleapis.com/storage/v1/b/cnpg-ogenki 12storageClass: STANDARD 13timeCreated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; 14updated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; Nous allons maintenant configurer les permissions afin que les pods (PostgreSQL Server) puissent permettant écrire/lire à partir du bucket grâce à Workload Identity.\nNote Workload Identity doit être activé au niveau du cluster GKE. Afin de vérifier que le cluster est bien configuré, vous pouvez lancer la commande suivante:\n1gcloud container clusters describe \u0026lt;cluster_name\u0026gt; --format json --zone \u0026lt;zone\u0026gt; | jq .workloadIdentityConfig 2{ 3 \u0026#34;workloadPool\u0026#34;: \u0026#34;{{ gcp_project }}.svc.id.goog\u0026#34; 4} Créer un compte de service Google Cloud\n1gcloud iam service-accounts create cloudnative-pg --project={{ gcp_project }} 2Created service account [cloudnative-pg]. Attribuer au compte de service la permission storage.admin\n1gcloud projects add-iam-policy-binding {{ gcp_project }} \\ 2--member \u0026#34;serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com\u0026#34; \\ 3--role \u0026#34;roles/storage.admin\u0026#34; 4[...] 5- members: 6 - serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 7 role: roles/storage.admin 8etag: BwXrGA_VRd4= 9version: 1 Autoriser le compte de service (Attention il s'agit là du compte de service au niveau Kubernetes) afin d'usurper le compte de service IAM. \u0026#x2139;\u0026#xfe0f; Assurez-vous d'utiliser le format approprié serviceAccount:{{ gcp_project }}.svc.id.goog[{{ kubernetes_namespace }}/{{ kubernetes_serviceaccount }}]\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 7 role: roles/iam.workloadIdentityUser 8etag: BwXrGBjt5kQ= 9version: 1 Nous sommes prêts à créer les ressources Kubernetes \u0026#x1f4aa;\n\u0026#x1f511; Créer les secrets pour les utilisateurs PostgreSQL Nous devons créer les paramètres d'authentification des utilisateurs qui seront créés pendant la phase de \u0026quot;bootstrap\u0026quot; (nous y reviendrons par la suite): le superutilisateur et le propriétaire de base de données nouvellement créé.\n1kubectl create secret generic cnpg-mydb-superuser --from-literal=username=postgres --from-literal=password=foobar --namespace demo 2secret/cnpg-mydb-superuser created 1kubectl create secret generic cnpg-mydb-user --from-literal=username=smana --from-literal=password=barbaz --namespace demo 2secret/cnpg-mydb-user created \u0026#x1f6e0;\u0026#xfe0f; Déployer l'opérateur CloudNativePG avec Helm Ici nous utiliserons le chart Helm pour déployer CloudNativePG:\n1helm repo add cnpg https://cloudnative-pg.github.io/charts 2 3helm upgrade --install cnpg --namespace cnpg-system \\ 4--create-namespace charts/cloudnative-pg 5 6kubectl get po -n cnpg-system 7NAME READY STATUS RESTARTS AGE 8cnpg-74488f5849-8lhjr 1/1 Running 0 6h17m Cela installe aussi quelques resources personnalisées (Custom Resources Definitions)\n1kubectl get crds | grep cnpg.io 2backups.postgresql.cnpg.io 2022-10-08T16:15:14Z 3clusters.postgresql.cnpg.io 2022-10-08T16:15:14Z 4poolers.postgresql.cnpg.io 2022-10-08T16:15:14Z 5scheduledbackups.postgresql.cnpg.io 2022-10-08T16:15:14Z Pour une liste complète des paramètres possibles, veuillez vous référer à la doc de l'API.\n\u0026#x1f680; Créer un serveur PostgreSQL Nous pouvons désormais créer notre première instance en utilisant une resource personnalisée Cluster. La définition suivante est assez simple: Nous souhaitons démarrer un serveur PostgreSQL, créer automatiquement une base de données nommée mydb et configurer les informations d'authentification en utilisant les secrets créés précédemment.\n1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki 5 namespace: demo 6spec: 7 description: \u0026#34;PostgreSQL Demo Ogenki\u0026#34; 8 imageName: ghcr.io/cloudnative-pg/postgresql:14.5 9 instances: 1 10 11 bootstrap: 12 initdb: 13 database: mydb 14 owner: smana 15 secret: 16 name: cnpg-mydb-user 17 18 serviceAccountTemplate: 19 metadata: 20 annotations: 21 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 22 23 superuserSecret: 24 name: cnpg-mydb-superuser 25 26 storage: 27 storageClass: standard 28 size: 10Gi 29 30 backup: 31 barmanObjectStore: 32 destinationPath: \u0026#34;gs://cnpg-ogenki\u0026#34; 33 googleCredentials: 34 gkeEnvironment: true 35 retentionPolicy: \u0026#34;30d\u0026#34; 36 37 resources: 38 requests: 39 memory: \u0026#34;1Gi\u0026#34; 40 cpu: \u0026#34;500m\u0026#34; 41 limits: 42 memory: \u0026#34;1Gi\u0026#34; Créer le namespace où notre instance postgresql sera déployée\n1kubectl create ns demo 2namespace/demo created Adapdez le fichier YAML ci-dessus vos besoins et appliquez comme suit:\n1kubectl apply -f cluster.yaml 2cluster.postgresql.cnpg.io/ogenki created Vous remarquerez que le cluster sera en phase Initializing. Nous allons utiliser le plugin CNPG pour la première fois afin de vérifier son état. Cet outil deviendra par la suite notre meilleur ami pour afficher une vue synthétique de l'état du cluster.\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Primary server is initializing 4Name: ogenki 5Namespace: demo 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: (switching to ogenki-1) 8Status: Setting up primary Creating primary instance ogenki-1 9Instances: 1 10Ready instances: 0 11 12Certificates Status 13Certificate Name Expiration Date Days Left Until Expiration 14---------------- --------------- -------------------------- 15ogenki-ca 2023-01-13 20:02:40 +0000 UTC 90.00 16ogenki-replication 2023-01-13 20:02:40 +0000 UTC 90.00 17ogenki-server 2023-01-13 20:02:40 +0000 UTC 90.00 18 19Continuous Backup status 20First Point of Recoverability: Not Available 21No Primary instance found 22Streaming Replication status 23Not configured 24 25Instances status 26Name Database Size Current LSN Replication role Status QoS Manager Version Node 27---- ------------- ----------- ---------------- ------ --- --------------- ---- immédiatement après la déclaration de notre nouveau Cluster, une action de bootstrap est lancée. Dans notre exemple, nous créons une toute nouvelle base de données nommée mydb avec un propriétaire smana dont les informations d'authentification viennent du secret créé précédemment.\n1[...] 2 bootstrap: 3 initdb: 4 database: mydb 5 owner: smana 6 secret: 7 name: cnpg-mydb-user 8[...] 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 0/1 Running 0 55s 4ogenki-1-initdb-q75cz 0/1 Completed 0 2m32s Après quelques secondes, le cluster change de statut et devient Ready (configuré et prêt à l'usage) \u0026#x1f44f;\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7154833472216277012 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 1 10Ready instances: 1 11 12[...] 13 14Instances status 15Name Database Size Current LSN Replication role Status QoS Manager Version Node 16---- ------------- ----------- ---------------- ------ --- --------------- ---- 17ogenki-1 33 MB 0/17079F8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xczh Info Il existe de nombreuses façons de bootstrap un cluster. Par exemple, la restauration d'une sauvegarde dans une toute nouvelle instance ou en exécutant du code SQL ... Plus d'infos ici.\n🩹 Instance de secours et résilience Info Dans les architectures postgresql traditionnelles, nous trouvons généralement un composant supplémentaire pour gérer la haute disponibilité (ex: Patroni). Un particularité de l'opérateur CloudNativePG est qu'il bénéficie des fonctionnalités de base de Kubernetes et s'appuie sur un composant nommé Postgres instance manager.\nAjoutez une instance de secours (\u0026quot;standby\u0026quot;) en définissant le nombre de répliques sur 2.\n1kubectl edit cluster -n demo ogenki 2cluster.postgresql.cnpg.io/ogenki edited 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3[...] 4spec: 5 instances: 2 6[...] L'opérateur remarque immédiatement le changement, ajoute une instance de secours et démarre le processus de réplication.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Creating a new replica Creating replica ogenki-2-join 9Instances: 2 10Ready instances: 1 11Current Write LSN: 0/1707A30 (Timeline: 1 - WAL File: 000000010000000000000001) 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 0 3m16s 4ogenki-2-join-xxrwx 0/1 Pending 0 82s Après un certain temps (qui dépend de la quantité de données à répliquer), l'instance de secours devient opérationnelle et nous pouvons voir les statistiques de réplication.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 2 10Ready instances: 2 11Current Write LSN: 0/3000060 (Timeline: 1 - WAL File: 000000010000000000000003) 12 13[...] 14 15Streaming Replication status 16Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 17---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 18ogenki-2 0/3000060 0/3000060 0/3000060 0/3000060 00:00:00 00:00:00 00:00:00 streaming async 0 19 20Instances status 21Name Database Size Current LSN Replication role Status QoS Manager Version Node 22---- ------------- ----------- ---------------- ------ --- --------------- ---- 23ogenki-1 33 MB 0/3000060 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 24ogenki-2 33 MB 0/3000060 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc Nous allons désormais éffectuer ce que l'on appelle un \u0026quot;Switchover\u0026quot;: Nous allons promouvoir l'instance de secours en instance primaire.\nLe plugin cnpg permet de le faire de façon impérative, en utilisant la ligne de commande suivante:\n1kubectl cnpg promote ogenki ogenki-2 -n demo 2Node ogenki-2 in cluster ogenki will be promoted Dans mon cas, le basculement était vraiment rapide. Nous pouvons vérifier que l'instance ogenki-2 est devenu primaire et que la réplication est effectuée dans l'autre sens.\n1kubectl cnpg status -n demo ogenki 2[...] 3Status: Switchover in progress Switching over to ogenki-2 4Instances: 2 5Ready instances: 1 6[...] 7Streaming Replication status 8Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 9---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 10ogenki-1 0/4004CA0 0/4004CA0 0/4004CA0 0/4004CA0 00:00:00 00:00:00 00:00:00 streaming async 0 11 12Instances status 13Name Database Size Current LSN Replication role Status QoS Manager Version Node 14---- ------------- ----------- ---------------- ------ --- --------------- ---- 15ogenki-2 33 MB 0/4004CA0 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc 16ogenki-1 33 MB 0/4004CA0 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 Maintenant, provoquons un Failover en supprimant le pod principal\n1kubectl delete po -n demo --grace-period 0 --force ogenki-2 2Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 3pod \u0026#34;ogenki-2\u0026#34; force deleted 1Cluster Summary 2Name: ogenki 3Namespace: demo 4System ID: 7155095145869606932 5PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 6Primary instance: ogenki-1 7Status: Failing over Failing over from ogenki-2 to ogenki-1 8Instances: 2 9Ready instances: 1 10Current Write LSN: 0/4005D98 (Timeline: 3 - WAL File: 000000030000000000000004) 11 12[...] 13Instances status 14Name Database Size Current LSN Replication role Status QoS Manager Version Node 15---- ------------- ----------- ---------------- ------ --- --------------- ---- 16ogenki-1 33 MB 0/40078D8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 17ogenki-2 - - - pod not available Burstable - gke-kcdfrance-main-np-0e87115b-xszc Quelques secondes plus tard le cluster devient opérationnel à nouveau.\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 13m 2 2 Cluster in healthy state ogenki-1 Jusqu'ici tout va bien, nous avons pu faire quelques tests de la haute disponibilité et c'était assez probant 😎.\n👁️ Supervision Nous allons utiliser la Stack Prometheus. Nous ne couvrirons pas son installation dans cet article. Si vous voulez voir comment l'installer avec Flux, vous pouvez jeter un oeil à cet exemple.\nPour récupérer les métriques de notre instance, nous devons créer un PodMonitor.\n1apiVersion: monitoring.coreos.com/v1 2kind: PodMonitor 3metadata: 4 labels: 5 prometheus-instance: main 6 name: cnpg-ogenki 7 namespace: demo 8spec: 9 namespaceSelector: 10 matchNames: 11 - demo 12 podMetricsEndpoints: 13 - port: metrics 14 selector: 15 matchLabels: 16 postgresql: ogenki Nous pouvons ensuite ajouter le tableau de bord Grafana disponible ici.\nEnfin, vous souhaiterez peut-être configurer des alertes et vous pouvez créer un PrometheusRule en utilisant ces règles.\n\u0026#x1f525; Performances and benchmark Info Mise à jour: Il est désormais possible de faire un test de performance avec le plugin cnpg\nAfin de connaitre les limites de votre serveur, vous devriez faire un test de performances et de conserver une base de référence pour de futures améliorations.\nNote Au sujet des performances, il existe de nombreux domaines d'amélioration sur lesquels nous pouvons travailler.Cela dépend principalement de l'objectif que nous voulons atteindre. En effet, nous ne voulons pas perdre du temps et de l'argent pour les performances dont nous n'aurons probablement jamais besoin.\nVoici les principaux éléments à analyser:\nTuning de la configuration PostgreSQL Resources systèmes (cpu et mémoire) Types de Disque : IOPS, stockage locale (local-volume-provisioner), Disques dédiées pour les WAL et les données PG_DATA \u0026quot;Pooling\u0026quot; de connexions PGBouncer. CloudNativePG fourni une resource personnalisée Pooler qui permet de configurer cela facilement. Optimisation de la base de données, analyser les plans d'exécution grâce à explain, utiliser l'extension pg_stat_statement ... Tout d'abord, nous ajouterons des \u0026quot;labels\u0026quot; aux nœuds afin d'exécuter la commande pgbench sur différentes machines de celles hébergeant la base de données.\n1PG_NODE=$(kubectl get po -n demo -l postgresql=ogenki,role=primary -o jsonpath={.items[0].spec.nodeName}) 2kubectl label node ${PG_NODE} workload=postgresql 3node/gke-kcdfrance-main-np-0e87115b-vlzm labeled 4 5 6# Choose any other node different than the ${PG_NODE} 7kubectl label node gke-kcdfrance-main-np-0e87115b-p5d7 workload=pgbench 8node/gke-kcdfrance-main-np-0e87115b-p5d7 labeled Et nous déploierons le chart Helm comme suit\n1git clone git@github.com:EnterpriseDB/cnp-bench.git 2cd cnp-bench 3 4cat \u0026gt; pgbench-benchmark/myvalues.yaml \u0026lt;\u0026lt;EOF 5cnp: 6 existingCluster: true 7 existingHost: ogenki-rw 8 existingCredentials: cnpg-mydb-superuser 9 existingDatabase: mydb 10 11pgbench: 12 # Node where to run pgbench 13 nodeSelector: 14 workload: pgbench 15 initialize: true 16 scaleFactor: 1 17 time: 600 18 clients: 10 19 jobs: 1 20 skipVacuum: false 21 reportLatencies: false 22EOF 23 24helm upgrade --install -n demo pgbench -f pgbench-benchmark/myvalues.yaml pgbench-benchmark/ Info Il existe différents services selon que vous souhaitez lire et écrire ou de la lecture seule.\n1kubectl get ep -n demo 2NAME ENDPOINTS AGE 3ogenki-any 10.64.1.136:5432,10.64.1.3:5432 15d 4ogenki-r 10.64.1.136:5432,10.64.1.3:5432 15d 5ogenki-ro 10.64.1.136:5432 15d 6ogenki-rw 10.64.1.3:5432 15d 1kubectl logs -n demo job/pgbench-pgbench-benchmark -f 2Defaulted container \u0026#34;pgbench\u0026#34; out of: pgbench, wait-for-cnp (init), pgbench-init (init) 3pgbench (14.1, server 14.5 (Debian 14.5-2.pgdg110+2)) 4starting vacuum...end. 5transaction type: \u0026lt;builtin: TPC-B (sort of)\u0026gt; 6scaling factor: 1 7query mode: simple 8number of clients: 10 9number of threads: 1 10duration: 600 s 11number of transactions actually processed: 545187 12latency average = 11.004 ms 13initial connection time = 111.585 ms 14tps = 908.782896 (without initial connection time) 💽 Sauvegarde and restaurations Note Le fait de pouvoir stocker des sauvegarde et fichiers WAL dans le bucket GCP est possible car nous avons attribué les autorisations en utilisant une annotation présente dans le ServiceAccount utilisé par le cluster\n1serviceAccountTemplate: 2 metadata: 3 annotations: 4 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com Nous pouvons d'abord déclencher une sauvegarde on demand à l'aide de la ressource personnalisée Backup\n1apiVersion: postgresql.cnpg.io/v1 2kind: Backup 3metadata: 4 name: ogenki-now 5 namespace: demo 6spec: 7 cluster: 8 name: ogenki 1kubectl apply -f backup.yaml 2backup.postgresql.cnpg.io/ogenki-now created 3 4kubectl get backup -n demo 5NAME AGE CLUSTER PHASE ERROR 6ogenki-now 36s ogenki completed Si vous jetez un œil au contenu du bucket GCS, vous verrez un nouveau répertoire qui stocke les sauvegardes de base (\u0026quot;base backups\u0026quot;).\n1gcloud storage ls gs://cnpg-ogenki/ogenki/base 2gs://cnpg-ogenki/ogenki/base/20221023T130327/ Mais la plupart du temps, nous préfererons configurer une sauvegarde planifiée (\u0026quot;scheduled\u0026quot;). Ci-dessous un exemple pour une sauvegarde quotidienne:\n1apiVersion: postgresql.cnpg.io/v1 2kind: ScheduledBackup 3metadata: 4 name: ogenki-daily 5 namespace: demo 6spec: 7 backupOwnerReference: self 8 cluster: 9 name: ogenki 10 schedule: 0 0 0 * * * Les restaurations ne peuvent être effectuées que sur de nouvelles instances. Ici, nous utiliserons la sauvegarde que nous avions créée précédemment afin d'initialiser une toute nouvelle instance.\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore] 7 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 8 role: roles/iam.workloadIdentityUser 9etag: BwXrs755FPA= 10version: 1 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki-restore 5 namespace: demo 6spec: 7 instances: 1 8 9 serviceAccountTemplate: 10 metadata: 11 annotations: 12 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 13 14 storage: 15 storageClass: standard 16 size: 10Gi 17 18 resources: 19 requests: 20 memory: \u0026#34;1Gi\u0026#34; 21 cpu: \u0026#34;500m\u0026#34; 22 limits: 23 memory: \u0026#34;1Gi\u0026#34; 24 25 superuserSecret: 26 name: cnpg-mydb-superuser 27 28 bootstrap: 29 recovery: 30 backup: 31 name: ogenki-now Nous notons qu'un pod se charge immédiatement de la restauration complète (\u0026quot;full recovery\u0026quot;).\n1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 1 (18h ago) 18h 4ogenki-2 1/1 Running 0 18h 5ogenki-restore-1 0/1 Init:0/1 0 0s 6ogenki-restore-1-full-recovery-5p4ct 0/1 Completed 0 51s Le nouveau cluster devient alors opérationnel (\u0026quot;Ready\u0026quot;).\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 18h 2 2 Cluster in healthy state ogenki-1 4ogenki-restore 80s 1 1 Cluster in healthy state ogenki-restore-1 \u0026#x1f9f9; Nettoyage Suppression du cluster\n1kubectl delete cluster -n demo ogenki ogenki-restore 2cluster.postgresql.cnpg.io \u0026#34;ogenki\u0026#34; deleted 3cluster.postgresql.cnpg.io \u0026#34;ogenki-restore\u0026#34; deleted Supprimer le service IAM\n1gcloud iam service-accounts delete cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 2You are about to delete service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 3 4Do you want to continue (Y/n)? y 5 6deleted service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com] 💭 Conclusion Je viens tout juste de découvrir CloudNativePG et je n'ai fait qu'en percevoir la surface, mais une chose est sûre: la gestion d'une instance PostgreSQL est vraiment facilitée. Cependant, le choix d'une solution de base de données est une décision complexe. Il faut prendre en compte le cas d'usage, les contraintes de l'entreprise, la criticité de l'application et les compétences des équipes opérationnelles. Il existe de nombreuses options: bases de données gérées par le fournisseur Cloud, installation traditionnelle sur serveur baremetal, solutions distribuées ...\nNous pouvons également envisager d'utiliser Crossplane et une Composition pour fournir un niveau d'abstraction supplémentaire afin de déclarer des bases de données des fournisseurs Cloud, mais cela nécessite plus de configuration.\nCloudNativePG sort du lot par sa simplicité: Super facile à exécuter et à comprendre. De plus, la documentation est excellente (l'une des meilleures que j'aie jamais vues!), Surtout pour un si jeune projet open source (cela aidera peut être pour être accepté en tant que projet \u0026quot;Sandbox\u0026quot; CNCF 🤞).\nSi vous voulez en savoir plus, il y avait une présentation à ce sujet à KubeCon NA 2022.\n","link":"https://blog.ogenki.io/fr/post/cnpg/","section":"post","tags":["data"],"title":"`CloudNativePG`: et PostgreSQL devient facile sur Kubernetes"},{"body":"","link":"https://blog.ogenki.io/fr/tags/data/","section":"tags","tags":null,"title":"data"},{"body":"In a previous article, we've seen how to use Crossplane so that we can manage cloud resources the same way as our applications. \u0026#x2764;\u0026#xfe0f; Declarative approach! There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.\nHere we'll achieve exactly the same thing but we'll do that in the GitOps way. According to the OpenGitOps working group there are 4 GitOps principles:\nThe desired state of our system must be expressed declaratively. This state must be stored in a versioning system. Changes are pulled and applied automatically in the target platform whenever the desired state changes. If, for any reason, the current state is modified, it will be automatically reconciled with the desired state. There are several GitOps engine options. The most famous ones are ArgoCD and Flux. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).\nLearn more about GitOps toolkit components here.\n🎯 Our target Here we want to declare our desired infrastructure components only by adding git changes. By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance. We'll discover Flux basics and how to use it in order to build a complete GitOps CD workflow.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Install required tools First of all we need to install a few tools using asdf\nCreate a local file .tool-versions\n1cd ~/sources/devflux/ 2 3cat \u0026gt; .tool-versions \u0026lt;\u0026lt;EOF 4flux2 0.31.3 5kubectl 1.24.3 6kubeseal 0.18.1 7kustomize 4.5.5 8EOF 1for PLUGIN in $(cat .tool-versions | awk \u0026#39;{print $1}\u0026#39;); do asdf plugin-add $PLUGIN; done 2 3asdf install 4Downloading ... 100.0% 5Copying Binary 6... Check that all the required tools are actually installed.\n1asdf current 2flux2 0.31.3 /home/smana/sources/devflux/.tool-versions 3kubectl 1.24.3 /home/smana/sources/devflux/.tool-versions 4kubeseal 0.18.1 /home/smana/sources/devflux/.tool-versions 5kustomize 4.5.5 /home/smana/sources/devflux/.tool-versions \u0026#x1f511; Create a Github personal access token In this article the git repository is hosted in Github. In order to be able to use the flux bootstrap a personnal access token is required.\nPlease follow this procedure.\nWarning Store the Github token in a safe place for later use\n\u0026#x1f9d1;\u0026zwj;\u0026#x1f4bb; Clone the devflux repository All the files used for the upcoming steps can be retrieved from this repository. You should clone it, that will be easier to copy them into your own repository.\n1git clone https://github.com/Smana/devflux.git \u0026#x1f680; Bootstrap flux in the Crossplane cluster As we will often be using the flux CLI you may want to configure the bash|zsh completion\n1source \u0026lt;(flux completion bash) Warning Here we consider that you already have a local k3d instance. If not you may want to either go through the whole previous article or just run the local cluster creation.\nEnsure that you're working in the right context\n1kubectl config current-context 2k3d-crossplane Run the bootstrap command that will basically deploy all Flux's components in the namespace flux-system. Here I'll create a repository named devflux using my personal Github account.\n1export GITHUB_USER=\u0026lt;YOUR_ACCOUNT\u0026gt; 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/k3d-crossplane 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/k3d-crossplane\u0026#34; for \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux\u0026#34; 9... 10✔ all components are healthy Check that all the pods are running properly and that the kustomization flux-system has been successfully reconciled.\n1kubectl get po -n flux-system 2NAME READY STATUS RESTARTS AGE 3helm-controller-5985c795f8-gs2pc 1/1 Running 0 86s 4notification-controller-6b7d7485fc-lzlpg 1/1 Running 0 86s 5kustomize-controller-6d4669f847-9x844 1/1 Running 0 86s 6source-controller-5fb4888d8f-wgcqv 1/1 Running 0 86s 7 8flux get kustomizations 9NAME REVISION SUSPENDED READY MESSAGE 10flux-system main/33ebef1 False True Applied revision: main/33ebef1 Running the bootstap command actually creates a github repository if it doesn't exist yet. Clone it now for our upcoming changes. You'll notice that the first commit has been made by Flux.\n1git clone https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git 2Cloning into \u0026#39;devflux\u0026#39;... 3 4cd devflux 5 6git log -1 7commit 2beb6aafea67f3386b50cbc706fb34575844040d (HEAD -\u0026gt; main, origin/main, origin/HEAD) 8Author: Flux \u0026lt;\u0026gt; 9Date: Thu Jul 14 17:13:27 2022 +0200 10 11 Add Flux sync manifests 12 13ls clusters/k3d-crossplane/flux-system/ 14gotk-components.yaml gotk-sync.yaml kustomization.yaml \u0026#x1f4c2; Flux repository structure There are several options for organizing your resources in the Flux configuration repository. Here is a proposition for the sake of this article.\n1tree -d -L 2 2. 3├── apps 4│ ├── base 5│ └── dev-cluster 6├── clusters 7│ ├── dev-cluster 8│ └── k3d-crossplane 9├── infrastructure 10│ ├── base 11│ ├── dev-cluster 12│ └── k3d-crossplane 13├── observability 14│ ├── base 15│ ├── dev-cluster 16│ └── k3d-crossplane 17└── security 18 ├── base 19 ├── dev-cluster 20 └── k3d-crossplane Directory Description Example /apps our applications Here we'll deploy a demo application \u0026quot;online-boutique\u0026quot; /infrastructure base infrastructure/network components Crossplane as it will be used to provision cloud resources but we can also find CSI/CNI/EBS drivers... /observability All metrics/apm/logging tools Prometheus of course, Opentelemetry ... /security Any component that enhance our security level SealedSecrets (see below) Info For the upcoming steps please refer to the demo repository here\nLet's use this structure and begin to deploy applications \u0026#x1f680;.\n\u0026#x1f510; SealedSecrets There are plenty of alternatives when it comes to secrets management in Kubernetes. In order to securely store secrets in a git repository the GitOps way we'll make use of SealedSecrets. It uses a custom resource definition named SealedSecrets in order to encrypt the Kubernetes secret at the client side then the controller is in charge of decrypting and generating the expected secret in the cluster.\n\u0026#x1f6e0;\u0026#xfe0f; Deploy the controller using Helm The first thing to do is to declare the kustomization that handles all the security tools.\nclusters/k3d-crossplane/security.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 2kind: Kustomization 3metadata: 4 name: security 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 sourceRef: 10 kind: GitRepository 11 name: flux-system 12 path: ./security/k3d-crossplane 13 healthChecks: 14 - apiVersion: helm.toolkit.fluxcd.io/v1beta1 15 kind: HelmRelease 16 name: sealed-secrets 17 namespace: kube-system Info A Kustomization is a custom resource that comes with Flux. It basically points to a set of Kubernetes resources managed with kustomize The above security kustomization points to a local directory where the kustomize resources are.\n1... 2spec: 3 path: ./security/k3d-crossplane 4... Note This is worth noting that there are two types on kustomizations. That can be confusing when you start playing with Flux.\nOne managed by flux's kustomize controller. Its API is kustomization.kustomize.toolkit.fluxcd.io The other kustomization.kustomize.config.k8s.io is for the kustomize overlay The kustomization.yaml file is always used for the kustomize overlay. Flux itself doesn't need this overlay in all cases, but if you want to use features of a Kustomize overlay you will occasionally need to create it in order to access them. It provides instructions to the Kustomize CLI.\nWe will deploy SealedSecrets using the Helm chart. So we need to declare the source of this chart. Using the kustomize overlay system, we'll first create the base files that will be inherited at the cluster level.\nsecurity/base/sealed-secrets/source.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: sealed-secrets 5 namespace: flux-system 6spec: 7 interval: 30m 8 url: https://bitnami-labs.github.io/sealed-secrets Then we'll define the HelmRelease which references the above source. Put the values you want to apply to the Helm chart under spec.values\nsecurity/base/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 releaseName: sealed-secrets 8 chart: 9 spec: 10 chart: sealed-secrets 11 sourceRef: 12 kind: HelmRepository 13 name: sealed-secrets 14 namespace: flux-system 15 version: \u0026#34;2.4.0\u0026#34; 16 interval: 10m0s 17 install: 18 remediation: 19 retries: 3 20 values: 21 fullnameOverride: sealed-secrets-controller 22 resources: 23 requests: 24 cpu: 80m 25 memory: 100Mi If you're starting your repository from scratch you'll need to generate the kustomization.yaml file (kustomize overlay).\n1kustomize create --autodetect security/base/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4- helmrelease.yaml 5- source.yaml Now we declare the sealed-secret kustomization at the cluster level. Just for the example we'll overwrite a value at the cluster level using kustomize's overlay system.\nsecurity/k3d-crossplane/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 values: 8 resources: 9 requests: 10 cpu: 100m security/k3d-crossplane/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3bases: 4 - ../../base 5patches: 6 - helmrelease.yaml Pushing our changes is the only thing to do in order to get sealed-secrets deployed in the target cluster.\n1git commit -m \u0026#34;security: deploy sealed-secrets in k3d-crossplane\u0026#34; 2[security/sealed-secrets 283648e] security: deploy sealed-secrets in k3d-crossplane 3 6 files changed, 66 insertions(+) 4 create mode 100644 clusters/k3d-crossplane/security.yaml 5 create mode 100644 security/base/sealed-secrets/helmrelease.yaml 6 create mode 100644 security/base/sealed-secrets/kustomization.yaml 7 create mode 100644 security/base/sealed-secrets/source.yaml 8 create mode 100644 security/k3d-crossplane/sealed-secrets/helmrelease.yaml 9 create mode 100644 security/k3d-crossplane/sealed-secrets/kustomization.yaml After a few seconds (1 minutes by default) a new kustomization will appear.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3flux-system main/d36a33c False True Applied revision: main/d36a33c 4security main/d36a33c False True Applied revision: main/d36a33c And all the resources that we declared in the flux repository should be available and READY.\n1flux get sources helm 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee False True stored artifact for revision \u0026#39;4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee\u0026#39; 1flux get helmrelease -n kube-system 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 2.2.0 False True Release reconciliation succeeded 🧪 A first test SealedSecret Let's use the CLI kubeseal to test it out. We'll create a SealedSecret that will be decrypted by the sealed-secrets controller in the cluster and create the expected secret foobar\n1kubectl create secret generic foobar -n default --dry-run=client -o yaml --from-literal=foo=bar \\ 2| kubeseal --namespace default --format yaml | kubectl apply -f - 3sealedsecret.bitnami.com/foobar created 4 5kubectl get secret -n default foobar 6NAME TYPE DATA AGE 7foobar Opaque 1 3m13s 8 9kubectl delete sealedsecrets.bitnami.com foobar 10sealedsecret.bitnami.com \u0026#34;foobar\u0026#34; deleted \u0026#x2601;\u0026#xfe0f; Deploy and configure Crossplane \u0026#x1f511; Create the Google service account secret The first thing we need to do in order to get Crossplane working is to create the GCP serviceaccount. The steps have been covered here in the previous article. We'll create a SealedSecret gcp-creds that contains the serviceaccount file crossplane.json.\ninfrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml\n1kubectl create secret generic gcp-creds --context k3d-crossplane -n crossplane-system --from-file=creds=./crossplane.json --dry-run=client -o yaml \\ 2| kubeseal --format yaml --namespace crossplane-system - \u0026gt; infrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml 🔄 Crossplane dependencies Now we will deploy Crossplane with Flux. I won't put the manifests here you'll find all of them in this repository. However it's important to understand that, in order to deploy and configure Crossplane properly we need to do that in a specific order. Indeed several CRD's (custom resource definitions) are required:\nFirst of all we'll install the crossplane controller. Then we'll configure the provider because the custom resource is now available thanks to the crossplane controller installation. Finally a provider installation deploys several CRDs that can be used to configure the provider itself and cloud resources. The dependencies between kustomizations can be controlled using the parameters dependsOn. Looking at the file clusters/k3d-crossplane/infrastructure.yaml, we can see for example that the kustomization infrastructure-custom-resources depends on the kustomization crossplane_provider which itself depends on crossplane-configuration....\n1--- 2apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 3kind: Kustomization 4metadata: 5 name: crossplane-provider 6spec: 7... 8 dependsOn: 9 - name: crossplane-core 10--- 11apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 12kind: Kustomization 13metadata: 14 name: crossplane-configuration 15spec: 16... 17 dependsOn: 18 - name: crossplane-provider 19--- 20apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 21kind: Kustomization 22metadata: 23 name: infrastructure-custom-resources 24spec: 25... 26 dependsOn: 27 - name: crossplane-configuration Commit and push the changes for the kustomisations to appear. Note that they'll be reconciled in the defined order.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3infrastructure-custom-resources False False dependency \u0026#39;flux-system/crossplane-configuration\u0026#39; is not ready 4crossplane-configuration False False dependency \u0026#39;flux-system/crossplane-provider\u0026#39; is not ready 5security main/666f85a False True Applied revision: main/666f85a 6flux-system main/666f85a False True Applied revision: main/666f85a 7crossplane-core main/666f85a False True Applied revision: main/666f85a 8crossplane-provider main/666f85a False True Applied revision: main/666f85a Then all Crossplane components will be deployed, we can have a look to the HelmRelease status for instance.\n1kubectl describe helmrelease -n crossplane-system crossplane 2... 3Status: 4 Conditions: 5 Last Transition Time: 2022-07-15T19:12:04Z 6 Message: Release reconciliation succeeded 7 Reason: ReconciliationSucceeded 8 Status: True 9 Type: Ready 10 Last Transition Time: 2022-07-15T19:12:04Z 11 Message: Helm upgrade succeeded 12 Reason: UpgradeSucceeded 13 Status: True 14 Type: Released 15 Helm Chart: crossplane-system/crossplane-system-crossplane 16 Last Applied Revision: 1.9.0 17 Last Attempted Revision: 1.9.0 18 Last Attempted Values Checksum: 056dc1c6029b3a644adc7d6a69a93620afd25b65 19 Last Release Revision: 2 20 Observed Generation: 1 21Events: 22 Type Reason Age From Message 23 ---- ------ ---- ---- ------- 24 Normal info 20m helm-controller HelmChart \u0026#39;crossplane-system/crossplane-system-crossplane\u0026#39; is not ready 25 Normal info 20m helm-controller Helm upgrade has started 26 Normal info 19m helm-controller Helm upgrade succeeded And our GKE cluster should also be created because we defined a bunch of crossplane custom resources in infrastructure/k3d-crossplane/custom-resources/crossplane\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RUNNING 34.x.x.190 europe-west9-a 22m \u0026#x1f680; Bootstrap flux in the dev cluster Our local Crossplane cluster is now ready and it created our dev cluster and we also want it to be managed with Flux. So let's configure Flux for this dev cluster using the same bootstrap command.\nAuthenticate to the newly created cluster. The following command will automatically change your current context.\n1gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project \u0026lt;your_project\u0026gt; 2Fetching cluster endpoint and auth data. 3kubeconfig entry generated for dev-cluster. 4 5kubectl config current-context 6gke_\u0026lt;your_project\u0026gt;_europe-west9-a_dev-cluster Run the bootstrap command for the dev-cluster.\n1export GITHUB_USER=Smana 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/dev-cluster 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/Smana/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/dev-cluster\u0026#34; for \u0026#34;https://github.com/Smana/devflux\u0026#34; 9... 10✔ all components are healthy Note It's worth noting that each Kubernetes cluster generates its own sealing keys. That means that if you recreate the dev-cluster, you must regenerate all the sealedsecrets. In our example we declared a secret in order to set the Grafana credentials. Here's the command you need to run in order to create a new version of the sealedsecret and don't forget to use the proper context \u0026#x1f609;.\n1kubectl create secret generic kube-prometheus-stack-grafana \\ 2--from-literal=admin-user=admin --from-literal=admin-password=\u0026lt;yourpassword\u0026gt; --namespace observability --dry-run=client -o yaml \\ 3| kubeseal --namespace observability --format yaml \u0026gt; observability/dev-cluster/kube-prometheus-stack/sealedsecrets.yaml After a few seconds we'll get the following kustomizations deployed.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3apps main/1380eaa False True Applied revision: main/1380eaa 4flux-system main/1380eaa False True Applied revision: main/1380eaa 5observability main/1380eaa False True Applied revision: main/1380eaa 6security main/1380eaa False True Applied revision: main/1380eaa Here we configured the prometheus stack and deployed a demo microservices stack named \u0026quot;online-boutique\u0026quot; This demo application exposes the frontend through a service of type LoadBalancer.\n1kubectl get svc -n demo frontend-external 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3frontend-external LoadBalancer 10.140.174.201 34.155.121.2 80:31943/TCP 7m44s Use the EXTERNAL_IP\n\u0026#x1f575;\u0026#xfe0f; Troubleshooting The cheatsheet in Flux's documentation contains many ways for troubleshooting when something goes wrong. Here I'll just give a sample of my favorite command lines.\nObjects that aren't ready\n1flux get all -A --status-selector ready=false Checking the logs of a given kustomization\n1flux logs --kind kustomization --name infrastructure-custom-resources 22022-07-15T19:38:52.996Z info Kustomization/infrastructure-custom-resources.flux-system - server-side apply completed 32022-07-15T19:38:53.016Z info Kustomization/infrastructure-custom-resources.flux-system - Reconciliation finished in 66.12266ms, next run in 4m0s 42022-07-15T19:11:34.697Z info Kustomization/infrastructure-custom-resources.flux-system - Discarding event, no alerts found for the involved object Show how a given pod is managed by Flux.\n1flux trace -n crossplane-system pod/crossplane-5dc8d888d7-g95qx 2 3Object: Pod/crossplane-5dc8d888d7-g95qx 4Namespace: crossplane-system 5Status: Managed by Flux 6--- 7HelmRelease: crossplane 8Namespace: crossplane-system 9Revision: 1.9.0 10Status: Last reconciled at 2022-07-15 21:12:04 +0200 CEST 11Message: Release reconciliation succeeded 12--- 13HelmChart: crossplane-system-crossplane 14Namespace: crossplane-system 15Chart: crossplane 16Version: 1.9.0 17Revision: 1.9.0 18Status: Last reconciled at 2022-07-15 21:11:36 +0200 CEST 19Message: pulled \u0026#39;crossplane\u0026#39; chart with version \u0026#39;1.9.0\u0026#39; 20--- 21HelmRepository: crossplane 22Namespace: crossplane-system 23URL: https://charts.crossplane.io/stable 24Revision: 362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d 25Status: Last reconciled at 2022-07-15 21:11:35 +0200 CEST 26Message: stored artifact for revision \u0026#39;362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d\u0026#39; If you want to check what would be the changes before pushing your commit. In thi given example I just increased the cpu requests for the sealed-secrets controller.\n1flux diff kustomization security --path security/k3d-crossplane 2✓ Kustomization diffing... 3► HelmRelease/kube-system/sealed-secrets drifted 4 5metadata.generation 6 ± value change 7 - 6 8 + 7 9 10spec.values.resources.requests.cpu 11 ± value change 12 - 100m 13 + 120m 14 15⚠️ identified at least one change, exiting with non-zero exit code \u0026#x1f9f9; Cleanup Don't forget to delete the Cloud resources if you don't want to have a bad suprise \u0026#x1f4b5;! Just comment the file infrastructure/k3d-crossplane/custom-resources/crossplane/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4 # - cluster.yaml 5 - network.yaml \u0026#x1f44f; Achievements With our current setup everything is configured using the GitOps approach:\nWe can manage infrastructure resources using Crossplane. Our secrets are securely stored in our git repository. We have a dev-cluster that we can enable or disable just but commenting a yaml file. Our demo application can be deployed from scratch in seconds. 💭 final thoughts Flux is probably the tool I'm using the most on a daily basis. It's really amazing!\nWhen you get familiar with its concepts and the command line it becomes really easy to use and troubleshoot. You can use either Helm when a chart is available or Kustomize.\nHowever we faced a few issues:\nIt's not straightforward to find an efficient structure depending on the company needs. Especially when you have several Kubernetes controllers that depend on other CRDs. The Helm controller doesn't maintain a state of the Kubernetes resources deployed by the Helm chart. That means that if you delete a resource which has been deployed through a Helm chart, it won't be reconciled (It will change soon. Being discussed here) Flux doesn't provide itself a web UI and switching between CLIs (kubectl, flux ...) can be annoying from a developer perspective. (I'm going to test weave-gitops ) I've been using Flux in production for more than a year and we configured it with the image automation so that the only thing a developer has to do is to merge a pull request and the new version of the application is automatically deployed in the target cluster.\nI should probably give another try to ArgoCD in order to be able to compare these precisely 🤔.\n","link":"https://blog.ogenki.io/fr/post/devflux/","section":"post","tags":["gitops","devxp"],"title":"100% `GitOps` using Flux"},{"body":"Qui suis-je? Je suis un ingénieur système / SRE senior. Je porte un intérêt particulier aux conteneurs Linux et les technologies dîtes \u0026quot;Cloud Native\u0026quot;. Au fil de ma carrière, j'ai eu l'occasion de collaborer avec une variété d'entreprises, des startups dynamiques aux grandes structures, où j'ai contribué à optimiser la fiabilité et la disponibilité des plateformes, tout en améliorant l'expérience des développeurs. Mon parcours inclut l'accompagnement de plusieurs sociétés dans leur migration vers des solutions Cloud. J'ai également pu diriger des équipes SRE/DevOps composées de profils variés, incluant développeurs et ingénieurs SRE, avec pour objectif un engagement collectif vers des objectifs communs.\nEn parallèle de mon parcours professionnel, je m'investis dans l'organisation du meetup Cloud Native Computing à Paris ainsi que les Kubernetes Community Days France , reflétant mon engagement continu dans l'écosystème du Cloud.\nAutres centre d'intéret : Lecture de romans de science-fiction, Kickboxing, Surf, Longboard et Rollers..\n","link":"https://blog.ogenki.io/fr/about/","section":"","tags":null,"title":"About"},{"body":"","link":"https://blog.ogenki.io/fr/tags/devxp/","section":"tags","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/fr/tags/gitops/","section":"tags","tags":null,"title":"gitops"},{"body":"La cible de cette documentation est de pouvoir créer et gérer un cluster GKE en utilisant Crossplane.\nCrossplane exploite les principes de base de Kubernetes afin de fournir des ressources cloud et bien plus encore: une ** approche déclarative ** avec ** Drift Detections ** et ** réconciliations ** Utilisation de boucles de contrôle: Exploseding_head:.En d'autres termes, nous déclarons les ressources cloud que nous voulons et Crossplane garantit que l'état cible correspond à celui appliqué via l'API Kubernetes.\nVoici les étapes que nous suivrons afin d'obtenir un cluster Kubernetes pour le développement et les cas d'utilisation des expérimentations.\n\u0026#x1f433; Create the local k3d cluster for Crossplane's control plane k3d est un cluster Kubernetes léger qui exploite K3S qui s'exécute dans notre ordinateur portable local. Il existe plusieurs modèles de déploiement pour Crossplane, nous pourrions par exemple déployer le plan de contrôle sur un cluster de gestion sur Kubernetes ou un plan de contrôle par cluster Kubernetes. Ici, j'ai choisi une méthode simple qui est bien pour un cas d'utilisation personnelle: une instance Kubernetes locale ** dans laquelle je vais déployer Crossplane.\nLet's install k3d using asdf.\n1asdf plugin-add k3d 2 3asdf install k3d $(asdf latest k3d) 4* Downloading k3d release 5.4.1... 5k3d 5.4.1 installation was successful! Create a single node Kubernetes cluster.\n1k3d cluster create crossplane 2... 3INFO[0043] You can now use it like this: 4kubectl cluster-info 5 6k3d cluster list 7crossplane 1/1 0/0 true Check that the cluster is reachable using the kubectl CLI.\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:40643 3CoreDNS is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy We only need a single node for our Crossplane use case.\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-crossplane-server-0 Ready control-plane,master 26h v1.22.7+k3s1 \u0026#x2601;\u0026#xfe0f; Generate the Google Cloud service account Warning Store the downloaded crossplane.json credentials file in a safe place.\nCreate a service account\n1GCP_PROJECT=\u0026lt;your_project\u0026gt; 2gcloud iam service-accounts create crossplane --display-name \u0026#34;Crossplane\u0026#34; --project=${GCP_PROJECT} 3Created service account [crossplane]. Assign the proper permissions to the service account.\nCompute Network Admin Kubernetes Engine Admin Service Account User 1SA_EMAIL=$(gcloud iam service-accounts list --filter=\u0026#34;email ~ ^crossplane\u0026#34; --format=\u0026#39;value(email)\u0026#39;) 2 3gcloud projects add-iam-policy-binding \u0026#34;${GCP_PROJECT}\u0026#34; --member=serviceAccount:\u0026#34;${SA_EMAIL}\u0026#34; \\ 4--role=roles/container.admin --role=roles/compute.networkAdmin --role=roles/iam.serviceAccountUser 5Updated IAM policy for project [\u0026lt;project\u0026gt;]. 6bindings: 7- members: 8 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 9 role: roles/compute.networkAdmin 10- members: 11 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 12... 13version: 1 Download the service account key (json format)\n1gcloud iam service-accounts keys create crossplane.json --iam-account ${SA_EMAIL} 2created key [ea2eb9ce2939127xxxxxxxxxx] of type [json] as [crossplane.json] for [crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com] \u0026#x1f6a7; Deploy and configure Crossplane Now that we have a credentials file for Google Cloud, we can deploy the Crossplane operator and configure the provider-gcp provider.\nInfo Most of the following steps are issued from the official documentation\nWe'll first use Helm in order to install the operator\n1helm repo add crossplane-master https://charts.crossplane.io/master/ 2\u0026#34;crossplane-master\u0026#34; has been added to your repositories 3 4helm repo update 5...Successfully got an update from the \u0026#34;crossplane-master\u0026#34; chart repository 6 7helm install crossplane --namespace crossplane-system --create-namespace \\ 8--version 1.18.1 crossplane-stable/crossplane 9 10NAME: crossplane 11LAST DEPLOYED: Mon Jun 6 22:00:02 2022 12NAMESPACE: crossplane-system 13STATUS: deployed 14REVISION: 1 15TEST SUITE: None 16NOTES: 17Release: crossplane 18... Check that the operator is running properly.\n1kubectl get po -n crossplane-system 2NAME READY STATUS RESTARTS AGE 3crossplane-rbac-manager-54d96cd559-222hc 1/1 Running 0 3m37s 4crossplane-688c575476-lgklq 1/1 Running 0 3m37s Info All the files used for the upcoming steps are stored within this blog repository. So you should clone and change the current directory:\n1git clone https://github.com/Smana/smana.github.io.git 2 3cd smana.github.io/content/resources/crossplane_k3d Now we'll configure Crossplane so that it will be able to create and manage GCP resources. This is done by configuring the provider provider-gcp as follows.\nprovider.yaml\n1apiVersion: pkg.crossplane.io/v1 2kind: Provider 3metadata: 4 name: crossplane-provider-gcp 5spec: 6 package: crossplane/provider-gcp:v0.21.0 1kubectl apply -f provider.yaml 2provider.pkg.crossplane.io/crossplane-provider-gcp created 3 4kubectl get providers 5NAME INSTALLED HEALTHY PACKAGE AGE 6crossplane-provider-gcp True True crossplane/provider-gcp:v0.21.0 10s Create the Kubernetes secret that holds the GCP credentials file created above\n1kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./crossplane.json 2secret/gcp-creds created Then we need to create a resource named ProviderConfig and reference the newly created secret.\nprovider-config.yaml\n1apiVersion: gcp.crossplane.io/v1beta1 2kind: ProviderConfig 3metadata: 4 name: default 5spec: 6 projectID: ${GCP_PROJECT} 7 credentials: 8 source: Secret 9 secretRef: 10 namespace: crossplane-system 11 name: gcp-creds 12 key: creds 1kubectl apply -f provider-config.yaml 2providerconfig.gcp.crossplane.io/default created Info If the serviceaccount has the proper permissions we can create resources in GCP. In order to learn about all the available resources and parameters we can have a look to the provider's API reference.\nThe first resource we'll create is the network that will host our Kubernetes cluster.\nnetwork.yaml\n1apiVersion: compute.gcp.crossplane.io/v1beta1 2kind: Network 3metadata: 4 name: dev-network 5 labels: 6 service: vpc 7 creation: crossplane 8spec: 9 forProvider: 10 autoCreateSubnetworks: false 11 description: \u0026#34;Network used for experimentations and POCs\u0026#34; 12 routingConfig: 13 routingMode: REGIONAL 1kubectl get network 2NAME READY SYNCED 3dev-network True True You can even get more details by describing this resource. For instance if something fails you would see the message returned by the Cloud provider in the events.\n1kubectl describe network dev-network | grep -A 20 \u0026#39;^Status:\u0026#39; 2Status: 3 At Provider: 4 Creation Timestamp: 2022-06-28T09:45:30.703-07:00 5 Id: 3005424280727359173 6 Self Link: https://www.googleapis.com/compute/v1/projects/${GCP_PROJECT}/global/networks/dev-network 7 Conditions: 8 Last Transition Time: 2022-06-28T16:45:31Z 9 Reason: Available 10 Status: True 11 Type: Ready 12 Last Transition Time: 2022-06-30T16:36:59Z 13 Reason: ReconcileSuccess 14 Status: True 15 Type: Synced \u0026#x1f680; Create a GKE cluster Everything is ready so that we can create our GKE cluster. Applying the file cluster.yaml will create a cluster and attach a node group to it.\ncluster.yaml\n1--- 2apiVersion: container.gcp.crossplane.io/v1beta2 3kind: Cluster 4metadata: 5 name: dev-cluster 6spec: 7 forProvider: 8 description: \u0026#34;Kubernetes cluster for experimentations and POCs\u0026#34; 9 initialClusterVersion: \u0026#34;1.24\u0026#34; 10 releaseChannel: 11 channel: \u0026#34;RAPID\u0026#34; 12 location: europe-west9-a 13 addonsConfig: 14 gcePersistentDiskCsiDriverConfig: 15 enabled: true 16 networkPolicyConfig: 17 disabled: false 18 networkRef: 19 name: dev-network 20 ipAllocationPolicy: 21 createSubnetwork: true 22 useIpAliases: true 23 defaultMaxPodsConstraint: 24 maxPodsPerNode: 110 25 networkPolicy: 26 enabled: false 27 writeConnectionSecretToRef: 28 namespace: default 29 name: gke-conn 30--- 31apiVersion: container.gcp.crossplane.io/v1beta1 32kind: NodePool 33metadata: 34 name: main-np 35spec: 36 forProvider: 37 initialNodeCount: 1 38 autoscaling: 39 autoprovisioned: false 40 enabled: true 41 maxNodeCount: 4 42 minNodeCount: 1 43 clusterRef: 44 name: dev-cluster 45 config: 46 machineType: n2-standard-2 47 diskSizeGb: 120 48 diskType: pd-standard 49 imageType: cos_containerd 50 preemptible: true 51 labels: 52 environment: dev 53 managed-by: crossplane 54 oauthScopes: 55 - \u0026#34;https://www.googleapis.com/auth/devstorage.read_only\u0026#34; 56 - \u0026#34;https://www.googleapis.com/auth/logging.write\u0026#34; 57 - \u0026#34;https://www.googleapis.com/auth/monitoring\u0026#34; 58 - \u0026#34;https://www.googleapis.com/auth/servicecontrol\u0026#34; 59 - \u0026#34;https://www.googleapis.com/auth/service.management.readonly\u0026#34; 60 - \u0026#34;https://www.googleapis.com/auth/trace.append\u0026#34; 61 metadata: 62 disable-legacy-endpoints: \u0026#34;true\u0026#34; 63 shieldedInstanceConfig: 64 enableIntegrityMonitoring: true 65 enableSecureBoot: true 66 management: 67 autoRepair: true 68 autoUpgrade: true 69 maxPodsConstraint: 70 maxPodsPerNode: 60 71 locations: 72 - \u0026#34;europe-west9-a\u0026#34; 1kubectl apply -f cluster.yaml 2cluster.container.gcp.crossplane.io/dev-cluster created 3nodepool.container.gcp.crossplane.io/main-np created Note that it takes around 10 minutes for the Kubernetes API and the nodes to be available. The STATE will transition from PROVISIONING to RUNNING and when a change is being applied the cluster status is RECONCILING\n1watch \u0026#39;kubectl get cluster,nodepool\u0026#39; 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3cluster.container.gcp.crossplane.io/dev-cluster False True PROVISIONING 34.155.122.6 europe-west9-a 3m15s 4 5NAME READY SYNCED STATE CLUSTER-REF AGE 6nodepool.container.gcp.crossplane.io/main-np False False dev-cluster 3m15s When the column READY switches to True you can download the cluster's credentials.\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RECONCILING 34.42.42.42 europe-west9-a 6m23s 4 5gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project ${GCP_PROJECT} 6Fetching cluster endpoint and auth data. 7kubeconfig entry generated for dev-cluster. For better readability you may want to rename the context id for the newly created cluster\n1kubectl config rename-context gke_${GCP_PROJECT}_europe-west9-a_dev-cluster dev-cluster 2Context \u0026#34;gke_${GCP_PROJECT}_europe-west9-a_dev-cluster\u0026#34; renamed to \u0026#34;dev-cluster\u0026#34;. 3 4kubectl config get-contexts 5CURRENT NAME CLUSTER AUTHINFO NAMESPACE 6* dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster 7 k3d-crossplane k3d-crossplane admin@k3d-crossplane Check that you can call our brand new GKE API\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3gke-dev-cluster-main-np-d0d978f9-5fc0 Ready \u0026lt;none\u0026gt; 10m v1.24.1-gke.1400 That's great \u0026#x1f389; we know have a GKE cluster up and running.\n💭 final thoughts I've been using Crossplane for a few months now in a production environment.\nEven if I'm conviced about the declarative approach using the Kubernetes API, we decided to move with caution with it. It clearly doesn't have Terraform's community and maturity. We're still declaring our resources using the deletionPolicy: Orphan so that even if something goes wrong on the controller side the resource won't be deleted.\nFurthermore we limited to a specific list of usual AWS resources requested by our developers. Nevertheless our target has always been to empower developers and we had really positive feedback from them. That's the best indicator for us. As the project matures, we'll move more and more resources from Terraform to Crossplane.\nIMHO the key success of Crossplane depends on the providers maintenance and evolution. The Cloud providers interest and involvement is really important.\nIn our next article we'll see how to use a GitOps engine to run all the above steps.\n","link":"https://blog.ogenki.io/fr/post/crossplane_k3d/","section":"post","tags":["kubernetes","infrastructure"],"title":"Mon cluster Kubernetes (GKE) avec `Crossplane`"},{"body":"Afin d'installer des binaires et de pouvoir passer d'une version à une autre, j'aime utiliser asdf.\n\u0026#x1f4e5; Installation L'installation recommandée consiste à utiliser Git comme suit\n1git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.0 Il y a quelques étapes supplémentaires qui dépendent de votre shell. Voici celles que j'utilise pour bash:\n1. $HOME/.asdf/asdf.sh Vous voudrez probablement configurer la completion du shell comme suit\n1. $HOME/.asdf/completions/asdf.bash \u0026#x1f680; Prenons un exemple Listons tous les plugins disponibles pour trouver k3d\n1asdf plugin-list-all | grep k3d 2k3d https://github.com/spencergilbert/asdf-k3d.git Installons k3d\n1asdf plugin-add k3d Vérifier les versions disponibles\n1asdf list-all k3d| tail -n 3 25.4.0-dev.3 35.4.0 45.4.1 Nous installerons la dernière version\n1asdf install k3d latest 2* Downloading k3d release 5.4.1... 3k3d 5.4.1 installation was successful! Enfin, nous pouvons passer d'une version à une autre. Nous pouvons définir une version \u0026quot;globale\u0026quot; qui serait utilisée sur tous les répertoires.\n1asdf global k3d 5.4.1 ou utilisez une version locale en fonction du répertoire actuel\n1cd /tmp 2asdf local k3d 5.4.1 3 4asdf current k3d 5k3d 5.4.1 /tmp/.tool-versions \u0026#x1f9f9; Faire le ménage Désinstaller une version donnée\n1asdf uninstall k3d 5.4.1 Retirer un plugin\n1asdf plugin remove k3d ","link":"https://blog.ogenki.io/fr/post/asdf/asdf/","section":"post","tags":["tooling","local"],"title":"Gérer les versions d'outils avec `asdf`"},{"body":"","link":"https://blog.ogenki.io/fr/tags/local/","section":"tags","tags":null,"title":"local"},{"body":"","link":"https://blog.ogenki.io/fr/tags/tooling/","section":"tags","tags":null,"title":"tooling"},{"body":"","link":"https://blog.ogenki.io/fr/categories/","section":"categories","tags":null,"title":"Categories"},{"body":"","link":"https://blog.ogenki.io/fr/tags/index/","section":"tags","tags":null,"title":"index"},{"body":"","link":"https://blog.ogenki.io/fr/series/","section":"series","tags":null,"title":"Series"}]
\ No newline at end of file
diff --git a/fr/post/cilium-gateway-api/index.html b/fr/post/cilium-gateway-api/index.html
index b508690..42538e3 100644
--- a/fr/post/cilium-gateway-api/index.html
+++ b/fr/post/cilium-gateway-api/index.html
@@ -8,8 +8,8 @@
 </a><a href=https://blog.ogenki.io/fr/tags/infrastructure/ title=infrastructure class="post_tag button button_translucent">infrastructure
 </a><a href=https://blog.ogenki.io/fr/tags/network/ title=network class="post_tag button button_translucent">network
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=%60Gateway%20API%60%3a%20Remplacer%20mon%20Ingress%20Controller%20avec%20%60Cilium%60%3f&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcilium-gateway-api%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcilium-gateway-api%2f&t=%60Gateway%20API%60%3a%20Remplacer%20mon%20Ingress%20Controller%20avec%20%60Cilium%60%3f" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/cilium-gateway-api/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#bullseye-notre-objectif>:bullseye: Notre objectif</a></li><li><a href=#-introduction-à-gateway-api>☸ Introduction à Gateway API</a></li><li><a href=#ballot_box_with_check-prérequis>Prérequis</a></li><li><a href=#-la-porte-dentrée-gatewayclass-et-gateway>🚪 La porte d'entrée: GatewayClass et Gateway</a></li><li><a href=#arrow_right_hook-les-règles-de-routage-httproute>Les règles de routage: HTTPRoute</a><ul><li><a href=#un-routage-simple>Un routage simple</a></li><li><a href=#exposer-un-service-en-utilisant-un-certificat-tls>Exposer un service en utilisant un certificat TLS</a></li><li><a href=#une-gateway-partagée-par-plusieurs-namespaces>Une Gateway partagée par plusieurs namespaces</a></li><li><a href=#traffic-splitting>Traffic splitting</a></li><li><a href=#manipulation-des-entêtes-http-headers>Manipulation des entêtes HTTP (Headers)</a></li></ul></li><li><a href=#-les-roles-et-permissions>🪪 Les roles et permissions</a></li><li><a href=#-un-périmètre-pas-évident>🤔 Un périmètre pas évident</a></li><li><a href=#-dernières-remarques>💭 Dernières remarques</a></li><li><a href=#-references>🔖 References</a></li></ul></nav></div><div class=post_body><p>Lorsque l'on déploie une application sur Kubernetes, l'étape suivante consiste généralement à l'exposer aux utilisateurs. On utilise habituellement des "<a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/><strong>Ingress controllers</strong></a>", comme Nginx, Haproxy, Traefik ou encore ceux des différents Clouders afin de diriger le trafic entrant vers l'application, gérer l'équilibrage de charge, la terminaison TLS et j'en passe.</p><p>Il faut alors choisir parmi la pléthore d'<a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/>options disponibles</a> 🤯 la solution qui sera en charge de tous ces aspects et Cilium est, depuis relativement récemment, l'une d'entre elles.</p><p><a href=https://cilium.io/><strong>Cilium</strong></a> est une solution Open-Source de connectivité réseau et de sécurité basée sur eBPF dont l'adoption est grandissante.
-Il s'agit probablement du plugin réseau qui fournit le plus de fonctionnalités. Nous n'allons pas toutes les parcourir mais l'une d'entre elles consiste à gérer le trafic entrant en utilisant le standard <a href=https://gateway-api.sigs.k8s.io/><strong>Gateway API</strong></a> (<code>GAPI</code>).</p><h2 id=bullseye-notre-objectif>:bullseye: Notre objectif</h2><ul><li>Comprendre ce qu'est exactement <strong>Gateway API</strong> et en quoi il s'agit d'une évolution par rapport à l'API <code>Ingress</code>.</li><li>Démonstrations de cas concrets à la sauce GitOps.</li><li>Les limitations actuelles et les évolutions à venir.</li></ul><div class="notices tip"><div class=label>Tip</div><p>Toutes les étapes réalisées dans cet article proviennent de ce <a href=https://github.com/Smana/cilium-gateway-api><strong>dépôt git</strong></a></p><p>Je vous invite à le parcourir car, il va bien au delà du contexte de cet article:</p><ul><li>Installation d'un cluster <strong>EKS</strong> avec <code>Cilium</code> configuré en mode sans kube-proxy et un Daemonset dédié à <code>Envoy</code></li><li>Proposition de structure de configuration <code>Flux</code> avec une gestion des dépendances et une factorisation que je trouve efficaces.</li><li><code>Crossplane</code> et composition <a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> qui simplifie la gestion des permissions IAM pour les composants plateforme</li><li>Gestion des noms de domaine ainsi que des certificats automatisés avec <code>External-DNS</code> et <code>Let's Encrypt</code></li></ul><p>L'idée étant d'avoir l'ensemble configuré au bout de quelques minutes, en une seule ligne de commande 🤩.</p></div><h2 id=-introduction-à-gateway-api>☸ Introduction à Gateway API</h2><p>Comme évoqué précédemment, il y a de nombreuses options qui font office d' <em>Ingress controller</em> et chacune a ses propres spécificités et des fonctionnalités particulières, rendant leur utilisation parfois complexe. Par ailleurs, l'API <code>Ingress</code>, utilisée historiquement dans Kubernetes possède très peu d'options. Certaines solutions ont d'ailleurs créé des <a href=https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>CRDs</a> (Ressources personalisées) quand d'autres font usage des <code>annotations</code> pour lever ces limites.</p><p>C'est dans ce contexte que <strong>Gateway API</strong> fait son apparition. Il s'agit d'un <strong>standard</strong> qui permet de définir des fonctionnalités réseau avancées sans nécessiter d'extensions spécifiques au contrôleur sous-jacent. De plus étant donné que tous les contrôleurs utilisent la même API , il est possible de passer d'une solution à une autre <strong>sans changer de configuration</strong> (les ressources qui gèrent le trafic entrant restent les mêmes).</p><p>Parmi les concepts que nous allons explorer la GAPI introduit un schéma de répartition des responsabilités. Elle définit des <strong>roles explicites avec des permissions bien distinctes</strong>. (Plus d'informations sur le modèle de sécurité GAPI <a href=https://gateway-api.sigs.k8s.io/concepts/security-model/#roles-and-personas>ici</a>).</p><p>Enfin il est important de noter que ce projet est dirigé par le groupe de travail <a href=https://github.com/kubernetes/community/tree/master/sig-network>sig-network-kubernetes</a> et un canal slack vous permettra de les solliciter si nécessaire.</p><p>Voyons comment cela s'utilise concrètement 🚀!</p><h2 id=ballot_box_with_check-prérequis>&#9745;&#xfe0f; Prérequis</h2><p>Pour le reste de cet article nous considérons qu'un cluster EKS a été déployé. Si vous n'utilisez pas la <a href=https://github.com/Smana/cilium-gateway-api/tree/main/terraform/eks>méthode proposée dans le repo de démo</a> servant de socle à cet article, il y a <strong>certains points à valider</strong> pour que GAPI puisse être utilisé.</p><p>ℹ️ La méthode d'installation decrite ici se base sur <code>Helm</code>, l'ensemble des <code>values</code> peuvent être consultées <a href=https://github.com/Smana/cilium-gateway-api/blob/main/terraform/eks/helm_values/cilium.yaml>ici</a>.</p><ul><li><p><strong>Installer</strong> les <code>CRDs</code> (resources personnalisés) disponibles dans le repository <a href=https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd>Gateway API</a><div class="notices note"><div class=label>Note</div><p>Si Cilium est configuré avec le support <code>GAPI</code> (voir ci-dessous) et que les CRDs sont absentes, il ne démarrera pas.</p><p>Dans le <a href=https://github.com/Smana/cilium-gateway-api>repo de demo</a> les <strong>CRDs</strong> <code>GAPI</code> sont installées une première fois lors de la création du cluster afin que Cilium puisse démarrer puis elles sont ensuite gérées par Flux.</p></div></p></li><li><p><strong>Remplacer</strong> <code>kube-proxy</code> par les fonctionnalités de transfert réseau apportées par Cilium et <code>eBPF</code>.</p></li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=ln>1</span><span class=cl><span class=nt>kubeProxyReplacement</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
+<a href="https://twitter.com/intent/tweet?text=%60Gateway%20API%60%3a%20Remplacer%20mon%20Ingress%20Controller%20avec%20%60Cilium%60%3f&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcilium-gateway-api%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcilium-gateway-api%2f&t=%60Gateway%20API%60%3a%20Remplacer%20mon%20Ingress%20Controller%20avec%20%60Cilium%60%3f" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/cilium-gateway-api/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#-notre-objectif>🎯 Notre objectif</a></li><li><a href=#-introduction-à-gateway-api>☸ Introduction à Gateway API</a></li><li><a href=#ballot_box_with_check-prérequis>Prérequis</a></li><li><a href=#-la-porte-dentrée-gatewayclass-et-gateway>🚪 La porte d'entrée: GatewayClass et Gateway</a></li><li><a href=#arrow_right_hook-les-règles-de-routage-httproute>Les règles de routage: HTTPRoute</a><ul><li><a href=#un-routage-simple>Un routage simple</a></li><li><a href=#exposer-un-service-en-utilisant-un-certificat-tls>Exposer un service en utilisant un certificat TLS</a></li><li><a href=#une-gateway-partagée-par-plusieurs-namespaces>Une Gateway partagée par plusieurs namespaces</a></li><li><a href=#traffic-splitting>Traffic splitting</a></li><li><a href=#manipulation-des-entêtes-http-headers>Manipulation des entêtes HTTP (Headers)</a></li></ul></li><li><a href=#-les-roles-et-permissions>🪪 Les roles et permissions</a></li><li><a href=#-un-périmètre-pas-évident>🤔 Un périmètre pas évident</a></li><li><a href=#-dernières-remarques>💭 Dernières remarques</a></li><li><a href=#-references>🔖 References</a></li></ul></nav></div><div class=post_body><p>Lorsque l'on déploie une application sur Kubernetes, l'étape suivante consiste généralement à l'exposer aux utilisateurs. On utilise habituellement des "<a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/><strong>Ingress controllers</strong></a>", comme Nginx, Haproxy, Traefik ou encore ceux des différents Clouders afin de diriger le trafic entrant vers l'application, gérer l'équilibrage de charge, la terminaison TLS et j'en passe.</p><p>Il faut alors choisir parmi la pléthore d'<a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/>options disponibles</a> 🤯 la solution qui sera en charge de tous ces aspects et Cilium est, depuis relativement récemment, l'une d'entre elles.</p><p><a href=https://cilium.io/><strong>Cilium</strong></a> est une solution Open-Source de connectivité réseau et de sécurité basée sur eBPF dont l'adoption est grandissante.
+Il s'agit probablement du plugin réseau qui fournit le plus de fonctionnalités. Nous n'allons pas toutes les parcourir mais l'une d'entre elles consiste à gérer le trafic entrant en utilisant le standard <a href=https://gateway-api.sigs.k8s.io/><strong>Gateway API</strong></a> (<code>GAPI</code>).</p><h2 id=-notre-objectif>🎯 Notre objectif</h2><ul><li>Comprendre ce qu'est exactement <strong>Gateway API</strong> et en quoi il s'agit d'une évolution par rapport à l'API <code>Ingress</code>.</li><li>Démonstrations de cas concrets à la sauce GitOps.</li><li>Les limitations actuelles et les évolutions à venir.</li></ul><div class="notices tip"><div class=label>Tip</div><p>Toutes les étapes réalisées dans cet article proviennent de ce <a href=https://github.com/Smana/cilium-gateway-api><strong>dépôt git</strong></a></p><p>Je vous invite à le parcourir car, il va bien au delà du contexte de cet article:</p><ul><li>Installation d'un cluster <strong>EKS</strong> avec <code>Cilium</code> configuré en mode sans kube-proxy et un Daemonset dédié à <code>Envoy</code></li><li>Proposition de structure de configuration <code>Flux</code> avec une gestion des dépendances et une factorisation que je trouve efficaces.</li><li><code>Crossplane</code> et composition <a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> qui simplifie la gestion des permissions IAM pour les composants plateforme</li><li>Gestion des noms de domaine ainsi que des certificats automatisés avec <code>External-DNS</code> et <code>Let's Encrypt</code></li></ul><p>L'idée étant d'avoir l'ensemble configuré au bout de quelques minutes, en une seule ligne de commande 🤩.</p></div><h2 id=-introduction-à-gateway-api>☸ Introduction à Gateway API</h2><p>Comme évoqué précédemment, il y a de nombreuses options qui font office d' <em>Ingress controller</em> et chacune a ses propres spécificités et des fonctionnalités particulières, rendant leur utilisation parfois complexe. Par ailleurs, l'API <code>Ingress</code>, utilisée historiquement dans Kubernetes possède très peu d'options. Certaines solutions ont d'ailleurs créé des <a href=https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>CRDs</a> (Ressources personalisées) quand d'autres font usage des <code>annotations</code> pour lever ces limites.</p><p>C'est dans ce contexte que <strong>Gateway API</strong> fait son apparition. Il s'agit d'un <strong>standard</strong> qui permet de définir des fonctionnalités réseau avancées sans nécessiter d'extensions spécifiques au contrôleur sous-jacent. De plus étant donné que tous les contrôleurs utilisent la même API , il est possible de passer d'une solution à une autre <strong>sans changer de configuration</strong> (les ressources qui gèrent le trafic entrant restent les mêmes).</p><p>Parmi les concepts que nous allons explorer la GAPI introduit un schéma de répartition des responsabilités. Elle définit des <strong>roles explicites avec des permissions bien distinctes</strong>. (Plus d'informations sur le modèle de sécurité GAPI <a href=https://gateway-api.sigs.k8s.io/concepts/security-model/#roles-and-personas>ici</a>).</p><p>Enfin il est important de noter que ce projet est dirigé par le groupe de travail <a href=https://github.com/kubernetes/community/tree/master/sig-network>sig-network-kubernetes</a> et un canal slack vous permettra de les solliciter si nécessaire.</p><p>Voyons comment cela s'utilise concrètement 🚀!</p><h2 id=ballot_box_with_check-prérequis>&#9745;&#xfe0f; Prérequis</h2><p>Pour le reste de cet article nous considérons qu'un cluster EKS a été déployé. Si vous n'utilisez pas la <a href=https://github.com/Smana/cilium-gateway-api/tree/main/terraform/eks>méthode proposée dans le repo de démo</a> servant de socle à cet article, il y a <strong>certains points à valider</strong> pour que GAPI puisse être utilisé.</p><p>ℹ️ La méthode d'installation decrite ici se base sur <code>Helm</code>, l'ensemble des <code>values</code> peuvent être consultées <a href=https://github.com/Smana/cilium-gateway-api/blob/main/terraform/eks/helm_values/cilium.yaml>ici</a>.</p><ul><li><p><strong>Installer</strong> les <code>CRDs</code> (resources personnalisés) disponibles dans le repository <a href=https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd>Gateway API</a><div class="notices note"><div class=label>Note</div><p>Si Cilium est configuré avec le support <code>GAPI</code> (voir ci-dessous) et que les CRDs sont absentes, il ne démarrera pas.</p><p>Dans le <a href=https://github.com/Smana/cilium-gateway-api>repo de demo</a> les <strong>CRDs</strong> <code>GAPI</code> sont installées une première fois lors de la création du cluster afin que Cilium puisse démarrer puis elles sont ensuite gérées par Flux.</p></div></p></li><li><p><strong>Remplacer</strong> <code>kube-proxy</code> par les fonctionnalités de transfert réseau apportées par Cilium et <code>eBPF</code>.</p></li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=ln>1</span><span class=cl><span class=nt>kubeProxyReplacement</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
 </span></span></span></code></pre></div><ul><li><strong>Activer</strong> le support de Gateway API</li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=ln>1</span><span class=cl><span class=nt>gatewayAPI</span><span class=p>:</span><span class=w>
 </span></span></span><span class=line><span class=ln>2</span><span class=cl><span class=w>  </span><span class=nt>enabled</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
 </span></span></span></code></pre></div><ul><li><p><strong>Vérifier</strong> L'installation
diff --git a/fr/post/cnpg/index.html b/fr/post/cnpg/index.html
index 1ee3e66..9163142 100644
--- a/fr/post/cnpg/index.html
+++ b/fr/post/cnpg/index.html
@@ -6,7 +6,7 @@
 <a href=https://blog.ogenki.io/fr/ class="nav_child nav_item">Français</a></div></div><div class=follow><a href=https://github.com/Smana><svg class="icon"><title>github</title><use xlink:href="#github"/></svg></a><a href=https://twitter.com/_smana_><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href=https://www.linkedin.com/in/sma%C3%AFne-kahlouch-44374110/><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><div class=color_mode><input type=checkbox class=color_choice id=mode></div></div></div></nav></header><main><div class="grid-inverse wrap content"><article class=post_content><h1 class=post_title><code>CloudNativePG</code>: et PostgreSQL devient facile sur Kubernetes</h1><div class=post_meta><span><svg class="icon"><title>calendar</title><use xlink:href="#calendar"/></svg></span><span class=post_date>23-10-2022</span>
 <span class=post_time>· 16 mins de lecture</span><span>&nbsp;· <a href=https://blog.ogenki.io/fr/tags/data/ title=data class="post_tag button button_translucent">data
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=%60CloudNativePG%60%3a%20et%20PostgreSQL%20devient%20facile%20sur%20Kubernetes&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcnpg%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcnpg%2f&t=%60CloudNativePG%60%3a%20et%20PostgreSQL%20devient%20facile%20sur%20Kubernetes" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/cnpg/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/cnpg/cnpg.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#bullseye-notre-objectif>:bullseye: Notre objectif</a></li><li><a href=#ballot_box_with_check-prérequis>Prérequis</a><ul><li><a href=#inbox_tray-outils>Outils</a></li><li><a href=#-créer-les-resources-google-cloud>☁️ Créer les resources Google Cloud</a></li><li><a href=#key-créer-les-secrets-pour-les-utilisateurs-postgresql>Créer les secrets pour les utilisateurs PostgreSQL</a></li></ul></li><li><a href=#hammer_and_wrench-déployer-lopérateur-cloudnativepg-avec-helm>Déployer l'opérateur CloudNativePG avec Helm</a></li><li><a href=#rocket-créer-un-serveur-postgresql>Créer un serveur PostgreSQL</a></li><li><a href=#-instance-de-secours-et-résilience>🩹 Instance de secours et résilience</a></li><li><a href=#-supervision>👁️ Supervision</a></li><li><a href=#fire-performances-and-benchmark>Performances and benchmark</a></li><li><a href=#-sauvegarde-and-restaurations>💽 Sauvegarde and restaurations</a></li><li><a href=#broom-nettoyage>Nettoyage</a></li><li><a href=#-conclusion>💭 Conclusion</a></li></ul></nav></div><div class=post_body><p>Kubernetes est désormais la plate-forme privilégiée pour orchestrer les applications "sans état" aussi appelé "stateless". Les conteneurs qui ne stockent pas de données peuvent être détruits et recréés ailleurs sans impact. En revanche, la gestion d'applications "stateful" dans un environnement dynamique tel que Kubernetes peut être un véritable <strong>défi</strong>. Malgré le fait qu'il existe un nombre croissant de solutions de base de données "Cloud Native" (comme CockroachDB, TiDB, K8ssandra, Strimzi ...) et il y a de nombreux <strong>éléments à considérer</strong> lors de leur évaluation:</p><ul><li>Quelle est la maturité de l'opérateur? (Dynamisme et contributeurs, gouvernance du projet)</li><li>Quels sont les resources personalisées disponibles ("custom resources"), quelles opérations permettent t-elles de réaliser?</li><li>Quels sont les type de stockage disponibles: HDD / SSD, stockage local / distant?</li><li>Que se passe-t-il lorsque quelque chose se passe mal: Quelle est le niveau de résilience de la solution?</li><li>Sauvegarde et restauration: est-il facile d'effectuer et de planifier des sauvegardes?</li><li>Quelles options de réplication et de mise à l'échelle sont disponibles?</li><li>Qu'en est-il des limites de connexion et de concurrence, les pools de connexion?</li><li>A propos de la supervision, quelles sont les métriques exposées et comment les exploiter?</li></ul><p>J'étais à la recherche d'une solution permettant de gérer un serveur <strong>PostgreSQL</strong>. La base de données qui y serait hébergée est nécessaire pour un logiciel de réservation de billets nommé <a href=https://alf.io/>Alf.io</a>. Nous sommes en effet en train d'organiser les <a href=https://kcdfrance.fr>Kubernetes Community Days France</a> vous êtes tous conviés! 👐.</p><p>Je cherchais spécifiquement une solution indépendante d'un clouder (cloud agnostic) et l'un des principaux critères était la simplicité d'utilisation. Je connaissais déjà plusieurs opérateurs Kubernetes, et j'ai fini par évaluer une solution relativement récente: <a href=https://cloudnative-pg.io/><strong>CloudNativePG</strong></a>.</p><blockquote><p>CloudNativepg est l'opérateur de Kubernetes qui couvre le cycle de vie complet d'un cluster de base de données PostgreSQL hautement disponible avec une architecture de réplication native en streaming.</p></blockquote><p>Ce projet été créé par l'entreprise <a href=https://www.enterprisedb.com/>EnterpriseDB</a> et a été soumis à la <strong>CNCF</strong> afin de rejoindre les projets <em>Sandbox</em>.</p><h2 id=bullseye-notre-objectif>:bullseye: Notre objectif</h2><p>Je vais donner ici une <strong>introduction</strong> aux principales fonctionnalités de CloudNativePG.</p><p>L'objectif est de:</p><ul><li>Créer une base de données PostgreSQL sur un cluster GKE,</li><li>Ajouter une instance secondaire (réplication)</li><li>Exécuter quelques tests de résilience.</li></ul><p>Nous verrons également comment tout cela se comporte en terme de performances et quels sont les outils de supervision disponibles.
+<a href="https://twitter.com/intent/tweet?text=%60CloudNativePG%60%3a%20et%20PostgreSQL%20devient%20facile%20sur%20Kubernetes&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcnpg%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fcnpg%2f&t=%60CloudNativePG%60%3a%20et%20PostgreSQL%20devient%20facile%20sur%20Kubernetes" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/cnpg/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/cnpg/cnpg.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#-notre-objectif>🎯 Notre objectif</a></li><li><a href=#ballot_box_with_check-prérequis>Prérequis</a><ul><li><a href=#inbox_tray-outils>Outils</a></li><li><a href=#-créer-les-resources-google-cloud>☁️ Créer les resources Google Cloud</a></li><li><a href=#key-créer-les-secrets-pour-les-utilisateurs-postgresql>Créer les secrets pour les utilisateurs PostgreSQL</a></li></ul></li><li><a href=#hammer_and_wrench-déployer-lopérateur-cloudnativepg-avec-helm>Déployer l'opérateur CloudNativePG avec Helm</a></li><li><a href=#rocket-créer-un-serveur-postgresql>Créer un serveur PostgreSQL</a></li><li><a href=#-instance-de-secours-et-résilience>🩹 Instance de secours et résilience</a></li><li><a href=#-supervision>👁️ Supervision</a></li><li><a href=#fire-performances-and-benchmark>Performances and benchmark</a></li><li><a href=#-sauvegarde-and-restaurations>💽 Sauvegarde and restaurations</a></li><li><a href=#broom-nettoyage>Nettoyage</a></li><li><a href=#-conclusion>💭 Conclusion</a></li></ul></nav></div><div class=post_body><p>Kubernetes est désormais la plate-forme privilégiée pour orchestrer les applications "sans état" aussi appelé "stateless". Les conteneurs qui ne stockent pas de données peuvent être détruits et recréés ailleurs sans impact. En revanche, la gestion d'applications "stateful" dans un environnement dynamique tel que Kubernetes peut être un véritable <strong>défi</strong>. Malgré le fait qu'il existe un nombre croissant de solutions de base de données "Cloud Native" (comme CockroachDB, TiDB, K8ssandra, Strimzi ...) et il y a de nombreux <strong>éléments à considérer</strong> lors de leur évaluation:</p><ul><li>Quelle est la maturité de l'opérateur? (Dynamisme et contributeurs, gouvernance du projet)</li><li>Quels sont les resources personalisées disponibles ("custom resources"), quelles opérations permettent t-elles de réaliser?</li><li>Quels sont les type de stockage disponibles: HDD / SSD, stockage local / distant?</li><li>Que se passe-t-il lorsque quelque chose se passe mal: Quelle est le niveau de résilience de la solution?</li><li>Sauvegarde et restauration: est-il facile d'effectuer et de planifier des sauvegardes?</li><li>Quelles options de réplication et de mise à l'échelle sont disponibles?</li><li>Qu'en est-il des limites de connexion et de concurrence, les pools de connexion?</li><li>A propos de la supervision, quelles sont les métriques exposées et comment les exploiter?</li></ul><p>J'étais à la recherche d'une solution permettant de gérer un serveur <strong>PostgreSQL</strong>. La base de données qui y serait hébergée est nécessaire pour un logiciel de réservation de billets nommé <a href=https://alf.io/>Alf.io</a>. Nous sommes en effet en train d'organiser les <a href=https://kcdfrance.fr>Kubernetes Community Days France</a> vous êtes tous conviés! 👐.</p><p>Je cherchais spécifiquement une solution indépendante d'un clouder (cloud agnostic) et l'un des principaux critères était la simplicité d'utilisation. Je connaissais déjà plusieurs opérateurs Kubernetes, et j'ai fini par évaluer une solution relativement récente: <a href=https://cloudnative-pg.io/><strong>CloudNativePG</strong></a>.</p><blockquote><p>CloudNativepg est l'opérateur de Kubernetes qui couvre le cycle de vie complet d'un cluster de base de données PostgreSQL hautement disponible avec une architecture de réplication native en streaming.</p></blockquote><p>Ce projet été créé par l'entreprise <a href=https://www.enterprisedb.com/>EnterpriseDB</a> et a été soumis à la <strong>CNCF</strong> afin de rejoindre les projets <em>Sandbox</em>.</p><h2 id=-notre-objectif>🎯 Notre objectif</h2><p>Je vais donner ici une <strong>introduction</strong> aux principales fonctionnalités de CloudNativePG.</p><p>L'objectif est de:</p><ul><li>Créer une base de données PostgreSQL sur un cluster GKE,</li><li>Ajouter une instance secondaire (réplication)</li><li>Exécuter quelques tests de résilience.</li></ul><p>Nous verrons également comment tout cela se comporte en terme de performances et quels sont les outils de supervision disponibles.
 Enfin, nous allons jeter un œil aux méthodes de sauvegarde/restauration.</p><div class="notices info"><div class=label>Info</div><p>Dans cet article, nous allons tout créer et tout mettre à jour manuellement. Mais dans un environnement de production, il est conseillé d'utiliser un moteur <strong>GitOps</strong>, par exemple Flux (sujet couvert dans un <a href=/fr/post/devflux/>article précédent</a>).</p><p>Si vous souhaitez voir un <strong>exemple complet</strong>, vous pouvez consulter le dépôt git <a href=https://github.com/cncfparis/kcdfrance-gitops>KCD France infrastructure</a>.</p><p>Toutes les resources de cet article sont dans <a href=https://github.com/Smana/smana.github.io/tree/main/content/resources/cnpg>ce dépôt</a>.</p></div><h2 id=ballot_box_with_check-prérequis>&#9745;&#xfe0f; Prérequis</h2><h3 id=inbox_tray-outils>&#x1f4e5; Outils</h3><ul><li><p><strong>gcloud SDK:</strong> Nous allons déployer sur Google Cloud (en particulier sur GKE) et, pour ce faire, nous devrons créer quelques ressources dans notre projet GCP. Nous aurons donc besoin du SDK et de la CLI Google Cloud. Il est donc nécessaire de l'installer en suivant <a href=https://cloud.google.com/sdk/docs/install-sdk>cette documentation</a>.</p></li><li><p><strong>kubectl plugin:</strong> Pour faciliter la gestion des clusters, il existe un plugin <code>kubectl</code> qui donne des informations synthétiques sur l'instance PostgreSQL et permet aussi d'effectuer certaines opérations.
 Ce plugin peut être installé en utilisant <a href=https://krew.sigs.k8s.io/>krew</a>:</p></li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>kubectl krew install cnpg
 </span></span></span></code></pre></div><h3 id=-créer-les-resources-google-cloud>☁️ Créer les resources Google Cloud</h3><p>Avant de créer notre instance PostgreSQL, nous devons configurer certaines choses:</p><ul><li>Nous avons besoin d'un cluster Kubernetes. (Cet article suppose que vous avez déjà pris soin de provisionner un cluster <strong>GKE</strong>)</li><li>Nous allons créer un bucket (Google Cloud Storage) pour stocker les sauvegardes et <a href=https://www.postgresql.org/docs/15/wal-intro.html>Fichiers WAL</a>.</li><li>Nous configurerons <strong>les permissions</strong> pour nos pods afin qu'ils puissent écrire dans ce bucket.</li></ul><p>Créer le bucket à l'aide de CLI <code>gcloud</code></p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln> 1</span><span class=cl><span class=go>gcloud storage buckets create --location=eu --default-storage-class=coldline gs://cnpg-ogenki
diff --git a/fr/post/devflux/index.html b/fr/post/devflux/index.html
index cd2071e..c7304c6 100644
--- a/fr/post/devflux/index.html
+++ b/fr/post/devflux/index.html
@@ -7,9 +7,9 @@
 <span class=post_time>· 14 mins de lecture</span><span>&nbsp;· <a href=https://blog.ogenki.io/fr/tags/gitops/ title=gitops class="post_tag button button_translucent">gitops
 </a><a href=https://blog.ogenki.io/fr/tags/devxp/ title=devxp class="post_tag button button_translucent">devxp
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=100%25%20%60GitOps%60%20using%20Flux&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fdevflux%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fdevflux%2f&t=100%25%20%60GitOps%60%20using%20Flux" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/devflux/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/devflux/devflux.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-install-required-tools>Install required tools</a></li><li><a href=#key-create-a-github-personal-access-token>Create a Github personal access token</a></li><li><a href=#technologist-clone-the-devflux-repository>Clone the devflux repository</a></li></ul></li><li><a href=#rocket-bootstrap-flux-in-the-crossplane-cluster>Bootstrap flux in the Crossplane cluster</a></li><li><a href=#open_file_folder-flux-repository-structure>Flux repository structure</a></li><li><a href=#closed_lock_with_key-sealedsecrets>SealedSecrets</a><ul><li><a href=#hammer_and_wrench-deploy-the-controller-using-helm>Deploy the controller using Helm</a></li><li><a href=#-a-first-test-sealedsecret>🧪 A first test SealedSecret</a></li></ul></li><li><a href=#cloud-deploy-and-configure-crossplane>Deploy and configure Crossplane</a><ul><li><a href=#key-create-the-google-service-account-secret>Create the Google service account secret</a></li><li><a href=#-crossplane-dependencies>🔄 Crossplane dependencies</a></li><li><a href=#rocket-bootstrap-flux-in-the-dev-cluster>Bootstrap flux in the dev cluster</a></li></ul></li><li><a href=#detective-troubleshooting>Troubleshooting</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#clap-achievements>Achievements</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>In a <a href=/post/crossplane_k3d/>previous article</a>, we've seen how to use <strong>Crossplane</strong> so that we can manage cloud resources the same way as our applications. &#x2764;&#xfe0f; Declarative approach!
+<a href="https://twitter.com/intent/tweet?text=100%25%20%60GitOps%60%20using%20Flux&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fdevflux%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fdevflux%2f&t=100%25%20%60GitOps%60%20using%20Flux" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/devflux/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/devflux/devflux.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-install-required-tools>Install required tools</a></li><li><a href=#key-create-a-github-personal-access-token>Create a Github personal access token</a></li><li><a href=#technologist-clone-the-devflux-repository>Clone the devflux repository</a></li></ul></li><li><a href=#rocket-bootstrap-flux-in-the-crossplane-cluster>Bootstrap flux in the Crossplane cluster</a></li><li><a href=#open_file_folder-flux-repository-structure>Flux repository structure</a></li><li><a href=#closed_lock_with_key-sealedsecrets>SealedSecrets</a><ul><li><a href=#hammer_and_wrench-deploy-the-controller-using-helm>Deploy the controller using Helm</a></li><li><a href=#-a-first-test-sealedsecret>🧪 A first test SealedSecret</a></li></ul></li><li><a href=#cloud-deploy-and-configure-crossplane>Deploy and configure Crossplane</a><ul><li><a href=#key-create-the-google-service-account-secret>Create the Google service account secret</a></li><li><a href=#-crossplane-dependencies>🔄 Crossplane dependencies</a></li><li><a href=#rocket-bootstrap-flux-in-the-dev-cluster>Bootstrap flux in the dev cluster</a></li></ul></li><li><a href=#detective-troubleshooting>Troubleshooting</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#clap-achievements>Achievements</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>In a <a href=/post/crossplane_k3d/>previous article</a>, we've seen how to use <strong>Crossplane</strong> so that we can manage cloud resources the same way as our applications. &#x2764;&#xfe0f; Declarative approach!
 There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.</p><p>Here we'll achieve exactly the same thing but we'll do that in the <strong>GitOps</strong> way.
-According to the <a href=https://opengitops.dev/>OpenGitOps</a> working group there are 4 GitOps principles:</p><ul><li>The desired state of our system must be expressed <strong>declaratively</strong>.</li><li>This state must be stored in a <strong>versioning</strong> system.</li><li>Changes are pulled and <strong>applied automatically</strong> in the target platform whenever the desired state changes.</li><li>If, for any reason, the current state is modified, it will be <strong>automatically reconciled</strong> with the desired state.</li></ul><p>There are several GitOps engine options. The most famous ones are <a href=https://argo-cd.readthedocs.io/en/stable/>ArgoCD</a> and <a href=https://fluxcd.io/>Flux</a>. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).</p><p><figure><picture><img loading=lazy decoding=async alt=flux_components class="image_figure image_internal image_processed" width=1352 height=575 src=/fr/post/devflux/flux_components.png></picture></figure></p><p>Learn more about <strong>GitOps toolkit</strong> components <a href=https://fluxcd.io/docs/components/>here</a>.</p><h2 id=bullseye-our-target>:bullseye: Our target</h2><p>Here we want to declare our desired infrastructure components only by adding <strong>git</strong> changes.
+According to the <a href=https://opengitops.dev/>OpenGitOps</a> working group there are 4 GitOps principles:</p><ul><li>The desired state of our system must be expressed <strong>declaratively</strong>.</li><li>This state must be stored in a <strong>versioning</strong> system.</li><li>Changes are pulled and <strong>applied automatically</strong> in the target platform whenever the desired state changes.</li><li>If, for any reason, the current state is modified, it will be <strong>automatically reconciled</strong> with the desired state.</li></ul><p>There are several GitOps engine options. The most famous ones are <a href=https://argo-cd.readthedocs.io/en/stable/>ArgoCD</a> and <a href=https://fluxcd.io/>Flux</a>. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).</p><p><figure><picture><img loading=lazy decoding=async alt=flux_components class="image_figure image_internal image_processed" width=1352 height=575 src=/fr/post/devflux/flux_components.png></picture></figure></p><p>Learn more about <strong>GitOps toolkit</strong> components <a href=https://fluxcd.io/docs/components/>here</a>.</p><h2 id=-our-target>🎯 Our target</h2><p>Here we want to declare our desired infrastructure components only by adding <strong>git</strong> changes.
 By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance.
 We'll discover <a href=https://fluxcd.io/><strong>Flux</strong></a> basics and how to use it in order to build a complete GitOps CD workflow.</p><br><h2 id=ballot_box_with_check-requirements>&#9745;&#xfe0f; Requirements</h2><h3 id=inbox_tray-install-required-tools>&#x1f4e5; Install required tools</h3><p>First of all we need to install a few tools using <a href=/post/asdf/asdf/>asdf</a></p><p>Create a local file
 .tool-versions</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>cd ~/sources/devflux/
diff --git a/fr/post/tailscale/index.html b/fr/post/tailscale/index.html
index dbc3685..a0e9d35 100644
--- a/fr/post/tailscale/index.html
+++ b/fr/post/tailscale/index.html
@@ -7,7 +7,7 @@
 <span class=post_time>· 14 mins de lecture</span><span>&nbsp;· <a href=https://blog.ogenki.io/fr/tags/security/ title=security class="post_tag button button_translucent">security
 </a><a href=https://blog.ogenki.io/fr/tags/network/ title=network class="post_tag button button_translucent">network
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=S%c3%a9curiser%20le%20Cloud%20avec%20%60Tailscale%60%20%3a%20Mise%20en%20%c5%93uvre%20d%27un%20VPN%20simplifi%c3%a9e&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2ftailscale%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2ftailscale%2f&t=S%c3%a9curiser%20le%20Cloud%20avec%20%60Tailscale%60%20%3a%20Mise%20en%20%c5%93uvre%20d%27un%20VPN%20simplifi%c3%a9e" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/tailscale/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#bullseye-nos-objectifs>:bullseye: Nos objectifs</a></li><li><a href=#-sous-le-capot>💡 Sous le capot</a></li><li><a href=#inbox_tray-installation-du-client>Installation du client</a></li><li><a href=#-accéder-à-aws-en-privé>☁️ Accéder à AWS en privé</a><ul><li><a href=#-déployer-un-subnet-router>🚀 Déployer un Subnet router</a></li><li><a href=#-une-autre-façon-de-faire-du-ssh>💻 Une autre façon de faire du SSH</a></li><li><a href=#-quen-est-il-de-kubernetes>☸ Qu'en est-il de Kubernetes?</a></li></ul></li><li><a href=#-dernières-remarques>💭 Dernières remarques</a></li></ul></nav></div><div class=post_body><p>Lorsqu'on parle de sécurisation de l'accès aux ressources Cloud, l'une des règles d'or est d'éviter les expositions directes à Internet. La question qui se pose alors pour les Devs/Ops est : comment, par exemple, accéder à une base de données, un cluster Kubernetes ou un serveur via SSH sans compromettre la sécurité?</br></p><p>Les <strong>réseaux privés virtuels (VPN)</strong> offrent une réponse en établissant un lien sécurisé entre différents éléments d'un réseau, indépendamment de leur localisation géographique. De nombreuses solutions existent, allant de modèles en SaaS aux solutions que l'on peut héberger soi-même, utilisant divers protocoles et étant soit open source, soit propriétaires.</p><p>Parmi ces options, je souhaitais vous parler de <a href=https://tailscale.com/><strong>Tailscale</strong></a>. Cette solution utilise le protocole <code>WireGuard</code>, réputé pour sa simplicité et sa performance. Avec Tailscale, il est possible de connecter des appareils ou serveurs de manière sécurisée, comme s'ils étaient sur un même réseau local, bien qu'ils soient répartis à travers le monde.</p><h2 id=bullseye-nos-objectifs>:bullseye: Nos objectifs</h2><ul><li>Comprendre comment fonctionne <code>Tailscale</code></li><li>Mise en oeuvre d'une connexion sécurisée avec AWS en quelques minutes</li><li>Interragir avec l'API d'un cluster EKS via un réseau privé</li><li>Accéder à des services hébergés sur Kubernetes en utilisant le réseau privé</li></ul><center><img src=admin_console.png width=400></center><p>Pour le reste de cet article il faudra évidemment <strong>créer un compte Tailscale</strong>. A noter que l'authentification est déléguée à des fournisseurs d'identité tiers (ex: Okta, Onelogin, Google ...).</p><p>Lorsque le compte est crée, on a directement accès à la console de gestion ci-dessus. Elle permet notamment de lister les appareils connectés, de consulter les logs, de modifier la plupart des paramètres...</p><h2 id=-sous-le-capot>💡 Sous le capot</h2><div class="notices info"><div class=label>Terminologie</div><p><strong>Mesh VPN</strong>: Un <em>mesh VPN</em> est un type de réseau VPN où chaque nœud (c'est-à-dire chaque appareil ou machine) est connecté à tous les autres nœuds du réseau, formant ainsi un maillage. À distinguer des configurations VPN traditionnelles qui sont conçues généralement "en étoile", où plusieurs clients se connectent à un serveur central.</p><p><strong>Zero trust</strong>: Signifie que chaque demande d'accès à un réseau est traitée comme si elle venait d'une source non fiable. Une application ou utilisateur doit prouver son identité et être autorisée avant d'accéder à une ressource. On ne fait pas confiance simplement parce qu'une machine ou un utilisateur provient d'un réseau interne ou d'une certaine zone géographique.</p><p><strong>Tailnet</strong>: Dès la première utilisation de Tailscale, un <em>Tailnet</em> est crée pour vous et correspond à votre propre réseau privé. Chaque appareil dans un tailnet reçoit une IP Tailscale unique, permettant une communication directe entre eux.</p></div><center><img src=mesh.png width=500></center><p>L'architecture de Tailscale est conçue de telle sorte que le <em>Control plane</em> et le <em>Data plane</em> sont clairement séparés:</p><ul><li><p>D'une part, il y a le <strong>serveur de coordination</strong>. Son rôle est d'échanger des métadonnées et des clés publiques entre tous les participants d'un Tailnet (La clé privée étant gardée en toute sécurité son nœud d'origine).</p></li><li><p>D'autre part, les nœuds du Tailnet s'organisent en un <strong>réseau maillé</strong> (<em>Mesh</em>). Au lieu de passer par le serveur de coordination pour échanger des données, ces nœuds communiquent directement les uns avec les autres en mode <strong>point à point</strong>. Chaque nœud dispose d'une identité unique pour s'authentifier et rejoindre le Tailnet.</p></li></ul><h2 id=inbox_tray-installation-du-client>&#x1f4e5; Installation du client</h2><p>La majorité des plateformes sont supportées et les procédures d'installation sont listées <a href=https://tailscale.com/kb/installation/>ici</a>.
+<a href="https://twitter.com/intent/tweet?text=S%c3%a9curiser%20le%20Cloud%20avec%20%60Tailscale%60%20%3a%20Mise%20en%20%c5%93uvre%20d%27un%20VPN%20simplifi%c3%a9e&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2ftailscale%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2ftailscale%2f&t=S%c3%a9curiser%20le%20Cloud%20avec%20%60Tailscale%60%20%3a%20Mise%20en%20%c5%93uvre%20d%27un%20VPN%20simplifi%c3%a9e" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/tailscale/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#-nos-objectifs>🎯 Nos objectifs</a></li><li><a href=#-sous-le-capot>💡 Sous le capot</a></li><li><a href=#inbox_tray-installation-du-client>Installation du client</a></li><li><a href=#-accéder-à-aws-en-privé>☁️ Accéder à AWS en privé</a><ul><li><a href=#-déployer-un-subnet-router>🚀 Déployer un Subnet router</a></li><li><a href=#-une-autre-façon-de-faire-du-ssh>💻 Une autre façon de faire du SSH</a></li><li><a href=#-quen-est-il-de-kubernetes>☸ Qu'en est-il de Kubernetes?</a></li></ul></li><li><a href=#-dernières-remarques>💭 Dernières remarques</a></li></ul></nav></div><div class=post_body><p>Lorsqu'on parle de sécurisation de l'accès aux ressources Cloud, l'une des règles d'or est d'éviter les expositions directes à Internet. La question qui se pose alors pour les Devs/Ops est : comment, par exemple, accéder à une base de données, un cluster Kubernetes ou un serveur via SSH sans compromettre la sécurité?</br></p><p>Les <strong>réseaux privés virtuels (VPN)</strong> offrent une réponse en établissant un lien sécurisé entre différents éléments d'un réseau, indépendamment de leur localisation géographique. De nombreuses solutions existent, allant de modèles en SaaS aux solutions que l'on peut héberger soi-même, utilisant divers protocoles et étant soit open source, soit propriétaires.</p><p>Parmi ces options, je souhaitais vous parler de <a href=https://tailscale.com/><strong>Tailscale</strong></a>. Cette solution utilise le protocole <code>WireGuard</code>, réputé pour sa simplicité et sa performance. Avec Tailscale, il est possible de connecter des appareils ou serveurs de manière sécurisée, comme s'ils étaient sur un même réseau local, bien qu'ils soient répartis à travers le monde.</p><h2 id=-nos-objectifs>🎯 Nos objectifs</h2><ul><li>Comprendre comment fonctionne <code>Tailscale</code></li><li>Mise en oeuvre d'une connexion sécurisée avec AWS en quelques minutes</li><li>Interragir avec l'API d'un cluster EKS via un réseau privé</li><li>Accéder à des services hébergés sur Kubernetes en utilisant le réseau privé</li></ul><center><img src=admin_console.png width=400></center><p>Pour le reste de cet article il faudra évidemment <strong>créer un compte Tailscale</strong>. A noter que l'authentification est déléguée à des fournisseurs d'identité tiers (ex: Okta, Onelogin, Google ...).</p><p>Lorsque le compte est crée, on a directement accès à la console de gestion ci-dessus. Elle permet notamment de lister les appareils connectés, de consulter les logs, de modifier la plupart des paramètres...</p><h2 id=-sous-le-capot>💡 Sous le capot</h2><div class="notices info"><div class=label>Terminologie</div><p><strong>Mesh VPN</strong>: Un <em>mesh VPN</em> est un type de réseau VPN où chaque nœud (c'est-à-dire chaque appareil ou machine) est connecté à tous les autres nœuds du réseau, formant ainsi un maillage. À distinguer des configurations VPN traditionnelles qui sont conçues généralement "en étoile", où plusieurs clients se connectent à un serveur central.</p><p><strong>Zero trust</strong>: Signifie que chaque demande d'accès à un réseau est traitée comme si elle venait d'une source non fiable. Une application ou utilisateur doit prouver son identité et être autorisée avant d'accéder à une ressource. On ne fait pas confiance simplement parce qu'une machine ou un utilisateur provient d'un réseau interne ou d'une certaine zone géographique.</p><p><strong>Tailnet</strong>: Dès la première utilisation de Tailscale, un <em>Tailnet</em> est crée pour vous et correspond à votre propre réseau privé. Chaque appareil dans un tailnet reçoit une IP Tailscale unique, permettant une communication directe entre eux.</p></div><center><img src=mesh.png width=500></center><p>L'architecture de Tailscale est conçue de telle sorte que le <em>Control plane</em> et le <em>Data plane</em> sont clairement séparés:</p><ul><li><p>D'une part, il y a le <strong>serveur de coordination</strong>. Son rôle est d'échanger des métadonnées et des clés publiques entre tous les participants d'un Tailnet (La clé privée étant gardée en toute sécurité son nœud d'origine).</p></li><li><p>D'autre part, les nœuds du Tailnet s'organisent en un <strong>réseau maillé</strong> (<em>Mesh</em>). Au lieu de passer par le serveur de coordination pour échanger des données, ces nœuds communiquent directement les uns avec les autres en mode <strong>point à point</strong>. Chaque nœud dispose d'une identité unique pour s'authentifier et rejoindre le Tailnet.</p></li></ul><h2 id=inbox_tray-installation-du-client>&#x1f4e5; Installation du client</h2><p>La majorité des plateformes sont supportées et les procédures d'installation sont listées <a href=https://tailscale.com/kb/installation/>ici</a>.
 En ce qui me concerne je suis sur Archlinux:</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo pacman -S tailscale
 </span></span></span></code></pre></div><p>Il est possible de démarrer le service automatiquement au démarrage de la machine.</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo systemctl enable --now tailscaled
 </span></span></span></code></pre></div><p>Pour enregistrer son ordinateur perso, lancer la commande suivante:</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo tailscale up --accept-routes
diff --git a/fr/post/terraform-controller/index.html b/fr/post/terraform-controller/index.html
index 701778f..4226bab 100644
--- a/fr/post/terraform-controller/index.html
+++ b/fr/post/terraform-controller/index.html
@@ -6,10 +6,10 @@
 <a href=https://blog.ogenki.io/fr/ class="nav_child nav_item">Français</a></div></div><div class=follow><a href=https://github.com/Smana><svg class="icon"><title>github</title><use xlink:href="#github"/></svg></a><a href=https://twitter.com/_smana_><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href=https://www.linkedin.com/in/sma%C3%AFne-kahlouch-44374110/><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><div class=color_mode><input type=checkbox class=color_choice id=mode></div></div></div></nav></header><main><div class="grid-inverse wrap content"><article class=post_content><h1 class=post_title>Appliquer les principes de GitOps à l'infrastructure: Introduction à <code>tf-controller</code></h1><div class=post_meta><span><svg class="icon"><title>calendar</title><use xlink:href="#calendar"/></svg></span><span class=post_date>01-06-2023</span>
 <span class=post_time>· 16 mins de lecture</span><span>&nbsp;· <a href=https://blog.ogenki.io/fr/tags/infrastructure/ title=infrastructure class="post_tag button button_translucent">infrastructure
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Partager sur:
-<a href="https://twitter.com/intent/tweet?text=Appliquer%20les%20principes%20de%20GitOps%20%c3%a0%20l%27infrastructure%3a%20Introduction%20%c3%a0%20%60tf-controller%60&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fterraform-controller%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fterraform-controller%2f&t=Appliquer%20les%20principes%20de%20GitOps%20%c3%a0%20l%27infrastructure%3a%20Introduction%20%c3%a0%20%60tf-controller%60" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/terraform-controller/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/terraform-controller/tf-controller.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#bullseye-notre-objectif>:bullseye: Notre objectif</a></li><li><a href=#-installer-le-controleur-terraform>🛠️ Installer le controleur Terraform</a><ul><li><a href=#-le-cluster-control-plane>☸ Le cluster "Control Plane"</a></li><li><a href=#-le-chart-helm-et-flux>📦 Le chart Helm et Flux</a></li></ul></li><li><a href=#-appliquer-un-changement>🚀 Appliquer un changement</a><ul><li><a href=#-détection-de-la-dérive>🚨 Détection de la dérive</a></li><li><a href=#-application-manuelle>🔧 Application manuelle</a></li><li><a href=#-application-automatique>🤖 Application automatique</a></li><li><a href=#-entrées-et-sorties-dépendances-entre-modules>🔄 Entrées et sorties: dépendances entre modules</a></li></ul></li><li><a href=#-sauvegarder-et-restaurer-un-tfstate>💾 Sauvegarder et restaurer un tfstate</a></li><li><a href=#-focus-sur-certaines-fonctionnalités-de-flux>🔍 Focus sur certaines fonctionnalités de Flux</a><ul><li><a href=#substition-de-variables>Substition de variables</a></li><li><a href=#web-ui-weave-gitops>Web UI (Weave GitOps)</a></li></ul></li><li><a href=#-remarques>💭 Remarques</a></li></ul></nav></div><div class=post_body><p><strong>Terraform</strong> est probablement l'outil "Infrastructure As Code" le plus utilisé pour construire, modifier et versionner les changements d'infrastructure Cloud.
+<a href="https://twitter.com/intent/tweet?text=Appliquer%20les%20principes%20de%20GitOps%20%c3%a0%20l%27infrastructure%3a%20Introduction%20%c3%a0%20%60tf-controller%60&url=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fterraform-controller%2f&tw_p=tweetbutton" class=twitter title="Partager sur Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2ffr%2fpost%2fterraform-controller%2f&t=Appliquer%20les%20principes%20de%20GitOps%20%c3%a0%20l%27infrastructure%3a%20Introduction%20%c3%a0%20%60tf-controller%60" class=facebook title="Partager sur Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Partager sur LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/fr/post/terraform-controller/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/fr/post/terraform-controller/tf-controller.png></picture></figure></div><div class=post_toc><h2>Sommaire</h2><nav id=TableOfContents><ul><li><a href=#-notre-objectif>🎯 Notre objectif</a></li><li><a href=#-installer-le-controleur-terraform>🛠️ Installer le controleur Terraform</a><ul><li><a href=#-le-cluster-control-plane>☸ Le cluster "Control Plane"</a></li><li><a href=#-le-chart-helm-et-flux>📦 Le chart Helm et Flux</a></li></ul></li><li><a href=#-appliquer-un-changement>🚀 Appliquer un changement</a><ul><li><a href=#-détection-de-la-dérive>🚨 Détection de la dérive</a></li><li><a href=#-application-manuelle>🔧 Application manuelle</a></li><li><a href=#-application-automatique>🤖 Application automatique</a></li><li><a href=#-entrées-et-sorties-dépendances-entre-modules>🔄 Entrées et sorties: dépendances entre modules</a></li></ul></li><li><a href=#-sauvegarder-et-restaurer-un-tfstate>💾 Sauvegarder et restaurer un tfstate</a></li><li><a href=#-focus-sur-certaines-fonctionnalités-de-flux>🔍 Focus sur certaines fonctionnalités de Flux</a><ul><li><a href=#substition-de-variables>Substition de variables</a></li><li><a href=#web-ui-weave-gitops>Web UI (Weave GitOps)</a></li></ul></li><li><a href=#-remarques>💭 Remarques</a></li></ul></nav></div><div class=post_body><p><strong>Terraform</strong> est probablement l'outil "Infrastructure As Code" le plus utilisé pour construire, modifier et versionner les changements d'infrastructure Cloud.
 Il s'agit d'un projet Open Source développé par Hashicorp et qui utilise le langage <a href=https://github.com/hashicorp/hcl>HCL</a> pour déclarer l'état souhaité de ressources Cloud.
 L'état des ressources créées est stocké dans un fichier d'état (terraform state).</p><p>On peut considérer que Terraform est un outil "semi-déclaratif" car il n'y a pas de fonctionnalité de <strong>réconciliation automatique</strong> intégrée. Il existe différentes approches pour répondre à cette problématique, mais en règle générale, une modification sera appliquée en utilisant <code>terraform apply</code>. Le code est bien décrit dans des fichiers de configuration HCL (déclaratif) mais l'exécution est faite de manière impérative.
-De ce fait, il peut y avoir de la dérive entre l'état déclaré et le réel (par exemple, un collègue qui serait passé sur la console pour changer un paramètre 😉).</p><p>❓❓ Alors, comment m'assurer que ce qui est commit dans mon repo git est vraiment appliqué. Comment être alerté s'il y a un changement par rapport à l'état désiré et comment appliquer automatiquement ce qui est dans mon code (GitOps) ?</p><p>C'est la promesse de <a href=https://github.com/weaveworks/tf-controller><strong>tf-controller</strong></a>, un operateur Kubernetes Open Source de Weaveworks, étroitement lié à Flux (un moteur GitOps de la même société). <a href=https://fluxcd.io/><strong>Flux</strong></a> est l'une des solutions que je plébiscite, et je vous invite donc à lire un <a href=https://blog.ogenki.io/post/devflux/>précédent article</a>.</p><div class="notices info"><div class=label>Info</div><p>L'ensemble des étapes décrites ci-dessous sont faites avec ce <a href=https://github.com/Smana/demo-tf-controller><strong>repo Git</strong></a></p></div><h2 id=bullseye-notre-objectif>:bullseye: Notre objectif</h2><p>En suivant les étapes de cet article nous visons les objectifs suivant:</p><ul><li>Déployer un cluster Kubernetes qui servira de "<strong>Control plane</strong>". Pour résumer il hébergera le controlleur Terraform qui nous permettra de déclarer tous les éléments d'infrastructure souhaités.</li><li>Utiliser <strong>Flux</strong> comme moteur GitOps pour toutes les ressources Kubernetes.</li></ul><p>Concernant le controleur Terraform, nous allons voir:</p><ul><li>Quelle est le moyen de définir des <strong>dépendances</strong> entre modules</li><li>Création de <strong>plusieurs ressources</strong> AWS: Zone route53, Certificat ACM, réseau, cluster EKS.</li><li>Les différentes options de <strong>reconciliation</strong> (automatique, nécessitant une confirmation)</li><li>Comment sauvegarder et <strong>restaurer</strong> un fichier d'état (tfstate)</li></ul><h2 id=-installer-le-controleur-terraform>🛠️ Installer le controleur Terraform</h2><h3 id=-le-cluster-control-plane>☸ Le cluster "Control Plane"</h3><p>Afin de pouvoir utiliser le controleur Kubernetes <code>tf-controller</code>, il nous faut d'abord un cluster Kubernetes 😆.
+De ce fait, il peut y avoir de la dérive entre l'état déclaré et le réel (par exemple, un collègue qui serait passé sur la console pour changer un paramètre 😉).</p><p>❓❓ Alors, comment m'assurer que ce qui est commit dans mon repo git est vraiment appliqué. Comment être alerté s'il y a un changement par rapport à l'état désiré et comment appliquer automatiquement ce qui est dans mon code (GitOps) ?</p><p>C'est la promesse de <a href=https://github.com/weaveworks/tf-controller><strong>tf-controller</strong></a>, un operateur Kubernetes Open Source de Weaveworks, étroitement lié à Flux (un moteur GitOps de la même société). <a href=https://fluxcd.io/><strong>Flux</strong></a> est l'une des solutions que je plébiscite, et je vous invite donc à lire un <a href=https://blog.ogenki.io/post/devflux/>précédent article</a>.</p><div class="notices info"><div class=label>Info</div><p>L'ensemble des étapes décrites ci-dessous sont faites avec ce <a href=https://github.com/Smana/demo-tf-controller><strong>repo Git</strong></a></p></div><h2 id=-notre-objectif>🎯 Notre objectif</h2><p>En suivant les étapes de cet article nous visons les objectifs suivant:</p><ul><li>Déployer un cluster Kubernetes qui servira de "<strong>Control plane</strong>". Pour résumer il hébergera le controlleur Terraform qui nous permettra de déclarer tous les éléments d'infrastructure souhaités.</li><li>Utiliser <strong>Flux</strong> comme moteur GitOps pour toutes les ressources Kubernetes.</li></ul><p>Concernant le controleur Terraform, nous allons voir:</p><ul><li>Quelle est le moyen de définir des <strong>dépendances</strong> entre modules</li><li>Création de <strong>plusieurs ressources</strong> AWS: Zone route53, Certificat ACM, réseau, cluster EKS.</li><li>Les différentes options de <strong>reconciliation</strong> (automatique, nécessitant une confirmation)</li><li>Comment sauvegarder et <strong>restaurer</strong> un fichier d'état (tfstate)</li></ul><h2 id=-installer-le-controleur-terraform>🛠️ Installer le controleur Terraform</h2><h3 id=-le-cluster-control-plane>☸ Le cluster "Control Plane"</h3><p>Afin de pouvoir utiliser le controleur Kubernetes <code>tf-controller</code>, il nous faut d'abord un cluster Kubernetes 😆.
 Nous allons donc créer un cluster <strong>control plane</strong> en utilisant la ligne de commande <code>terraform</code> et les bonnes pratiques EKS.</p><div class="notices warning"><div class=label>Warning</div><p>Il est primordial que ce cluster soit résiliant, sécurisé et supervisé car il sera responsable de la gestion de l'ensemble des ressources AWS créées par la suite.</p></div><p>Sans entrer dans le détail, le cluster "control plane" a été créé un utilisant <a href=https://github.com/Smana/demo-tf-controller/tree/main/terraform/controlplane>ce code</a>. Celà-dit, il est important de noter que toutes les opérations de déploiement d'application se font en utilisant Flux.</p><div class="notices info"><div class=label>Info</div><p>En suivant les instructions du <a href=https://github.com/Smana/demo-tf-controller/blob/main/terraform/controlplane/README.md>README</a>, un cluster EKS sera créé mais pas uniquement!</br>Il faut en effet donner les permissions au controlleur Terraform pour appliquer les changements d'infrastructure.
 De plus, Flux doit être installé et configuré afin d'appliquer la configuration définie <a href=https://github.com/Smana/demo-tf-controller/tree/main/clusters/controlplane-0>ici</a>.</p><p>Au final on se retrouve donc avec plusieurs éléments installés et configurés:</p><ul><li>les addons quasi indispensables que sont <code>aws-loadbalancer-controller</code> et <code>external-dns</code></li><li>les roles <a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> pour ces mêmes composants sont installés en utilisant <code>tf-controller</code></li><li>La stack de supervision Prometheus / Grafana.</li><li><code>external-secrets</code> pour pouvoir récupérer des éléments sensibles depuis AWS secretsmanager.</li><li>Afin de démontrer tout cela au bout de quelques minutes l'interface web pour Flux est accessible via l'URL <code>gitops-&lt;cluster_name>.&lt;domain_name</code>></li></ul><p>Vérifier toute de même que le cluster est accessible et que Flux fonctionne correctement</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0
 </span></span></span><span class=line><span class=ln>2</span><span class=cl><span class=go>Updated context controlplane-0 in /home/smana/.kube/config
diff --git a/index.json b/index.json
index 776975d..64feae5 100644
--- a/index.json
+++ b/index.json
@@ -1 +1 @@
-[{"body":"","link":"https://blog.ogenki.io/","section":"","tags":null,"title":""},{"body":"When we talk about securing access to cloud resources, one of the golden rules is to avoid direct exposure to the Internet. This raises a question for Dev/Ops: How can we , for instance, access a database, a Kubernetes cluster, or a server via SSH without compromising security? Virtual Private Networks (VPN) offer an answer by establishing a secure link between different elements of a network, regardless of their geographical location. Solutions range from SaaS models to self-hosted platforms, using various protocols, either open-source or proprietary.\nAmong these options, I wanted to introduce you to Tailscale. This solution uses WireGuard under the hood, known for its simplicity and performance. With Tailscale, you can securely connect devices or servers as if they were on the same local network, even though they are spread across the globe.\n:bullseye: Our target Understanding how Tailscale works Implementing a secure connection with AWS in a few minutes Interacting with an EKS cluster API via a private network Accessing services hosted on Kubernetes using the private network. Throughout this article, you will obviously need to create a Tailscale account. It's worth noting that authentication is delegated to third-party identity providers (e.g., Okta, OneLogin, Google).\nOnce the account is created, you'll have instant access to the management console. It allows you to list connected devices, visualize logs, and change most of the settings.\n💡 Under the Hood Terminology Mesh VPN: A mesh VPN refers to a VPN network configuration in which every node (i.e., each device or machine) connects directly to every other node, creating a mesh-like structure. This stands in contrast to traditional VPN configurations that usually adopt a \u0026quot;hub and spoke\u0026quot; topology, where multiple clients connect to one central server.\nZero Trust: This approach operates on the principle that every access request to a network is inherently untrusted. Whether an application or user, identity must be proven and authorization given before accessing a resource. Trust isn't assumed based solely on originating from an internal network or a specific geographic location.\nTailnet: When you first use Tailscale, a Tailnet is automatically generated for you, representing your personal private network. Every device within this tailnet is assigned a unique Tailscale IP.\nTailscale's architecture clearly distinguishes between the Control plane and the Data plane:\nThe coordination server primarily manages the exchange of metadata and public keys among all Tailnet members. Importantly, by Tailscale's design, only the public key is shared, with the private key securely retained on its originating node.\nThe nodes in the Tailnet establish a mesh network: Instead of passing data through the coordination server, these nodes communicate directly in a peer-to-peer manner. Each node has a unique identity, allowing it to authenticate and become part of the Tailnet.\n\u0026#x1f4e5; Client Installation Most platforms are supported, and the installation options are listed here. As for me, I am on Archlinux:\n1sudo pacman -S tailscale It's possible to start the service automatically upon machine startup.\n1sudo systemctl enable --now tailscaled To register your personal computer, run the following command:\n1sudo tailscale up --accept-routes 2 3To authenticate, visit: 4 5 https://login.tailscale.com/a/f50... ℹ️ The --accept-routes option is required on Linux in order to configure the routes announced by Subnet routers. We'll delve deeper into this later on in this post.\nEnsure that you've indeed acquired an IP from the Tailscale network:\n1tailscale ip -4 2100.118.83.67 3 4tailscale status 5100.118.83.67 ogenki smainklh@ linux - ℹ️ For Linux users, ensure that Tailscale works well with your DNS configuration: Follow this documentation.\nSources All the steps performed in this article come from this git repository.\nIt will allow you to create all the components aiming to set up an EKS Lab cluster, following a previous blog post on Cilium and Gateway API.\n☁️ Accessing AWS Privately To securely access all resources available on AWS, one can deploy a Subnet router.\nA Subnet router is a Tailscale instance that provides access to subnets not directly linked to Tailscale. It acts as a bridge between Tailscale's virtual private network (Tailnet) and other local networks.\nWe can then reach Cloud subnets through Tailscale's VPN.\n⚠️ To do so, on AWS, you'll need to configure the security groups appropriately to allow Subnet routers.\n🚀 Deploying a Subnet Router Let's dive in and deploy a Subnet router on an AWS network! Everything is done using the Terraform code present in the directory terraform/network. We will analyze the Tailscale-specific configuration present in the tailscale.tf file before deploying.\nThe Terraform provider You can configure several settings via the API Tailscale using the Terraform provider. First, you need to generate an API key 🔑 from the admin console:\nYou should keep this key in a secure store as it is used to deploy the Subnet router.\n1provider \u0026#34;tailscale\u0026#34; { 2 api_key = var.tailscale.api_key 3 tailnet = var.tailscale.tailnet 4} The ACL's\nAccess Control Lists allow you to define who is authorized to communicate with whom (user or device). Upon account creation, they are very permissive, and there are no restrictions (everyone can communicate with everyone).\n1resource \u0026#34;tailscale_acl\u0026#34; \u0026#34;this\u0026#34; { 2 acl = jsonencode({ 3 acls = [ 4 { 5 action = \u0026#34;accept\u0026#34; 6 src = [\u0026#34;*\u0026#34;] 7 dst = [\u0026#34;*:*\u0026#34;] 8 } 9 ] 10... 11} Note For my Lab environment, I kept this default configuration because I'm the only person accessing it. Furthermore, the only devices connected to my Tailnet are my laptop and the Subnet router. However, in a corporate setting, you need to think this through. It's then possible to define a security policy based on groups of people or node tags.\nCheck out this doc for more info.\nDomain Names\nThere are various ways to manage DNS with Tailscale:\nMagic DNS: When a device joins the Tailnet, it registers with a name, and this can be immediately used to communicate with the device.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.115.31.152 ip-10-0-43-98 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 3044 rx 2588 4 5ping ip-10-0-43-98 6PING ip-10-0-43-98.tail9c382.ts.net (100.115.31.152) 56(84) bytes of data. 764 bytes from ip-10-0-43-98.tail9c382.ts.net (100.115.31.152): icmp_seq=1 ttl=64 time=11.4 ms AWS: To use AWS internal domain names, you can utilize the second IP of the VPC, which always corresponds to the DNS server (In our case it would be 10.0.0.2). This enables the use of potential private zones on route53 or connection to resources using domain names.\nThe simplest configuration is thus to declare the list of DNS servers to use and add the AWS one. Here's an example with Cloudflare's public DNS.\n1resource \u0026#34;tailscale_dns_nameservers\u0026#34; \u0026#34;this\u0026#34; { 2 nameservers = [ 3 \u0026#34;1.1.1.1\u0026#34;, 4 cidrhost(module.vpc.vpc_cidr_block, 2) 5 ] 6} The Authentication Key\nFor a device to join the Tailnet at startup, Tailscale must be started using an authentication key. It is generated as follows:\n1resource \u0026#34;tailscale_tailnet_key\u0026#34; \u0026#34;this\u0026#34; { 2 reusable = true 3 ephemeral = false 4 preauthorized = true 5} reusable: As we're using an autoscaling group which can be composed of multiple instances, this key should be reusable multiple times. ephemeral: For this demo, we have created a non-expiring key. In a production environment, enabling expiration would be preferable. preauthorized: This key needs to be valid and preauthorized so that the instance can automatically join Tailscale. The generated key is then used to launch tailscale with the --auth-key parameter.\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; Announcing Routes for AWS subnets\nIt's also necessary to announce the network that we wish to route through the Subnet router. In our example, we choose to route the entire VPC network with the CIDR 10.0.0.0/16.\nTo automate this process, an autoApprovers rule needs to be added. This indicates that the routes announced by the user smainklh@gmail.com are authorized without requiring an approval step.\n1 autoApprovers = { 2 routes = { 3 \u0026#34;10.0.0.0/16\u0026#34; = [\u0026#34;smainklh@gmail.com\u0026#34;] 4 } 5 } Here is the Taiscale command run in the Subnet router:\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; --advertise-routes=\u0026#34;10.0.0.0/16\u0026#34; The Terraform Module I've created a module that is straightforward and allows deploying an autoscaling group on AWS and configuring Tailscale. Upon the instance's startup, it will authenticate using an auth_key and announce the specified networks. In the example below, the instance announces the CIDR of the VPC on AWS.\n1module \u0026#34;tailscale_subnet_router\u0026#34; { 2 source = \u0026#34;Smana/tailscale-subnet-router/aws\u0026#34; 3 version = \u0026#34;1.0.4\u0026#34; 4 5 region = var.region 6 env = var.env 7 8 name = var.tailscale.subnet_router_name 9 auth_key = tailscale_tailnet_key.this.key 10 11 vpc_id = module.vpc.vpc_id 12 subnet_ids = module.vpc.private_subnets 13 advertise_routes = [module.vpc.vpc_cidr_block] 14... 15} Now that we've examined the various parameters, it's time to start our Subnet router 🚀 !! First, you need to create a variable.tfvars file in the terraform/network directory.\n1env = \u0026#34;dev\u0026#34; 2region = \u0026#34;eu-west-3\u0026#34; 3private_domain_name = \u0026#34;priv.cloud.ogenki.io\u0026#34; 4 5tailscale = { 6 subnet_router_name = \u0026#34;ogenki\u0026#34; 7 tailnet = \u0026#34;smainklh@gmail.com\u0026#34; 8 api_key = \u0026#34;tskey-api-...\u0026#34; 9} 10 11tags = { 12 project = \u0026#34;demo-secured-eks\u0026#34; 13 owner = \u0026#34;Smana\u0026#34; 14} Then run the following command:\n1tofu plan --var-file variables.tfvars After checking the plan, apply the changes as follows\n1tofu apply --var-file variables.tfvars When the instance starts up, it will appear in the list of devices on the Tailnet.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.68.109.138 ip-10-0-26-99 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 33868 rx 32292 We can also check that the route is correctly announced as follows:\n1tailscale status --json|jq \u0026#39;.Peer[] | select(.HostName == \u0026#34;ip-10-0-26-99\u0026#34;) .PrimaryRoutes\u0026#39; 2[ 3 \u0026#34;10.0.0.0/16\u0026#34; 4] ⚠️ For security reasons, do not forget to delete the variables.tfvars file as it contains the API key.\n👏 That's it! We are now able to access the network on AWS, provided we've also set up the filtering rules, such as ACLs and security groups. For instance, we can access a database from the workstation.\n1psql -h demo-tailscale.cymnaynfchjt.eu-west-3.rds.amazonaws.com -U postgres 2Password for user postgres: 3psql (15.4, server 15.3) 4SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off) 5Type \u0026#34;help\u0026#34; for help. 6 7postgres=\u0026gt; 💻 A Different Way to SSH Traditionally, we often connect to servers using the SSH protocol. This involves generating a private key and distributing the corresponding public key to remote servers.\nUnlike traditional SSH key usage, Tailscale uses Wireguard for authentication and encrypted connections, eliminating the need to re-authenticate the client. Furthermore, Tailscale handles the distribution of SSH host keys. Through ACL rules, user access can be revoked without the need to delete SSH keys. There's also a mode that can be activated to enhance security by requiring periodic re-authentication. It's evident that Tailscale SSH simplifies authentication, streamlines SSH connection management, and enhance security levels.\nTo achieve this, one must also initiate Tailscale with the --ssh option. Permissions for SSH are managed at the ACL level as follows:\n1... 2 ssh = [ 3 { 4 action = \u0026#34;check\u0026#34; 5 src = [\u0026#34;autogroup:member\u0026#34;] 6 dst = [\u0026#34;autogroup:self\u0026#34;] 7 users = [\u0026#34;autogroup:nonroot\u0026#34;] 8 } 9 ] 10... The rule mentioned above allows all users to access their own devices via SSH. When trying to connect, they must use a user account other than root. Each connection attempt mandates an additional authentication (action=check). This authentication is carried out by visiting a specific web link.\n1ssh ubuntu@ip-10-0-26-99 2... 3# Tailscale SSH requires an additional check. 4# To authenticate, visit: https://login.tailscale.com/a/f1f09a548cc6 5... 6ubuntu@ip-10-0-26-99:~$ Access logs to the machine can be viewed using journalctl.\n1ubuntu@ip-10-0-26-99:~$ journalctl -aeu tailscaled|grep ssh 2Oct 15 15:51:34 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155130-00ede660b8: handling conn: 100.118.83.67:55098-\u0026gt;ubuntu@100.68.109.138:22 3Oct 15 15:51:56 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: handling conn: 100.118.83.67:44560-\u0026gt;ubuntu@100.68.109.138:22 4Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: starting session: sess-20231015T155252-5b2acc170e 5Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): handling new SSH connection from smainklh@gmail.com (100.118.83.67) to ssh-user \u0026#34;ubuntu\u0026#34; 6Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): access granted to smainklh@gmail.com as ssh-user \u0026#34;ubuntu\u0026#34; 7Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): starting pty command: [/usr/sbin/tailscaled be-child ssh --uid=1000 --gid=1000 --groups=1000,4,20,24,25,27,29,30,44,46,115,116 --local-user=ubuntu --remote-user=smainklh@gmail.com --remote-ip=100.118.83.67 --has-tty=true --tty-name=pts/0 --shell --login-cmd=/usr/bin/login --cmd=/bin/bash -- -l] ℹ️ With Tailscale SSH, you can SSH into a device no matter where it's located. However, in a 100% AWS context, one would likely prefer using AWS SSM.\nLogs 💾 In security, retaining logs for future reference is vital. There are various types of logs:\nAudit Logs: These provide insights into who did what. They can be accessed through the admin console and can also be forwarded to a SIEM.\nDevice Logs: Specific commands to each device can be used to view these, such as (journalctl -u tailscaled on Linux).\nNetwork Logs: These are useful for visualizing the connections between devices.\n☸ What About Kubernetes? In the context of Kubernetes, there are several options for accessing a Service:\nProxy: This method involves an additional pod that forwards calls to an existing Service. Sidecar: This allows a pod to connect to the Tailnet. As a result, end-to-end connectivity is established, making bi-directional communication possible (from the pod to the Tailnet nodes). Operator: This approach facilitates exposing Kubernetes services and the API (ingress) and permits pods to access Tailnet nodes (egress). Configuration is achieved by modifying existing resources: Services and Ingresses. In our setup, we already have a Subnet router that routes the entire VPC network. As such, it's sufficient for our service to be exposed via a private IP.\nThe Kubernetes API To access the Kubernetes API, it's essential to authorize the Subnet router. This is accomplished by setting the following rule for the source security group.\nterraform/eks/main.tf\n1module \u0026#34;eks\u0026#34; { 2... 3 cluster_security_group_additional_rules = { 4 ingress_source_security_group_id = { 5 description = \u0026#34;Ingress from the Tailscale security group to the API server\u0026#34; 6 protocol = \u0026#34;tcp\u0026#34; 7 from_port = 443 8 to_port = 443 9 type = \u0026#34;ingress\u0026#34; 10 source_security_group_id = data.aws_security_group.tailscale.id 11 } 12 } 13... 14} Let's ensure that the API is indeed accessible on a private IP.\n1CLUSTER_URL=$(TERM=dumb kubectl cluster-info | grep \u0026#34;Kubernetes control plane\u0026#34; | awk \u0026#39;{print $NF}\u0026#39;) 2 3curl -s -o /dev/null -w \u0026#39;%{remote_ip}\\n\u0026#39; ${CLUSTER_URL} 410.228.244.167 5 6kubectl get ns 7NAME STATUS AGE 8cilium-secrets Active 5m46s 9crossplane-system Active 4m1s 10default Active 23m 11flux-system Active 5m29s 12infrastructure Active 4m1s 13... Accessing Kubernetes Services Privately An exposed Kubernetes Service is just another AWS network resource 😉. We simply need to ensure that this service uses a private IP. In my example, I'm using the Gateway API to configure Clouder's load balancing, and I encourage you to read my previous article on the subject.\nAll that's needed is to create an internal NLB, ensuring that the Service has the annotation service.beta.kubernetes.io/aws-load-balancer-scheme set to internal. With Cilium Gateway API, this is achieved via the Kyverno clusterPolicy.\n1 metadata: 2 annotations: 3 external-dns.alpha.kubernetes.io/hostname: gitops-${cluster_name}.priv.${domain_name},grafana-${cluster_name}.priv.${domain_name} 4 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internal\u0026#34; 5 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 6 spec: 7 loadBalancerClass: service.k8s.aws/nlb However, there's an additional prerequisite since we can't use Let's Encrypt for private DNS zones. I've thus generated an internal PKI that produces self-signed certificates with Cert-manager.\nIn this post, I won't delve into the details of the EKS cluster deployment or the configuration of Flux. Once the cluster is created and all the Kubernetes resources have been reconciled, we have a service that is exposed via an internal AWS LoadBalancer.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2dig +short ${NLB_DOMAIN} 310.0.33.5 410.0.26.228 510.0.9.183 An DNS record is also automatically created for the exposed services, allowing us to access privately through the Flux web interface.\n1dig +short gitops-mycluster-0.priv.cloud.ogenki.io 210.0.9.183 310.0.26.228 410.0.33.5 💭 Final Thoughts Some time ago, in a professional context, I implemented Cloudflare Zero Trust. I've noticed here that Tailscale shares many similarities with this solution. Of course, there are other solutions like Teleport, and the choice of a solution for private access to one's infrastructure depends on the context and security goals.\nFor me, I was particularly convinced by the simplicity of Tailscale's implementation, perfectly meeting my need to access the Clouder network.\nPart of Tailscale's code is open source, especially the client which is under the BSD 3-Clause license. The proprietary part mainly concerns the coordination platform. It's worth noting that there's an open-source alternative named Headscale. This is a distinct initiative with no connection to the Tailscale company.\nFor personal use, Tailscale is quite generous, offering free access for up to 100 devices and 3 users. That said, Tailscale is a serious option to consider for businesses, and in my opinion, it's essential to support companies that have a clear open-source policy and a quality product.\n","link":"https://blog.ogenki.io/post/tailscale/","section":"post","tags":["security","network"],"title":"Beyond Traditional VPNs: Simplifying Cloud Access with `Tailscale`"},{"body":"","link":"https://blog.ogenki.io/tags/network/","section":"tags","tags":null,"title":"network"},{"body":"","link":"https://blog.ogenki.io/post/","section":"post","tags":["index"],"title":"Posts"},{"body":"","link":"https://blog.ogenki.io/tags/security/","section":"tags","tags":null,"title":"security"},{"body":"","link":"https://blog.ogenki.io/tags/","section":"tags","tags":null,"title":"Tags"},{"body":"When deploying an application on Kubernetes, the next step usually involves making it accessible to users. We commonly use Ingress controllers, such as Nginx, Haproxy, Traefik, or those from Cloud providers, to direct incoming traffic to the application, manage load balancing, TLS termination, and more.\nThen we have to choose from the plethora of available options 🤯. Cilium is, relatively recently, one of them and aims to handle all these networking aspects.\nCilium is an Open-Source networking and security solution based on eBPF whose adoption is growing rapidly. It's probably the network plugin that provides the most features. We won't cover all of them, but one such feature involves managing incoming traffic using the Gateway API (GAPI).\n:bullseye: Our target Understand exactly what the Gateway API is and how it represents an evolution from the Ingress API. Demonstrations of real-world scenarios deployed the GitOps way. Current limitations and upcoming developments. Tip All the steps carried out in this article come from this git repository.\nI encourage you to explore it, as it goes far beyond the context of this article:\nInstallation of an EKS cluster with Cilium configured with the kube-proxy replacement enbled and a dedicated Daemonset for Envoy. Proposal of a Flux structure with dependency management and a DRY code I think is efficient. Crossplane and IRSA composition which simplifies the management of IAM permissions for platform components. Automated domain names and certificates management with External-DNS and Let's Encrypt. The idea being to have everything set up in just a few minutes, with a single command line 🤩.\n☸ Introduction to Gateway API As mentioned previously, there are many Ingress Controllers options, and each has its own specificities and particular features, sometimes making their use complex. Furthermore, the traditionnal Ingress API in Kubernetes has very limited parameters. Some solutions have even created their own CRDs (Kubernetes Custom Resources) while others use annotations to overcome these limitations.\nHere comes the Gateway API! This is actually a standard that allows declaring advanced networking features without requiring specific extensions to the underlying controller. Moreover, since all controllers use the same API, it is possible to switch from one solution to another without changing the configuration (The Kubenetes manifests which describe how the incoming traffic should be routed).\nAmong the concepts that we will explore, GAPI brings a granular authorization model which defines explicit roles with distinct permissions. (More information on the GAPI security model here).\nThis is worth noting that this project is driven by the sig-network-kubernetes working group, and there's a slack channel where you can reach out to them if needed.\nLet's see how GAPI is used in practice with Cilium 🚀!\n\u0026#x2611;\u0026#xfe0f; Prerequisites For the remainder of this article, we assume an EKS cluster has been deployed. If you're not using the method suggested in the demo repo as the basis for this article, there are a few points to check for GAPI to be usable.\nℹ️ The installation method described here is based on Helm, all the values can be viewed here.\nInstall the CRDs available in the Gateway API repository. Note If Cilium is set up with GAPI support (see below) and the CRDs are missing, it won't start. In the demo repo, the GAPI CRDs are installed once during the cluster creation so that Cilium can start, and then they are managed by Flux.\nReplace kube-proxy with the network forwarding features provided by Cilium and eBPF.\n1kubeProxyReplacement: true Enable Gateway API support. 1gatewayAPI: 2 enabled: true Check the installation For that you need to install the command line tool cilium. I personnaly use asdf:\n1asdf plugin-add cilium-cli 2asdf install cilium-cli 0.15.7 3asdf global cilium 0.15.7 The following command allows to ensure that all the components are up and running:\n1cilium status --wait 2 /¯¯\\ 3/¯¯\\__/¯¯\\ Cilium: OK 4\\__/¯¯\\__/ Operator: OK 5/¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6\\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled 8 9Deployment cilium-operator Desired: 2, Ready: 2/2, Available: 2/2 10DaemonSet cilium Desired: 2, Ready: 2/2, Available: 2/2 11DaemonSet cilium-envoy Desired: 2, Ready: 2/2, Available: 2/2 12Containers: cilium Running: 2 13 cilium-operator Running: 2 14 cilium-envoy Running: 2 15Cluster Pods: 33/33 managed by Cilium 16Helm chart version: 1.14.2 17Image versions cilium quay.io/cilium/cilium:v1.14.2@sha256:6263f3a3d5d63b267b538298dbeb5ae87da3efacf09a2c620446c873ba807d35: 2 18 cilium-operator quay.io/cilium/operator-aws:v1.14.2@sha256:8d514a9eaa06b7a704d1ccead8c7e663334975e6584a815efe2b8c15244493f1: 2 19 cilium-envoy quay.io/cilium/cilium-envoy:v1.25.9-e198a2824d309024cb91fb6a984445e73033291d@sha256:52541e1726041b050c5d475b3c527ca4b8da487a0bbb0309f72247e8127af0ec: 2 Finally you can check that the Gateway API support is enabled by running\n1cilium config view | grep -w \u0026#34;enable-gateway-api\u0026#34; 2enable-gateway-api true 3enable-gateway-api-secrets-sync true You could also run end to end tests as follows\n1cilium connectivity test ⚠️ However this command (connectivity test) currently throws errors with Envoy as a DaemonSet enabled. (Github Issue).\nInfo as DaemonSet\nBy default, the Cilium agent also runs Envoy within the same pod and delegates to it level 7 network operations. Since the version v1.14, it is possible to deploy Envoy separately, which brings several benefits:\nIf one modifies/restarts a component (whether it's Cilium or Envoy), it doesn't affect the other. Better allocate resources to each component to optimize performance. Limits the attack surface in case of compromise of one of the pods. Envoy logs and Cilium agent logs are not mixed. You can use the following command to check that this feature is indeed active:\n1cilium status 2 /¯¯\\ 3 /¯¯\\__/¯¯\\ Cilium: OK 4 \\__/¯¯\\__/ Operator: OK 5 /¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6 \\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled More info.\n🚪 The Entry Point: GatewayClass and Gateway Once the conditions are met, we have access to several elements. We can make use of the custom resources defined by the Gateway API CRDs. Moreover, right after installing Cilium, a GatewayClass is immediately available.\n1kubectl get gatewayclasses.gateway.networking.k8s.io 2NAME CONTROLLER ACCEPTED AGE 3cilium io.cilium/gateway-controller True 7m59s On a Kubernetes cluster, you could configure multiple GatewayClasses, thus having the ability to use different implementations. For instance, we can use Linkerd by referencing the GatewayClass in the Gateway configuration.\nThe Gateway is the resource that allows triggering the creation of load balancing components in the Cloud provider.\nHere's a simple example: apps/base/echo/gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo-gateway 5 namespace: echo 6spec: 7 gatewayClassName: cilium 8 listeners: 9 - protocol: HTTP 10 port: 80 11 name: echo-1-echo-server 12 allowedRoutes: 13 namespaces: 14 from: Same On AWS (EKS), when configuring a Gateway, Cilium creates a Service of type LoadBalancer. Then another controller (The AWS Load Balancer Controller) handles the creation of the Cloud load balancer (NLB)\n1kubectl get svc -n echo cilium-gateway-echo 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3cilium-gateway-echo LoadBalancer 172.20.19.82 k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com 80:30395/TCP 2m58s This is worth noting that the load balancer address is also linked to the Gateway.\n1kubectl get gateway -n echo echo 2NAME CLASS ADDRESS PROGRAMMED AGE 3echo cilium k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com True 16m \u0026#x21aa;\u0026#xfe0f; Routing rules: HTTPRoute A basic rule To summarize the above diagram in a few words: An HTTPRoute allows configuring the routing to the service by referencing the Gateway and defining the desired routing parameters.\nNote workaround\nAs of now, it is not possible to configure the annotations of services generated by the Gateways (Github Issue). A workaround has been proposed to modify the service generated by the Gateway as soon as it is created.\nKyverno is a tool that ensures configuration compliance with best practices and security requirements. We are using it here solely for its ability to easily describe a mutation rule.\nsecurity/mycluster-0/echo-gw-clusterpolicy.yaml\n1spec: 2 rules: 3 - name: mutate-svc-annotations 4 match: 5 any: 6 - resources: 7 kinds: 8 - Service 9 namespaces: 10 - echo 11 name: cilium-gateway-echo 12 mutate: 13 patchStrategicMerge: 14 metadata: 15 annotations: 16 external-dns.alpha.kubernetes.io/hostname: echo.${domain_name} 17 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internet-facing\u0026#34; 18 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 19 spec: 20 loadBalancerClass: service.k8s.aws/nlb The service cilium-gateway-echo will therefore have the AWS controller's annotations added, as well as an annotation allowing for automatic DNS record configuration.\napps/base/echo/httproute.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 rules: 11 - matches: 12 - path: 13 type: PathPrefix 14 value: / 15 backendRefs: 16 - name: echo-1-echo-server 17 port: 80 The example used above is very simple: all requests are forwarded to the echo-1-echo-server service.\nparentRefs indicates which Gateway to use and then the routing rules are defined under the rules section.\nThe routing rules could also be based on the path.\n1... 2spec: 3 hostnames: 4 - foo.bar.com 5 rules: 6 - matches: 7 - path: 8 type: PathPrefix 9 value: /login Or based on an HTTP Header\n1... 2spec: 3 rules: 4 - matches: 5 headers: 6 - name: \u0026#34;version\u0026#34; 7 value: \u0026#34;2\u0026#34; 8... Let's check if the service is reachable.:\n1curl -s http://echo.cloud.ogenki.io | jq -rc \u0026#39;.environment.HOSTNAME\u0026#39; 2echo-1-echo-server-fd88497d-w6sgn As you can see, the service is exposed in HTTP without a certificate. Let's try to fix that 😉\nConfigure a TLS certificate There are several methods to configure TLS with GAPI. Here, we will use the most common case: HTTPS protocol and TLS termination at the Gateway.\nLet's assume we want to configure the domain name echo.cloud.ogenki.io used earlier. The configuration is mainly done by configuring the Gateway.\napps/base/echo/tls-gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo 5 namespace: echo 6 annotations: 7 cert-manager.io/cluster-issuer: letsencrypt-prod 8spec: 9 gatewayClassName: cilium 10 listeners: 11 - name: http 12 hostname: \u0026#34;echo.${domain_name}\u0026#34; 13 port: 443 14 protocol: HTTPS 15 allowedRoutes: 16 namespaces: 17 from: Same 18 tls: 19 mode: Terminate 20 certificateRefs: 21 - name: echo-tls The essential point here is the reference to a secret containing the certificate named echo-tls. This certificate can be created manually, but for this article, I chose to automate this with Let's Encrypt and cert-manager.\nInfo cert-manager\nWith cert-manager, it's pretty straightforward to automate the creation and update of certificates exposed by the Gateway. For this, you need to allow the controller to access route53 in order to solve a DNS01 challenge (A mechanism that ensures that clients can only request certificates for domains they own).\nA ClusterIssuer resource describes the required configuration to generate certificates with cert-manager.\nNext, we just need to add an annotation cert-manager.io/cluster-issuer and set the Kubernetes secret where the certificate will be stored.\nMore information\nℹ️ In the demo repo, permissions are assigned using Crossplane, which takes care of configuring these IAM perms in AWS.\nFor routing to work correctly, you also need to attach the HTTPRoute to the right Gateway and specify the domain name.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 hostnames: 11 - \u0026#34;echo.${domain_name}\u0026#34; 12... After a few seconds the certificate will be created.\n1kubectl get cert -n echo 2NAME READY SECRET AGE 3echo-tls True echo-tls 43m Finally, we can check that the certificate indeed comes from Let's Encrypt as follows:\n1curl https://echo.cloud.ogenki.io -v 2\u0026gt;\u0026amp;1 | grep -A 6 \u0026#39;Server certificate\u0026#39; 2* Server certificate: 3* subject: CN=echo.cloud.ogenki.io 4* start date: Sep 15 14:43:00 2023 GMT 5* expire date: Dec 14 14:42:59 2023 GMT 6* subjectAltName: host \u0026#34;echo.cloud.ogenki.io\u0026#34; matched cert\u0026#39;s \u0026#34;echo.cloud.ogenki.io\u0026#34; 7* issuer: C=US; O=Let\u0026#39;s Encrypt; CN=R3 8* SSL certificate verify ok. Info GAPI also allows you to configure end-to-end TLS, all the way to the container. This is done by configuring the Gateway in Passthrough mode and using a TLSRoute resource. The certificate must also be carried by the pod that performs the TLS termination.\nSharing a Gateway accross multiple namespaces With GAPI, you can route traffic across Namespaces. This is made possible thanks to distinct resources for each function: A Gateway that allows configuring the infrastructure, and the *Routes. These routes can be attached to a Gateway located in another namespace. It is thus possible for different teams/projects to share the same infrastructure components.\nHowever, this requires to specify which route is allowed to reference the Gateway. Here we assume that we have a Gateway dedicated to internal tools called platform. By using the allowedRoutes parameter, we explicitly specify which namespaces are allowed to be attached to this Gateway.\ninfrastructure/base/gapi/platform-gateway.yaml\n1... 2 allowedRoutes: 3 namespaces: 4 from: Selector 5 selector: 6 matchExpressions: 7 - key: kubernetes.io/metadata.name 8 operator: In 9 values: 10 - observability 11 - flux-system 12 tls: 13 mode: Terminate 14 certificateRefs: 15 - name: platform-tls The HTTPRoutes configured in the namespaces observability and flux-system are attached to this unique Gateway.\n1... 2spec: 3 parentRefs: 4 - name: platform 5 namespace: infrastructure And therefore, use the same load balancer from the Cloud provider.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2 3dig +short ${NLB_DOMAIN} 413.36.89.108 5 6dig +short grafana-mycluster-0.cloud.ogenki.io 713.36.89.108 8 9dig +short gitops-mycluster-0.cloud.ogenki.io 1013.36.89.108 Note 🔒 These internal tools shouldn't be exposed on the Internet, but you know: this is just a demo 🙏. For instance, we could use an internal Gateway (private IP) by playing with the annotations and make use of a private connection system (VPN, tunnels...).\nTraffic splitting One feature that is commonly brought by Service Meshes is the ability to test an application on a portion of the traffic when a new version is available (A/B testing or Canary deployment). GAPI makes this quite simple by using weights.\nHere's an example that forwards 5% of the traffic to the service echo-2-echo-server:\napps/base/echo/httproute-split.yaml\n1... 2 hostnames: 3 - \u0026#34;split-echo.${domain_name}\u0026#34; 4 rules: 5 - matches: 6 - path: 7 type: PathPrefix 8 value: / 9 backendRefs: 10 - name: echo-1-echo-server 11 port: 80 12 weight: 95 13 - name: echo-2-echo-server 14 port: 80 15 weight: 5 Let's check that the distribution happens as expected:\nscripts/check-split.sh\n1./scripts/check-split.sh https://split-echo.cloud.ogenki.io 2Number of requests for echo-1: 95 3Number of requests for echo-2: 5 Headers modifications It is also possible to change HTTP Headers: to add, modify, or delete them. These modifications can be applied to either request or response headers through the use of filters in the HTTPRoute manifest.\nFor instance, we will add a Header to the request.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7... 8 rules: 9 - matches: 10 - path: 11 type: PathPrefix 12 value: /req-header-add 13 filters: 14 - type: RequestHeaderModifier 15 requestHeaderModifier: 16 add: 17 - name: foo 18 value: bar 19 backendRefs: 20 - name: echo-1-echo-server 21 port: 80 22... This command allows to that the header is indeed added:\n1curl -s https://echo.cloud.ogenki.io/req-header-add | jq \u0026#39;.request.headers\u0026#39; 2{ 3 \u0026#34;host\u0026#34;: \u0026#34;echo.cloud.ogenki.io\u0026#34;, 4 \u0026#34;user-agent\u0026#34;: \u0026#34;curl/8.2.1\u0026#34;, 5 \u0026#34;accept\u0026#34;: \u0026#34;*/*\u0026#34;, 6 \u0026#34;x-forwarded-for\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 7 \u0026#34;x-forwarded-proto\u0026#34;: \u0026#34;https\u0026#34;, 8 \u0026#34;x-envoy-external-address\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 9 \u0026#34;x-request-id\u0026#34;: \u0026#34;320ba4d2-3bd6-4c2f-8a97-74296a9f3f26\u0026#34;, 10 \u0026#34;foo\u0026#34;: \u0026#34;bar\u0026#34; 11} 🔒 Assign the proper permissions GAPI offers a clear permission-sharing model between the traffic routing infrastructure (managed by cluster administrators) and the applications (managed by developers).\nThe availability of multiple custom resources allows to use Kubernete's RBAC configuration to assign permissions in a declarative way. I've added a few examples which have no effect in my demo cluster but might give you an idea.\nThe configuration below grants members of the developers group the ability to manage HTTPRoutes within the echo namespace, while only providing them read access to the Gateways.\n1--- 2apiVersion: rbac.authorization.k8s.io/v1 3kind: Role 4metadata: 5 namespace: echo 6 name: gapi-developer 7rules: 8 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 9 resources: [\u0026#34;httproutes\u0026#34;] 10 verbs: [\u0026#34;*\u0026#34;] 11 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 12 resources: [\u0026#34;gateways\u0026#34;] 13 verbs: [\u0026#34;get\u0026#34;, \u0026#34;list\u0026#34;] 14--- 15apiVersion: rbac.authorization.k8s.io/v1 16kind: RoleBinding 17metadata: 18 name: gapi-developer 19 namespace: echo 20subjects: 21 - kind: Group 22 name: \u0026#34;developers\u0026#34; 23 apiGroup: rbac.authorization.k8s.io 24roleRef: 25 kind: Role 26 name: gapi-developer 27 apiGroup: rbac.authorization.k8s.io 🤔 A somewhat unclear scope at first glance One could be confused with what's commonly referred to as an API Gateway. A section of the FAQ has been created to clarify its difference with the Gateway API. Although GAPI offers features typically found in an API Gateway, it primarily serves as a specific implementation for Kubernetes. However, the choice of this name can indeed cause confusion.\nMoreover please note that this article focuses solely on inbound traffic, termed north-south, traditionally managed by Ingress Controllers. This traffic is actually GAPI's initial scope. A recent initiative named GAMMA aims to also handle east-west routing, which will standardize certain features commonly provided by Service Meshes solutions in the future. (See this article for more details).\n💭 Final thoughts To be honest, I've known about the Gateway API for some time. Although I've read a few articles, I hadn't truly dived deep. I'd think, \u0026quot;Why bother? My Ingress Controller works, and there's a learning curve with this.\u0026quot;\nGAPI is on the rise and nearing its GA release. Several projects have embraced it, and this API for managing traffic within Kubernetes will quickly become the standard.\nI must say, configuring GAPI felt intuitive and explicit ❤️. Its security model strikes a balance, empowering developers without compromising security. And the seamless infrastructure management? You can switch between implementations without touching the *Routes.\nWould I swap my Ingress Controller for Cilium today? Not yet, but it's on the horizon.\nIt's worth highlighting Cilium's broad range of capabilities: With Kubernetes surrounded by a plethora of tools, Cilium stands out, promising features like metrics, tracing, service-mesh, security, and, yes, Ingress Controller with GAPI.\nHowever, there are a few challenges to note:\nTCP and UDP support GRPC support The need to use a mutation rule to configure cloud components (Github Issue). Many of the features discussed in this blog are still in the experimental stage. For instance, the extended functions, which have been supported since the most recent release at the time of my writing (v1.14.2). I attempted to set up a straightforward HTTP\u0026gt;HTTPS redirect but ran into this issue. Consequently, I expect some modifications to the API in the near future. While I've only scratched the surface of what Cilium's GAPI can offer (honestly, this post is already quite long 😜), I am hopeful that we can consider its use in production soon. But considering the points mentioned earlier, I would advise waiting a bit longer. That said if you want to prepare the future, now's the time 😉!\n🔖 References https://gateway-api.sigs.k8s.io/ https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#gs-gateway-api https://isovalent.com/blog/post/cilium-gateway-api/ https://isovalent.com/blog/post/tutorial-getting-started-with-the-cilium-gateway-api/ Isovalent's labs are great to start playing with Gateway API and you'll get new badges to add to your collection 😄 ","link":"https://blog.ogenki.io/post/cilium-gateway-api/","section":"post","tags":["kubernetes","infrastructure","network"],"title":"`Gateway API`: Can I replace my Ingress Controller with `Cilium`?"},{"body":"","link":"https://blog.ogenki.io/tags/infrastructure/","section":"tags","tags":null,"title":"infrastructure"},{"body":"","link":"https://blog.ogenki.io/tags/kubernetes/","section":"tags","tags":null,"title":"kubernetes"},{"body":"Terraform is probably the most used \u0026quot;Infrastructure As Code\u0026quot; tool for building, modifying, and versioning Cloud infrastructure changes. It is an Open Source project developed by Hashicorp that uses the HCL language to declare the desired state of Cloud resources. The state of the created resources is stored in a file called terraform state.\nTerraform can be considered a \u0026quot;semi-declarative\u0026quot; tool as there is no built-in automatic reconciliation feature. There are several solutions to address this issue, but generally speaking, a modification will be applied using terraform apply. The code is actually written using the HCL configuration files (declarative), but the execution is done imperatively. As a result, there can be a drift between the declared and actual state (for example, a colleague who would have changed something directly into the console 😉).\n❓❓ So, how can I ensure that what is committed using Git is really applied. How to be notified if there is a change compared to the desired state and how to automatically apply what is in my code (GitOps)?\nThis is the promise of tf-controller, an Open Source Kubernetes operator from Weaveworks, tightly related to Flux (a GitOps engine from the same company). Flux is one of the solutions I really appreciate, that's why I invite you to have a look on my previous article\nInfo All the steps described in this article come from this Git repo\n:bullseye: Our target By following the steps in this article, we aim to achieve the following things:\nDeploy a Control plane EKS cluster. Long story short, it will host the Terraform controller that will be in charge of managing all the desired infrastructure components. Use Flux as the GitOps engine for all Kubernetes resources. Regarding the Terraform controller, we will see:\nHow to define dependencies between modules Creation of several AWS resources: Route53 Zone, ACM Certificate, network, EKS cluster. The different reconciliation options (automatic, requiring confirmation) How to backup and restore a Terraform state. 🛠️ Install the tf-controller ☸ The control plane In order to be able to use the Kubernetes controller tf-controller, we first need a Kubernetes cluster 😆. So we are going to create a control plane cluster using the terraform command line and EKS best practices.\nWarning It is crucial that this cluster is resilient, secure, and supervised as it will be responsible for managing all the AWS resources created subsequently.\nWithout going into detail, the control plane cluster was created using this code. That said, it is important to note that all application deployment operations are done using Flux.\nInfo By following the instructions in the README, an EKS cluster will be created but not only! Indeed, it is required to give permissions to the Terraform controller so it will able to apply infrastructure changes. Furthermore, Flux must be installed and configured to apply the configuration defined here.\nWe end up with several components installed and configured:\nThe almost unavoidable addons: aws-loadbalancer-controller and external-dns IRSA roles for these same components are installed using tf-controller Prometheus / Grafana monitoring stack. external-secrets to be able to retrieve sensitive data from AWS secretsmanager. To demonstrate all this after a few minutes the web interface for Flux is accessible via the URL gitops-\u0026lt;cluster_name\u0026gt;.\u0026lt;domain_name\u0026gt;\nStill you should check that Flux is working properly\n1aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0 2Updated context controlplane-0 in /home/smana/.kube/config 1flux check 2... 3✔ all checks passed 4 5flux get kustomizations 6NAME REVISION SUSPENDED READY MESSAGE 7flux-config main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 8flux-system main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 9infrastructure main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 10security main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 11tf-controller main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 12... 📦 The Helm chart and Flux Now that the control plane cluster is available we can add the Terraform controller and this is just the matter of using the Helm chart as follows.\nWe must declare the its Source first:\nsource.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: tf-controller 5spec: 6 interval: 30m 7 url: https://weaveworks.github.io/tf-controller Then we need to define the HelmRelease:\nrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: tf-controller 5spec: 6 releaseName: tf-controller 7 chart: 8 spec: 9 chart: tf-controller 10 sourceRef: 11 kind: HelmRepository 12 name: tf-controller 13 namespace: flux-system 14 version: \u0026#34;0.12.0\u0026#34; 15 interval: 10m0s 16 install: 17 remediation: 18 retries: 3 19 values: 20 resources: 21 limits: 22 memory: 1Gi 23 requests: 24 cpu: 200m 25 memory: 500Mi 26 runner: 27 serviceAccount: 28 annotations: 29 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/tfcontroller_${cluster_name}\u0026#34; When this change is actually written into Git, the HelmRelease will be deployed and the tf-controller started:\n1kubectl get hr -n flux-system 2NAME AGE READY STATUS 3tf-controller 67m True Release reconciliation succeeded 4 5kubectl get po -n flux-system -l app.kubernetes.io/instance=tf-controller 6NAME READY STATUS RESTARTS AGE 7tf-controller-7ffdc69b54-c2brg 1/1 Running 0 2m6s In this demo, there are already a several AWS resources declared. Therefore, after a few minutes, the cluster takes care of creating these: Info Although the majority of operations are performed declaratively or via the CLIs kubectl and flux, another tool allows to manage Terraform resources: tfctl\n🚀 Apply a change One of the Terraform's best practices is to use modules. A module is a set of logically linked Terraform resources bundled into a single reusable unit. They allow to abstract complexity, take inputs, perform specific actions, and produce outputs.\nYou can create your own modules and make them available as Sources or use the many modules shared and maintained by communities. You just need to specify a few variables in order to fit to your context.\nWith tf-controller, the first step is therefore to define the Source of the module. Here we are going to configure the AWS base networking components (vpc, subnets...) using the terraform-aws-vpc module.\n1apiVersion: source.toolkit.fluxcd.io/v1 2kind: GitRepository 3metadata: 4 name: terraform-aws-vpc 5 namespace: flux-system 6spec: 7 interval: 30s 8 ref: 9 tag: v5.0.0 10 url: https://github.com/terraform-aws-modules/terraform-aws-vpc Then we can make use of this Source within a Terraform resource:\nvpc/dev.yaml\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6 interval: 8m 7 path: . 8 destroyResourcesOnDeletion: true # You wouldn\u0026#39;t do that on a prod env ;) 9 storeReadablePlan: human 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-vpc 13 namespace: flux-system 14 vars: 15 - name: name 16 value: vpc-dev 17 - name: cidr 18 value: \u0026#34;10.42.0.0/16\u0026#34; 19 - name: azs 20 value: 21 - \u0026#34;eu-west-3a\u0026#34; 22 - \u0026#34;eu-west-3b\u0026#34; 23 - \u0026#34;eu-west-3c\u0026#34; 24 - name: private_subnets 25 value: 26 - \u0026#34;10.42.0.0/19\u0026#34; 27 - \u0026#34;10.42.32.0/19\u0026#34; 28 - \u0026#34;10.42.64.0/19\u0026#34; 29 - name: public_subnets 30 value: 31 - \u0026#34;10.42.96.0/24\u0026#34; 32 - \u0026#34;10.42.97.0/24\u0026#34; 33 - \u0026#34;10.42.98.0/24\u0026#34; 34 - name: enable_nat_gateway 35 value: true 36 - name: single_nat_gateway 37 value: true 38 - name: private_subnet_tags 39 value: 40 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 41 \u0026#34;karpenter.sh/discovery\u0026#34;: dev 42 - name: public_subnet_tags 43 value: 44 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 45 writeOutputsToSecret: 46 name: vpc-dev In summary: the terraform code from the terraform-aws-vpc source is applied using the variables defined within vars.\nThere are then several parameters that influence the tf-controller behavior. The main parameters that control how modifications are applied are .spec.approvePlan and .spec.autoApprove\n🚨 Drift detection Setting spec.approvePlan to disable only notifies that the current state of resources has drifted from the Terraform code. This allows you to choose when and how to apply the changes.\nNote This is worth noting that there is a missing section on notifications: Drift, pending plans, reconciliation problems. I'm trying to identify possible methods (preferably with Prometheus) and update this article as soon as possible.\n🔧 Manual execution The given example above (vpc-dev) does not contain the .spec.approvePlan parameter and therefore inherits the default value which is false. In other words, the actual execution of changes (apply) is not done automatically.\nA plan is executed and will be waiting for validation:\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system vpc-dev Unknown Plan generated: set approvePlan: \u0026#34;plan-v5.0.0-26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. true 2 minutes I also advise to configure the storeReadablePlan parameter to human. This allows you to easily visualize the pending modifications using tfctl:\n1tfctl show plan vpc-dev 2 3Terraform used the selected providers to generate the following execution 4plan. Resource actions are indicated with the following symbols: 5 + create 6 7Terraform will perform the following actions: 8 9 # aws_default_network_acl.this[0] will be created 10 + resource \u0026#34;aws_default_network_acl\u0026#34; \u0026#34;this\u0026#34; { 11 + arn = (known after apply) 12 + default_network_acl_id = (known after apply) 13 + id = (known after apply) 14 + owner_id = (known after apply) 15 + tags = { 16 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 17 } 18 + tags_all = { 19 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 20 } 21 + vpc_id = (known after apply) 22 23 + egress { 24 + action = \u0026#34;allow\u0026#34; 25 + from_port = 0 26 + ipv6_cidr_block = \u0026#34;::/0\u0026#34; 27 + protocol = \u0026#34;-1\u0026#34; 28 + rule_no = 101 29 + to_port = 0 30 } 31 + egress { 32... 33Plan generated: set approvePlan: \u0026#34;plan-v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. 34To set the field, you can also run: 35 36 tfctl approve vpc-dev -f filename.yaml After reviewing the above modifications, you just need to add the identifier of the plan to validate and push the change to git as follows:\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6... 7 approvePlan: plan-v5.0.0-26c38a66f1 8... After a few seconds, a runner will be launched and will apply the changes:\n1kubectl logs -f -n flux-system vpc-dev-tf-runner 22023/07/01 15:33:36 Starting the runner... version sha 3... 4aws_vpc.this[0]: Creating... 5aws_vpc.this[0]: Still creating... [10s elapsed] 6... 7aws_route_table_association.private[1]: Creation complete after 0s [id=rtbassoc-01b7347a7e9960a13] 8aws_nat_gateway.this[0]: Still creating... [10s elapsed] As soon as the apply is finished the status of the Terraform resource becomes \u0026quot;READY\u0026quot;\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m 🤖 Automatic reconciliation We can also enable automatic reconciliation. To do this, set the .spec.autoApprove parameter to true.\nAll IRSA resources are configured in this way:\nexternal-secrets.yaml\n1piVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: irsa-external-secrets 5spec: 6 approvePlan: auto 7 destroyResourcesOnDeletion: true 8 interval: 8m 9 path: ./modules/iam-role-for-service-accounts-eks 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-iam 13 namespace: flux-system 14 vars: 15 - name: role_name 16 value: ${cluster_name}-external-secrets 17 - name: attach_external_secrets_policy 18 value: true 19 - name: oidc_providers 20 value: 21 main: 22 provider_arn: ${oidc_provider_arn} 23 namespace_service_accounts: [\u0026#34;security:external-secrets\u0026#34;] So if I make any change on the AWS console for example, it will be quickly overwritten by the one managed by tf-controller.\nInfo The deletion policy of components created by a Terraform resource is controlled by the setting destroyResourcesOnDeletion. By default anything created is not destroyed by the controller. If you want to destroy the resources when the Terraform object is deleted you must set this parameter to true.\nHere we want to be able to delete IRSA roles because they're tightly linked to a given EKS cluster\n🔄 Inputs/Outputs and modules dependencies When using Terraform, we often need to share data from one module to another. This is done using the outputs that are defined within modules. So we need a way to store them somewhere and import them into another module.\nLet's take again the given example above (vpc-dev). We can see at the bottom of the YAML file, the following block:\n1... 2 writeOutputsToSecret: 3 name: vpc-dev When this resource is applied, we will get a message confirming that the outputs are available (\u0026quot;Outputs written\u0026quot;):\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m Indeed this module exports many information (126).\n1kubectl get secrets -n flux-system vpc-dev 2NAME TYPE DATA AGE 3vpc-dev Opaque 126 15s 4 5kubectl get secret -n flux-system vpc-dev --template=\u0026#39;{{.data.vpc_id}}\u0026#39; | base64 -d 6vpc-0c06a6d153b8cc4db Some of these are then used to create a dev EKS cluster. Note that you don't have to read them all, you can cherry pick a few chosen outputs from the secret:\nvpc/dev.yaml\n1... 2 varsFrom: 3 - kind: Secret 4 name: vpc-dev 5 varsKeys: 6 - vpc_id 7 - private_subnets 8... 💾 Backup and restore a tfstate For my demos, I don't want to recreate the zone and the certificate each time the control plane is destroyed (The DNS propagation and certificate validation take time). Here is an example of the steps to take so that I can restore the state of these resources when I use this demo.\nNote This is a manual procedure to demonstrate the behavior of tf-controller with respect to state files. By default, these tfstates are stored in secrets, but we would prefer to configure a GCS or S3 backend.\nThe initial creation of the demo environment allowed me to save the state files (tfstate) as follows.\n1WORKSPACE=\u0026#34;default\u0026#34; 2STACK=\u0026#34;route53-cloud-hostedzone\u0026#34; 3BACKUPDIR=\u0026#34;${HOME}/tf-controller-backup\u0026#34; 4 5mkdir -p ${BACKUPDIR} 6 7kubectl get secrets -n flux-system tfstate-${WORKSPACE}-${STACK} -o jsonpath=\u0026#39;{.data.tfstate}\u0026#39; | \\ 8base64 -d \u0026gt; ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz When the cluster is created again, tf-controller tries to create the zone because the state file is empty.\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system route53-cloud-hostedzone Unknown Plan generated: set approvePlan: \u0026#34;plan-main@sha1:345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. true 16 minutes 5 6tfctl show plan route53-cloud-hostedzone 7 8Terraform used the selected providers to generate the following execution 9plan. Resource actions are indicated with the following symbols: 10 + create 11 12Terraform will perform the following actions: 13 14 # aws_route53_zone.this will be created 15 + resource \u0026#34;aws_route53_zone\u0026#34; \u0026#34;this\u0026#34; { 16 + arn = (known after apply) 17 + comment = \u0026#34;Experimentations for blog.ogenki.io\u0026#34; 18 + force_destroy = false 19 + id = (known after apply) 20 + name = \u0026#34;cloud.ogenki.io\u0026#34; 21 + name_servers = (known after apply) 22 + primary_name_server = (known after apply) 23 + tags = { 24 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 25 } 26 + tags_all = { 27 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 28 } 29 + zone_id = (known after apply) 30 } 31 32Plan: 1 to add, 0 to change, 0 to destroy. 33 34Changes to Outputs: 35 + domain_name = \u0026#34;cloud.ogenki.io\u0026#34; 36 + nameservers = (known after apply) 37 + zone_arn = (known after apply) 38 + zone_id = (known after apply) 39 40Plan generated: set approvePlan: \u0026#34;plan-main@345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. 41To set the field, you can also run: 42 43 tfctl approve route53-cloud-hostedzone -f filename.yaml So we need to restore the terraform state as it was when the cloud resources where initially created.\n1cat \u0026lt;\u0026lt;EOF | kubectl apply -f - 2apiVersion: v1 3kind: Secret 4metadata: 5 name: tfstate-${WORKSPACE}-${STACK} 6 namespace: flux-system 7 annotations: 8 encoding: gzip 9type: Opaque 10data: 11 tfstate: $(cat ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz | base64 -w 0) 12EOF You will also need to trigger a plan manually\n1tfctl replan route53-cloud-hostedzone 2 Replan requested for flux-system/route53-cloud-hostedzone 3Error: timed out waiting for the condition We can then check that the state file has been updated\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3flux-system route53-cloud-hostedzone True Outputs written: main@sha1:d0934f979d832feb870a8741ec01a927e9ee6644 false 19 minutes 🔍 Focus on Key Features of Flux Well, I lied a bit about the agenda 😝. Indeed I want to highlight two features that I hadn't explored until now and that are very useful!\nVariable Substitution When Flux is initialized, some cluster-specific Kustomization files are applied. It is possible to specify variable substitutions within these files, so that they can be used in all resources deployed by this Kustomization. This helps to minimize code duplication.\nI recently discovered the efficiency of this feature. Here is how I use it:\nThe Terraform code that creates an EKS cluster also generates a ConfigMap that contains cluster-specific variables such as the cluster name, as well as all the parameters that vary between clusters.\nflux.tf\n1resource \u0026#34;kubernetes_config_map\u0026#34; \u0026#34;flux_clusters_vars\u0026#34; { 2 metadata { 3 name = \u0026#34;eks-${var.cluster_name}-vars\u0026#34; 4 namespace = \u0026#34;flux-system\u0026#34; 5 } 6 7 data = { 8 cluster_name = var.cluster_name 9 oidc_provider_arn = module.eks.oidc_provider_arn 10 aws_account_id = data.aws_caller_identity.this.account_id 11 region = var.region 12 environment = var.env 13 vpc_id = module.vpc.vpc_id 14 } 15 depends_on = [flux_bootstrap_git.this] 16} As mentioned previously, variable substitutions are defined in the Kustomization files. Let's take a concrete example. Below, we define the Kustomization that deploys all the resources controlled by the tf-controller. Here, we declare the eks-controlplane-0-vars ConfigMap that was generated during the EKS cluster creation.\ninfrastructure.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1 2kind: Kustomization 3metadata: 4 name: tf-custom-resources 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 path: ./infrastructure/controlplane-0/terraform/custom-resources 10 postBuild: 11 substitute: 12 domain_name: \u0026#34;cloud.ogenki.io\u0026#34; 13 substituteFrom: 14 - kind: ConfigMap 15 name: eks-controlplane-0-vars 16 - kind: Secret 17 name: eks-controlplane-0-vars 18 optional: true 19 sourceRef: 20 kind: GitRepository 21 name: flux-system 22 dependsOn: 23 - name: tf-controller Finally, below is an example of a Kubernetes resource that makes use of it. This single manifest can be used by all clusters!.\nexternal-dns/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: external-dns 5spec: 6... 7 values: 8 global: 9 imageRegistry: public.ecr.aws 10 fullnameOverride: external-dns 11 aws: 12 region: ${region} 13 zoneType: \u0026#34;public\u0026#34; 14 batchChangeSize: 1000 15 domainFilters: [\u0026#34;${domain_name}\u0026#34;] 16 logFormat: json 17 txtOwnerId: \u0026#34;${cluster_name}\u0026#34; 18 serviceAccount: 19 annotations: 20 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/${cluster_name}-external-dns\u0026#34; This will reduce significantly the number of overlays that were used to patch with cluster-specific parameters.\nWeb UI (Weave GitOps) In my previous article on Flux, I mentioned one of its downsides (when compared to its main competitor, ArgoCD): the lack of a Web interface. While I am a command line guy, this is sometimes useful to have a consolidated view and the ability to perform some operations with just a few clicks \u0026#x1f5b1;\u0026#xfe0f;\nThis is now possible with Weave GitOps! Of course, it is not comparable to ArgoCD's UI, but the essentials are there: pausing reconciliation, visualizing manifests, dependencies, events...\nThere is also the VSCode plugin as an alternative.\n💭 Final thoughts One might say \u0026quot;yet another infrastructure management tool from Kubernetes\u0026quot;. Well this is true but despite a few minor issues faced along the way, which I shared on the project's Git repo, I really enjoyed the experience. tf-controller provides a concrete answer to a common question: how to manage our infrastructure like we manage our code?\nI really like the GitOps approach applied to infrastructure, and I had actually written an article on Crossplane. tf-controller tackles the problem from a different angle: using Terraform directly. This means that we can leverage our existing knowledge and code. There's no need to learn a new way of declaring our resources. This is an important criterion to consider because migrating to a new tool when you already have an existing infrastructure represents a significant effort. However, I would also add that tf-controller is only targeted at Flux users, which restricts its target audience.\nCurrently, I'm using a combination of Terraform, Terragrunt and RunAtlantis. tf-controller could become a serious alternative: We've talked about the value of Kustomize combined with variable substitions to avoid code deduplication. The project's roadmap also aims to display plans in pull-requests. Another frequent need is to pass sensitive information to modules. Using a Terraform resource, we can inject variables from Kubernetes secrets. This makes it possible to use common secrets management tools, such as external-secrets, sealed-secrets ...\nSo, I encourage you to try tf-controller yourself, and perhaps even contribute to it 🙂\nNote The demo I used create quite a few resources, some of which are quite critical (like the network). So, keep in mind that this is just for the demo! I suggest taking a gradual approach if you plan to implement it: start by using drift detection, then create simple resources. I also took some shortcuts in terms of security that should be avoided, such as giving admin rights to the controller. ","link":"https://blog.ogenki.io/post/terraform-controller/","section":"post","tags":["infrastructure"],"title":"Applying GitOps Principles to Infrastructure: An overview of `tf-controller`"},{"body":"Kubernetes is now the de facto platform for orchestrating stateless applications. Containers that don't store data can be destroyed and easily recreated elsewhere. On the other hand, running persistent applications in an ephemeral environment can be quite challenging. There is an increasing number of mature cloud-native database solutions (like CockroachDB, TiDB, K8ssandra, Strimzi...) and there are a lot of things to consider when evaluating them:\nHow mature is the operator? What do the CRDs look like, which options, settings, and status do they expose? Which Kubernetes storage APIs does it leverage? (PV/PVC, CSI, snapshots...) Can it differentiate HDD and SSD, local/remote storage? What happens when something goes wrong: how resilient is the system? Backup and recovery: how easy is it to perform and schedule backups? What replication and scaling options are available? What about connection and concurrency limits, connection pooling, bouncers? Observability: what metrics are exposed and how? I was looking for a solution to host a PostgreSQL database. This database is a requirement for a ticket reservation software named Alf.io that's being used for an upcoming event: The Kubernetes Community Days France. (By the way you're welcome to submit a talk 👐, the CFP closes soon).\nI was specifically looking for a cloud-agnostic solution, with emphasis on ease of use. I was already familiar with several Kubernetes operators, and I ended up evaluating a fairly new kid on the block: CloudNativePG.\nCloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.\nIt has been created by the company EnterpriseDB, who submitted it to the CNCF in order to join the Sandbox projects.\n:bullseye: Our target I'm going to give you an introduction to the main CloudNativePG features. The plan is to:\ncreate a PostgreSQL database on a GKE cluster, add a standby instance, run a few resiliency tests. We will also see how it behaves in terms of performances and what are the observability tools available. Finally we'll have a look to the backup/restore methods.\nInfo In this article, we will create and update everything manually; but in production, we probably should use a GitOps engine, for instance Flux (which has been covered in a previous article).\nIf you want to see a complete end-to-end example, you can look at the KCD France infrastructure repository.\nAll the manifests shown in this article can be found in this repository.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Tooling gcloud SDK: we're going to deploy on Google Cloud (specifically, on GKE) and we will need to create a few resources in our GCP project; so we'll need the Google Cloud SDK and CLI. If needed, you can install and configure it using this documentation.\nkubectl plugin: to facilitate cluster management, CloudNativePG comes with a handy kubectl plugin that gives insights of your PostgreSQL instance and allows to perform some operations. It can be installed using krew as follows:\n1kubectl krew install cnpg ☁️ Create the Google cloud requirements Before creating our PostgreSQL instance, we need to configure a few things:\nWe need a Kubernetes cluster. This article assumes that you have already taken care of provisioning a GKE cluster. We'll create a bucket to store the backups and WAL files. We'll grant permissions to our pods so that they can write to that bucket. Create the bucket using gcloud CLI\n1gcloud storage buckets create --location=eu --default-storage-class=coldline gs://cnpg-ogenki 2Creating gs://cnpg-ogenki/... 3 4gcloud storage buckets describe gs://cnpg-ogenki 5[...] 6name: cnpg-ogenki 7owner: 8 entity: project-owners-xxxx0008 9projectNumber: \u0026#39;xxx00008\u0026#39; 10rpo: DEFAULT 11selfLink: https://www.googleapis.com/storage/v1/b/cnpg-ogenki 12storageClass: STANDARD 13timeCreated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; 14updated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; Now, we're going to give the permissions to the pods (PostgreSQL server) to write/read from the bucket using Workload Identity.\nNote The GKE cluster should be configured with Workload Identity enabled. Check your cluster configuration, you should get something with this command:\n1gcloud container clusters describe \u0026lt;cluster_name\u0026gt; --format json --zone \u0026lt;zone\u0026gt; | jq .workloadIdentityConfig 2{ 3 \u0026#34;workloadPool\u0026#34;: \u0026#34;{{ gcp_project }}.svc.id.goog\u0026#34; 4} Create a Google Cloud service account\n1gcloud iam service-accounts create cloudnative-pg --project={{ gcp_project }} 2Created service account [cloudnative-pg]. Assign the storage.admin permission to the serviceaccount\n1gcloud projects add-iam-policy-binding {{ gcp_project }} \\ 2--member \u0026#34;serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com\u0026#34; \\ 3--role \u0026#34;roles/storage.admin\u0026#34; 4[...] 5- members: 6 - serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 7 role: roles/storage.admin 8etag: BwXrGA_VRd4= 9version: 1 Allow the Kubernetes service account to impersonate the IAM service account. \u0026#x2139;\u0026#xfe0f; ensure you use the proper format serviceAccount:{{ gcp_project }}.svc.id.goog[{{ kubernetes_namespace }}/{{ kubernetes_serviceaccount }}]\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 7 role: roles/iam.workloadIdentityUser 8etag: BwXrGBjt5kQ= 9version: 1 We're ready to create the Kubernetes resources \u0026#x1f4aa;\n\u0026#x1f511; Create the users secrets We need to create the users credentials that will be used during the bootstrap process (more info later on): The superuser and the newly created database owner.\n1kubectl create secret generic cnpg-mydb-superuser --from-literal=username=postgres --from-literal=password=foobar --namespace demo 2secret/cnpg-mydb-superuser created 1kubectl create secret generic cnpg-mydb-user --from-literal=username=smana --from-literal=password=barbaz --namespace demo 2secret/cnpg-mydb-user created \u0026#x1f6e0;\u0026#xfe0f; Deploy the CloudNativePG operator using Helm CloudNativePG is basically a Kubernetes operator which comes with some CRDs. We'll use the Helm chart as follows\n1helm repo add cnpg https://cloudnative-pg.github.io/charts 2 3helm upgrade --install cnpg --namespace cnpg-system \\ 4--create-namespace charts/cloudnative-pg 5 6kubectl get po -n cnpg-system 7NAME READY STATUS RESTARTS AGE 8cnpg-74488f5849-8lhjr 1/1 Running 0 6h17m Here are the Custom Resource Definitions installed along with the operator.\n1kubectl get crds | grep cnpg.io 2backups.postgresql.cnpg.io 2022-10-08T16:15:14Z 3clusters.postgresql.cnpg.io 2022-10-08T16:15:14Z 4poolers.postgresql.cnpg.io 2022-10-08T16:15:14Z 5scheduledbackups.postgresql.cnpg.io 2022-10-08T16:15:14Z For a full list of the available parameters for these CRDs please refer to the API reference.\n\u0026#x1f680; Create a PostgreSQL server Now we can create our first instance using a custom resource Cluster. The following definition is pretty simple: we want to start a PostgreSQL server, automatically create a database named mydb and configure the credentials based on the secrets created previously.\n1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki 5 namespace: demo 6spec: 7 description: \u0026#34;PostgreSQL Demo Ogenki\u0026#34; 8 imageName: ghcr.io/cloudnative-pg/postgresql:14.5 9 instances: 1 10 11 bootstrap: 12 initdb: 13 database: mydb 14 owner: smana 15 secret: 16 name: cnpg-mydb-user 17 18 serviceAccountTemplate: 19 metadata: 20 annotations: 21 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 22 23 superuserSecret: 24 name: cnpg-mydb-superuser 25 26 storage: 27 storageClass: standard 28 size: 10Gi 29 30 backup: 31 barmanObjectStore: 32 destinationPath: \u0026#34;gs://cnpg-ogenki\u0026#34; 33 googleCredentials: 34 gkeEnvironment: true 35 retentionPolicy: \u0026#34;30d\u0026#34; 36 37 resources: 38 requests: 39 memory: \u0026#34;1Gi\u0026#34; 40 cpu: \u0026#34;500m\u0026#34; 41 limits: 42 memory: \u0026#34;1Gi\u0026#34; Create the namespace where our PostgreSQL instance will be deployed\n1kubectl create ns demo 2namespace/demo created Change the above cluster manifest to fit your needs and apply it.\n1kubectl apply -f cluster.yaml 2cluster.postgresql.cnpg.io/ogenki created You'll notice that the cluster will be in initializing phase. Let's use the cnpg plugin for the first time, it will become our best friend to display a neat view of the cluster's status.\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Primary server is initializing 4Name: ogenki 5Namespace: demo 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: (switching to ogenki-1) 8Status: Setting up primary Creating primary instance ogenki-1 9Instances: 1 10Ready instances: 0 11 12Certificates Status 13Certificate Name Expiration Date Days Left Until Expiration 14---------------- --------------- -------------------------- 15ogenki-ca 2023-01-13 20:02:40 +0000 UTC 90.00 16ogenki-replication 2023-01-13 20:02:40 +0000 UTC 90.00 17ogenki-server 2023-01-13 20:02:40 +0000 UTC 90.00 18 19Continuous Backup status 20First Point of Recoverability: Not Available 21No Primary instance found 22Streaming Replication status 23Not configured 24 25Instances status 26Name Database Size Current LSN Replication role Status QoS Manager Version Node 27---- ------------- ----------- ---------------- ------ --- --------------- ---- The first thing that runs under the hood is the bootstrap process. In our example we create a brand new database named mydb with an owner smana and credentials are retrieved from the secret we created previously.\n1[...] 2 bootstrap: 3 initdb: 4 database: mydb 5 owner: smana 6 secret: 7 name: cnpg-mydb-user 8[...] 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 0/1 Running 0 55s 4ogenki-1-initdb-q75cz 0/1 Completed 0 2m32s After a few seconds the cluster becomes ready \u0026#x1f44f;\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7154833472216277012 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 1 10Ready instances: 1 11 12[...] 13 14Instances status 15Name Database Size Current LSN Replication role Status QoS Manager Version Node 16---- ------------- ----------- ---------------- ------ --- --------------- ---- 17ogenki-1 33 MB 0/17079F8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xczh Info There are many ways of bootstrapping your cluster. For instance, restoring a backup into a brand new instance, running SQL scripts... more info here.\n🩹 Standby instance and resiliency Info In traditional PostgreSQL architectures we usually find an additional component to handle high availability (e.g. Patroni). A specific aspect of the CloudNativePG operator is that it leverages built-in Kubernetes features and relies on a component named Postgres instance manager.\nAdd a standby instance by setting the number of replicas to 2.\n1kubectl edit cluster -n demo ogenki 2cluster.postgresql.cnpg.io/ogenki edited 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3[...] 4spec: 5 instances: 2 6[...] The operator immediately notices the change, adds a standby instance, and starts the replication process.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Creating a new replica Creating replica ogenki-2-join 9Instances: 2 10Ready instances: 1 11Current Write LSN: 0/1707A30 (Timeline: 1 - WAL File: 000000010000000000000001) 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 0 3m16s 4ogenki-2-join-xxrwx 0/1 Pending 0 82s After a while (depending on the amount of data to replicate), the standby instance will be up and running and we can see the replication statistics.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 2 10Ready instances: 2 11Current Write LSN: 0/3000060 (Timeline: 1 - WAL File: 000000010000000000000003) 12 13[...] 14 15Streaming Replication status 16Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 17---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 18ogenki-2 0/3000060 0/3000060 0/3000060 0/3000060 00:00:00 00:00:00 00:00:00 streaming async 0 19 20Instances status 21Name Database Size Current LSN Replication role Status QoS Manager Version Node 22---- ------------- ----------- ---------------- ------ --- --------------- ---- 23ogenki-1 33 MB 0/3000060 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 24ogenki-2 33 MB 0/3000060 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc Let's promote the standby instance to primary (perform a Switchover).\nThe cnpg plugin allows to do this imperatively by running this command\n1kubectl cnpg promote ogenki ogenki-2 -n demo 2Node ogenki-2 in cluster ogenki will be promoted In my case the switchover was really fast. We can check that the instance ogenki-2 is now the primary and that the replication is done the other way around.\n1kubectl cnpg status -n demo ogenki 2[...] 3Status: Switchover in progress Switching over to ogenki-2 4Instances: 2 5Ready instances: 1 6[...] 7Streaming Replication status 8Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 9---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 10ogenki-1 0/4004CA0 0/4004CA0 0/4004CA0 0/4004CA0 00:00:00 00:00:00 00:00:00 streaming async 0 11 12Instances status 13Name Database Size Current LSN Replication role Status QoS Manager Version Node 14---- ------------- ----------- ---------------- ------ --- --------------- ---- 15ogenki-2 33 MB 0/4004CA0 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc 16ogenki-1 33 MB 0/4004CA0 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 Now let's simulate a Failover by deleting the primary pod\n1kubectl delete po -n demo --grace-period 0 --force ogenki-2 2Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 3pod \u0026#34;ogenki-2\u0026#34; force deleted 1Cluster Summary 2Name: ogenki 3Namespace: demo 4System ID: 7155095145869606932 5PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 6Primary instance: ogenki-1 7Status: Failing over Failing over from ogenki-2 to ogenki-1 8Instances: 2 9Ready instances: 1 10Current Write LSN: 0/4005D98 (Timeline: 3 - WAL File: 000000030000000000000004) 11 12[...] 13Instances status 14Name Database Size Current LSN Replication role Status QoS Manager Version Node 15---- ------------- ----------- ---------------- ------ --- --------------- ---- 16ogenki-1 33 MB 0/40078D8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 17ogenki-2 - - - pod not available Burstable - gke-kcdfrance-main-np-0e87115b-xszc After a few seconds, the cluster becomes healthy again\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 13m 2 2 Cluster in healthy state ogenki-1 So far so good, we've been able to test the high availability and the experience is pretty smooth 😎.\n👁️ Monitoring We're going to use Prometheus Stack. We won't cover its installation in this article. If you want to see how to install it \u0026quot;the GitOps way\u0026quot; you can check this example.\nTo scrape our instance's metrics, we need to create a PodMonitor.\n1apiVersion: monitoring.coreos.com/v1 2kind: PodMonitor 3metadata: 4 labels: 5 prometheus-instance: main 6 name: cnpg-ogenki 7 namespace: demo 8spec: 9 namespaceSelector: 10 matchNames: 11 - demo 12 podMetricsEndpoints: 13 - port: metrics 14 selector: 15 matchLabels: 16 postgresql: ogenki We can then add the Grafana dashboard available here.\nFinally, you may want to configure alerts and you can create a PrometheusRule using these rules.\n\u0026#x1f525; Performances and benchmark Info Update: It is now possible to use the cnpg plugin. The following method is deprecated, I'll update it asap.\nThis is worth running a performance test in order to know the limits of your current server and keep a baseline for further improvements.\nNote When it comes to performance there are many improvement areas we can work on. It mostly depends on the target we want to achieve. Indeed we don't want to waste time and money for performance we'll likely never need.\nHere are the main things to look at:\nPostgreSQL configuration tuning Compute resources (cpu and memory) Disk type IOPS, local storage (local-volume-provisioner), Dedicated disks for WAL and PG_DATA Connection pooling PGBouncer. The CloudNativePG comes with a CRD Pooler to handle that. Database optimization, analyzing the query plans using explain, use the extension pg_stat_statement ... First of all we'll add labels to the nodes in order to run the pgbench command on different machines than the ones hosting the database.\n1PG_NODE=$(kubectl get po -n demo -l postgresql=ogenki,role=primary -o jsonpath={.items[0].spec.nodeName}) 2kubectl label node ${PG_NODE} workload=postgresql 3node/gke-kcdfrance-main-np-0e87115b-vlzm labeled 4 5 6# Choose any other node different than the ${PG_NODE} 7kubectl label node gke-kcdfrance-main-np-0e87115b-p5d7 workload=pgbench 8node/gke-kcdfrance-main-np-0e87115b-p5d7 labeled And we'll deploy the Helm chart as follows\n1git clone git@github.com:EnterpriseDB/cnp-bench.git 2cd cnp-bench 3 4cat \u0026gt; pgbench-benchmark/myvalues.yaml \u0026lt;\u0026lt;EOF 5cnp: 6 existingCluster: true 7 existingHost: ogenki-rw 8 existingCredentials: cnpg-mydb-superuser 9 existingDatabase: mydb 10 11pgbench: 12 # Node where to run pgbench 13 nodeSelector: 14 workload: pgbench 15 initialize: true 16 scaleFactor: 1 17 time: 600 18 clients: 10 19 jobs: 1 20 skipVacuum: false 21 reportLatencies: false 22EOF 23 24helm upgrade --install -n demo pgbench -f pgbench-benchmark/myvalues.yaml pgbench-benchmark/ Info There are different services depending on wether you want to read and write or read only.\n1kubectl get ep -n demo 2NAME ENDPOINTS AGE 3ogenki-any 10.64.1.136:5432,10.64.1.3:5432 15d 4ogenki-r 10.64.1.136:5432,10.64.1.3:5432 15d 5ogenki-ro 10.64.1.136:5432 15d 6ogenki-rw 10.64.1.3:5432 15d 1kubectl logs -n demo job/pgbench-pgbench-benchmark -f 2Defaulted container \u0026#34;pgbench\u0026#34; out of: pgbench, wait-for-cnp (init), pgbench-init (init) 3pgbench (14.1, server 14.5 (Debian 14.5-2.pgdg110+2)) 4starting vacuum...end. 5transaction type: \u0026lt;builtin: TPC-B (sort of)\u0026gt; 6scaling factor: 1 7query mode: simple 8number of clients: 10 9number of threads: 1 10duration: 600 s 11number of transactions actually processed: 545187 12latency average = 11.004 ms 13initial connection time = 111.585 ms 14tps = 908.782896 (without initial connection time) 💽 Backup and Restore Note Writing backups and WAL files to the GCP bucket is possible because we gave the permissions using an annotation in the pod's serviceaccount\n1 serviceAccountTemplate: 2 metadata: 3 annotations: 4 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com We can first trigger an on-demand backup using the custom resource Backup\n1apiVersion: postgresql.cnpg.io/v1 2kind: Backup 3metadata: 4 name: ogenki-now 5 namespace: demo 6spec: 7 cluster: 8 name: ogenki 1kubectl apply -f backup.yaml 2backup.postgresql.cnpg.io/ogenki-now created 3 4kubectl get backup -n demo 5NAME AGE CLUSTER PHASE ERROR 6ogenki-now 36s ogenki completed If you take a look at the Google Cloud Storage content you'll see an new directory that stores the base backups\n1gcloud storage ls gs://cnpg-ogenki/ogenki/base 2gs://cnpg-ogenki/ogenki/base/20221023T130327/ But most of the time we would want to have a scheduled backup. So let's configure a daily schedule.\n1apiVersion: postgresql.cnpg.io/v1 2kind: ScheduledBackup 3metadata: 4 name: ogenki-daily 5 namespace: demo 6spec: 7 backupOwnerReference: self 8 cluster: 9 name: ogenki 10 schedule: 0 0 0 * * * Recoveries can only be done on new instances. Here we'll use the backup we've created previously to bootstrap a new instance with it.\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore] 7 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 8 role: roles/iam.workloadIdentityUser 9etag: BwXrs755FPA= 10version: 1 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki-restore 5 namespace: demo 6spec: 7 instances: 1 8 9 serviceAccountTemplate: 10 metadata: 11 annotations: 12 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 13 14 storage: 15 storageClass: standard 16 size: 10Gi 17 18 resources: 19 requests: 20 memory: \u0026#34;1Gi\u0026#34; 21 cpu: \u0026#34;500m\u0026#34; 22 limits: 23 memory: \u0026#34;1Gi\u0026#34; 24 25 superuserSecret: 26 name: cnpg-mydb-superuser 27 28 bootstrap: 29 recovery: 30 backup: 31 name: ogenki-now We can notice a first pod that performs the full recovery from the backup.\n1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 1 (18h ago) 18h 4ogenki-2 1/1 Running 0 18h 5ogenki-restore-1 0/1 Init:0/1 0 0s 6ogenki-restore-1-full-recovery-5p4ct 0/1 Completed 0 51s Then the new cluster becomes ready.\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 18h 2 2 Cluster in healthy state ogenki-1 4ogenki-restore 80s 1 1 Cluster in healthy state ogenki-restore-1 \u0026#x1f9f9; Cleanup Delete the cluster\n1kubectl delete cluster -n demo ogenki ogenki-restore 2cluster.postgresql.cnpg.io \u0026#34;ogenki\u0026#34; deleted 3cluster.postgresql.cnpg.io \u0026#34;ogenki-restore\u0026#34; deleted Cleanup the IAM serviceaccount\n1gcloud iam service-accounts delete cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 2You are about to delete service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 3 4Do you want to continue (Y/n)? y 5 6deleted service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com] 💭 final thoughts I just discovered CloudNativePG and I only scratched the surface but one thing for sure is that managing PostgreSQL is really made easy. However choosing a database solution is a tough decision. Depending on the use case, the company constraints, the criticity of the application and the ops teams skills, there are plenty of options: Cloud managed databases, traditional bare metal installations, building the architecture with an Infrastructure As Code tool...\nWe may also consider using Crossplane and composition to give an opinionated way of declaring managed databases in cloud providers but that requires more configuration.\nCloudNativePG shines by its simplicity: it is easy to run and easy to understand. Furthermore the documentation is excellent (one of the best I ever seen!), especially for such a young open source project (Hopefuly this will help in the CNCF Sandbox acceptance process 🤞).\nIf you want to learn more about it, there was a presentation on about it at KubeCon NA 2022.\n","link":"https://blog.ogenki.io/post/cnpg/","section":"post","tags":["data"],"title":"`CloudNativePG`: An easy way to run PostgreSQL on Kubernetes"},{"body":"","link":"https://blog.ogenki.io/tags/data/","section":"tags","tags":null,"title":"data"},{"body":"In a previous article, we've seen how to use Crossplane so that we can manage cloud resources the same way as our applications. \u0026#x2764;\u0026#xfe0f; Declarative approach! There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.\nHere we'll achieve exactly the same thing but we'll do that in the GitOps way. According to the OpenGitOps working group there are 4 GitOps principles:\nThe desired state of our system must be expressed declaratively. This state must be stored in a versioning system. Changes are pulled and applied automatically in the target platform whenever the desired state changes. If, for any reason, the current state is modified, it will be automatically reconciled with the desired state. There are several GitOps engine options. The most famous ones are ArgoCD and Flux. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).\nLearn more about GitOps toolkit components here.\n:bullseye: Our target Here we want to declare our desired infrastructure components only by adding git changes. By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance. We'll discover Flux basics and how to use it in order to build a complete GitOps CD workflow.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Install required tools First of all we need to install a few tools using asdf\nCreate a local file .tool-versions\n1cd ~/sources/devflux/ 2 3cat \u0026gt; .tool-versions \u0026lt;\u0026lt;EOF 4flux2 0.31.3 5kubectl 1.24.3 6kubeseal 0.18.1 7kustomize 4.5.5 8EOF 1for PLUGIN in $(cat .tool-versions | awk \u0026#39;{print $1}\u0026#39;); do asdf plugin-add $PLUGIN; done 2 3asdf install 4Downloading ... 100.0% 5Copying Binary 6... Check that all the required tools are actually installed.\n1asdf current 2flux2 0.31.3 /home/smana/sources/devflux/.tool-versions 3kubectl 1.24.3 /home/smana/sources/devflux/.tool-versions 4kubeseal 0.18.1 /home/smana/sources/devflux/.tool-versions 5kustomize 4.5.5 /home/smana/sources/devflux/.tool-versions \u0026#x1f511; Create a Github personal access token In this article the git repository is hosted in Github. In order to be able to use the flux bootstrap a personnal access token is required.\nPlease follow this procedure.\nWarning Store the Github token in a safe place for later use\n\u0026#x1f9d1;\u0026zwj;\u0026#x1f4bb; Clone the devflux repository All the files used for the upcoming steps can be retrieved from this repository. You should clone it, that will be easier to copy them into your own repository.\n1git clone https://github.com/Smana/devflux.git \u0026#x1f680; Bootstrap flux in the Crossplane cluster As we will often be using the flux CLI you may want to configure the bash|zsh completion\n1source \u0026lt;(flux completion bash) Warning Here we consider that you already have a local k3d instance. If not you may want to either go through the whole previous article or just run the local cluster creation.\nEnsure that you're working in the right context\n1kubectl config current-context 2k3d-crossplane Run the bootstrap command that will basically deploy all Flux's components in the namespace flux-system. Here I'll create a repository named devflux using my personal Github account.\n1export GITHUB_USER=\u0026lt;YOUR_ACCOUNT\u0026gt; 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/k3d-crossplane 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/k3d-crossplane\u0026#34; for \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux\u0026#34; 9... 10✔ all components are healthy Check that all the pods are running properly and that the kustomization flux-system has been successfully reconciled.\n1kubectl get po -n flux-system 2NAME READY STATUS RESTARTS AGE 3helm-controller-5985c795f8-gs2pc 1/1 Running 0 86s 4notification-controller-6b7d7485fc-lzlpg 1/1 Running 0 86s 5kustomize-controller-6d4669f847-9x844 1/1 Running 0 86s 6source-controller-5fb4888d8f-wgcqv 1/1 Running 0 86s 7 8flux get kustomizations 9NAME REVISION SUSPENDED READY MESSAGE 10flux-system main/33ebef1 False True Applied revision: main/33ebef1 Running the bootstap command actually creates a github repository if it doesn't exist yet. Clone it now for our upcoming changes. You'll notice that the first commit has been made by Flux.\n1git clone https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git 2Cloning into \u0026#39;devflux\u0026#39;... 3 4cd devflux 5 6git log -1 7commit 2beb6aafea67f3386b50cbc706fb34575844040d (HEAD -\u0026gt; main, origin/main, origin/HEAD) 8Author: Flux \u0026lt;\u0026gt; 9Date: Thu Jul 14 17:13:27 2022 +0200 10 11 Add Flux sync manifests 12 13ls clusters/k3d-crossplane/flux-system/ 14gotk-components.yaml gotk-sync.yaml kustomization.yaml \u0026#x1f4c2; Flux repository structure There are several options for organizing your resources in the Flux configuration repository. Here is a proposition for the sake of this article.\n1tree -d -L 2 2. 3├── apps 4│ ├── base 5│ └── dev-cluster 6├── clusters 7│ ├── dev-cluster 8│ └── k3d-crossplane 9├── infrastructure 10│ ├── base 11│ ├── dev-cluster 12│ └── k3d-crossplane 13├── observability 14│ ├── base 15│ ├── dev-cluster 16│ └── k3d-crossplane 17└── security 18 ├── base 19 ├── dev-cluster 20 └── k3d-crossplane Directory Description Example /apps our applications Here we'll deploy a demo application \u0026quot;online-boutique\u0026quot; /infrastructure base infrastructure/network components Crossplane as it will be used to provision cloud resources but we can also find CSI/CNI/EBS drivers... /observability All metrics/apm/logging tools Prometheus of course, Opentelemetry ... /security Any component that enhance our security level SealedSecrets (see below) Info For the upcoming steps please refer to the demo repository here\nLet's use this structure and begin to deploy applications \u0026#x1f680;.\n\u0026#x1f510; SealedSecrets There are plenty of alternatives when it comes to secrets management in Kubernetes. In order to securely store secrets in a git repository the GitOps way we'll make use of SealedSecrets. It uses a custom resource definition named SealedSecrets in order to encrypt the Kubernetes secret at the client side then the controller is in charge of decrypting and generating the expected secret in the cluster.\n\u0026#x1f6e0;\u0026#xfe0f; Deploy the controller using Helm The first thing to do is to declare the kustomization that handles all the security tools.\nclusters/k3d-crossplane/security.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 2kind: Kustomization 3metadata: 4 name: security 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 sourceRef: 10 kind: GitRepository 11 name: flux-system 12 path: ./security/k3d-crossplane 13 healthChecks: 14 - apiVersion: helm.toolkit.fluxcd.io/v1beta1 15 kind: HelmRelease 16 name: sealed-secrets 17 namespace: kube-system Info A Kustomization is a custom resource that comes with Flux. It basically points to a set of Kubernetes resources managed with kustomize The above security kustomization points to a local directory where the kustomize resources are.\n1... 2spec: 3 path: ./security/k3d-crossplane 4... Note This is worth noting that there are two types on kustomizations. That can be confusing when you start playing with Flux.\nOne managed by flux's kustomize controller. Its API is kustomization.kustomize.toolkit.fluxcd.io The other kustomization.kustomize.config.k8s.io is for the kustomize overlay The kustomization.yaml file is always used for the kustomize overlay. Flux itself doesn't need this overlay in all cases, but if you want to use features of a Kustomize overlay you will occasionally need to create it in order to access them. It provides instructions to the Kustomize CLI.\nWe will deploy SealedSecrets using the Helm chart. So we need to declare the source of this chart. Using the kustomize overlay system, we'll first create the base files that will be inherited at the cluster level.\nsecurity/base/sealed-secrets/source.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: sealed-secrets 5 namespace: flux-system 6spec: 7 interval: 30m 8 url: https://bitnami-labs.github.io/sealed-secrets Then we'll define the HelmRelease which references the above source. Put the values you want to apply to the Helm chart under spec.values\nsecurity/base/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 releaseName: sealed-secrets 8 chart: 9 spec: 10 chart: sealed-secrets 11 sourceRef: 12 kind: HelmRepository 13 name: sealed-secrets 14 namespace: flux-system 15 version: \u0026#34;2.4.0\u0026#34; 16 interval: 10m0s 17 install: 18 remediation: 19 retries: 3 20 values: 21 fullnameOverride: sealed-secrets-controller 22 resources: 23 requests: 24 cpu: 80m 25 memory: 100Mi If you're starting your repository from scratch you'll need to generate the kustomization.yaml file (kustomize overlay).\n1kustomize create --autodetect security/base/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4- helmrelease.yaml 5- source.yaml Now we declare the sealed-secret kustomization at the cluster level. Just for the example we'll overwrite a value at the cluster level using kustomize's overlay system.\nsecurity/k3d-crossplane/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 values: 8 resources: 9 requests: 10 cpu: 100m security/k3d-crossplane/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3bases: 4 - ../../base 5patches: 6 - helmrelease.yaml Pushing our changes is the only thing to do in order to get sealed-secrets deployed in the target cluster.\n1git commit -m \u0026#34;security: deploy sealed-secrets in k3d-crossplane\u0026#34; 2[security/sealed-secrets 283648e] security: deploy sealed-secrets in k3d-crossplane 3 6 files changed, 66 insertions(+) 4 create mode 100644 clusters/k3d-crossplane/security.yaml 5 create mode 100644 security/base/sealed-secrets/helmrelease.yaml 6 create mode 100644 security/base/sealed-secrets/kustomization.yaml 7 create mode 100644 security/base/sealed-secrets/source.yaml 8 create mode 100644 security/k3d-crossplane/sealed-secrets/helmrelease.yaml 9 create mode 100644 security/k3d-crossplane/sealed-secrets/kustomization.yaml After a few seconds (1 minutes by default) a new kustomization will appear.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3flux-system main/d36a33c False True Applied revision: main/d36a33c 4security main/d36a33c False True Applied revision: main/d36a33c And all the resources that we declared in the flux repository should be available and READY.\n1flux get sources helm 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee False True stored artifact for revision \u0026#39;4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee\u0026#39; 1flux get helmrelease -n kube-system 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 2.2.0 False True Release reconciliation succeeded 🧪 A first test SealedSecret Let's use the CLI kubeseal to test it out. We'll create a SealedSecret that will be decrypted by the sealed-secrets controller in the cluster and create the expected secret foobar\n1kubectl create secret generic foobar -n default --dry-run=client -o yaml --from-literal=foo=bar \\ 2| kubeseal --namespace default --format yaml | kubectl apply -f - 3sealedsecret.bitnami.com/foobar created 4 5kubectl get secret -n default foobar 6NAME TYPE DATA AGE 7foobar Opaque 1 3m13s 8 9kubectl delete sealedsecrets.bitnami.com foobar 10sealedsecret.bitnami.com \u0026#34;foobar\u0026#34; deleted \u0026#x2601;\u0026#xfe0f; Deploy and configure Crossplane \u0026#x1f511; Create the Google service account secret The first thing we need to do in order to get Crossplane working is to create the GCP serviceaccount. The steps have been covered here in the previous article. We'll create a SealedSecret gcp-creds that contains the serviceaccount file crossplane.json.\ninfrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml\n1kubectl create secret generic gcp-creds --context k3d-crossplane -n crossplane-system --from-file=creds=./crossplane.json --dry-run=client -o yaml \\ 2| kubeseal --format yaml --namespace crossplane-system - \u0026gt; infrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml 🔄 Crossplane dependencies Now we will deploy Crossplane with Flux. I won't put the manifests here you'll find all of them in this repository. However it's important to understand that, in order to deploy and configure Crossplane properly we need to do that in a specific order. Indeed several CRD's (custom resource definitions) are required:\nFirst of all we'll install the crossplane controller. Then we'll configure the provider because the custom resource is now available thanks to the crossplane controller installation. Finally a provider installation deploys several CRDs that can be used to configure the provider itself and cloud resources. The dependencies between kustomizations can be controlled using the parameters dependsOn. Looking at the file clusters/k3d-crossplane/infrastructure.yaml, we can see for example that the kustomization infrastructure-custom-resources depends on the kustomization crossplane_provider which itself depends on crossplane-configuration....\n1--- 2apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 3kind: Kustomization 4metadata: 5 name: crossplane-provider 6spec: 7... 8 dependsOn: 9 - name: crossplane-core 10--- 11apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 12kind: Kustomization 13metadata: 14 name: crossplane-configuration 15spec: 16... 17 dependsOn: 18 - name: crossplane-provider 19--- 20apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 21kind: Kustomization 22metadata: 23 name: infrastructure-custom-resources 24spec: 25... 26 dependsOn: 27 - name: crossplane-configuration Commit and push the changes for the kustomisations to appear. Note that they'll be reconciled in the defined order.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3infrastructure-custom-resources False False dependency \u0026#39;flux-system/crossplane-configuration\u0026#39; is not ready 4crossplane-configuration False False dependency \u0026#39;flux-system/crossplane-provider\u0026#39; is not ready 5security main/666f85a False True Applied revision: main/666f85a 6flux-system main/666f85a False True Applied revision: main/666f85a 7crossplane-core main/666f85a False True Applied revision: main/666f85a 8crossplane-provider main/666f85a False True Applied revision: main/666f85a Then all Crossplane components will be deployed, we can have a look to the HelmRelease status for instance.\n1kubectl describe helmrelease -n crossplane-system crossplane 2... 3Status: 4 Conditions: 5 Last Transition Time: 2022-07-15T19:12:04Z 6 Message: Release reconciliation succeeded 7 Reason: ReconciliationSucceeded 8 Status: True 9 Type: Ready 10 Last Transition Time: 2022-07-15T19:12:04Z 11 Message: Helm upgrade succeeded 12 Reason: UpgradeSucceeded 13 Status: True 14 Type: Released 15 Helm Chart: crossplane-system/crossplane-system-crossplane 16 Last Applied Revision: 1.9.0 17 Last Attempted Revision: 1.9.0 18 Last Attempted Values Checksum: 056dc1c6029b3a644adc7d6a69a93620afd25b65 19 Last Release Revision: 2 20 Observed Generation: 1 21Events: 22 Type Reason Age From Message 23 ---- ------ ---- ---- ------- 24 Normal info 20m helm-controller HelmChart \u0026#39;crossplane-system/crossplane-system-crossplane\u0026#39; is not ready 25 Normal info 20m helm-controller Helm upgrade has started 26 Normal info 19m helm-controller Helm upgrade succeeded And our GKE cluster should also be created because we defined a bunch of crossplane custom resources in infrastructure/k3d-crossplane/custom-resources/crossplane\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RUNNING 34.x.x.190 europe-west9-a 22m \u0026#x1f680; Bootstrap flux in the dev cluster Our local Crossplane cluster is now ready and it created our dev cluster and we also want it to be managed with Flux. So let's configure Flux for this dev cluster using the same bootstrap command.\nAuthenticate to the newly created cluster. The following command will automatically change your current context.\n1gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project \u0026lt;your_project\u0026gt; 2Fetching cluster endpoint and auth data. 3kubeconfig entry generated for dev-cluster. 4 5kubectl config current-context 6gke_\u0026lt;your_project\u0026gt;_europe-west9-a_dev-cluster Run the bootstrap command for the dev-cluster.\n1export GITHUB_USER=Smana 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/dev-cluster 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/Smana/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/dev-cluster\u0026#34; for \u0026#34;https://github.com/Smana/devflux\u0026#34; 9... 10✔ all components are healthy Note It's worth noting that each Kubernetes cluster generates its own sealing keys. That means that if you recreate the dev-cluster, you must regenerate all the sealedsecrets. In our example we declared a secret in order to set the Grafana credentials. Here's the command you need to run in order to create a new version of the sealedsecret and don't forget to use the proper context \u0026#x1f609;.\n1kubectl create secret generic kube-prometheus-stack-grafana \\ 2--from-literal=admin-user=admin --from-literal=admin-password=\u0026lt;yourpassword\u0026gt; --namespace observability --dry-run=client -o yaml \\ 3| kubeseal --namespace observability --format yaml \u0026gt; observability/dev-cluster/kube-prometheus-stack/sealedsecrets.yaml After a few seconds we'll get the following kustomizations deployed.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3apps main/1380eaa False True Applied revision: main/1380eaa 4flux-system main/1380eaa False True Applied revision: main/1380eaa 5observability main/1380eaa False True Applied revision: main/1380eaa 6security main/1380eaa False True Applied revision: main/1380eaa Here we configured the prometheus stack and deployed a demo microservices stack named \u0026quot;online-boutique\u0026quot; This demo application exposes the frontend through a service of type LoadBalancer.\n1kubectl get svc -n demo frontend-external 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3frontend-external LoadBalancer 10.140.174.201 34.155.121.2 80:31943/TCP 7m44s Use the EXTERNAL_IP\n\u0026#x1f575;\u0026#xfe0f; Troubleshooting The cheatsheet in Flux's documentation contains many ways for troubleshooting when something goes wrong. Here I'll just give a sample of my favorite command lines.\nObjects that aren't ready\n1flux get all -A --status-selector ready=false Checking the logs of a given kustomization\n1flux logs --kind kustomization --name infrastructure-custom-resources 22022-07-15T19:38:52.996Z info Kustomization/infrastructure-custom-resources.flux-system - server-side apply completed 32022-07-15T19:38:53.016Z info Kustomization/infrastructure-custom-resources.flux-system - Reconciliation finished in 66.12266ms, next run in 4m0s 42022-07-15T19:11:34.697Z info Kustomization/infrastructure-custom-resources.flux-system - Discarding event, no alerts found for the involved object Show how a given pod is managed by Flux.\n1flux trace -n crossplane-system pod/crossplane-5dc8d888d7-g95qx 2 3Object: Pod/crossplane-5dc8d888d7-g95qx 4Namespace: crossplane-system 5Status: Managed by Flux 6--- 7HelmRelease: crossplane 8Namespace: crossplane-system 9Revision: 1.9.0 10Status: Last reconciled at 2022-07-15 21:12:04 +0200 CEST 11Message: Release reconciliation succeeded 12--- 13HelmChart: crossplane-system-crossplane 14Namespace: crossplane-system 15Chart: crossplane 16Version: 1.9.0 17Revision: 1.9.0 18Status: Last reconciled at 2022-07-15 21:11:36 +0200 CEST 19Message: pulled \u0026#39;crossplane\u0026#39; chart with version \u0026#39;1.9.0\u0026#39; 20--- 21HelmRepository: crossplane 22Namespace: crossplane-system 23URL: https://charts.crossplane.io/stable 24Revision: 362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d 25Status: Last reconciled at 2022-07-15 21:11:35 +0200 CEST 26Message: stored artifact for revision \u0026#39;362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d\u0026#39; If you want to check what would be the changes before pushing your commit. In thi given example I just increased the cpu requests for the sealed-secrets controller.\n1flux diff kustomization security --path security/k3d-crossplane 2✓ Kustomization diffing... 3► HelmRelease/kube-system/sealed-secrets drifted 4 5metadata.generation 6 ± value change 7 - 6 8 + 7 9 10spec.values.resources.requests.cpu 11 ± value change 12 - 100m 13 + 120m 14 15⚠️ identified at least one change, exiting with non-zero exit code \u0026#x1f9f9; Cleanup Don't forget to delete the Cloud resources if you don't want to have a bad suprise \u0026#x1f4b5;! Just comment the file infrastructure/k3d-crossplane/custom-resources/crossplane/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4 # - cluster.yaml 5 - network.yaml \u0026#x1f44f; Achievements With our current setup everything is configured using the GitOps approach:\nWe can manage infrastructure resources using Crossplane. Our secrets are securely stored in our git repository. We have a dev-cluster that we can enable or disable just but commenting a yaml file. Our demo application can be deployed from scratch in seconds. 💭 final thoughts Flux is probably the tool I'm using the most on a daily basis. It's really amazing!\nWhen you get familiar with its concepts and the command line it becomes really easy to use and troubleshoot. You can use either Helm when a chart is available or Kustomize.\nHowever we faced a few issues:\nIt's not straightforward to find an efficient structure depending on the company needs. Especially when you have several Kubernetes controllers that depend on other CRDs. The Helm controller doesn't maintain a state of the Kubernetes resources deployed by the Helm chart. That means that if you delete a resource which has been deployed through a Helm chart, it won't be reconciled (It will change soon. Being discussed here) Flux doesn't provide itself a web UI and switching between CLIs (kubectl, flux ...) can be annoying from a developer perspective. (I'm going to test weave-gitops ) I've been using Flux in production for more than a year and we configured it with the image automation so that the only thing a developer has to do is to merge a pull request and the new version of the application is automatically deployed in the target cluster.\nI should probably give another try to ArgoCD in order to be able to compare these precisely 🤔.\n","link":"https://blog.ogenki.io/post/devflux/","section":"post","tags":["gitops","devxp"],"title":"100% `GitOps` using Flux"},{"body":"Who am I? I'm a senior Site Reliability Engineer with a particular interest in Linux containers and cloud technologies. I worked in different companies (small startups and large scale) and I've been working in different areas in order to improve the reliability, availability of the platform as well as the developer experience. I helped several companies in their transition to the Cloud. I was leading SRE/DevOps teams (diverse profiles with developers and SREs) and I really enjoy seeing them engaged in the same direction.\nAs side activities, I am an organizer of the Cloud Native Computing meetup in Paris and the Kubernetes Community Days France.\nHobbies: Reading SF books, Kick Boxing, Surfing/Skating/Inline Roller\n","link":"https://blog.ogenki.io/about/","section":"","tags":null,"title":"About"},{"body":"","link":"https://blog.ogenki.io/tags/devxp/","section":"tags","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/tags/gitops/","section":"tags","tags":null,"title":"gitops"},{"body":"The target of this documentation is to be able to create and manage a GKE cluster using Crossplane.\nCrossplane leverages Kubernetes base principles in order to provision cloud resources and much more: a declarative approach with drift detections and reconciliations using control loops \u0026#x1f92f;. In other words, we declare what cloud resources we want and Crossplane ensures that the target state matches the one applied through the Kubernetes API.\nHere are the steps we'll follow in order to get a Kubernetes cluster for development and experimentations use cases.\n\u0026#x1f433; Create the local k3d cluster for Crossplane's control plane k3d is a lightweight kubernetes cluster that leverages k3s that runs in our local laptop. There are several deployment models for Crossplane, we could for instance deploy the control plane on a management cluster on Kubernetes or a control plane per Kubernetes cluster.\nHere I chose a simple method which is fine for a personal use case: A local Kubernetes instance in which I'll deploy Crossplane.\nLet's install k3d using asdf.\n1asdf plugin-add k3d 2 3asdf install k3d $(asdf latest k3d) 4* Downloading k3d release 5.4.1... 5k3d 5.4.1 installation was successful! Create a single node Kubernetes cluster.\n1k3d cluster create crossplane 2... 3INFO[0043] You can now use it like this: 4kubectl cluster-info 5 6k3d cluster list 7crossplane 1/1 0/0 true Check that the cluster is reachable using the kubectl CLI.\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:40643 3CoreDNS is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy We only need a single node for our Crossplane use case.\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-crossplane-server-0 Ready control-plane,master 26h v1.22.7+k3s1 \u0026#x2601;\u0026#xfe0f; Generate the Google Cloud service account Warning Store the downloaded crossplane.json credentials file in a safe place.\nCreate a service account\n1GCP_PROJECT=\u0026lt;your_project\u0026gt; 2gcloud iam service-accounts create crossplane --display-name \u0026#34;Crossplane\u0026#34; --project=${GCP_PROJECT} 3Created service account [crossplane]. Assign the proper permissions to the service account.\nCompute Network Admin Kubernetes Engine Admin Service Account User 1SA_EMAIL=$(gcloud iam service-accounts list --filter=\u0026#34;email ~ ^crossplane\u0026#34; --format=\u0026#39;value(email)\u0026#39;) 2 3gcloud projects add-iam-policy-binding \u0026#34;${GCP_PROJECT}\u0026#34; --member=serviceAccount:\u0026#34;${SA_EMAIL}\u0026#34; \\ 4--role=roles/container.admin --role=roles/compute.networkAdmin --role=roles/iam.serviceAccountUser 5Updated IAM policy for project [\u0026lt;project\u0026gt;]. 6bindings: 7- members: 8 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 9 role: roles/compute.networkAdmin 10- members: 11 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 12... 13version: 1 Download the service account key (json format)\n1gcloud iam service-accounts keys create crossplane.json --iam-account ${SA_EMAIL} 2created key [ea2eb9ce2939127xxxxxxxxxx] of type [json] as [crossplane.json] for [crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com] \u0026#x1f6a7; Deploy and configure Crossplane Now that we have a credentials file for Google Cloud, we can deploy the Crossplane operator and configure the provider-gcp provider.\nInfo Most of the following steps are issued from the official documentation\nWe'll first use Helm in order to install the operator\n1helm repo add crossplane-master https://charts.crossplane.io/master/ 2\u0026#34;crossplane-master\u0026#34; has been added to your repositories 3 4helm repo update 5...Successfully got an update from the \u0026#34;crossplane-master\u0026#34; chart repository 6 7helm install crossplane --namespace crossplane-system --create-namespace \\ 8--version 1.18.1 crossplane-stable/crossplane 9 10NAME: crossplane 11LAST DEPLOYED: Mon Jun 6 22:00:02 2022 12NAMESPACE: crossplane-system 13STATUS: deployed 14REVISION: 1 15TEST SUITE: None 16NOTES: 17Release: crossplane 18... Check that the operator is running properly.\n1kubectl get po -n crossplane-system 2NAME READY STATUS RESTARTS AGE 3crossplane-rbac-manager-54d96cd559-222hc 1/1 Running 0 3m37s 4crossplane-688c575476-lgklq 1/1 Running 0 3m37s Info All the files used for the upcoming steps are stored within this blog repository. So you should clone and change the current directory:\n1git clone https://github.com/Smana/smana.github.io.git 2 3cd smana.github.io/content/resources/crossplane_k3d Now we'll configure Crossplane so that it will be able to create and manage GCP resources. This is done by configuring the provider provider-gcp as follows.\nprovider.yaml\n1apiVersion: pkg.crossplane.io/v1 2kind: Provider 3metadata: 4 name: crossplane-provider-gcp 5spec: 6 package: crossplane/provider-gcp:v0.21.0 1kubectl apply -f provider.yaml 2provider.pkg.crossplane.io/crossplane-provider-gcp created 3 4kubectl get providers 5NAME INSTALLED HEALTHY PACKAGE AGE 6crossplane-provider-gcp True True crossplane/provider-gcp:v0.21.0 10s Create the Kubernetes secret that holds the GCP credentials file created above\n1kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./crossplane.json 2secret/gcp-creds created Then we need to create a resource named ProviderConfig and reference the newly created secret.\nprovider-config.yaml\n1apiVersion: gcp.crossplane.io/v1beta1 2kind: ProviderConfig 3metadata: 4 name: default 5spec: 6 projectID: ${GCP_PROJECT} 7 credentials: 8 source: Secret 9 secretRef: 10 namespace: crossplane-system 11 name: gcp-creds 12 key: creds 1kubectl apply -f provider-config.yaml 2providerconfig.gcp.crossplane.io/default created Info If the serviceaccount has the proper permissions we can create resources in GCP. In order to learn about all the available resources and parameters we can have a look to the provider's API reference.\nThe first resource we'll create is the network that will host our Kubernetes cluster.\nnetwork.yaml\n1apiVersion: compute.gcp.crossplane.io/v1beta1 2kind: Network 3metadata: 4 name: dev-network 5 labels: 6 service: vpc 7 creation: crossplane 8spec: 9 forProvider: 10 autoCreateSubnetworks: false 11 description: \u0026#34;Network used for experimentations and POCs\u0026#34; 12 routingConfig: 13 routingMode: REGIONAL 1kubectl get network 2NAME READY SYNCED 3dev-network True True You can even get more details by describing this resource. For instance if something fails you would see the message returned by the Cloud provider in the events.\n1kubectl describe network dev-network | grep -A 20 \u0026#39;^Status:\u0026#39; 2Status: 3 At Provider: 4 Creation Timestamp: 2022-06-28T09:45:30.703-07:00 5 Id: 3005424280727359173 6 Self Link: https://www.googleapis.com/compute/v1/projects/${GCP_PROJECT}/global/networks/dev-network 7 Conditions: 8 Last Transition Time: 2022-06-28T16:45:31Z 9 Reason: Available 10 Status: True 11 Type: Ready 12 Last Transition Time: 2022-06-30T16:36:59Z 13 Reason: ReconcileSuccess 14 Status: True 15 Type: Synced \u0026#x1f680; Create a GKE cluster Everything is ready so that we can create our GKE cluster. Applying the file cluster.yaml will create a cluster and attach a node group to it.\ncluster.yaml\n1--- 2apiVersion: container.gcp.crossplane.io/v1beta2 3kind: Cluster 4metadata: 5 name: dev-cluster 6spec: 7 forProvider: 8 description: \u0026#34;Kubernetes cluster for experimentations and POCs\u0026#34; 9 initialClusterVersion: \u0026#34;1.24\u0026#34; 10 releaseChannel: 11 channel: \u0026#34;RAPID\u0026#34; 12 location: europe-west9-a 13 addonsConfig: 14 gcePersistentDiskCsiDriverConfig: 15 enabled: true 16 networkPolicyConfig: 17 disabled: false 18 networkRef: 19 name: dev-network 20 ipAllocationPolicy: 21 createSubnetwork: true 22 useIpAliases: true 23 defaultMaxPodsConstraint: 24 maxPodsPerNode: 110 25 networkPolicy: 26 enabled: false 27 writeConnectionSecretToRef: 28 namespace: default 29 name: gke-conn 30--- 31apiVersion: container.gcp.crossplane.io/v1beta1 32kind: NodePool 33metadata: 34 name: main-np 35spec: 36 forProvider: 37 initialNodeCount: 1 38 autoscaling: 39 autoprovisioned: false 40 enabled: true 41 maxNodeCount: 4 42 minNodeCount: 1 43 clusterRef: 44 name: dev-cluster 45 config: 46 machineType: n2-standard-2 47 diskSizeGb: 120 48 diskType: pd-standard 49 imageType: cos_containerd 50 preemptible: true 51 labels: 52 environment: dev 53 managed-by: crossplane 54 oauthScopes: 55 - \u0026#34;https://www.googleapis.com/auth/devstorage.read_only\u0026#34; 56 - \u0026#34;https://www.googleapis.com/auth/logging.write\u0026#34; 57 - \u0026#34;https://www.googleapis.com/auth/monitoring\u0026#34; 58 - \u0026#34;https://www.googleapis.com/auth/servicecontrol\u0026#34; 59 - \u0026#34;https://www.googleapis.com/auth/service.management.readonly\u0026#34; 60 - \u0026#34;https://www.googleapis.com/auth/trace.append\u0026#34; 61 metadata: 62 disable-legacy-endpoints: \u0026#34;true\u0026#34; 63 shieldedInstanceConfig: 64 enableIntegrityMonitoring: true 65 enableSecureBoot: true 66 management: 67 autoRepair: true 68 autoUpgrade: true 69 maxPodsConstraint: 70 maxPodsPerNode: 60 71 locations: 72 - \u0026#34;europe-west9-a\u0026#34; 1kubectl apply -f cluster.yaml 2cluster.container.gcp.crossplane.io/dev-cluster created 3nodepool.container.gcp.crossplane.io/main-np created Note that it takes around 10 minutes for the Kubernetes API and the nodes to be available. The STATE will transition from PROVISIONING to RUNNING and when a change is being applied the cluster status is RECONCILING\n1watch \u0026#39;kubectl get cluster,nodepool\u0026#39; 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3cluster.container.gcp.crossplane.io/dev-cluster False True PROVISIONING 34.155.122.6 europe-west9-a 3m15s 4 5NAME READY SYNCED STATE CLUSTER-REF AGE 6nodepool.container.gcp.crossplane.io/main-np False False dev-cluster 3m15s When the column READY switches to True you can download the cluster's credentials.\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RECONCILING 34.42.42.42 europe-west9-a 6m23s 4 5gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project ${GCP_PROJECT} 6Fetching cluster endpoint and auth data. 7kubeconfig entry generated for dev-cluster. For better readability you may want to rename the context id for the newly created cluster\n1kubectl config rename-context gke_${GCP_PROJECT}_europe-west9-a_dev-cluster dev-cluster 2Context \u0026#34;gke_${GCP_PROJECT}_europe-west9-a_dev-cluster\u0026#34; renamed to \u0026#34;dev-cluster\u0026#34;. 3 4kubectl config get-contexts 5CURRENT NAME CLUSTER AUTHINFO NAMESPACE 6* dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster 7 k3d-crossplane k3d-crossplane admin@k3d-crossplane Check that you can call our brand new GKE API\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3gke-dev-cluster-main-np-d0d978f9-5fc0 Ready \u0026lt;none\u0026gt; 10m v1.24.1-gke.1400 That's great \u0026#x1f389; we know have a GKE cluster up and running.\n💭 final thoughts I've been using Crossplane for a few months now in a production environment.\nEven if I'm conviced about the declarative approach using the Kubernetes API, we decided to move with caution with it. It clearly doesn't have Terraform's community and maturity. We're still declaring our resources using the deletionPolicy: Orphan so that even if something goes wrong on the controller side the resource won't be deleted.\nFurthermore we limited to a specific list of usual AWS resources requested by our developers. Nevertheless our target has always been to empower developers and we had really positive feedback from them. That's the best indicator for us. As the project matures, we'll move more and more resources from Terraform to Crossplane.\nIMHO the key success of Crossplane depends on the providers maintenance and evolution. The Cloud providers interest and involvement is really important.\nIn our next article we'll see how to use a GitOps engine to run all the above steps.\n","link":"https://blog.ogenki.io/post/crossplane_k3d/","section":"post","tags":["kubernetes","infrastructure"],"title":"My Kubernetes cluster (GKE) with `Crossplane`"},{"body":"","link":"https://blog.ogenki.io/tags/local/","section":"tags","tags":null,"title":"local"},{"body":"In order to install binaries and to be able to switch from a version to another I like to use asdf.\n\u0026#x1f4e5; Installation The recommended installation is to use Git as follows\n1git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.0 Then depending on your shell here are the remaining steps to follow\n1. $HOME/.asdf/asdf.sh And you may want to configure the shell completion\n1. $HOME/.asdf/completions/asdf.bash \u0026#x1f680; Let's take an example List all available plugins and look for k3d\n1asdf plugin-list-all | grep k3d 2k3d https://github.com/spencergilbert/asdf-k3d.git Let's install k3d\n1asdf plugin-add k3d Check the versions available\n1asdf list-all k3d| tail -n 3 25.4.0-dev.3 35.4.0 45.4.1 We'll install the latest version\n1asdf install k3d latest 2* Downloading k3d release 5.4.1... 3k3d 5.4.1 installation was successful! Finally we can switch from a version to another. We can set a global version that would be used on all directories.\n1asdf global k3d 5.4.1 or use a local version depending on the current directory\n1cd /tmp 2asdf local k3d 5.4.1 3 4asdf current k3d 5k3d 5.4.1 /tmp/.tool-versions \u0026#x1f9f9; Cleanup Uninstall a given version\n1asdf uninstall k3d 5.4.1 Remove a plugin\n1asdf plugin remove k3d ","link":"https://blog.ogenki.io/post/asdf/asdf/","section":"post","tags":["tooling","local"],"title":"Manage tools versions with `asdf`"},{"body":"","link":"https://blog.ogenki.io/tags/tooling/","section":"tags","tags":null,"title":"tooling"},{"body":"","link":"https://blog.ogenki.io/categories/","section":"categories","tags":null,"title":"Categories"},{"body":"","link":"https://blog.ogenki.io/categories/devxp/","section":"categories","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/tags/helm/","section":"tags","tags":null,"title":"Helm"},{"body":"Template challenge Here you’ll be able to practice in order to get familiar with some of the possibilities offered by a templating language.\n(pro tip: Don't forget the testing)\nThese examples may seem useless but the purpose of this is just playing with templates.\n1 - \u0026quot;Configuration depends on region\u0026quot; Create a secret that contains a key 'secret' and a value \u0026quot;myEuropeanSecret\u0026quot;. Set an environment variable from this secret only if the value global.region is 'eu-west1' So the first step is to add the values into the values.yaml file.\n1global: 2 region: eu-west-1 2 - \u0026quot;Create only if\u0026quot; Create a job that prints the pod's IP on stdout based on a boolean value printIP.enabled. Use the \u0026quot;busybox\u0026quot; image and the command wget -qO - http://ipinfo.io/ip. The job should be created only if the value is True, before every other resource has been created (pre-install and pre-upgrade hooks)\n3 - \u0026quot;Looping\u0026quot; Given the following values, create a loop whether in the deployment or in the helpers.tpl file in order to add the environment variables.\n1envVars: 2 key1: value1 3 key2: value2 4 key3: value3 4 - \u0026quot;Playing with strings and sprigs\u0026quot; Add to the \u0026quot;common labels\u0026quot;, a new label \u0026quot;codename\u0026quot; with a value composed with the release name, the chart name and the date in the form 20061225. The release name must be at most 3 characters long. The whole string has to be in snakecase.\n(you should get something like codename: rel_web_20210215)\n5 - We want to create a list of etcd hosts in the form of \u0026quot;etcd-0,etcd-1,etcd-2\u0026quot; based on a integer that defines the number of etcd hosts 1etcd: 2 count: 5 This list has to be defined in an environment variable ETCD_HOSTS\nProposition of solutions try_first You should try to find a solution by your own to the above exercises before checking these solutions\n1 The following command generates a secrets in the templates directory\n1kubectl create secret generic --dry-run=client eu-secret --from-literal=secret=\u0026#39;myEuropeanSecret\u0026#39; -o yaml | kubectl neat \u0026gt; templates/secret.yaml Then we'll enclose the environment variable definition with a condition depending on the region in the deployment template.\n1 {{- if eq .Values.global.region \u0026#34;eu-west-1\u0026#34; }} 2 - name: eu-secret 3 valueFrom: 4 secretKeyRef: 5 name: eu-secret 6 key: secret 7 {{- end }} 2 First of all we need to add a new value\n1printIP: 2 enabled: True Then this command will generate a job yaml\n1kubectl create job my-ip --dry-run=client --image=busybox -o yaml -- wget -qO - http://ipinfo.io/ip | kubectl neat \u0026gt; templates/job.yaml If we enclose the whole yaml, it won't be created if the boolean is False. With the hook annotation here, the job will be created before any other resource will be applied. We defined a delete policy \u0026quot;hook-failed\u0026quot; in order to keep the job, otherwise it would have been deleted.\n1{{- if .Values.printIP.enabled -}} 2apiVersion: batch/v1 3kind: Job 4metadata: 5 name: my-ip 6 annotations: 7 \u0026#34;helm.sh/hook\u0026#34;: pre-install,pre-upgrade 8 \u0026#34;helm.sh/hook-weight\u0026#34;: \u0026#34;1\u0026#34; 9 \u0026#34;helm.sh/hook-delete-policy\u0026#34;: hook-failed 10... 11{{- end -}} 3 If we want to keep the deployment easy to read, we would prefer adding the code in the _helpers.tpl\n1{{/* 2Environment variables 3*/}} 4{{- define \u0026#34;web.envVars\u0026#34; -}} 5{{- range $key, $value := .Values.envVars }} 6- name: {{ $key }} 7 value: {{ $value }} 8{{- end }} 9{{- end -}} Then this new variable could be used in the deployment as follows\n1 env: 2 {{- include \u0026#34;web.envVars\u0026#34; . | nindent 12 }} 4 The common labels can be changed in the file templates/_helpers.tpl. Here's a proposal This one is tricky, I needed to dig back into the charts available in the stable github repository.\n1codename: {{ printf \u0026#34;%s %s %s\u0026#34; (.Release.Name | trunc 3) .Chart.Name (now | date \u0026#34;20060102\u0026#34;) | snakecase }} 5 Here's an option to achieve the expected results.\n1 env: 2 - name: ETCD_HOSTS 3 value: \u0026#34;{{ range $index, $e := until (.Values.etcd.count|int) }}{{- if $index }},{{end}}etcd-{{ $index }}{{- end }}\u0026#34; ","link":"https://blog.ogenki.io/post/series/workshop_helm/templating/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Templating exercises"},{"body":"","link":"https://blog.ogenki.io/series/","section":"series","tags":null,"title":"Series"},{"body":"","link":"https://blog.ogenki.io/series/workshop-helm/","section":"series","tags":null,"title":"Workshop Helm"},{"body":"Create a simple webserver chart In order to get familiar with a typical chart we will create a simple webserver chart.\n1$ helm create web 2Creating web The above command will create a chart directory named web\nweb/ charts directory that contains the subcharts Chart.yaml metadatas (author, version, description), dependencies and more templates contains all the templates basically kubernetes resources in the form of templated yaml files. (go template) deployment.yaml helpers.tpl helpers, functions that can be used from the templates. hpa.yaml ingress.yaml NOTES.txt This file is used to print information after a release has been successfully installed. serviceaccount.yaml service.yaml tests contains a job that will run a command to check the application after it has been installed. test-connection.yaml values.yaml Maybe the most important file. We’ll play with the values to define how the kubernetes resources will be rendered. Testing the chart Here’s a combo if you want to check properly your chart before actually deploying it:\ntemplate + lint + kubeval + test\nGolang errors When you add templating changes, you should run the command helm template --debug \u0026lt;chart_dir\u0026gt;\n1$ helm template --debug web 2install.go:173: [debug] Original chart version: \u0026#34;\u0026#34; 3install.go:190: [debug] CHART PATH: /tmp/web 4 5Error: parse error at (web/templates/_helpers.tpl:73): unexpected EOF 6helm.go:81: [debug] parse error at (web/templates/_helpers.tpl:73): unexpected EOF Read carefully if there are error messages. Always use the option --debug to see the template rendering.\nChart linting The command helm lint \u0026lt;chart_dir\u0026gt; verifies that the chart is well-formed.\n1$ helm lint web/ 2==\u0026gt; Linting web/ 3[ERROR] Chart.yaml: apiVersion \u0026#39;v3\u0026#39; is not valid. The value must be either \u0026#34;v1\u0026#34; or \u0026#34;v2\u0026#34; 4[INFO] Chart.yaml: icon is recommended 5[ERROR] Chart.yaml: chart type is not valid in apiVersion \u0026#39;v3\u0026#39;. It is valid in apiVersion \u0026#39;v2\u0026#39; 6 7Error: 1 chart(s) linted, 1 chart(s) failed Validate Kubernetes resources In order to validate that the rendered kubernetes objects are well-formed we’ll make use of a tool named kubeval.\nThis is even easier by using the Helm plugin.\nInstall the plugin:\n1$ helm plugin install https://github.com/instrumenta/helm-kubeval 2Installing helm-kubeval v0.13.0 ... 3helm-kubeval 0.13.0 is installed. Then check the chart as follows\n1$ helm kubeval web 2The file web/templates/serviceaccount.yaml contains a valid ServiceAccount 3The file web/templates/secret.yaml contains a valid Secret 4... Now you can safely install the chart\n1$ helm upgrade --install web web 2Release \u0026#34;web\u0026#34; has been upgraded. Happy Helming! 3NAME: web 4LAST DEPLOYED: Mon Feb 15 18:22:23 2021 5NAMESPACE: default 6STATUS: deployed 7REVISION: 1 Check that the application works as expected This is a good practice to add tests under the directory template/tests.\nBasically, this is achieved with a job that you can call when the release is already installed (just after)\nIt returns a code 0 if the command succeeds.\nIn the chart we’ve already generated there’s a job that checks the webserver availability.\nCheck that the release is already installed\n1helm list 2NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION 3web default 1 2021-02-15 15:11:09.036602795 +0100 CET deployed web-0.1.0 1.16.0 1helm test web 2NAME: web 3LAST DEPLOYED: Mon Feb 15 15:11:09 2021 4NAMESPACE: default 5STATUS: deployed 6REVISION: 1 7TEST SUITE: web-test-connection 8Last Started: Mon Feb 15 16:55:17 2021 9Last Completed: Mon Feb 15 16:55:19 2021 10Phase: Succeeded Dependencies Sometimes, the application requires another component to work (caching, database, persistence …).\nThis dependency system has to be used with caution because this is generally recommended to manage the applications lifecycles independently from each other.\nLet’s say that our webserver need to store the information related to the sessions in a Redis server.\nWe’ll add a redis server to our web application by declaring the dependency in the file chart.yaml.\n1dependencies: 2 - name: redis 3 version: \u0026#34;12.6.4\u0026#34; 4 repository: https://charts.bitnami.com/bitnami 5 condition: redis.enabled As you may have noticed, this dependency will be pulled only if the condition redis.enabled is True.\nSo we need to change our values.yaml accordingly:\n1redis: 2 enabled: True 3 master: 4 persistence: 5 enabled: False Check all the available values for this chart here.\nWhenever you add a dependency and you’re using a local chart (on your laptop), you must run the following command to pull it\n1helm dep update web 2Hang tight while we grab the latest from your chart repositories... 3…. 4...Successfully got an update from the \u0026#34;bitnami\u0026#34; chart repository 5Update Complete. ⎈Happy Helming!⎈ 6Saving 1 charts 7Downloading redis from repo https://charts.bitnami.com/bitnami 8Deleting outdated charts The dependencies are stored in the directory charts.\nAfter testing your changes you can install the release with the command\nhelm upgrade --install \u0026lt;release_name\u0026gt; \u0026lt;chart_dir\u0026gt;\n1helm upgrade --install web web 2Release \u0026#34;web\u0026#34; has been upgraded. Happy Helming! 3NAME: web 4LAST DEPLOYED: Mon Feb 15 18:22:23 2021 5NAMESPACE: default 6STATUS: deployed 7REVISION: 2 You can notice that your webserver has been successfully installed along with a HA Redis cluster\n1kubectl get po 2NAME READY STATUS RESTARTS AGE 3web-74bf5c6c66-fjsmb 1/1 Running 0 3h14m 4web-test-connection 0/1 Completed 0 90m 5web-redis-master-0 1/1 Running 0 3m14s 6web-redis-slave-0 1/1 Running 0 3m14s 7web-redis-slave-1 1/1 Running 0 2m42s Hooks Helm comes with a hook system that allows it to run jobs at given times of the lifecycle.\nThe description is crystal clear in the documentation and you’ll have the opportunity to add one later on during this workshop.\nMastering the Golang template The main challenge when you start using Helm is to learn all the tips and tricks of the Golang template\nThe official Helm documentation is very useful for that.\nYour best friends when you write Helm templates are the Sprig functions, you should definitely add this to your bookmarks.\nFurthermore, even if it has been deprecated, you should clone/fork the original stable chart repository. Indeed it has a wide range of examples.\nNote that most of the time, if you want to keep the kubernetes manifests readable, you would put most of the code in what we call helpers files. There’s often at least one named _helpers.tpl.\nNote: Even if you can do pretty advanced things with this templating language, you shouldn’t overuse it in order to keep the kubernetes resources readable and the chart maintainable.\n\u0026#x27a1;\u0026#xfe0f; Next: Application lifecycle using Helm\n","link":"https://blog.ogenki.io/post/series/workshop_helm/build_chart/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Build your first chart"},{"body":"Apply a change, anything. For example we will add a label stage: dev. Edit the file templates/_helpers.tpl\n1{{- define \u0026#34;web.labels\u0026#34; -}} 2stage: \u0026#34;dev\u0026#34; 3... Deploy a new revision with the same command we ran previously\n1helm upgrade --install web web Now we can have a look to the changes we’ve made so far to the release\n1$ helm history web 2REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 31 Mon Feb 15 15:11:09 2021 superseded web-0.1.0 1.16.0 Install complete 4... 54 Mon Feb 15 21:15:25 2021 superseded web-0.1.0 1.16.0 Upgrade complete 65 Mon Feb 15 21:21:21 2021 deployed web-0.1.0 1.16.0 Upgrade complete We can then check what would be the changes if we rollback to the previous revision\n1helm diff rollback web 4 2default, web, Deployment (apps) has changed: 3 # Source: web/templates/deployment.yaml 4 apiVersion: apps/v1 5 kind: Deployment 6 metadata: 7 name: web 8 labels: 9- stage: \u0026#34;dev\u0026#34; 10 helm.sh/chart: web-0.1.0 11... Now that we’re sure we can safely rollback to the previous revision\n1helm rollback web 4 2Rollback was a success! Happy Helming! \u0026#x27a1;\u0026#xfe0f; Next: Helm templating challenge\n","link":"https://blog.ogenki.io/post/series/workshop_helm/lifecycle/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Lifecycle operations"},{"body":"Helm’s configuration is stored in the environment variable $HELM_CONFIG_HOME , by default $HOME/.config/helm\nAll the environment variables are described in the documentation there.\nHere are a few tools (with a wide adoption) that will add capabilities to Helm.\nPlugins There are several plugins in order to extend Helm’s features.\nSome of them are really useful (kubeval, diff, secrets).\nHelmfile Helmfile is a really useful tool that allows you to declare the state of the releases on your cluster.\nIt helps keeping a central view of the releases deployed on a given cluster.\nIt automatically configures repositories, pulls dependencies and it is very helpful to build CI/CD workflows.\nOf course it uses Helm under the hood and a few modules/plugins such as secrets decryption, helm diff\nThese steps are very basic, you should have a look at the documentation for further details.\nInstall the helmdiff plugin (used by helmfile)\n1helm plugin install https://github.com/databus23/helm-diff We’ll make use of some examples provided by CloudPosse\n1git clone git@github.com:cloudposse/helmfiles.git 2cd helmfiles Let’s say we want to install the kubernetes dashboard and the reloader tool.\n1cat \u0026gt; releases/kubernetes-dashboard/dev.yaml \u0026lt;\u0026lt;EOF 2installed: True 3banner: \u0026#34;Workshop cluster\u0026#34; 4EOF Now we’ll create our main helmfile.yaml that describes all the releases we want to install\n1cat \u0026gt; helmfile.yaml \u0026lt;\u0026lt;EOF 2helmfiles: 3 - path: \u0026#34;releases/kubernetes-dashboard/helmfile.yaml\u0026#34; 4 values: 5 - releases/kubernetes-dashboard/dev.yaml 6 - path: \u0026#34;releases/reloader/helmfile.yaml\u0026#34; 7 values: 8 - installed: True 9EOF Now we can see what changes will be applied.\n1helmfile diff 2Adding repo stable https://charts.helm.sh/stable 3\u0026#34;stable\u0026#34; has been added to your repositories 4 5Comparing release=kubernetes-dashboard, chart=stable/kubernetes-dashboard 6******************** 7 8 Release was not present in Helm. Diff will show entire contents as new. 9 10… The command helm sync will install the releases\n1helmfile sync 2Adding repo stable https://charts.helm.sh/stable 3\u0026#34;stable\u0026#34; has been added to your repositories 4 5Affected releases are: 6 kubernetes-dashboard (stable/kubernetes-dashboard) UPDATED 7 8Upgrading release=kubernetes-dashboard, chart=stable/kubernetes-dashboard 9Release \u0026#34;kubernetes-dashboard\u0026#34; does not exist. Installing it now. 10NAME: kubernetes-dashboard 11... You can list all the releases managed by the local helmfile.\n1helmfile list 2NAME NAMESPACE ENABLED LABELS 3kubernetes-dashboard kube-system true chart:kubernetes-dashboard,component:monitoring,namespace:kube-system,repo:stable,vendor:kubernetes 4reloader reloader true chart:stakater/reloader,component:reloader,namespace:reloader,repo:stakater,vendor:stakater Delete all the releases\n1helmfile delete 2Listing releases matching ^reloader$ 3reloader reloader 1 2021-02-16 10:10:35.378800455 +0100 CET deployed reloader-v0.0.68 v0.0.68 4 5Deleting reloader 6release \u0026#34;reloader\u0026#34; uninstalled \u0026#x27a1;\u0026#xfe0f; Next: Build a Helm chart\n","link":"https://blog.ogenki.io/post/series/workshop_helm/ecosystem/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Ecosystem"},{"body":"Looking for a chart Helm works with what is called a “chart”. A chart is basically a package of yaml resources that support a templating language.\nBefore building our own chart we should always have a look of what is available in the community. There are often charts that fits our needs.\nThese charts can be installed from different sources: a Helm chart repository, a local archive or chart directory.\nFirst of all, let’s say that we want to install a Wordpress instance on an empty infrastructure.\nWe’ll need to provision a database as well as the Wordpress application.\nLet’s look for a wordpress chart !\nIf you just installed Helm, your repositories list should be empty\n1helm repo list We’re going to check what are the Wordpress charts available in the artifacthub.\nYou can either browse from the web page or use the command\n1helm search hub wordpress 2URL CHART VERSION APP VERSION DESCRIPTION 3https://artifacthub.io/packages/helm/bitnami/wo... 10.6.4 5.6.1 Web publishing platform for building blogs and ... 4https://artifacthub.io/packages/helm/groundhog2... 0.2.6 5.6.0-apache A Helm chart for Wordpress on Kubernetes 5https://artifacthub.io/packages/helm/seccurecod... 2.4.0 4.0 Insecure \u0026amp; Outdated Wordpress Instance: Never e... 6https://artifacthub.io/packages/helm/presslabs/... 0.10.5 0.10.5 Presslabs WordPress Operator Helm Chart Using the Hub there are a few things that can help to choose the best option.\nFirst of all the number of stars obviously and whether the artifact comes from a verified publisher or signed by the maintainer.\nWe’ll get the one provided by Bitnami. In the chart page you’ll be guided with the commands to add Bitnami’s repository.\n1helm repo add bitnami https://charts.bitnami.com/bitnami 2\u0026#34;bitnami\u0026#34; has been added to your repositories 3 4helm repo update From now on we can install all the charts published by Bitnami:\n1helm search repo bitnami 2NAME CHART VERSION APP VERSION DESCRIPTION 3bitnami/bitnami-common 0.0.9 0.0.9 DEPRECATED Chart with custom templates used in ... 4bitnami/airflow 8.0.3 2.0.1 Apache Airflow is a platform to programmaticall... 5bitnami/apache 8.2.3 2.4.46 Chart for Apache HTTP Server 6bitnami/aspnet-core 1.2.3 3.1.9 ASP.NET Core is an open-source framework create... 7bitnami/cassandra 7.3.2 3.11.10 Apache Cassandra is a free and open-source dist... Inspect the chart OK let’s get back to what we want to achieve: Installing a Wordpress instance.\nNow that we identified the chart, we’re going to check what it actually does. You should always check what will be installed.\nyou can download the chart on your laptop and have a look to its content 1helm pull --untar bitnami/wordpress 2 3tree -L 2 wordpress/ 4wordpress/ 5├── Chart.lock 6├── charts 7│ ├── common 8│ └── mariadb 9├── Chart.yaml 10├── ci 11│ ├── ct-values.yaml 12│ ├── ingress-wildcard-values.yaml 13│ ├── values-hpa-pdb.yaml 14│ └── values-metrics-and-ingress.yaml 15├── README.md 16├── templates 17│ ├── configmap.yaml 18… 19│ ├── tests 20│ └── tls-secrets.yaml 21├── values.schema.json 22└── values.yaml read carefully the readme check what are the dependencies pulled by this chart 1helm show chart bitnami/wordpress 2annotations: 3 category: CMS 4apiVersion: v2 5appVersion: 5.6.1 6dependencies: 7- condition: mariadb.enabled 8 name: mariadb 9 repository: https://charts.bitnami.com/bitnami 10 version: 9.x.x 11- name: common 12 repository: https://charts.bitnami.com/bitnami 13 tags: 14 - bitnami-common 15 version: 1.x.x 16... Note: that the wordpress chart defines the mariadb chart as dependency\nLook at the available values 1helm show values bitnami/wordpress Our first release Our next step will be to set our desired values. Indeed you mentioned that Helm uses a templating language to render the manifests. This will help us to configure our instance according to our environment.\nno persistency at all, this is just a workshop 2 replicas for the wordpress instance a database named “foodb” an owner “foobar” for this database passwords All the charts have a file named values.yaml that contains the default values.\nThese values can be overridden at the command line with --set or we can put them in a yaml file that we’ll use with the -f parameter.\nFor this exercise we’ll create a file named “override-values.yaml” and we’ll use the command line for sensitive information.\n1wordpressUsername: foobar 2wordpressPassword: \u0026#34;\u0026#34; 3wordpressBlogName: Foo\u0026#39;s Blog! 4replicaCount: 2 5persistence: 6 enabled: false 7service: 8 type: ClusterIP 9mariadb: 10 auth: 11 rootPassword: \u0026#34;\u0026#34; 12 database: foodb 13 username: foobar 14 password: \u0026#34;\u0026#34; 15 primary: 16 persistence: 17 enabled: false Note: In order to define the values of a subchart you must put the chart name as the first key. here mariadb.values of the mariadb chart.\nHere we go!\nFirst of all we’ll run it in dry-run mode in order to check the yaml rendering (be careful, the passwords are printed in plain text)\n1helm install foo-blog bitnami/wordpress \\ 2-f override-values.yaml \\ 3--set mariadb.auth.rootPassword=r00tP4ss \\ 4--set mariadb.auth.password=us3rP4ss \\ 5--set wordpressPassword=azerty123 \\ 6--dry-run Another word you need to know is Release.\n“A Release is an instance of a chart running in a Kubernetes cluster”. Our release name here is foo-blog\nIf the output looks OK we can install our wordpress, just remove the --dry-run parameter\n1helm install foo-blog bitnami/wordpress -f override-values.yaml --set mariadb.auth.rootPassword=\u0026#34;r00tP4ss\u0026#34; --set mariadb.auth.password=\u0026#34;us3rP4ss\u0026#34; --set wordpressPassword=\u0026#34;azerty123\u0026#34; 2NAME: foo-blog 3LAST DEPLOYED: Fri Feb 12 16:33:21 2021 4NAMESPACE: default 5STATUS: deployed 6REVISION: 1 7NOTES: 8** Please be patient while the chart is being deployed ** 9 10Your WordPress site can be accessed through the following DNS name from within your cluster: 11 12 foo-blog-wordpress.default.svc.cluster.local (port 80) 13 14To access your WordPress site from outside the cluster follow the steps below: 15 161. Get the WordPress URL by running these commands: 17 18 NOTE: It may take a few minutes for the LoadBalancer IP to be available. 19 Watch the status with: \u0026#39;kubectl get svc --namespace default -w foo-blog-wordpress\u0026#39; 20 21 export SERVICE_IP=$(kubectl get svc --namespace default foo-blog-wordpress --template \u0026#34;{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}\u0026#34;) 22 echo \u0026#34;WordPress URL: http://$SERVICE_IP/\u0026#34; 23 echo \u0026#34;WordPress Admin URL: http://$SERVICE_IP/admin\u0026#34; 24 252. Open a browser and access WordPress using the obtained URL. 26 273. Login with the following credentials below to see your blog: 28 29 echo Username: foobar 30 echo Password: $(kubectl get secret --namespace default foo-blog-wordpress -o jsonpath=\u0026#34;{.data.wordpress-password}\u0026#34; | base64 --decode) When the release has been successfully installed you’ll get the above “NOTES” that are very useful to get access to your application. You just have to copy/paste.\nBut first of all we’re going to check that the pods are actually running\n1kubectl get deploy,sts 2NAME READY UP-TO-DATE AVAILABLE AGE 3deployment.apps/foo-blog-wordpress 2/2 2 2 55m 4 5NAME READY AGEkubectl get deploy,sts 6statefulset.apps/foo-blog-mariadb 1/1 55m We didn’t define an ingress for the purpose of the workshop, therefore we’ll use a port-forward\n1kubectl port-forward svc/foo-blog-wordpress 9090:80 Then open a browser using the URL http://localhost:9090/admin, you’ll be prompted to fill in the credentials you defined above. (wordpressPassword)\nWe’ll check the database credentials too as follows\n1MARIADB=$(kubectl get po -l app.kubernetes.io/name=mariadb -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 1kubectl exec -ti ${MARIADB} -- bash -c \u0026#39;mysql -u foobar -pus3rP4ss\u0026#39; 2Welcome to the MariaDB monitor. Commands end with ; or \\g. 3Your MariaDB connection id is 372 4Server version: 10.5.8-MariaDB Source distribution 5 6Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. 7 8Type \u0026#39;help;\u0026#39; or \u0026#39;\\h\u0026#39; for help. Type \u0026#39;\\c\u0026#39; to clear the current input statement. 9 10MariaDB [(none)]\u0026gt; SHOW GRANTS; 11+-------------------------------------------------------------------------------------------------------+ 12| Grants for foobar@% | 13+-------------------------------------------------------------------------------------------------------+ 14| GRANT USAGE ON *.* TO `foobar`@`%` IDENTIFIED BY PASSWORD \u0026#39;*CD5BE357349BDA710A444B0BD741E8EB12B8BC2C\u0026#39; | 15| GRANT ALL PRIVILEGES ON `foodb`.* TO `foobar`@`%` | 16+-------------------------------------------------------------------------------------------------------+ 172 rows in set (0.000 sec) Delete the wordpress release\n1helm uninstall foo-blog Deploy a complete monitoring stack with a single command! The purpose of this step is to show that, even if the stack is composed of dozens of manifest, Helm makes things easy.\n1helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 2\u0026#34;prometheus-community\u0026#34; has been added to your repositories 3 4helm repo update 1helm install kube-prometheus prometheus-community/kube-prometheus-stack --create-namespace --namespace monitoring 2NAME: kube-prometheus 3LAST DEPLOYED: Fri Feb 12 18:03:05 2021 4NAMESPACE: monitoring 5STATUS: deployed 6REVISION: 1 7NOTES: 8kube-prometheus-stack has been installed. Check its status by running: 9 kubectl --namespace monitoring get pods -l \u0026#34;release=kube-prometheus\u0026#34; Check that all the pods are running and run a port-forward\n1kubectl port-forward -n monitoring svc/kube-prometheus-grafana 9090:80 Then open a browser using the URL http://localhost:9090/admin\ndefault credentials: admin / prom-operator\nYou should browse a few minutes over all the dashboards available. There is pretty useful info.\nYou can then have a look to the resources that have been applied with a single command line as follows\n1helm get manifest -n monitoring kube-prometheus Well for a production ready prometheus we would have played a bit with the values but you get the point.\nDelete the kube-prometheus stack\n1helm uninstall -n monitoring kube-prometheus \u0026#x27a1;\u0026#xfe0f; Next: Helm ecosystem\n","link":"https://blog.ogenki.io/post/series/workshop_helm/third_party/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Third party charts"},{"body":"Requirements docker k3d \u0026gt;5.x.x helm \u0026gt;3.x.x helmfile In order to have an easily provisioned temporary playground we’ll make use of k3d which is a lightweight local Kubernetes instance.\nAfter installing the binary you should enable the completion (bash or zsh) as follows (do the same for both helm and k3d).\n1source \u0026lt;(k3d completion bash) Then create the sandbox cluster named “helm-workshop”\n1k3d cluster create helm-workshop 2INFO[0000] Created network \u0026#39;k3d-helm-workshop\u0026#39; 3INFO[0000] Created volume \u0026#39;k3d-helm-workshop-images\u0026#39; 4INFO[0001] Creating node \u0026#39;k3d-helm-workshop-server-0\u0026#39; 5INFO[0006] Creating LoadBalancer \u0026#39;k3d-helm-workshop-serverlb\u0026#39; 6INFO[0007] (Optional) Trying to get IP of the docker host and inject it into the cluster as \u0026#39;host.k3d.internal\u0026#39; for easy access 7INFO[0010] Successfully added host record to /etc/hosts in 2/2 nodes and to the CoreDNS ConfigMap 8INFO[0010] Cluster \u0026#39;helm-workshop\u0026#39; created successfully! 9INFO[0010] You can now use it like this: 10kubectl cluster-info Note that your current configuration should be automatically switched to the newly created cluster.\n1$ kubectl config current-context 2k3d-helm-workshop Playing with third party charts Environment and ecosystem Build your first chart Application lifecycle Templating challenge Other considerations Hosting and versioning Most of the time we would want to share the charts in order to be used on different systems or to pull the dependencies.\nThere are multiple options for that, here are the ones that are generally used.\nChartmuseum is the official solution. This is a pretty simple webserver that exposes a Rest API. Harbor. Its main purpose is to store images (containers), but it offers many other features such as vulnerability scanning, images signing and integrates chartmuseum. Artifactory can be used to stored Helm charts too An OCI store (container registry). Pushing the charts into a central location requires to manage the versions of the charts. Any changes should trigger a version bump in the Chart.yaml file.\nSecrets management One sensitive topic that we didn’t talk about is how to handle secrets.\nThis is not directly related to Helm but this is a general issue on Kubernetes.\nThere are many options, some of them work great with Helm, some others require managing secrets apart from Helm releases.\nIn the ArgoCD documentation they tried to reference all the options available.\nCleanup Pretty simple we’ll drop the whole k3d cluster\n1k3d cluster delete helm-workshop ","link":"https://blog.ogenki.io/post/series/workshop_helm/intro/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop"},{"body":"","link":"https://blog.ogenki.io/categories/containers/","section":"categories","tags":null,"title":"containers"},{"body":"RBAC is the method used by Kubernetes to authorize access to API resources.\n\u0026#x2139;\u0026#xfe0f; When it makes sense you can use the default roles that are available in all Kubernetes installation instead of having to maintain custom ones.\nFor this lab what we want to achieve is to give permissions the following permissions to an application myapp:\nread the configmaps in the namespace foo List the pods in all the namespaces It is a good practice to configure your pod to make use of a serviceaccount. A serviceaccount are used to identify applications and give them permissions if necessary.\nCreate a service account\n1kubectl create serviceaccount myapp 2serviceaccount/myapp created When a service account is created, a token is automatically generated and stored in a secret.\n1kubectl describe sa myapp | grep -i token 2Mountable secrets: myapp-token-bz2zq 3Tokens: myapp-token-bz2zq 4 5kubectl get secret myapp-token-bz2zq --template={{.data.token}} | base64 -d 6eyJhb...EYxhjI_ckZ74A Using a tool to decode the JWT token you should see the following content\n1kubectl get secret myapp-token-bz2zq --template={{.data.token}} | base64 -d | jwt decode - 2 3Token header 4------------ 5{ 6 \u0026#34;alg\u0026#34;: \u0026#34;RS256\u0026#34;, 7 \u0026#34;kid\u0026#34;: \u0026#34;IdsXYO6E93xozgJg-LY2oETTPEHBJjydTU4vF2wy-wg\u0026#34; 8} 9 10Token claims 11------------ 12{ 13 \u0026#34;iss\u0026#34;: \u0026#34;kubernetes/serviceaccount\u0026#34;, 14 \u0026#34;kubernetes.io/serviceaccount/namespace\u0026#34;: \u0026#34;foo\u0026#34;, 15 \u0026#34;kubernetes.io/serviceaccount/secret.name\u0026#34;: \u0026#34;myapp-token-bz2zq\u0026#34;, 16 \u0026#34;kubernetes.io/serviceaccount/service-account.name\u0026#34;: \u0026#34;myapp\u0026#34;, 17 \u0026#34;kubernetes.io/serviceaccount/service-account.uid\u0026#34;: \u0026#34;eb606bdc-b713-4b7c-8da8-c4f71075995e\u0026#34;, 18 \u0026#34;sub\u0026#34;: \u0026#34;system:serviceaccount:foo:myapp\u0026#34; 19} We're going to create a deployment that will be configured to used this serviceaccount. In the yaml you'll notice that we defined the serviceAccountName.\n1kubectl apply -f content/resources/kubernetes_workshop/rbac/deployment.yaml 2deployment.apps/myapp created As we didn't assigned any permissions to this serviceaccount, our application won't be able to call any of the API endpoints\n1POD_NAME=$(kubectl get po -l app=myapp -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 2 3kubectl exec ${POD_NAME} -- kubectl auth can-i -n foo --list 4Resources Non-Resource URLs Resource Names Verbs 5selfsubjectaccessreviews.authorization.k8s.io [] [] [create] 6selfsubjectrulesreviews.authorization.k8s.io [] [] [create] 7 [/.well-known/openid-configuration] [] [get] 8 [/api/*] [] [get] 9 [/api] [] [get] 10 [/apis/*] [] [get] 11 [/apis] [] [get] 12 [/healthz] [] [get] 13 [/healthz] [] [get] 14 [/livez] [] [get] 15 [/livez] [] [get] 16 [/openapi/*] [] [get] 17 [/openapi] [] [get] 18 [/openid/v1/jwks] [] [get] 19 [/readyz] [] [get] 20 [/readyz] [] [get] 21 [/version/] [] [get] 22 [/version/] [] [get] 23 [/version] [] [get] 24 [/version] [] [get] In order to allow it to read configmaps in the namespace foo, we're going to create 2 resources:\nA role which will describe the permissions and which is bounded to a namespace A rolebinding to assign this role to our application (serviceaccount) Create the role\n1kubectl apply -f content/resources/kubernetes_workshop/rbac/role.yaml 2role.rbac.authorization.k8s.io/read-configmaps created And assign it to the serviceaccount we've created previously\n1kubectl create rolebinding -n foo myapp-configmap --serviceaccount=foo:myapp --role=read-configmaps 2rolebinding.rbac.authorization.k8s.io/myapp-configmap created Note that in the above command the serviceaccount must be specified with the namespace as a prefix and separated by a semicolon.\nYou don't have to restart the pod to get the permissions enabled.\n1kubectl exec ${POD_NAME} -- kubectl auth can-i get configmaps -n foo 2yes 3 4kubectl exec ${POD_NAME} -- kubectl get cm 5NAME DATA AGE 6kube-root-ca.crt 1 3d20h 7helloworld 2 2d2h This is possible thanks to the token mounted within the container\n1kubectl exec -ti ${POD_NAME} -- bash -c \u0026#39;curl -skH \u0026#34;Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\u0026#34; https://kubernetes.default/api/v1/namespaces/foo/configmaps\u0026#39; 2{ 3 \u0026#34;kind\u0026#34;: \u0026#34;ConfigMapList\u0026#34;, 4 \u0026#34;apiVersion\u0026#34;: \u0026#34;v1\u0026#34;, 5 \u0026#34;metadata\u0026#34;: { 6 \u0026#34;resourceVersion\u0026#34;: \u0026#34;76334\u0026#34; 7 }, 8 \u0026#34;items\u0026#34;: [ 9 { 10 \u0026#34;metadata\u0026#34;: { 11 \u0026#34;name\u0026#34;: \u0026#34;kube-root-ca.crt\u0026#34;, 12 \u0026#34;namespace\u0026#34;: \u0026#34;foo\u0026#34;, 13 \u0026#34;uid\u0026#34;: \u0026#34;c352e4cd-3b88-4400-80a0-cbba318794e4\u0026#34;, 14... Finally we want to list the pods in all the namespaces of our cluster. we need:\nA clusterrole which will describe the permissions that are cluster wide. A clusterrolebinding to assign this clusterrole to our application (serviceaccount) 1$ kubectl apply -f content/resources/kubernetes_workshop/rbac/clusterrole.yaml 2clusterrole.rbac.authorization.k8s.io/list-pods created 1kubectl create clusterrolebinding -n foo myapp-pods --serviceaccount=foo:myapp --clusterrole=list-pods 2clusterrolebinding.rbac.authorization.k8s.io/myapp-pods created Now lets have a look to the permissions our applications has in the namespace foo\n1kubectl exec ${POD_NAME} -- kubectl auth can-i -n foo --list 2Resources Non-Resource URLs Resource Names Verbs 3selfsubjectaccessreviews.authorization.k8s.io [] [] [create] 4selfsubjectrulesreviews.authorization.k8s.io [] [] [create] 5pods [] [] [get list] 6configmaps [] [] [get watch list] 7 [/.well-known/openid-configuration] [] [get] 8 [/api/*] [] [get] 9 [/api] [] [get] 10 [/apis/*] [] [get] 11... ","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/rbac/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Manage permissions in Kubernetes"},{"body":"","link":"https://blog.ogenki.io/series/workshop-kubernetes/","section":"series","tags":null,"title":"Workshop Kubernetes"},{"body":"Events The first source of information when something goes wrong is the event stream. Note that you may want to sort them by creation time\n1kubectl get events -n foo --sort-by=.metadata.creationTimestamp 2... 320m Normal Created pod/web-85575f4476-5pbqv Created container nginx 420m Normal Started pod/web-85575f4476-5pbqv Started container nginx 520m Normal SuccessfulDelete replicaset/web-987f6cf9 Deleted pod: web-987f6cf9-mzsxd 620m Normal ScalingReplicaSet deployment/web Scaled down replica set web-987f6cf9 to 0 Logs Having a look to a pod's logs is just the matter of running\n1kubectl logs -f --tail=7 -c mysql wordpress-mysql-6c597b98bd-4mbbd 22021-06-24 08:27:38 1 [Note] - \u0026#39;::\u0026#39; resolves to \u0026#39;::\u0026#39;; 32021-06-24 08:27:38 1 [Note] Server socket created on IP: \u0026#39;::\u0026#39;. 42021-06-24 08:27:38 1 [Warning] Insecure configuration for --pid-file: Location \u0026#39;/var/run/mysqld\u0026#39; in the path is accessible to all OS users. Consider choosing a different directory. 52021-06-24 08:27:38 1 [Warning] \u0026#39;proxies_priv\u0026#39; entry \u0026#39;@ root@wordpress-mysql-6c597b98bd-4mbbd\u0026#39; ignored in --skip-name-resolve mode. 62021-06-24 08:27:38 1 [Note] Event Scheduler: Loaded 0 events 72021-06-24 08:27:38 1 [Note] mysqld: ready for connections. 8Version: \u0026#39;5.6.51\u0026#39; socket: \u0026#39;/var/run/mysqld/mysqld.sock\u0026#39; port: 3306 MySQL Community Server (GPL) Alternatively you can use a tool made to display logs from multiple pods: stern. A better way to explore logs is to send them to a central location using a tool such as Loki or the well know EFK stack.\nHealth checks Kubernetes self healing system is mostly based on health checks. There are different types of health checks (please have a look to the official documentation).\nWe'll add a new plugin to kubectl which is really useful to export a resource while cleaning useless metadatas: neat\n1kubectl krew install neat 2Updated the local copy of plugin index. 3Installing plugin: neat 4Installed plugin: neat 5... Let's create a new deployment using the image nginx\n1kubectl create deploy web --image=nginx --dry-run=client -o yaml | kubectl neat \u0026gt; /tmp/web.yaml Edit its content and add an HTTP health check on port 80. The endpoint must return a code ranging between 200 and 400 and it has to be a relevant test that shows the actual availability of the service.\n1apiVersion: apps/v1 2kind: Deployment 3metadata: 4 labels: 5 app: web 6 name: web 7spec: 8 replicas: 1 9 selector: 10 matchLabels: 11 app: web 12 template: 13 metadata: 14 creationTimestamp: null 15 labels: 16 app: web 17 spec: 18 containers: 19 - image: nginx 20 name: nginx 21 livenessProbe: 22 httpGet: 23 path: / 24 port: 80 25 initialDelaySeconds: 3 26 periodSeconds: 3 1kubectl apply -f /tmp/web.yaml 2deployment.apps/web created 3 4kubectl describe deploy web | grep Liveness: 5 Liveness: http-get http://:80/ delay=3s timeout=1s period=3s #success=1 #failure=3 The pod should be up without any error\n1kubectl get po -l app=web 2NAME READY STATUS RESTARTS AGE 3web-85575f4476-6qvd5 1/1 Running 0 92s We're going to simulate a service being unavailable, just change the path being checked. Here we'll use another method to modify a resource by creating a patch and applying it.\nCreate a yaml /tmp/patch.yaml file\n1cat \u0026gt; /tmp/patch.yaml \u0026lt;\u0026lt;EOF 2spec: 3 template: 4 spec: 5 containers: 6 - name: nginx 7 livenessProbe: 8 httpGet: 9 path: /foobar 10EOF And we're going to apply our change as follows\n1kubectl patch deployment web --patch \u0026#34;$(cat /tmp/patch.yaml)\u0026#34; --record 2deployment.apps/web patched 3 4kubectl describe deployment web | grep Liveness: 5 Liveness: http-get http://:80/foobar delay=3s timeout=1s period=3s #success=1 #failure=3 Now our pod should start to fail, the number of restarts increases\n1kubectl get po -l app=web 2web-987f6cf9-n4rnb 1/1 Running 4 83s Until the pod enter in a CrashLoopBackOff, meaning that it constantly restarts.\n1kubectl get po -l app=web 2NAME READY STATUS RESTARTS AGE 3web-987f6cf9-n4rnb 0/1 CrashLoopBackOff 5 3m23s Describing the pod will give you a hint on the reason it restarts\n1kubectl describe po web-987f6cf9-n4rnb | tail -n 5 2Normal Created 4m7s (x3 over 4m30s) kubelet Created container nginx 3Normal Started 4m7s (x3 over 4m30s) kubelet Started container nginx 4Warning Unhealthy 3m56s (x9 over 4m26s) kubelet Liveness probe failed: HTTP probe failed with statuscode: 404 5Normal Killing 3m56s (x3 over 4m20s) kubelet Container nginx failed liveness probe, will be restarted 6Normal Pulling 3m56s (x4 over 4m35s) kubelet Pulling image \u0026#34;nginx\u0026#34; Rollback the latest change in order to return to a working state. Note that we used the option --record when we applied the patch. That helps saving changes history.\n1kubectl rollout history deployment web 2deployment.apps/web 3REVISION CHANGE-CAUSE 41 \u0026lt;none\u0026gt; 52 kubectl patch deployment web --patch=spec: 6 template: 7 spec: 8 containers: 9 - name: nginx 10 livenessProbe: 11 httpGet: 12 path: /foobar --record=true 13 14kubectl rollout undo deployment web 15deployment.apps/web rolled back Cleanup 1kubectl delete deploy web 2deployment.apps \u0026#34;web\u0026#34; deleted learnk8s documentation There is a great documentation that contains all the steps that help debugging a deployment: https://learnk8s.io/troubleshooting-deployments\n\u0026#x27a1;\u0026#xfe0f; Next: RBAC\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/troubleshoot/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Troubleshooting"},{"body":"Resources allocation in Kubernetes Resources allocation in Kubernetes is made using requests and limits in the container's definition.\nrequests: What the container is guaranteed to get. These values are used when the scheduler takes a decision on where (what node) to place a given pod. limits: Are values that cannot be exceeded \u0026#x2139;\u0026#xfe0f; You can use explain to have a look to the documentation of resources.\n1kubectl explain --recursive pod.spec.containers.resources.limits 2KIND: Pod 3VERSION: v1 4 5FIELD: limits \u0026lt;map[string]string\u0026gt; 6 7DESCRIPTION: 8 Limits describes the maximum amount of compute resources allowed. More 9... The wordpress we've created in the previous lab doesn't have resources definition. There are different ways to edit its current state (kubectl edit, apply, patch ...)\n1kubectl edit deploy wordpress replace resources: {} with this block\n1... 2 resources: 3 requests: 4 cpu: 100m 5 memory: 100Mi 6 limits: 7 cpu: 1000m 8 memory: 200Mi 9... The pods resources usage can be displayed using (this might take a few seconds)\n1kubectl top pods 2NAME CPU(cores) MEMORY(bytes) 3wordpress-694866c6b7-mqxdd 1m 171Mi 4wordpress-mysql-6c597b98bd-4mbbd 1m 531Mi Configure the autoscaling base on cpu usage. When a pod reaches 50% of its allocated cpu a new pod is created.\n1kubectl autoscale deployment wordpress --cpu-percent=50 --min=1 --max=5 2horizontalpodautoscaler.autoscaling/wordpress autoscaled It takes up to 15 seconds (default configuration) to get the first values\n1kubectl get hpa 2NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 3wordpress Deployment/wordpress \u0026lt;unknown\u0026gt;/50% 1 5 0 10s 4 5kubectl get hpa 6NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 7wordpress Deployment/wordpress 1%/50% 1 5 1 20s Now we'll run an HTTP bench using wrk. Open a new shell and run\n1kubectl run -ti --rm bench --image=jess/wrk -- /bin/sh -c \u0026#39;wrk -t12 -c100 -d180s http://wordpress\u0026#39; During the benchmark above (3 minutes duration) let's have a look to the hpa\n1watch kubectl get hpa 2Every 2.0s: kubectl get hpa 3hostname: Tue Jun 22 11:13:08 2021 4 5NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 6wordpress Deployment/wordpress 1%/50% 1 5 1 8m28s After a few seconds we'll see that the upscaling will be done automatically. Here the number of replicas will reach the maximum we defined (5 pods).\n1Every 2.0s: kubectl get hpa 2hostname: Tue Jun 22 11:14:13 2021 3 4NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 5wordpress Deployment/wordpress 998%/50% 1 5 5 9m33s That was a pretty simple configuration, basing the autoscaling on CPU usage for a webserver makes sense. You can also base the autoscaling on any other metrics that are reported by your application.\n\u0026#x27a1;\u0026#xfe0f; Next: Troubleshooting\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/autoscaling/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Resources allocation and autoscaling"},{"body":"\u0026#x2139;\u0026#xfe0f; This section is, for a most part, based on the official Kubernetes doc.\nBy the end of this lab we'll create the following components. You may want to come back to this schema from time to time in order to get the whole picture.\nA database with a persistent volume Check that your cluster is up and running and that your context is still configured with the namespace foo\n1kubectl config get-contexts 2CURRENT NAME CLUSTER AUTHINFO NAMESPACE 3* k3d-workshop k3d-workshop admin@k3d-workshop foo Create a persistent volume claim There are several options when it comes to persistent workloads on Kubernetes. For this workshop we'll use our local disks thanks to the local path provisionner.\nCreate a persistentVolumeClaim, it will stay pending until a pod consumes it\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/pvc.yaml 2persistentvolumeclaim/local-path-mysql created 3 4kubectl get pvc 5NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 6local-path-mysql Pending local-path 16s Create the MySQL secret In Kubernetes sensitive data are stored in Secrets. Here we'll create a secret that stores the MySQL root password\n1kubectl create secret generic mysql-pass --from-literal=password=YOUR_PASSWORD 2secret/mysql-pass created Note that a secret is stored in an base64 encoded format and can be easily decoded. (There are best practices to enforce safe access to the secrets that we're not going to cover there)\n1kubectl get secrets mysql-pass -o yaml 2apiVersion: v1 3data: 4 password: WU9VUl9QQVNTV09SRA== 5kind: Secret 6metadata: 7 creationTimestamp: \u0026#34;2021-06-20T09:11:59Z\u0026#34; 8 name: mysql-pass 9 namespace: foo 10 resourceVersion: \u0026#34;2809\u0026#34; 11 uid: c96c58d6-8472-4d68-8554-5dcfb69d834c 12type: Opaque 13 14echo -n \u0026#34;WU9VUl9QQVNTV09SRA==\u0026#34; | base64 -d 15YOUR_PASSWORD Run the MySQL deployment We will now create a MySQL deployment. It will be composed of a single replica as we're accessing to a local volume and it is configured to make use of the secret we've created previously.\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/deployment.yaml 2deployment.apps/wordpress-mysql created 3 4kubectl get po -w 5NAME READY STATUS RESTARTS AGE 6wordpress-mysql-6c597b98bd-vcm62 0/1 ContainerCreating 0 9s 7wordpress-mysql-6c597b98bd-vcm62 1/1 Running 0 13s 8^C Service discovery in Kubernetes In order to be able to call our MySQL deployment we may want to expose it using a service.\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/svc.yaml 2service/wordpress-mysql created 3 4kubectl get svc 5NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 6wordpress-mysql ClusterIP None \u0026lt;none\u0026gt; 3306/TCP 6s Kubernetes's service discovery is based on an internal DNS system. For instance a service A service is accessible using the following nomenclature: \u0026lt;service_name\u0026gt;.\u0026lt;Namespace\u0026gt;.svc.\u0026lt;Cluster_domain_name\u0026gt;\nLet's try to access to the database server using a mysql client pod and create a database named foobar\n1kubectl run -ti --rm mysql-client --restart=Never --image=mysql:5.7 -- /bin/bash 2If you don\u0026#39;t see a command prompt, try pressing enter. 3root@mysql-client:/# apt -qq update \u0026amp;\u0026amp; apt install -yq netcat 4... 5Setting up netcat (1.10-41.1) ... 6 7 8root@mysql-client:/# nc -vz wordpress-mysql.foo.svc.cluster.local 3306 9DNS fwd/rev mismatch: wordpress-mysql.foo.svc.cluster.local != 10-42-1-8.wordpress-mysql.foo.svc.cluster.local 10wordpress-mysql.foo.svc.cluster.local [10.42.1.8] 3306 (?) open 11 12root@mysql-client:/# mysql -u root -h wordpress-mysql -p 13Enter password: 14... 15 16mysql\u0026gt; show databases; 17+--------------------+ 18| Database | 19+--------------------+ 20| information_schema | 21| mysql | 22| performance_schema | 23+--------------------+ 243 rows in set (0.01 sec) 25 26mysql\u0026gt; create database foobar; 27Query OK, 1 row affected (0.00 sec) 28 29mysql\u0026gt; exit 30Bye Note: You can either use the service name wordpress-mysql, or if your source pod is in another namespace use wordpress-mysql.foo\nCheck how the data is persisted with the local-path-provisioner We may want to check how the data is stored. Now that we have a MySQL instance running and consuming the pvc, a persistent volume has been provision\n1kubectl get pvc 2NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 3local-path-mysql Bound pvc-4bb3c033-2261-4d5c-ba61-41e364769599 500Mi RWO local-path 14m Having a closer look we notice that the volume is actually a directory within a worker node.\n1kubectl describe pv pvc-4bb3c033-2261-4d5c-ba61-41e364769599 2Name: pvc-4bb3c033-2261-4d5c-ba61-41e364769599 3Labels: \u0026lt;none\u0026gt; 4Annotations: pv.kubernetes.io/provisioned-by: rancher.io/local-path 5Finalizers: [kubernetes.io/pv-protection] 6StorageClass: local-path 7Status: Bound 8Claim: foo/local-path-mysql 9Reclaim Policy: Delete 10Access Modes: RWO 11VolumeMode: Filesystem 12Capacity: 500Mi 13Node Affinity: 14 Required Terms: 15 Term 0: kubernetes.io/hostname in [k3d-workshop-agent-0] 16Message: 17Source: 18 Type: HostPath (bare host directory volume) 19 Path: /var/lib/rancher/k3s/storage/pvc-4bb3c033-2261-4d5c-ba61-41e364769599_foo_local-path-mysql 20 HostPathType: DirectoryOrCreate 21Events: \u0026lt;none\u0026gt; 1docker exec k3d-workshop-agent-0 ls /var/lib/rancher/k3s/storage/pvc-4bb3c033-2261-4d5c-ba61-41e364769599_foo_local-path-mysql 2auto.cnf 3foobar 4ib_logfile0 5ib_logfile1 6ibdata1 7mysql 8performance_schema That means that even if you restart your laptop you should retrieve the data (here the database foobar we've created previously)\n1k3d cluster stop workshop 2INFO[0000] Stopping cluster \u0026#39;workshop\u0026#39; 3 4k3d cluster list 5NAME SERVERS AGENTS LOADBALANCER 6workshop 0/1 0/1 true 7 8k3d cluster start workshop 9INFO[0000] Starting cluster \u0026#39;workshop\u0026#39; 10INFO[0000] Starting servers... 11INFO[0000] Starting Node \u0026#39;k3d-workshop-server-0\u0026#39; 12INFO[0006] Starting agents... 13INFO[0006] Starting Node \u0026#39;k3d-workshop-agent-0\u0026#39; 14INFO[0013] Starting helpers... 15INFO[0013] Starting Node \u0026#39;k3d-workshop-serverlb\u0026#39; 16 17kubectl run -ti --rm mysql-client --restart=Never --image=mysql:5.7 -- mysql -u root -h wordpress-mysql --password=\u0026#34;YOUR_PASSWORD\u0026#34; 18If you don\u0026#39;t see a command prompt, try pressing enter. 19 20mysql\u0026gt; show databases; 21+--------------------+ 22| Database | 23+--------------------+ 24| information_schema | 25| foobar | 26| mysql | 27| performance_schema | 28+--------------------+ 294 rows in set (0.00 sec) The Wordpress deployment Now we will deploy the wordpress instance with a persistent volume.\nSo first of all create a pvc as follows\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/pvc.yaml 2persistentvolumeclaim/wp-pv-claim created Then create the deployment. Note that it is configured with our mysql database as backend.\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/deployment.yaml 2deployment.apps/wordpress created 3 4$ kubectl get deploy 5NAME READY UP-TO-DATE AVAILABLE AGE 6wordpress-mysql 1/1 1 1 11h 7wordpress 1/1 1 1 4s Most of the time, when we want to expose an HTTP service to the outside world (outside of the cluster), we would create an ingress\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/svc.yaml 2service/wordpress created 3 4kubectl apply -f content/resources/kubernetes_workshop/wordpress/ingress.yaml 5ingress.networking.k8s.io/wordpress created With k3d the ingress endpoint has been defined when we've created the cluster. With the parameter -p \u0026quot;8081:80@loadbalancer\u0026quot; Our wordpress should therefore be accessible through http://localhost:8081\nConfigure your pods A ConfigMap is a kubernetes resource that stores non-sensitive data. Its content can be consumed as config files, environment variables or command args.\nLet's consider that we need a configfile to be mounted in our wordpress deployment as well as an environment variable made available.\nCreate a dumb \u0026quot;hello world\u0026quot; config file\n1echo \u0026#34;Hello World!\u0026#34; \u0026gt; /tmp/helloworld.conf Then we'll create a configmap that contains a file and environment variable we want to make use of. Note This following command doesn't actually apply the resource on our Kubernetes cluster. It just generate a local yaml file using --dry-run and -o yaml.\n1kubectl create configmap helloworld --from-file=/tmp/helloworld.conf --from-literal=HELLO=WORLD -o yaml --dry-run=client \u0026gt; /tmp/cm.yaml Check the configmap\n1apiVersion: v1 2data: 3 HELLO: WORLD 4 helloworld.conf: | 5 Hello World! 6kind: ConfigMap 7metadata: 8 creationTimestamp: null 9 name: helloworld And apply it\n1$ kubectl apply -f /tmp/cm.yaml 2configmap/helloworld created Now we're gonna make use of it by changing the wordpress deployment. For this kind of change it is recommended to use an IDE with a Kubernetes plugin that will highlight errors.\nEdit the file located here: content/resources/kubernetes_workshop/wordpress/deployment.yaml\n1... 2 env: 3 - name: WORDPRESS_DB_HOST 4 value: wordpress-mysql 5 - name: WORDPRESS_DB_PASSWORD 6 valueFrom: 7 secretKeyRef: 8 name: mysql-pass 9 key: password 10 - name: HELLO 11 valueFrom: 12 configMapKeyRef: 13 name: helloworld 14 key: HELLO 15 volumeMounts: 16 - name: wordpress-persistent-storage 17 mountPath: /var/www/html 18 - name: helloworld-config 19 mountPath: /config 20 volumes: 21 - name: wordpress-persistent-storage 22 persistentVolumeClaim: 23 claimName: wp-pv-claim 24 - name: helloworld-config 25 configMap: 26 name: helloworld 27 items: 28 - key: helloworld.conf 29 path: helloworld.conf Applying this change will trigger a rolling-update\n1$ kubectl apply -f content/resources/kubernetes_workshop/wordpress/deployment.yaml 2deployment.apps/wordpress configured 3 4$ kubectl get po 5NAME READY STATUS RESTARTS AGE 6wordpress-mysql-6c597b98bd-4mbbd 1/1 Running 2 41h 7wordpress-594f88c9c4-n9qqr 1/1 Running 0 5s And the configuration will be available in the newly created pod\n1$ kubectl exec -ti wordpress-594f88c9c4-n9qqr -- env | grep HELLO 2HELLO=WORLD 3 4$ kubectl exec -ti wordpress-594f88c9c4-n9qqr -- cat /config/helloworld.conf 5Hello World! \u0026#x26a0;\u0026#xfe0f; Do not delete anything, we'll make use of these resources in the next section.\n\u0026#x27a1;\u0026#xfe0f; Next: Resources in Kubernetes\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/application_stack/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Complete application stack"},{"body":"Prepare your local Kubernetes environment Goal: Having a running local Kubernetes environment\nEnsure that you fulfilled the requirements\nUsing k3d to create a cluster In order to have an easily provisioned temporary playground we’ll make use of k3d which is a lightweight local Kubernetes instance. (Note that they are alternatives to run a local Kubernetes cluster such as: kubeadm, microk8s, minikube)\nAfter installing the binary you should enable the completion (bash or zsh) as follows (do the same for both kubectl and k3d).\n1source \u0026lt;(k3d completion bash) Then create the sandbox cluster named \u0026quot;workshop\u0026quot; with an additional worker\n1k3d cluster create workshop -p \u0026#34;8081:80@loadbalancer\u0026#34; --agents 1 2INFO[0000] Prep: Network 3INFO[0000] Created network \u0026#39;k3d-workshop\u0026#39; (ce74508d3fe09d8622f1ae83effd412d754dfdb441aa9d550723805f9b528c6b) 4INFO[0000] Created volume \u0026#39;k3d-workshop-images\u0026#39; 5INFO[0001] Creating node \u0026#39;k3d-workshop-server-0\u0026#39; 6INFO[0001] Creating node \u0026#39;k3d-workshop-agent-0\u0026#39; 7INFO[0001] Creating LoadBalancer \u0026#39;k3d-workshop-serverlb\u0026#39; 8INFO[0001] Starting cluster \u0026#39;workshop\u0026#39; 9INFO[0001] Starting servers... 10INFO[0001] Starting Node \u0026#39;k3d-workshop-server-0\u0026#39; 11INFO[0006] Starting agents... 12INFO[0006] Starting Node \u0026#39;k3d-workshop-agent-0\u0026#39; 13INFO[0018] Starting helpers... 14INFO[0018] Starting Node \u0026#39;k3d-workshop-serverlb\u0026#39; 15INFO[0019] (Optional) Trying to get IP of the docker host and inject it into the cluster as \u0026#39;host.k3d.internal\u0026#39; for easy access 16INFO[0023] Successfully added host record to /etc/hosts in 3/3 nodes and to the CoreDNS ConfigMap 17INFO[0023] Cluster \u0026#39;workshop\u0026#39; created successfully! 18INFO[0023] --kubeconfig-update-default=false --\u0026gt; sets --kubeconfig-switch-context=false 19INFO[0023] You can now use it like this: 20kubectl config use-context k3d-workshop 21kubectl cluster-info As k3d is made to be used on top of docker you can see the status of the running containers. You should have 3 containers, one for the loadbalancing, one for the control-plane and an agent (worker).\n1docker ps 2CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 34b5847b265dd rancher/k3d-proxy:v4.4.6 \u0026#34;/bin/sh -c nginx-pr…\u0026#34; About a minute ago Up About a minute 80/tcp, 0.0.0.0:43903-\u0026gt;6443/tcp k3d-workshop-serverlb 4523a025087b3 rancher/k3s:v1.21.1-k3s1 \u0026#34;/bin/entrypoint.sh …\u0026#34; About a minute ago Up About a minute k3d-workshop-agent-0 5791b8a69bc1f rancher/k3s:v1.21.1-k3s1 \u0026#34;/bin/entrypoint.sh …\u0026#34; About a minute ago Up About a minute k3d-workshop-server-0 With kubectl you'll see 2 running pods: a control-plane and a worker\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-workshop-agent-0 Ready \u0026lt;none\u0026gt; 2m34s v1.21.1+k3s1 4k3d-workshop-server-0 Ready control-plane,master 2m44s v1.21.1+k3s1 You can also have a look to the default cluster's components that are all located in the namespace kube-system\n1kubectl get pods -n kube-system 2NAME READY STATUS RESTARTS AGE 3helm-install-traefik-crd-h5j7m 0/1 Completed 0 16h 4helm-install-traefik-8mzhk 0/1 Completed 0 16h 5svclb-traefik-gh4rk 2/2 Running 2 16h 6traefik-97b44b794-lcmh4 1/1 Running 1 16h 7coredns-7448499f4d-h7xvn 1/1 Running 2 16h 8local-path-provisioner-5ff76fc89d-qvpf7 1/1 Running 1 16h 9svclb-traefik-cbvmp 2/2 Running 2 16h 10metrics-server-86cbb8457f-5v9ls 1/1 Running 1 16h The CLI configuration The main interface to the Kubernetes API is kubectl. This CLI is configured with what we call a kubeconfig you can have a look at its content wether by having a look at its default location is ~/.kube/config or running the command\n1kubectl config view 2apiVersion: v1 3clusters: 4- cluster: 5 certificate-authority-data: DATA+OMITTED 6 server: https://0.0.... and you can check if the CLI is properly configured by running\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:43903 3CoreDNS is running at https://0.0.0.0:43903/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:43903/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy Kubectl plugins It is really easy to extend the capabilities of he kubectl CLI. Here is a basic \u0026quot;hello-world\u0026quot; example:\nWrite a dumb script, just ensure its name is prefixed with kubectl- and put it in your PATH\n1cat \u0026gt; kubectl-helloworld\u0026lt;\u0026lt;EOF 2#!/bin/bash 3echo \u0026#34;Hello world!\u0026#34; 4EOF 5 6chmod u+x kubectl-helloworld \u0026amp;\u0026amp; sudo mv kubectl-helloworld /usr/local/bin Then it can be used as an argument of kubectl\n1kubectl helloworld 2Hello world! Delete our test\n1sudo rm /usr/local/bin/kubectl-helloworld You can find more information on how to create a kubectl plugin here\nIn order to benefit from the plugins written by the community there's a tool named krew\nUpdate the local index\n1kubectl krew update 2Adding \u0026#34;default\u0026#34; plugin index from https://github.com/kubernetes-sigs/krew-index.git. 3Updated the local copy of plugin index. Browse the available plugins\n1kubectl krew search 2NAME DESCRIPTION INSTALLED 3access-matrix Show an RBAC access matrix for server resources no 4advise-psp Suggests PodSecurityPolicies for cluster. no 5allctx Run commands on contexts in your kubeconfig no 6apparmor-manager Manage AppArmor profiles for cluster. no 7... For the current workshop we'll make use of ctx ns\nctx: Switch between contexts in your kubeconfig (Really helpful when you have multiple clusters to manage) ns: Switch between Kubernetes namespaces (Avoid to specify the namespace for each kubectl commands when working on a given namespace) 1kubectl krew install ctx ns 2Updated the local copy of plugin index. 3Installing plugin: ctx 4... Then you'll be able to switch between contexts (clusters) and namespaces.\n1kubectl ns 2Context \u0026#34;k3d-workshop\u0026#34; modified. 3Active namespace is \u0026#34;kube-system\u0026#34;. \u0026#x27a1;\u0026#xfe0f; Next: Run an application on Kubernetes\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/local/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Local environment"},{"body":"Namespaces Namespaces allow to logically distribute your applications, generally based on teams, projects or applications stacks. Resources names are unique within a namespace. That means that you could have a service named webserver on 2 different namespaces.\nThey can be used to isolate applications using network policies, or to define quotas.\nFor this training we'll work on a namespace named foo\n1kubectl create ns foo 2namespace/foo created And we'll make use of the plugin ns installed in the previous section to set the default namespace as follows\n1kubectl ns 2Context \u0026#34;k3d-workshop\u0026#34; modified. 3Active namespace is \u0026#34;foo\u0026#34;. Check that your kubectl is properly configured, here you can see the cluter and the namespace:\n1kubectl config get-contexts 2CURRENT NAME CLUSTER AUTHINFO NAMESPACE 3* k3d-workshop k3d-workshop admin@k3d-workshop foo Create your first pod Creating resources in Kubernetes is often done by applying a yaml/json definition through the API.\nFirst of all we need to clone this repository and change the current path to its root\n1git clone https://github.com/Smana/workshop_kubernetes_2021.git 2 3cd workshop_kubernetes_2021.git Start by creating a pretty simple pod:\n1kubectl apply -f content/resources/kubernetes_workshop/pod.yaml --namespace foo 2pod/web created 3 4kubectl get po 5NAME READY STATUS RESTARTS AGE 6web 1/1 Running 0 98s We can get detailed information about the pod as follows\n1kubectl describe po web 2Name: web 3Namespace: foo 4Priority: 0 5Node: k3d-workshop-agent-0/172.20.0.3 6Start Time: Fri, 18 Jun 2021 17:05:46 +0200 7Labels: run=web 8Annotations: \u0026lt;none\u0026gt; 9Status: Running 10IP: 10.42.1.6 11... Or even get a specific attribute, here is an example to get the pod's IP\n1kubectl get po web --template={{.status.podIP}} 210.42.1.6 This is worth noting that a pod isn't controlled by a replicaset-controller. That means that when it is deleted, it is not restarted automatically.\n1kubectl delete po web 2pod \u0026#34;web\u0026#34; deleted 3 4kubectl get po 5No resources found in foo namespace. A pod with 2 containers Now create a new pod using the manifest content/resources/kubernetes_workshop/pod2containers.yaml. Look at its content, we will be using a shared temporary directory and we'll mount its content on both containers. That way we can share data between 2 containers of a given pod.\n1kubectl apply -f content/resources/kubernetes_workshop/pod2containers.yaml 2pod/web created 3 4kubectl get pod 5NAME READY STATUS RESTARTS AGE 6web 2/2 Running 0 36s We can check that the logs are accessible on the 2 containers\n1kubectl logs web -c logger --tail=6 -f 2Mon Jun 28 21:06:20 2021 3Mon Jun 28 21:06:21 2021 4Mon Jun 28 21:06:22 2021 5Mon Jun 28 21:06:23 2021 6Mon Jun 28 21:06:24 2021 7 8kubectl exec web -c web -- tail -n 5 /log/out.log 9Mon Jun 28 21:07:19 2021 10Mon Jun 28 21:07:20 2021 11Mon Jun 28 21:07:21 2021 12Mon Jun 28 21:07:22 2021 13Mon Jun 28 21:07:23 2021 Delete the pod\n1kubectl delete po web 2pod \u0026#34;web\u0026#34; deleted Create a simple webserver deployment A deployment is a resource that describes the desired state of an application. Kubernetes will ensure that its current status is aligned with the desired one.\nCreating a simple deployment can be done using kubectl\n1kubectl create deployment podinfo --image stefanprodan/podinfo 2deployment.apps/podinfo created After a few seconds the deployment will be up to date, meaning that the a pod is up and running.\n1kubectl get deploy 2NAME READY UP-TO-DATE AVAILABLE AGE 3podinfo 1/1 1 1 14s Replicas and scaling A deployment creates a replicaset under the hood in order to ensure that the number of replicas (pods) matches the desired one.\n1kubectl get replicasets 2NAME DESIRED CURRENT READY AGE 3podinfo-7fbb45ccfc 1 1 1 36s Creating a deployment without specifying the number of replicas will create a single replica. We can scale it on demand using\n1kubectl scale deploy podinfo --replicas 6 2deployment.apps/podinfo scaled 3 4kubectl rollout status deployment podinfo 5Waiting for deployment \u0026#34;podinfo\u0026#34; rollout to finish: 4 of 6 updated replicas are available... 6Waiting for deployment \u0026#34;podinfo\u0026#34; rollout to finish: 5 of 6 updated replicas are available... 7deployment \u0026#34;podinfo\u0026#34; successfully rolled out The default Kubernetes scheduler will try to spread evenly the pods according to the available resources on worker nodes.\n1kubectl get po -o wide 2NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES 3podinfo-7fbb45ccfc-dwxtx 1/1 Running 0 114s 10.42.1.8 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 4podinfo-7fbb45ccfc-p2djv 1/1 Running 0 34s 10.42.1.11 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 5podinfo-7fbb45ccfc-4fk9z 1/1 Running 0 34s 10.42.1.9 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 6podinfo-7fbb45ccfc-gqwz6 1/1 Running 0 34s 10.42.1.10 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 7podinfo-7fbb45ccfc-4qgvs 1/1 Running 0 34s 10.42.0.8 k3d-workshop-server-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 8podinfo-7fbb45ccfc-r6dn5 1/1 Running 0 34s 10.42.0.9 k3d-workshop-server-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; The deployment controller will ensure to start new pods if the number of replicas doesn't match its configuration.\n1kubectl delete po $(kubectl get po -l app=podinfo -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 2pod \u0026#34;podinfo-7fbb45ccfc-r6dn5\u0026#34; deleted 3 4kubectl describe rs podinfo-7fbb45ccfc 5Name: podinfo-7fbb45ccfc 6Namespace: foo 7Selector: app=podinfo,pod-template-hash=7fbb45ccfc 8Labels: app=podinfo 9 pod-template-hash=7fbb45ccfc 10Annotations: deployment.kubernetes.io/desired-replicas: 6 11 deployment.kubernetes.io/max-replicas: 8 12 deployment.kubernetes.io/revision: 5 13 deployment.kubernetes.io/revision-history: 1,3 14Controlled By: Deployment/podinfo 15Replicas: 6 current / 6 desired 16Pods Status: 6 Running / 0 Waiting / 0 Succeeded / 0 Failed 17... 18Events: 19 Type Reason Age From Message 20 ---- ------ ---- ---- ------- 21... 22 Normal SuccessfulCreate 16h (x3 over 16h) replicaset-controller (combined from similar events): Created pod: podinfo-7fbb45ccfc-pkt4r 23 Normal SuccessfulCreate 96s replicaset-controller Created pod: podinfo-7fbb45ccfc-bkm8n 24 25kubectl get deploy 26NAME READY UP-TO-DATE AVAILABLE AGE 27podinfo 6/6 6 6 18m Rolling update Using a deployment allows to manage the application lifecycle. Changing its configuration will trigger a rolling update.\nFirst of all we'll change the image tag of our deployment\n1kubectl set image deployment podinfo podinfo=stefanprodan/podinfo:5.2.1 2deployment.apps/podinfo image updated During a rolling update a new replicaset is created in order to update the application in place without any downtime. New pods (with the current deployment state) will be created in the new replicaset while they will be deleted progressively from the previous replicaset.\n1kubectl get rs -o wide 2NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR 3podinfo-7fbb45ccfc 0 0 0 21m podinfo stefanprodan/podinfo app=podinfo,pod-template-hash=7fbb45ccfc 4podinfo-564b4ddd7c 6 6 6 30s podinfo stefanprodan/podinfo:5.2.1 app=podinfo,pod-template-hash=564b4ddd7c Keeping the old replicaset makes very easy to rollback\n1kubectl rollout undo deployment podinfo 2deployment.apps/podinfo rolled back 3 4kubectl get rs -o wide 5NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR 6podinfo-7fbb45ccfc 6 6 6 22m podinfo stefanprodan/podinfo app=podinfo,pod-template-hash=7fbb45ccfc 7podinfo-564b4ddd7c 0 0 0 77s podinfo stefanprodan/podinfo:5.2.1 app=podinfo,pod-template-hash=564b4ddd7c Expose a deployment Now that we have a running web application we may want to access it. There are several ways to expose an app, here we'll use the easiest way: Create a service and run a port-forward.\nThe following command will create a service which will be in charge of forwarding calls through the tcp port 9898\n1kubectl expose deploy podinfo --port 9898 2service/podinfo exposed We can get more information on the service as follows\n1kubectl get svc -o yaml podinfo 2apiVersion: v1 3kind: Service 4metadata: 5 labels: 6 app: podinfo 7 name: podinfo 8 namespace: foo 9spec: 10 clusterIP: 10.43.47.17 11 clusterIPs: 12 - 10.43.47.17 13 ipFamilies: 14 - IPv4 15 ipFamilyPolicy: SingleStack 16 ports: 17 - port: 9898 18 selector: 19 app: podinfo A service uses the selector above to identify on which pod to forward the traffic and usually creates the endpoints accordingly.\n1kubectl get po -l app=podinfo 2NAME READY STATUS RESTARTS AGE 3podinfo-7fbb45ccfc-bkm8n 1/1 Running 1 146m 4podinfo-7fbb45ccfc-sbqht 1/1 Running 2 18h 5... 6 7kubectl get endpoints 8NAME ENDPOINTS AGE 9podinfo 10.42.0.16:9898,10.42.0.17:9898,10.42.1.18:9898 + 3 more... 92s The service we've created has an IP that's only accessible from within the cluster. Using the port-forward command we're able to forward the traffic from our local machine to the application (through the API server). Note that you can target either a deployment, a service or a single pod\n1 2kubectl port-forward svc/podinfo 9898 \u0026amp; 3Forwarding from 127.0.0.1:9898 -\u0026gt; 9898 4Forwarding from [::1]:9898 -\u0026gt; 9898 5 6curl http://localhost:9898 7Handling connection for 9898 8{ 9 \u0026#34;hostname\u0026#34;: \u0026#34;podinfo-7fbb45ccfc-sbqht\u0026#34;, 10 \u0026#34;version\u0026#34;: \u0026#34;6.0.0\u0026#34;, 11 \u0026#34;revision\u0026#34;: \u0026#34;\u0026#34;, 12 \u0026#34;color\u0026#34;: \u0026#34;#34577c\u0026#34;, 13 \u0026#34;logo\u0026#34;: \u0026#34;https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif\u0026#34;, 14 \u0026#34;message\u0026#34;: \u0026#34;greetings from podinfo v6.0.0\u0026#34;, 15 \u0026#34;goos\u0026#34;: \u0026#34;linux\u0026#34;, 16 \u0026#34;goarch\u0026#34;: \u0026#34;amd64\u0026#34;, 17 \u0026#34;runtime\u0026#34;: \u0026#34;go1.16.5\u0026#34;, 18 \u0026#34;num_goroutine\u0026#34;: \u0026#34;6\u0026#34;, 19 \u0026#34;num_cpu\u0026#34;: \u0026#34;16\u0026#34; 20} Cleanup In this section we created 2 resources: a deployment and a service.\n1fg 2kubectl port-forward svc/podinfo 9898 3^C 4 5kubectl delete svc,deploy podinfo 6service \u0026#34;podinfo\u0026#34; deleted 7deployment.apps \u0026#34;podinfo\u0026#34; deleted \u0026#x27a1;\u0026#xfe0f; Next: Deploy a Wordpress\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/run_app/","section":"post","tags":["Kubernetes"],"title":"Run an application on Kubernetes"},{"body":"This repository aims to quickly learn the basics of Kubernetes.\n\u0026#x26a0;\u0026#xfe0f; None of the examples given here are made for production.\nRequirements docker k3d \u0026gt;5.x.x kubectl krew (optional)fzf Agenda Prepare your local Kubernetes environment Run an application on Kubernetes Deploy a Wordpress Resources and autoscaling Troubleshooting RBAC Cleanup Pretty simple we’ll drop the whole k3d cluster\n1k3d cluster delete workshop 2INFO[0000] Deleting cluster \u0026#39;workshop\u0026#39; 3... 4INFO[0008] Successfully deleted cluster workshop! \u0026#x27a1;\u0026#xfe0f; You may want to continue with the Helm workshop\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/intro/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop"},{"body":"","link":"https://blog.ogenki.io/tags/index/","section":"tags","tags":null,"title":"index"}]
\ No newline at end of file
+[{"body":"","link":"https://blog.ogenki.io/","section":"","tags":null,"title":""},{"body":"When we talk about securing access to cloud resources, one of the golden rules is to avoid direct exposure to the Internet. This raises a question for Dev/Ops: How can we , for instance, access a database, a Kubernetes cluster, or a server via SSH without compromising security? Virtual Private Networks (VPN) offer an answer by establishing a secure link between different elements of a network, regardless of their geographical location. Solutions range from SaaS models to self-hosted platforms, using various protocols, either open-source or proprietary.\nAmong these options, I wanted to introduce you to Tailscale. This solution uses WireGuard under the hood, known for its simplicity and performance. With Tailscale, you can securely connect devices or servers as if they were on the same local network, even though they are spread across the globe.\n🎯 Our target Understanding how Tailscale works Implementing a secure connection with AWS in a few minutes Interacting with an EKS cluster API via a private network Accessing services hosted on Kubernetes using the private network. Throughout this article, you will obviously need to create a Tailscale account. It's worth noting that authentication is delegated to third-party identity providers (e.g., Okta, OneLogin, Google).\nOnce the account is created, you'll have instant access to the management console. It allows you to list connected devices, visualize logs, and change most of the settings.\n💡 Under the Hood Terminology Mesh VPN: A mesh VPN refers to a VPN network configuration in which every node (i.e., each device or machine) connects directly to every other node, creating a mesh-like structure. This stands in contrast to traditional VPN configurations that usually adopt a \u0026quot;hub and spoke\u0026quot; topology, where multiple clients connect to one central server.\nZero Trust: This approach operates on the principle that every access request to a network is inherently untrusted. Whether an application or user, identity must be proven and authorization given before accessing a resource. Trust isn't assumed based solely on originating from an internal network or a specific geographic location.\nTailnet: When you first use Tailscale, a Tailnet is automatically generated for you, representing your personal private network. Every device within this tailnet is assigned a unique Tailscale IP.\nTailscale's architecture clearly distinguishes between the Control plane and the Data plane:\nThe coordination server primarily manages the exchange of metadata and public keys among all Tailnet members. Importantly, by Tailscale's design, only the public key is shared, with the private key securely retained on its originating node.\nThe nodes in the Tailnet establish a mesh network: Instead of passing data through the coordination server, these nodes communicate directly in a peer-to-peer manner. Each node has a unique identity, allowing it to authenticate and become part of the Tailnet.\n\u0026#x1f4e5; Client Installation Most platforms are supported, and the installation options are listed here. As for me, I am on Archlinux:\n1sudo pacman -S tailscale It's possible to start the service automatically upon machine startup.\n1sudo systemctl enable --now tailscaled To register your personal computer, run the following command:\n1sudo tailscale up --accept-routes 2 3To authenticate, visit: 4 5 https://login.tailscale.com/a/f50... ℹ️ The --accept-routes option is required on Linux in order to configure the routes announced by Subnet routers. We'll delve deeper into this later on in this post.\nEnsure that you've indeed acquired an IP from the Tailscale network:\n1tailscale ip -4 2100.118.83.67 3 4tailscale status 5100.118.83.67 ogenki smainklh@ linux - ℹ️ For Linux users, ensure that Tailscale works well with your DNS configuration: Follow this documentation.\nSources All the steps performed in this article come from this git repository.\nIt will allow you to create all the components aiming to set up an EKS Lab cluster, following a previous blog post on Cilium and Gateway API.\n☁️ Accessing AWS Privately To securely access all resources available on AWS, one can deploy a Subnet router.\nA Subnet router is a Tailscale instance that provides access to subnets not directly linked to Tailscale. It acts as a bridge between Tailscale's virtual private network (Tailnet) and other local networks.\nWe can then reach Cloud subnets through Tailscale's VPN.\n⚠️ To do so, on AWS, you'll need to configure the security groups appropriately to allow Subnet routers.\n🚀 Deploying a Subnet Router Let's dive in and deploy a Subnet router on an AWS network! Everything is done using the Terraform code present in the directory terraform/network. We will analyze the Tailscale-specific configuration present in the tailscale.tf file before deploying.\nThe Terraform provider You can configure several settings via the API Tailscale using the Terraform provider. First, you need to generate an API key 🔑 from the admin console:\nYou should keep this key in a secure store as it is used to deploy the Subnet router.\n1provider \u0026#34;tailscale\u0026#34; { 2 api_key = var.tailscale.api_key 3 tailnet = var.tailscale.tailnet 4} The ACL's\nAccess Control Lists allow you to define who is authorized to communicate with whom (user or device). Upon account creation, they are very permissive, and there are no restrictions (everyone can communicate with everyone).\n1resource \u0026#34;tailscale_acl\u0026#34; \u0026#34;this\u0026#34; { 2 acl = jsonencode({ 3 acls = [ 4 { 5 action = \u0026#34;accept\u0026#34; 6 src = [\u0026#34;*\u0026#34;] 7 dst = [\u0026#34;*:*\u0026#34;] 8 } 9 ] 10... 11} Note For my Lab environment, I kept this default configuration because I'm the only person accessing it. Furthermore, the only devices connected to my Tailnet are my laptop and the Subnet router. However, in a corporate setting, you need to think this through. It's then possible to define a security policy based on groups of people or node tags.\nCheck out this doc for more info.\nDomain Names\nThere are various ways to manage DNS with Tailscale:\nMagic DNS: When a device joins the Tailnet, it registers with a name, and this can be immediately used to communicate with the device.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.115.31.152 ip-10-0-43-98 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 3044 rx 2588 4 5ping ip-10-0-43-98 6PING ip-10-0-43-98.tail9c382.ts.net (100.115.31.152) 56(84) bytes of data. 764 bytes from ip-10-0-43-98.tail9c382.ts.net (100.115.31.152): icmp_seq=1 ttl=64 time=11.4 ms AWS: To use AWS internal domain names, you can utilize the second IP of the VPC, which always corresponds to the DNS server (In our case it would be 10.0.0.2). This enables the use of potential private zones on route53 or connection to resources using domain names.\nThe simplest configuration is thus to declare the list of DNS servers to use and add the AWS one. Here's an example with Cloudflare's public DNS.\n1resource \u0026#34;tailscale_dns_nameservers\u0026#34; \u0026#34;this\u0026#34; { 2 nameservers = [ 3 \u0026#34;1.1.1.1\u0026#34;, 4 cidrhost(module.vpc.vpc_cidr_block, 2) 5 ] 6} The Authentication Key\nFor a device to join the Tailnet at startup, Tailscale must be started using an authentication key. It is generated as follows:\n1resource \u0026#34;tailscale_tailnet_key\u0026#34; \u0026#34;this\u0026#34; { 2 reusable = true 3 ephemeral = false 4 preauthorized = true 5} reusable: As we're using an autoscaling group which can be composed of multiple instances, this key should be reusable multiple times. ephemeral: For this demo, we have created a non-expiring key. In a production environment, enabling expiration would be preferable. preauthorized: This key needs to be valid and preauthorized so that the instance can automatically join Tailscale. The generated key is then used to launch tailscale with the --auth-key parameter.\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; Announcing Routes for AWS subnets\nIt's also necessary to announce the network that we wish to route through the Subnet router. In our example, we choose to route the entire VPC network with the CIDR 10.0.0.0/16.\nTo automate this process, an autoApprovers rule needs to be added. This indicates that the routes announced by the user smainklh@gmail.com are authorized without requiring an approval step.\n1 autoApprovers = { 2 routes = { 3 \u0026#34;10.0.0.0/16\u0026#34; = [\u0026#34;smainklh@gmail.com\u0026#34;] 4 } 5 } Here is the Taiscale command run in the Subnet router:\n1sudo tailscale up --authkey=\u0026lt;REDACTED\u0026gt; --advertise-routes=\u0026#34;10.0.0.0/16\u0026#34; The Terraform Module I've created a module that is straightforward and allows deploying an autoscaling group on AWS and configuring Tailscale. Upon the instance's startup, it will authenticate using an auth_key and announce the specified networks. In the example below, the instance announces the CIDR of the VPC on AWS.\n1module \u0026#34;tailscale_subnet_router\u0026#34; { 2 source = \u0026#34;Smana/tailscale-subnet-router/aws\u0026#34; 3 version = \u0026#34;1.0.4\u0026#34; 4 5 region = var.region 6 env = var.env 7 8 name = var.tailscale.subnet_router_name 9 auth_key = tailscale_tailnet_key.this.key 10 11 vpc_id = module.vpc.vpc_id 12 subnet_ids = module.vpc.private_subnets 13 advertise_routes = [module.vpc.vpc_cidr_block] 14... 15} Now that we've examined the various parameters, it's time to start our Subnet router 🚀 !! First, you need to create a variable.tfvars file in the terraform/network directory.\n1env = \u0026#34;dev\u0026#34; 2region = \u0026#34;eu-west-3\u0026#34; 3private_domain_name = \u0026#34;priv.cloud.ogenki.io\u0026#34; 4 5tailscale = { 6 subnet_router_name = \u0026#34;ogenki\u0026#34; 7 tailnet = \u0026#34;smainklh@gmail.com\u0026#34; 8 api_key = \u0026#34;tskey-api-...\u0026#34; 9} 10 11tags = { 12 project = \u0026#34;demo-secured-eks\u0026#34; 13 owner = \u0026#34;Smana\u0026#34; 14} Then run the following command:\n1tofu plan --var-file variables.tfvars After checking the plan, apply the changes as follows\n1tofu apply --var-file variables.tfvars When the instance starts up, it will appear in the list of devices on the Tailnet.\n1tailscale status 2100.118.83.67 ogenki smainklh@ linux - 3100.68.109.138 ip-10-0-26-99 smainklh@ linux active; relay \u0026#34;par\u0026#34;, tx 33868 rx 32292 We can also check that the route is correctly announced as follows:\n1tailscale status --json|jq \u0026#39;.Peer[] | select(.HostName == \u0026#34;ip-10-0-26-99\u0026#34;) .PrimaryRoutes\u0026#39; 2[ 3 \u0026#34;10.0.0.0/16\u0026#34; 4] ⚠️ For security reasons, do not forget to delete the variables.tfvars file as it contains the API key.\n👏 That's it! We are now able to access the network on AWS, provided we've also set up the filtering rules, such as ACLs and security groups. For instance, we can access a database from the workstation.\n1psql -h demo-tailscale.cymnaynfchjt.eu-west-3.rds.amazonaws.com -U postgres 2Password for user postgres: 3psql (15.4, server 15.3) 4SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off) 5Type \u0026#34;help\u0026#34; for help. 6 7postgres=\u0026gt; 💻 A Different Way to SSH Traditionally, we often connect to servers using the SSH protocol. This involves generating a private key and distributing the corresponding public key to remote servers.\nUnlike traditional SSH key usage, Tailscale uses Wireguard for authentication and encrypted connections, eliminating the need to re-authenticate the client. Furthermore, Tailscale handles the distribution of SSH host keys. Through ACL rules, user access can be revoked without the need to delete SSH keys. There's also a mode that can be activated to enhance security by requiring periodic re-authentication. It's evident that Tailscale SSH simplifies authentication, streamlines SSH connection management, and enhance security levels.\nTo achieve this, one must also initiate Tailscale with the --ssh option. Permissions for SSH are managed at the ACL level as follows:\n1... 2 ssh = [ 3 { 4 action = \u0026#34;check\u0026#34; 5 src = [\u0026#34;autogroup:member\u0026#34;] 6 dst = [\u0026#34;autogroup:self\u0026#34;] 7 users = [\u0026#34;autogroup:nonroot\u0026#34;] 8 } 9 ] 10... The rule mentioned above allows all users to access their own devices via SSH. When trying to connect, they must use a user account other than root. Each connection attempt mandates an additional authentication (action=check). This authentication is carried out by visiting a specific web link.\n1ssh ubuntu@ip-10-0-26-99 2... 3# Tailscale SSH requires an additional check. 4# To authenticate, visit: https://login.tailscale.com/a/f1f09a548cc6 5... 6ubuntu@ip-10-0-26-99:~$ Access logs to the machine can be viewed using journalctl.\n1ubuntu@ip-10-0-26-99:~$ journalctl -aeu tailscaled|grep ssh 2Oct 15 15:51:34 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155130-00ede660b8: handling conn: 100.118.83.67:55098-\u0026gt;ubuntu@100.68.109.138:22 3Oct 15 15:51:56 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: handling conn: 100.118.83.67:44560-\u0026gt;ubuntu@100.68.109.138:22 4Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-conn-20231015T155156-b6d1dc28c0: starting session: sess-20231015T155252-5b2acc170e 5Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): handling new SSH connection from smainklh@gmail.com (100.118.83.67) to ssh-user \u0026#34;ubuntu\u0026#34; 6Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): access granted to smainklh@gmail.com as ssh-user \u0026#34;ubuntu\u0026#34; 7Oct 15 15:52:52 ip-10-0-26-99 tailscaled[1768]: ssh-session(sess-20231015T155252-5b2acc170e): starting pty command: [/usr/sbin/tailscaled be-child ssh --uid=1000 --gid=1000 --groups=1000,4,20,24,25,27,29,30,44,46,115,116 --local-user=ubuntu --remote-user=smainklh@gmail.com --remote-ip=100.118.83.67 --has-tty=true --tty-name=pts/0 --shell --login-cmd=/usr/bin/login --cmd=/bin/bash -- -l] ℹ️ With Tailscale SSH, you can SSH into a device no matter where it's located. However, in a 100% AWS context, one would likely prefer using AWS SSM.\nLogs 💾 In security, retaining logs for future reference is vital. There are various types of logs:\nAudit Logs: These provide insights into who did what. They can be accessed through the admin console and can also be forwarded to a SIEM.\nDevice Logs: Specific commands to each device can be used to view these, such as (journalctl -u tailscaled on Linux).\nNetwork Logs: These are useful for visualizing the connections between devices.\n☸ What About Kubernetes? In the context of Kubernetes, there are several options for accessing a Service:\nProxy: This method involves an additional pod that forwards calls to an existing Service. Sidecar: This allows a pod to connect to the Tailnet. As a result, end-to-end connectivity is established, making bi-directional communication possible (from the pod to the Tailnet nodes). Operator: This approach facilitates exposing Kubernetes services and the API (ingress) and permits pods to access Tailnet nodes (egress). Configuration is achieved by modifying existing resources: Services and Ingresses. In our setup, we already have a Subnet router that routes the entire VPC network. As such, it's sufficient for our service to be exposed via a private IP.\nThe Kubernetes API To access the Kubernetes API, it's essential to authorize the Subnet router. This is accomplished by setting the following rule for the source security group.\nterraform/eks/main.tf\n1module \u0026#34;eks\u0026#34; { 2... 3 cluster_security_group_additional_rules = { 4 ingress_source_security_group_id = { 5 description = \u0026#34;Ingress from the Tailscale security group to the API server\u0026#34; 6 protocol = \u0026#34;tcp\u0026#34; 7 from_port = 443 8 to_port = 443 9 type = \u0026#34;ingress\u0026#34; 10 source_security_group_id = data.aws_security_group.tailscale.id 11 } 12 } 13... 14} Let's ensure that the API is indeed accessible on a private IP.\n1CLUSTER_URL=$(TERM=dumb kubectl cluster-info | grep \u0026#34;Kubernetes control plane\u0026#34; | awk \u0026#39;{print $NF}\u0026#39;) 2 3curl -s -o /dev/null -w \u0026#39;%{remote_ip}\\n\u0026#39; ${CLUSTER_URL} 410.228.244.167 5 6kubectl get ns 7NAME STATUS AGE 8cilium-secrets Active 5m46s 9crossplane-system Active 4m1s 10default Active 23m 11flux-system Active 5m29s 12infrastructure Active 4m1s 13... Accessing Kubernetes Services Privately An exposed Kubernetes Service is just another AWS network resource 😉. We simply need to ensure that this service uses a private IP. In my example, I'm using the Gateway API to configure Clouder's load balancing, and I encourage you to read my previous article on the subject.\nAll that's needed is to create an internal NLB, ensuring that the Service has the annotation service.beta.kubernetes.io/aws-load-balancer-scheme set to internal. With Cilium Gateway API, this is achieved via the Kyverno clusterPolicy.\n1 metadata: 2 annotations: 3 external-dns.alpha.kubernetes.io/hostname: gitops-${cluster_name}.priv.${domain_name},grafana-${cluster_name}.priv.${domain_name} 4 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internal\u0026#34; 5 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 6 spec: 7 loadBalancerClass: service.k8s.aws/nlb However, there's an additional prerequisite since we can't use Let's Encrypt for private DNS zones. I've thus generated an internal PKI that produces self-signed certificates with Cert-manager.\nIn this post, I won't delve into the details of the EKS cluster deployment or the configuration of Flux. Once the cluster is created and all the Kubernetes resources have been reconciled, we have a service that is exposed via an internal AWS LoadBalancer.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2dig +short ${NLB_DOMAIN} 310.0.33.5 410.0.26.228 510.0.9.183 An DNS record is also automatically created for the exposed services, allowing us to access privately through the Flux web interface.\n1dig +short gitops-mycluster-0.priv.cloud.ogenki.io 210.0.9.183 310.0.26.228 410.0.33.5 💭 Final Thoughts Some time ago, in a professional context, I implemented Cloudflare Zero Trust. I've noticed here that Tailscale shares many similarities with this solution. Of course, there are other solutions like Teleport, and the choice of a solution for private access to one's infrastructure depends on the context and security goals.\nFor me, I was particularly convinced by the simplicity of Tailscale's implementation, perfectly meeting my need to access the Clouder network.\nPart of Tailscale's code is open source, especially the client which is under the BSD 3-Clause license. The proprietary part mainly concerns the coordination platform. It's worth noting that there's an open-source alternative named Headscale. This is a distinct initiative with no connection to the Tailscale company.\nFor personal use, Tailscale is quite generous, offering free access for up to 100 devices and 3 users. That said, Tailscale is a serious option to consider for businesses, and in my opinion, it's essential to support companies that have a clear open-source policy and a quality product.\n","link":"https://blog.ogenki.io/post/tailscale/","section":"post","tags":["security","network"],"title":"Beyond Traditional VPNs: Simplifying Cloud Access with `Tailscale`"},{"body":"","link":"https://blog.ogenki.io/tags/network/","section":"tags","tags":null,"title":"network"},{"body":"","link":"https://blog.ogenki.io/post/","section":"post","tags":["index"],"title":"Posts"},{"body":"","link":"https://blog.ogenki.io/tags/security/","section":"tags","tags":null,"title":"security"},{"body":"","link":"https://blog.ogenki.io/tags/","section":"tags","tags":null,"title":"Tags"},{"body":"When deploying an application on Kubernetes, the next step usually involves making it accessible to users. We commonly use Ingress controllers, such as Nginx, Haproxy, Traefik, or those from Cloud providers, to direct incoming traffic to the application, manage load balancing, TLS termination, and more.\nThen we have to choose from the plethora of available options 🤯. Cilium is, relatively recently, one of them and aims to handle all these networking aspects.\nCilium is an Open-Source networking and security solution based on eBPF whose adoption is growing rapidly. It's probably the network plugin that provides the most features. We won't cover all of them, but one such feature involves managing incoming traffic using the Gateway API (GAPI).\n🎯 Our target Understand exactly what the Gateway API is and how it represents an evolution from the Ingress API. Demonstrations of real-world scenarios deployed the GitOps way. Current limitations and upcoming developments. Tip All the steps carried out in this article come from this git repository.\nI encourage you to explore it, as it goes far beyond the context of this article:\nInstallation of an EKS cluster with Cilium configured with the kube-proxy replacement enbled and a dedicated Daemonset for Envoy. Proposal of a Flux structure with dependency management and a DRY code I think is efficient. Crossplane and IRSA composition which simplifies the management of IAM permissions for platform components. Automated domain names and certificates management with External-DNS and Let's Encrypt. The idea being to have everything set up in just a few minutes, with a single command line 🤩.\n☸ Introduction to Gateway API As mentioned previously, there are many Ingress Controllers options, and each has its own specificities and particular features, sometimes making their use complex. Furthermore, the traditionnal Ingress API in Kubernetes has very limited parameters. Some solutions have even created their own CRDs (Kubernetes Custom Resources) while others use annotations to overcome these limitations.\nHere comes the Gateway API! This is actually a standard that allows declaring advanced networking features without requiring specific extensions to the underlying controller. Moreover, since all controllers use the same API, it is possible to switch from one solution to another without changing the configuration (The Kubenetes manifests which describe how the incoming traffic should be routed).\nAmong the concepts that we will explore, GAPI brings a granular authorization model which defines explicit roles with distinct permissions. (More information on the GAPI security model here).\nThis is worth noting that this project is driven by the sig-network-kubernetes working group, and there's a slack channel where you can reach out to them if needed.\nLet's see how GAPI is used in practice with Cilium 🚀!\n\u0026#x2611;\u0026#xfe0f; Prerequisites For the remainder of this article, we assume an EKS cluster has been deployed. If you're not using the method suggested in the demo repo as the basis for this article, there are a few points to check for GAPI to be usable.\nℹ️ The installation method described here is based on Helm, all the values can be viewed here.\nInstall the CRDs available in the Gateway API repository. Note If Cilium is set up with GAPI support (see below) and the CRDs are missing, it won't start. In the demo repo, the GAPI CRDs are installed once during the cluster creation so that Cilium can start, and then they are managed by Flux.\nReplace kube-proxy with the network forwarding features provided by Cilium and eBPF.\n1kubeProxyReplacement: true Enable Gateway API support. 1gatewayAPI: 2 enabled: true Check the installation For that you need to install the command line tool cilium. I personnaly use asdf:\n1asdf plugin-add cilium-cli 2asdf install cilium-cli 0.15.7 3asdf global cilium 0.15.7 The following command allows to ensure that all the components are up and running:\n1cilium status --wait 2 /¯¯\\ 3/¯¯\\__/¯¯\\ Cilium: OK 4\\__/¯¯\\__/ Operator: OK 5/¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6\\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled 8 9Deployment cilium-operator Desired: 2, Ready: 2/2, Available: 2/2 10DaemonSet cilium Desired: 2, Ready: 2/2, Available: 2/2 11DaemonSet cilium-envoy Desired: 2, Ready: 2/2, Available: 2/2 12Containers: cilium Running: 2 13 cilium-operator Running: 2 14 cilium-envoy Running: 2 15Cluster Pods: 33/33 managed by Cilium 16Helm chart version: 1.14.2 17Image versions cilium quay.io/cilium/cilium:v1.14.2@sha256:6263f3a3d5d63b267b538298dbeb5ae87da3efacf09a2c620446c873ba807d35: 2 18 cilium-operator quay.io/cilium/operator-aws:v1.14.2@sha256:8d514a9eaa06b7a704d1ccead8c7e663334975e6584a815efe2b8c15244493f1: 2 19 cilium-envoy quay.io/cilium/cilium-envoy:v1.25.9-e198a2824d309024cb91fb6a984445e73033291d@sha256:52541e1726041b050c5d475b3c527ca4b8da487a0bbb0309f72247e8127af0ec: 2 Finally you can check that the Gateway API support is enabled by running\n1cilium config view | grep -w \u0026#34;enable-gateway-api\u0026#34; 2enable-gateway-api true 3enable-gateway-api-secrets-sync true You could also run end to end tests as follows\n1cilium connectivity test ⚠️ However this command (connectivity test) currently throws errors with Envoy as a DaemonSet enabled. (Github Issue).\nInfo as DaemonSet\nBy default, the Cilium agent also runs Envoy within the same pod and delegates to it level 7 network operations. Since the version v1.14, it is possible to deploy Envoy separately, which brings several benefits:\nIf one modifies/restarts a component (whether it's Cilium or Envoy), it doesn't affect the other. Better allocate resources to each component to optimize performance. Limits the attack surface in case of compromise of one of the pods. Envoy logs and Cilium agent logs are not mixed. You can use the following command to check that this feature is indeed active:\n1cilium status 2 /¯¯\\ 3 /¯¯\\__/¯¯\\ Cilium: OK 4 \\__/¯¯\\__/ Operator: OK 5 /¯¯\\__/¯¯\\ Envoy DaemonSet: OK 6 \\__/¯¯\\__/ Hubble Relay: disabled 7 \\__/ ClusterMesh: disabled More info.\n🚪 The Entry Point: GatewayClass and Gateway Once the conditions are met, we have access to several elements. We can make use of the custom resources defined by the Gateway API CRDs. Moreover, right after installing Cilium, a GatewayClass is immediately available.\n1kubectl get gatewayclasses.gateway.networking.k8s.io 2NAME CONTROLLER ACCEPTED AGE 3cilium io.cilium/gateway-controller True 7m59s On a Kubernetes cluster, you could configure multiple GatewayClasses, thus having the ability to use different implementations. For instance, we can use Linkerd by referencing the GatewayClass in the Gateway configuration.\nThe Gateway is the resource that allows triggering the creation of load balancing components in the Cloud provider.\nHere's a simple example: apps/base/echo/gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo-gateway 5 namespace: echo 6spec: 7 gatewayClassName: cilium 8 listeners: 9 - protocol: HTTP 10 port: 80 11 name: echo-1-echo-server 12 allowedRoutes: 13 namespaces: 14 from: Same On AWS (EKS), when configuring a Gateway, Cilium creates a Service of type LoadBalancer. Then another controller (The AWS Load Balancer Controller) handles the creation of the Cloud load balancer (NLB)\n1kubectl get svc -n echo cilium-gateway-echo 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3cilium-gateway-echo LoadBalancer 172.20.19.82 k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com 80:30395/TCP 2m58s This is worth noting that the load balancer address is also linked to the Gateway.\n1kubectl get gateway -n echo echo 2NAME CLASS ADDRESS PROGRAMMED AGE 3echo cilium k8s-echo-ciliumga-64708ec85c-fcb7661f1ae4e4a4.elb.eu-west-3.amazonaws.com True 16m \u0026#x21aa;\u0026#xfe0f; Routing rules: HTTPRoute A basic rule To summarize the above diagram in a few words: An HTTPRoute allows configuring the routing to the service by referencing the Gateway and defining the desired routing parameters.\nNote workaround\nAs of now, it is not possible to configure the annotations of services generated by the Gateways (Github Issue). A workaround has been proposed to modify the service generated by the Gateway as soon as it is created.\nKyverno is a tool that ensures configuration compliance with best practices and security requirements. We are using it here solely for its ability to easily describe a mutation rule.\nsecurity/mycluster-0/echo-gw-clusterpolicy.yaml\n1spec: 2 rules: 3 - name: mutate-svc-annotations 4 match: 5 any: 6 - resources: 7 kinds: 8 - Service 9 namespaces: 10 - echo 11 name: cilium-gateway-echo 12 mutate: 13 patchStrategicMerge: 14 metadata: 15 annotations: 16 external-dns.alpha.kubernetes.io/hostname: echo.${domain_name} 17 service.beta.kubernetes.io/aws-load-balancer-scheme: \u0026#34;internet-facing\u0026#34; 18 service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp 19 spec: 20 loadBalancerClass: service.k8s.aws/nlb The service cilium-gateway-echo will therefore have the AWS controller's annotations added, as well as an annotation allowing for automatic DNS record configuration.\napps/base/echo/httproute.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 rules: 11 - matches: 12 - path: 13 type: PathPrefix 14 value: / 15 backendRefs: 16 - name: echo-1-echo-server 17 port: 80 The example used above is very simple: all requests are forwarded to the echo-1-echo-server service.\nparentRefs indicates which Gateway to use and then the routing rules are defined under the rules section.\nThe routing rules could also be based on the path.\n1... 2spec: 3 hostnames: 4 - foo.bar.com 5 rules: 6 - matches: 7 - path: 8 type: PathPrefix 9 value: /login Or based on an HTTP Header\n1... 2spec: 3 rules: 4 - matches: 5 headers: 6 - name: \u0026#34;version\u0026#34; 7 value: \u0026#34;2\u0026#34; 8... Let's check if the service is reachable.:\n1curl -s http://echo.cloud.ogenki.io | jq -rc \u0026#39;.environment.HOSTNAME\u0026#39; 2echo-1-echo-server-fd88497d-w6sgn As you can see, the service is exposed in HTTP without a certificate. Let's try to fix that 😉\nConfigure a TLS certificate There are several methods to configure TLS with GAPI. Here, we will use the most common case: HTTPS protocol and TLS termination at the Gateway.\nLet's assume we want to configure the domain name echo.cloud.ogenki.io used earlier. The configuration is mainly done by configuring the Gateway.\napps/base/echo/tls-gateway.yaml\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: Gateway 3metadata: 4 name: echo 5 namespace: echo 6 annotations: 7 cert-manager.io/cluster-issuer: letsencrypt-prod 8spec: 9 gatewayClassName: cilium 10 listeners: 11 - name: http 12 hostname: \u0026#34;echo.${domain_name}\u0026#34; 13 port: 443 14 protocol: HTTPS 15 allowedRoutes: 16 namespaces: 17 from: Same 18 tls: 19 mode: Terminate 20 certificateRefs: 21 - name: echo-tls The essential point here is the reference to a secret containing the certificate named echo-tls. This certificate can be created manually, but for this article, I chose to automate this with Let's Encrypt and cert-manager.\nInfo cert-manager\nWith cert-manager, it's pretty straightforward to automate the creation and update of certificates exposed by the Gateway. For this, you need to allow the controller to access route53 in order to solve a DNS01 challenge (A mechanism that ensures that clients can only request certificates for domains they own).\nA ClusterIssuer resource describes the required configuration to generate certificates with cert-manager.\nNext, we just need to add an annotation cert-manager.io/cluster-issuer and set the Kubernetes secret where the certificate will be stored.\nMore information\nℹ️ In the demo repo, permissions are assigned using Crossplane, which takes care of configuring these IAM perms in AWS.\nFor routing to work correctly, you also need to attach the HTTPRoute to the right Gateway and specify the domain name.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7 parentRefs: 8 - name: echo 9 namespace: echo 10 hostnames: 11 - \u0026#34;echo.${domain_name}\u0026#34; 12... After a few seconds the certificate will be created.\n1kubectl get cert -n echo 2NAME READY SECRET AGE 3echo-tls True echo-tls 43m Finally, we can check that the certificate indeed comes from Let's Encrypt as follows:\n1curl https://echo.cloud.ogenki.io -v 2\u0026gt;\u0026amp;1 | grep -A 6 \u0026#39;Server certificate\u0026#39; 2* Server certificate: 3* subject: CN=echo.cloud.ogenki.io 4* start date: Sep 15 14:43:00 2023 GMT 5* expire date: Dec 14 14:42:59 2023 GMT 6* subjectAltName: host \u0026#34;echo.cloud.ogenki.io\u0026#34; matched cert\u0026#39;s \u0026#34;echo.cloud.ogenki.io\u0026#34; 7* issuer: C=US; O=Let\u0026#39;s Encrypt; CN=R3 8* SSL certificate verify ok. Info GAPI also allows you to configure end-to-end TLS, all the way to the container. This is done by configuring the Gateway in Passthrough mode and using a TLSRoute resource. The certificate must also be carried by the pod that performs the TLS termination.\nSharing a Gateway accross multiple namespaces With GAPI, you can route traffic across Namespaces. This is made possible thanks to distinct resources for each function: A Gateway that allows configuring the infrastructure, and the *Routes. These routes can be attached to a Gateway located in another namespace. It is thus possible for different teams/projects to share the same infrastructure components.\nHowever, this requires to specify which route is allowed to reference the Gateway. Here we assume that we have a Gateway dedicated to internal tools called platform. By using the allowedRoutes parameter, we explicitly specify which namespaces are allowed to be attached to this Gateway.\ninfrastructure/base/gapi/platform-gateway.yaml\n1... 2 allowedRoutes: 3 namespaces: 4 from: Selector 5 selector: 6 matchExpressions: 7 - key: kubernetes.io/metadata.name 8 operator: In 9 values: 10 - observability 11 - flux-system 12 tls: 13 mode: Terminate 14 certificateRefs: 15 - name: platform-tls The HTTPRoutes configured in the namespaces observability and flux-system are attached to this unique Gateway.\n1... 2spec: 3 parentRefs: 4 - name: platform 5 namespace: infrastructure And therefore, use the same load balancer from the Cloud provider.\n1NLB_DOMAIN=$(kubectl get svc -n infrastructure cilium-gateway-platform -o jsonpath={.status.loadBalancer.ingress[0].hostname}) 2 3dig +short ${NLB_DOMAIN} 413.36.89.108 5 6dig +short grafana-mycluster-0.cloud.ogenki.io 713.36.89.108 8 9dig +short gitops-mycluster-0.cloud.ogenki.io 1013.36.89.108 Note 🔒 These internal tools shouldn't be exposed on the Internet, but you know: this is just a demo 🙏. For instance, we could use an internal Gateway (private IP) by playing with the annotations and make use of a private connection system (VPN, tunnels...).\nTraffic splitting One feature that is commonly brought by Service Meshes is the ability to test an application on a portion of the traffic when a new version is available (A/B testing or Canary deployment). GAPI makes this quite simple by using weights.\nHere's an example that forwards 5% of the traffic to the service echo-2-echo-server:\napps/base/echo/httproute-split.yaml\n1... 2 hostnames: 3 - \u0026#34;split-echo.${domain_name}\u0026#34; 4 rules: 5 - matches: 6 - path: 7 type: PathPrefix 8 value: / 9 backendRefs: 10 - name: echo-1-echo-server 11 port: 80 12 weight: 95 13 - name: echo-2-echo-server 14 port: 80 15 weight: 5 Let's check that the distribution happens as expected:\nscripts/check-split.sh\n1./scripts/check-split.sh https://split-echo.cloud.ogenki.io 2Number of requests for echo-1: 95 3Number of requests for echo-2: 5 Headers modifications It is also possible to change HTTP Headers: to add, modify, or delete them. These modifications can be applied to either request or response headers through the use of filters in the HTTPRoute manifest.\nFor instance, we will add a Header to the request.\n1apiVersion: gateway.networking.k8s.io/v1beta1 2kind: HTTPRoute 3metadata: 4 name: echo-1 5 namespace: echo 6spec: 7... 8 rules: 9 - matches: 10 - path: 11 type: PathPrefix 12 value: /req-header-add 13 filters: 14 - type: RequestHeaderModifier 15 requestHeaderModifier: 16 add: 17 - name: foo 18 value: bar 19 backendRefs: 20 - name: echo-1-echo-server 21 port: 80 22... This command allows to that the header is indeed added:\n1curl -s https://echo.cloud.ogenki.io/req-header-add | jq \u0026#39;.request.headers\u0026#39; 2{ 3 \u0026#34;host\u0026#34;: \u0026#34;echo.cloud.ogenki.io\u0026#34;, 4 \u0026#34;user-agent\u0026#34;: \u0026#34;curl/8.2.1\u0026#34;, 5 \u0026#34;accept\u0026#34;: \u0026#34;*/*\u0026#34;, 6 \u0026#34;x-forwarded-for\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 7 \u0026#34;x-forwarded-proto\u0026#34;: \u0026#34;https\u0026#34;, 8 \u0026#34;x-envoy-external-address\u0026#34;: \u0026#34;81.220.234.254\u0026#34;, 9 \u0026#34;x-request-id\u0026#34;: \u0026#34;320ba4d2-3bd6-4c2f-8a97-74296a9f3f26\u0026#34;, 10 \u0026#34;foo\u0026#34;: \u0026#34;bar\u0026#34; 11} 🔒 Assign the proper permissions GAPI offers a clear permission-sharing model between the traffic routing infrastructure (managed by cluster administrators) and the applications (managed by developers).\nThe availability of multiple custom resources allows to use Kubernete's RBAC configuration to assign permissions in a declarative way. I've added a few examples which have no effect in my demo cluster but might give you an idea.\nThe configuration below grants members of the developers group the ability to manage HTTPRoutes within the echo namespace, while only providing them read access to the Gateways.\n1--- 2apiVersion: rbac.authorization.k8s.io/v1 3kind: Role 4metadata: 5 namespace: echo 6 name: gapi-developer 7rules: 8 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 9 resources: [\u0026#34;httproutes\u0026#34;] 10 verbs: [\u0026#34;*\u0026#34;] 11 - apiGroups: [\u0026#34;gateway.networking.k8s.io\u0026#34;] 12 resources: [\u0026#34;gateways\u0026#34;] 13 verbs: [\u0026#34;get\u0026#34;, \u0026#34;list\u0026#34;] 14--- 15apiVersion: rbac.authorization.k8s.io/v1 16kind: RoleBinding 17metadata: 18 name: gapi-developer 19 namespace: echo 20subjects: 21 - kind: Group 22 name: \u0026#34;developers\u0026#34; 23 apiGroup: rbac.authorization.k8s.io 24roleRef: 25 kind: Role 26 name: gapi-developer 27 apiGroup: rbac.authorization.k8s.io 🤔 A somewhat unclear scope at first glance One could be confused with what's commonly referred to as an API Gateway. A section of the FAQ has been created to clarify its difference with the Gateway API. Although GAPI offers features typically found in an API Gateway, it primarily serves as a specific implementation for Kubernetes. However, the choice of this name can indeed cause confusion.\nMoreover please note that this article focuses solely on inbound traffic, termed north-south, traditionally managed by Ingress Controllers. This traffic is actually GAPI's initial scope. A recent initiative named GAMMA aims to also handle east-west routing, which will standardize certain features commonly provided by Service Meshes solutions in the future. (See this article for more details).\n💭 Final thoughts To be honest, I've known about the Gateway API for some time. Although I've read a few articles, I hadn't truly dived deep. I'd think, \u0026quot;Why bother? My Ingress Controller works, and there's a learning curve with this.\u0026quot;\nGAPI is on the rise and nearing its GA release. Several projects have embraced it, and this API for managing traffic within Kubernetes will quickly become the standard.\nI must say, configuring GAPI felt intuitive and explicit ❤️. Its security model strikes a balance, empowering developers without compromising security. And the seamless infrastructure management? You can switch between implementations without touching the *Routes.\nWould I swap my Ingress Controller for Cilium today? Not yet, but it's on the horizon.\nIt's worth highlighting Cilium's broad range of capabilities: With Kubernetes surrounded by a plethora of tools, Cilium stands out, promising features like metrics, tracing, service-mesh, security, and, yes, Ingress Controller with GAPI.\nHowever, there are a few challenges to note:\nTCP and UDP support GRPC support The need to use a mutation rule to configure cloud components (Github Issue). Many of the features discussed in this blog are still in the experimental stage. For instance, the extended functions, which have been supported since the most recent release at the time of my writing (v1.14.2). I attempted to set up a straightforward HTTP\u0026gt;HTTPS redirect but ran into this issue. Consequently, I expect some modifications to the API in the near future. While I've only scratched the surface of what Cilium's GAPI can offer (honestly, this post is already quite long 😜), I am hopeful that we can consider its use in production soon. But considering the points mentioned earlier, I would advise waiting a bit longer. That said if you want to prepare the future, now's the time 😉!\n🔖 References https://gateway-api.sigs.k8s.io/ https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#gs-gateway-api https://isovalent.com/blog/post/cilium-gateway-api/ https://isovalent.com/blog/post/tutorial-getting-started-with-the-cilium-gateway-api/ Isovalent's labs are great to start playing with Gateway API and you'll get new badges to add to your collection 😄 ","link":"https://blog.ogenki.io/post/cilium-gateway-api/","section":"post","tags":["kubernetes","infrastructure","network"],"title":"`Gateway API`: Can I replace my Ingress Controller with `Cilium`?"},{"body":"","link":"https://blog.ogenki.io/tags/infrastructure/","section":"tags","tags":null,"title":"infrastructure"},{"body":"","link":"https://blog.ogenki.io/tags/kubernetes/","section":"tags","tags":null,"title":"kubernetes"},{"body":"Terraform is probably the most used \u0026quot;Infrastructure As Code\u0026quot; tool for building, modifying, and versioning Cloud infrastructure changes. It is an Open Source project developed by Hashicorp that uses the HCL language to declare the desired state of Cloud resources. The state of the created resources is stored in a file called terraform state.\nTerraform can be considered a \u0026quot;semi-declarative\u0026quot; tool as there is no built-in automatic reconciliation feature. There are several solutions to address this issue, but generally speaking, a modification will be applied using terraform apply. The code is actually written using the HCL configuration files (declarative), but the execution is done imperatively. As a result, there can be a drift between the declared and actual state (for example, a colleague who would have changed something directly into the console 😉).\n❓❓ So, how can I ensure that what is committed using Git is really applied. How to be notified if there is a change compared to the desired state and how to automatically apply what is in my code (GitOps)?\nThis is the promise of tf-controller, an Open Source Kubernetes operator from Weaveworks, tightly related to Flux (a GitOps engine from the same company). Flux is one of the solutions I really appreciate, that's why I invite you to have a look on my previous article\nInfo All the steps described in this article come from this Git repo\n🎯 Our target By following the steps in this article, we aim to achieve the following things:\nDeploy a Control plane EKS cluster. Long story short, it will host the Terraform controller that will be in charge of managing all the desired infrastructure components. Use Flux as the GitOps engine for all Kubernetes resources. Regarding the Terraform controller, we will see:\nHow to define dependencies between modules Creation of several AWS resources: Route53 Zone, ACM Certificate, network, EKS cluster. The different reconciliation options (automatic, requiring confirmation) How to backup and restore a Terraform state. 🛠️ Install the tf-controller ☸ The control plane In order to be able to use the Kubernetes controller tf-controller, we first need a Kubernetes cluster 😆. So we are going to create a control plane cluster using the terraform command line and EKS best practices.\nWarning It is crucial that this cluster is resilient, secure, and supervised as it will be responsible for managing all the AWS resources created subsequently.\nWithout going into detail, the control plane cluster was created using this code. That said, it is important to note that all application deployment operations are done using Flux.\nInfo By following the instructions in the README, an EKS cluster will be created but not only! Indeed, it is required to give permissions to the Terraform controller so it will able to apply infrastructure changes. Furthermore, Flux must be installed and configured to apply the configuration defined here.\nWe end up with several components installed and configured:\nThe almost unavoidable addons: aws-loadbalancer-controller and external-dns IRSA roles for these same components are installed using tf-controller Prometheus / Grafana monitoring stack. external-secrets to be able to retrieve sensitive data from AWS secretsmanager. To demonstrate all this after a few minutes the web interface for Flux is accessible via the URL gitops-\u0026lt;cluster_name\u0026gt;.\u0026lt;domain_name\u0026gt;\nStill you should check that Flux is working properly\n1aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0 2Updated context controlplane-0 in /home/smana/.kube/config 1flux check 2... 3✔ all checks passed 4 5flux get kustomizations 6NAME REVISION SUSPENDED READY MESSAGE 7flux-config main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 8flux-system main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 9infrastructure main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 10security main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 11tf-controller main@sha1:e2cdaced False True Applied revision: main@sha1:e2cdaced 12... 📦 The Helm chart and Flux Now that the control plane cluster is available we can add the Terraform controller and this is just the matter of using the Helm chart as follows.\nWe must declare the its Source first:\nsource.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: tf-controller 5spec: 6 interval: 30m 7 url: https://weaveworks.github.io/tf-controller Then we need to define the HelmRelease:\nrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: tf-controller 5spec: 6 releaseName: tf-controller 7 chart: 8 spec: 9 chart: tf-controller 10 sourceRef: 11 kind: HelmRepository 12 name: tf-controller 13 namespace: flux-system 14 version: \u0026#34;0.12.0\u0026#34; 15 interval: 10m0s 16 install: 17 remediation: 18 retries: 3 19 values: 20 resources: 21 limits: 22 memory: 1Gi 23 requests: 24 cpu: 200m 25 memory: 500Mi 26 runner: 27 serviceAccount: 28 annotations: 29 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/tfcontroller_${cluster_name}\u0026#34; When this change is actually written into Git, the HelmRelease will be deployed and the tf-controller started:\n1kubectl get hr -n flux-system 2NAME AGE READY STATUS 3tf-controller 67m True Release reconciliation succeeded 4 5kubectl get po -n flux-system -l app.kubernetes.io/instance=tf-controller 6NAME READY STATUS RESTARTS AGE 7tf-controller-7ffdc69b54-c2brg 1/1 Running 0 2m6s In this demo, there are already a several AWS resources declared. Therefore, after a few minutes, the cluster takes care of creating these: Info Although the majority of operations are performed declaratively or via the CLIs kubectl and flux, another tool allows to manage Terraform resources: tfctl\n🚀 Apply a change One of the Terraform's best practices is to use modules. A module is a set of logically linked Terraform resources bundled into a single reusable unit. They allow to abstract complexity, take inputs, perform specific actions, and produce outputs.\nYou can create your own modules and make them available as Sources or use the many modules shared and maintained by communities. You just need to specify a few variables in order to fit to your context.\nWith tf-controller, the first step is therefore to define the Source of the module. Here we are going to configure the AWS base networking components (vpc, subnets...) using the terraform-aws-vpc module.\n1apiVersion: source.toolkit.fluxcd.io/v1 2kind: GitRepository 3metadata: 4 name: terraform-aws-vpc 5 namespace: flux-system 6spec: 7 interval: 30s 8 ref: 9 tag: v5.0.0 10 url: https://github.com/terraform-aws-modules/terraform-aws-vpc Then we can make use of this Source within a Terraform resource:\nvpc/dev.yaml\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6 interval: 8m 7 path: . 8 destroyResourcesOnDeletion: true # You wouldn\u0026#39;t do that on a prod env ;) 9 storeReadablePlan: human 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-vpc 13 namespace: flux-system 14 vars: 15 - name: name 16 value: vpc-dev 17 - name: cidr 18 value: \u0026#34;10.42.0.0/16\u0026#34; 19 - name: azs 20 value: 21 - \u0026#34;eu-west-3a\u0026#34; 22 - \u0026#34;eu-west-3b\u0026#34; 23 - \u0026#34;eu-west-3c\u0026#34; 24 - name: private_subnets 25 value: 26 - \u0026#34;10.42.0.0/19\u0026#34; 27 - \u0026#34;10.42.32.0/19\u0026#34; 28 - \u0026#34;10.42.64.0/19\u0026#34; 29 - name: public_subnets 30 value: 31 - \u0026#34;10.42.96.0/24\u0026#34; 32 - \u0026#34;10.42.97.0/24\u0026#34; 33 - \u0026#34;10.42.98.0/24\u0026#34; 34 - name: enable_nat_gateway 35 value: true 36 - name: single_nat_gateway 37 value: true 38 - name: private_subnet_tags 39 value: 40 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 41 \u0026#34;karpenter.sh/discovery\u0026#34;: dev 42 - name: public_subnet_tags 43 value: 44 \u0026#34;kubernetes.io/role/elb\u0026#34;: 1 45 writeOutputsToSecret: 46 name: vpc-dev In summary: the terraform code from the terraform-aws-vpc source is applied using the variables defined within vars.\nThere are then several parameters that influence the tf-controller behavior. The main parameters that control how modifications are applied are .spec.approvePlan and .spec.autoApprove\n🚨 Drift detection Setting spec.approvePlan to disable only notifies that the current state of resources has drifted from the Terraform code. This allows you to choose when and how to apply the changes.\nNote This is worth noting that there is a missing section on notifications: Drift, pending plans, reconciliation problems. I'm trying to identify possible methods (preferably with Prometheus) and update this article as soon as possible.\n🔧 Manual execution The given example above (vpc-dev) does not contain the .spec.approvePlan parameter and therefore inherits the default value which is false. In other words, the actual execution of changes (apply) is not done automatically.\nA plan is executed and will be waiting for validation:\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system vpc-dev Unknown Plan generated: set approvePlan: \u0026#34;plan-v5.0.0-26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. true 2 minutes I also advise to configure the storeReadablePlan parameter to human. This allows you to easily visualize the pending modifications using tfctl:\n1tfctl show plan vpc-dev 2 3Terraform used the selected providers to generate the following execution 4plan. Resource actions are indicated with the following symbols: 5 + create 6 7Terraform will perform the following actions: 8 9 # aws_default_network_acl.this[0] will be created 10 + resource \u0026#34;aws_default_network_acl\u0026#34; \u0026#34;this\u0026#34; { 11 + arn = (known after apply) 12 + default_network_acl_id = (known after apply) 13 + id = (known after apply) 14 + owner_id = (known after apply) 15 + tags = { 16 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 17 } 18 + tags_all = { 19 + \u0026#34;Name\u0026#34; = \u0026#34;vpc-dev-default\u0026#34; 20 } 21 + vpc_id = (known after apply) 22 23 + egress { 24 + action = \u0026#34;allow\u0026#34; 25 + from_port = 0 26 + ipv6_cidr_block = \u0026#34;::/0\u0026#34; 27 + protocol = \u0026#34;-1\u0026#34; 28 + rule_no = 101 29 + to_port = 0 30 } 31 + egress { 32... 33Plan generated: set approvePlan: \u0026#34;plan-v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671\u0026#34; to approve this plan. 34To set the field, you can also run: 35 36 tfctl approve vpc-dev -f filename.yaml After reviewing the above modifications, you just need to add the identifier of the plan to validate and push the change to git as follows:\n1apiVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: vpc-dev 5spec: 6... 7 approvePlan: plan-v5.0.0-26c38a66f1 8... After a few seconds, a runner will be launched and will apply the changes:\n1kubectl logs -f -n flux-system vpc-dev-tf-runner 22023/07/01 15:33:36 Starting the runner... version sha 3... 4aws_vpc.this[0]: Creating... 5aws_vpc.this[0]: Still creating... [10s elapsed] 6... 7aws_route_table_association.private[1]: Creation complete after 0s [id=rtbassoc-01b7347a7e9960a13] 8aws_nat_gateway.this[0]: Still creating... [10s elapsed] As soon as the apply is finished the status of the Terraform resource becomes \u0026quot;READY\u0026quot;\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m 🤖 Automatic reconciliation We can also enable automatic reconciliation. To do this, set the .spec.autoApprove parameter to true.\nAll IRSA resources are configured in this way:\nexternal-secrets.yaml\n1piVersion: infra.contrib.fluxcd.io/v1alpha2 2kind: Terraform 3metadata: 4 name: irsa-external-secrets 5spec: 6 approvePlan: auto 7 destroyResourcesOnDeletion: true 8 interval: 8m 9 path: ./modules/iam-role-for-service-accounts-eks 10 sourceRef: 11 kind: GitRepository 12 name: terraform-aws-iam 13 namespace: flux-system 14 vars: 15 - name: role_name 16 value: ${cluster_name}-external-secrets 17 - name: attach_external_secrets_policy 18 value: true 19 - name: oidc_providers 20 value: 21 main: 22 provider_arn: ${oidc_provider_arn} 23 namespace_service_accounts: [\u0026#34;security:external-secrets\u0026#34;] So if I make any change on the AWS console for example, it will be quickly overwritten by the one managed by tf-controller.\nInfo The deletion policy of components created by a Terraform resource is controlled by the setting destroyResourcesOnDeletion. By default anything created is not destroyed by the controller. If you want to destroy the resources when the Terraform object is deleted you must set this parameter to true.\nHere we want to be able to delete IRSA roles because they're tightly linked to a given EKS cluster\n🔄 Inputs/Outputs and modules dependencies When using Terraform, we often need to share data from one module to another. This is done using the outputs that are defined within modules. So we need a way to store them somewhere and import them into another module.\nLet's take again the given example above (vpc-dev). We can see at the bottom of the YAML file, the following block:\n1... 2 writeOutputsToSecret: 3 name: vpc-dev When this resource is applied, we will get a message confirming that the outputs are available (\u0026quot;Outputs written\u0026quot;):\n1kubectl get tf -n flux-system vpc-dev 2NAME READY STATUS AGE 3vpc-dev True Outputs written: v5.0.0@sha1:26c38a66f12e7c6c93b6a2ba127ad68981a48671 17m Indeed this module exports many information (126).\n1kubectl get secrets -n flux-system vpc-dev 2NAME TYPE DATA AGE 3vpc-dev Opaque 126 15s 4 5kubectl get secret -n flux-system vpc-dev --template=\u0026#39;{{.data.vpc_id}}\u0026#39; | base64 -d 6vpc-0c06a6d153b8cc4db Some of these are then used to create a dev EKS cluster. Note that you don't have to read them all, you can cherry pick a few chosen outputs from the secret:\nvpc/dev.yaml\n1... 2 varsFrom: 3 - kind: Secret 4 name: vpc-dev 5 varsKeys: 6 - vpc_id 7 - private_subnets 8... 💾 Backup and restore a tfstate For my demos, I don't want to recreate the zone and the certificate each time the control plane is destroyed (The DNS propagation and certificate validation take time). Here is an example of the steps to take so that I can restore the state of these resources when I use this demo.\nNote This is a manual procedure to demonstrate the behavior of tf-controller with respect to state files. By default, these tfstates are stored in secrets, but we would prefer to configure a GCS or S3 backend.\nThe initial creation of the demo environment allowed me to save the state files (tfstate) as follows.\n1WORKSPACE=\u0026#34;default\u0026#34; 2STACK=\u0026#34;route53-cloud-hostedzone\u0026#34; 3BACKUPDIR=\u0026#34;${HOME}/tf-controller-backup\u0026#34; 4 5mkdir -p ${BACKUPDIR} 6 7kubectl get secrets -n flux-system tfstate-${WORKSPACE}-${STACK} -o jsonpath=\u0026#39;{.data.tfstate}\u0026#39; | \\ 8base64 -d \u0026gt; ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz When the cluster is created again, tf-controller tries to create the zone because the state file is empty.\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3... 4flux-system route53-cloud-hostedzone Unknown Plan generated: set approvePlan: \u0026#34;plan-main@sha1:345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. true 16 minutes 5 6tfctl show plan route53-cloud-hostedzone 7 8Terraform used the selected providers to generate the following execution 9plan. Resource actions are indicated with the following symbols: 10 + create 11 12Terraform will perform the following actions: 13 14 # aws_route53_zone.this will be created 15 + resource \u0026#34;aws_route53_zone\u0026#34; \u0026#34;this\u0026#34; { 16 + arn = (known after apply) 17 + comment = \u0026#34;Experimentations for blog.ogenki.io\u0026#34; 18 + force_destroy = false 19 + id = (known after apply) 20 + name = \u0026#34;cloud.ogenki.io\u0026#34; 21 + name_servers = (known after apply) 22 + primary_name_server = (known after apply) 23 + tags = { 24 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 25 } 26 + tags_all = { 27 + \u0026#34;Name\u0026#34; = \u0026#34;cloud.ogenki.io\u0026#34; 28 } 29 + zone_id = (known after apply) 30 } 31 32Plan: 1 to add, 0 to change, 0 to destroy. 33 34Changes to Outputs: 35 + domain_name = \u0026#34;cloud.ogenki.io\u0026#34; 36 + nameservers = (known after apply) 37 + zone_arn = (known after apply) 38 + zone_id = (known after apply) 39 40Plan generated: set approvePlan: \u0026#34;plan-main@345394fb4a82b9b258014332ddd556dde87f73ab\u0026#34; to approve this plan. 41To set the field, you can also run: 42 43 tfctl approve route53-cloud-hostedzone -f filename.yaml So we need to restore the terraform state as it was when the cloud resources where initially created.\n1cat \u0026lt;\u0026lt;EOF | kubectl apply -f - 2apiVersion: v1 3kind: Secret 4metadata: 5 name: tfstate-${WORKSPACE}-${STACK} 6 namespace: flux-system 7 annotations: 8 encoding: gzip 9type: Opaque 10data: 11 tfstate: $(cat ${BACKUPDIR}/${WORKSPACE}-${STACK}.tfstate.gz | base64 -w 0) 12EOF You will also need to trigger a plan manually\n1tfctl replan route53-cloud-hostedzone 2 Replan requested for flux-system/route53-cloud-hostedzone 3Error: timed out waiting for the condition We can then check that the state file has been updated\n1tfctl get 2NAMESPACE NAME READY MESSAGE PLAN PENDING AGE 3flux-system route53-cloud-hostedzone True Outputs written: main@sha1:d0934f979d832feb870a8741ec01a927e9ee6644 false 19 minutes 🔍 Focus on Key Features of Flux Well, I lied a bit about the agenda 😝. Indeed I want to highlight two features that I hadn't explored until now and that are very useful!\nVariable Substitution When Flux is initialized, some cluster-specific Kustomization files are applied. It is possible to specify variable substitutions within these files, so that they can be used in all resources deployed by this Kustomization. This helps to minimize code duplication.\nI recently discovered the efficiency of this feature. Here is how I use it:\nThe Terraform code that creates an EKS cluster also generates a ConfigMap that contains cluster-specific variables such as the cluster name, as well as all the parameters that vary between clusters.\nflux.tf\n1resource \u0026#34;kubernetes_config_map\u0026#34; \u0026#34;flux_clusters_vars\u0026#34; { 2 metadata { 3 name = \u0026#34;eks-${var.cluster_name}-vars\u0026#34; 4 namespace = \u0026#34;flux-system\u0026#34; 5 } 6 7 data = { 8 cluster_name = var.cluster_name 9 oidc_provider_arn = module.eks.oidc_provider_arn 10 aws_account_id = data.aws_caller_identity.this.account_id 11 region = var.region 12 environment = var.env 13 vpc_id = module.vpc.vpc_id 14 } 15 depends_on = [flux_bootstrap_git.this] 16} As mentioned previously, variable substitutions are defined in the Kustomization files. Let's take a concrete example. Below, we define the Kustomization that deploys all the resources controlled by the tf-controller. Here, we declare the eks-controlplane-0-vars ConfigMap that was generated during the EKS cluster creation.\ninfrastructure.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1 2kind: Kustomization 3metadata: 4 name: tf-custom-resources 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 path: ./infrastructure/controlplane-0/terraform/custom-resources 10 postBuild: 11 substitute: 12 domain_name: \u0026#34;cloud.ogenki.io\u0026#34; 13 substituteFrom: 14 - kind: ConfigMap 15 name: eks-controlplane-0-vars 16 - kind: Secret 17 name: eks-controlplane-0-vars 18 optional: true 19 sourceRef: 20 kind: GitRepository 21 name: flux-system 22 dependsOn: 23 - name: tf-controller Finally, below is an example of a Kubernetes resource that makes use of it. This single manifest can be used by all clusters!.\nexternal-dns/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: external-dns 5spec: 6... 7 values: 8 global: 9 imageRegistry: public.ecr.aws 10 fullnameOverride: external-dns 11 aws: 12 region: ${region} 13 zoneType: \u0026#34;public\u0026#34; 14 batchChangeSize: 1000 15 domainFilters: [\u0026#34;${domain_name}\u0026#34;] 16 logFormat: json 17 txtOwnerId: \u0026#34;${cluster_name}\u0026#34; 18 serviceAccount: 19 annotations: 20 eks.amazonaws.com/role-arn: \u0026#34;arn:aws:iam::${aws_account_id}:role/${cluster_name}-external-dns\u0026#34; This will reduce significantly the number of overlays that were used to patch with cluster-specific parameters.\nWeb UI (Weave GitOps) In my previous article on Flux, I mentioned one of its downsides (when compared to its main competitor, ArgoCD): the lack of a Web interface. While I am a command line guy, this is sometimes useful to have a consolidated view and the ability to perform some operations with just a few clicks \u0026#x1f5b1;\u0026#xfe0f;\nThis is now possible with Weave GitOps! Of course, it is not comparable to ArgoCD's UI, but the essentials are there: pausing reconciliation, visualizing manifests, dependencies, events...\nThere is also the VSCode plugin as an alternative.\n💭 Final thoughts One might say \u0026quot;yet another infrastructure management tool from Kubernetes\u0026quot;. Well this is true but despite a few minor issues faced along the way, which I shared on the project's Git repo, I really enjoyed the experience. tf-controller provides a concrete answer to a common question: how to manage our infrastructure like we manage our code?\nI really like the GitOps approach applied to infrastructure, and I had actually written an article on Crossplane. tf-controller tackles the problem from a different angle: using Terraform directly. This means that we can leverage our existing knowledge and code. There's no need to learn a new way of declaring our resources. This is an important criterion to consider because migrating to a new tool when you already have an existing infrastructure represents a significant effort. However, I would also add that tf-controller is only targeted at Flux users, which restricts its target audience.\nCurrently, I'm using a combination of Terraform, Terragrunt and RunAtlantis. tf-controller could become a serious alternative: We've talked about the value of Kustomize combined with variable substitions to avoid code deduplication. The project's roadmap also aims to display plans in pull-requests. Another frequent need is to pass sensitive information to modules. Using a Terraform resource, we can inject variables from Kubernetes secrets. This makes it possible to use common secrets management tools, such as external-secrets, sealed-secrets ...\nSo, I encourage you to try tf-controller yourself, and perhaps even contribute to it 🙂\nNote The demo I used create quite a few resources, some of which are quite critical (like the network). So, keep in mind that this is just for the demo! I suggest taking a gradual approach if you plan to implement it: start by using drift detection, then create simple resources. I also took some shortcuts in terms of security that should be avoided, such as giving admin rights to the controller. ","link":"https://blog.ogenki.io/post/terraform-controller/","section":"post","tags":["infrastructure"],"title":"Applying GitOps Principles to Infrastructure: An overview of `tf-controller`"},{"body":"Kubernetes is now the de facto platform for orchestrating stateless applications. Containers that don't store data can be destroyed and easily recreated elsewhere. On the other hand, running persistent applications in an ephemeral environment can be quite challenging. There is an increasing number of mature cloud-native database solutions (like CockroachDB, TiDB, K8ssandra, Strimzi...) and there are a lot of things to consider when evaluating them:\nHow mature is the operator? What do the CRDs look like, which options, settings, and status do they expose? Which Kubernetes storage APIs does it leverage? (PV/PVC, CSI, snapshots...) Can it differentiate HDD and SSD, local/remote storage? What happens when something goes wrong: how resilient is the system? Backup and recovery: how easy is it to perform and schedule backups? What replication and scaling options are available? What about connection and concurrency limits, connection pooling, bouncers? Observability: what metrics are exposed and how? I was looking for a solution to host a PostgreSQL database. This database is a requirement for a ticket reservation software named Alf.io that's being used for an upcoming event: The Kubernetes Community Days France. (By the way you're welcome to submit a talk 👐, the CFP closes soon).\nI was specifically looking for a cloud-agnostic solution, with emphasis on ease of use. I was already familiar with several Kubernetes operators, and I ended up evaluating a fairly new kid on the block: CloudNativePG.\nCloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.\nIt has been created by the company EnterpriseDB, who submitted it to the CNCF in order to join the Sandbox projects.\n🎯 Our target I'm going to give you an introduction to the main CloudNativePG features. The plan is to:\ncreate a PostgreSQL database on a GKE cluster, add a standby instance, run a few resiliency tests. We will also see how it behaves in terms of performances and what are the observability tools available. Finally we'll have a look to the backup/restore methods.\nInfo In this article, we will create and update everything manually; but in production, we probably should use a GitOps engine, for instance Flux (which has been covered in a previous article).\nIf you want to see a complete end-to-end example, you can look at the KCD France infrastructure repository.\nAll the manifests shown in this article can be found in this repository.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Tooling gcloud SDK: we're going to deploy on Google Cloud (specifically, on GKE) and we will need to create a few resources in our GCP project; so we'll need the Google Cloud SDK and CLI. If needed, you can install and configure it using this documentation.\nkubectl plugin: to facilitate cluster management, CloudNativePG comes with a handy kubectl plugin that gives insights of your PostgreSQL instance and allows to perform some operations. It can be installed using krew as follows:\n1kubectl krew install cnpg ☁️ Create the Google cloud requirements Before creating our PostgreSQL instance, we need to configure a few things:\nWe need a Kubernetes cluster. This article assumes that you have already taken care of provisioning a GKE cluster. We'll create a bucket to store the backups and WAL files. We'll grant permissions to our pods so that they can write to that bucket. Create the bucket using gcloud CLI\n1gcloud storage buckets create --location=eu --default-storage-class=coldline gs://cnpg-ogenki 2Creating gs://cnpg-ogenki/... 3 4gcloud storage buckets describe gs://cnpg-ogenki 5[...] 6name: cnpg-ogenki 7owner: 8 entity: project-owners-xxxx0008 9projectNumber: \u0026#39;xxx00008\u0026#39; 10rpo: DEFAULT 11selfLink: https://www.googleapis.com/storage/v1/b/cnpg-ogenki 12storageClass: STANDARD 13timeCreated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; 14updated: \u0026#39;2022-10-15T19:27:54.364000+00:00\u0026#39; Now, we're going to give the permissions to the pods (PostgreSQL server) to write/read from the bucket using Workload Identity.\nNote The GKE cluster should be configured with Workload Identity enabled. Check your cluster configuration, you should get something with this command:\n1gcloud container clusters describe \u0026lt;cluster_name\u0026gt; --format json --zone \u0026lt;zone\u0026gt; | jq .workloadIdentityConfig 2{ 3 \u0026#34;workloadPool\u0026#34;: \u0026#34;{{ gcp_project }}.svc.id.goog\u0026#34; 4} Create a Google Cloud service account\n1gcloud iam service-accounts create cloudnative-pg --project={{ gcp_project }} 2Created service account [cloudnative-pg]. Assign the storage.admin permission to the serviceaccount\n1gcloud projects add-iam-policy-binding {{ gcp_project }} \\ 2--member \u0026#34;serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com\u0026#34; \\ 3--role \u0026#34;roles/storage.admin\u0026#34; 4[...] 5- members: 6 - serviceAccount:cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 7 role: roles/storage.admin 8etag: BwXrGA_VRd4= 9version: 1 Allow the Kubernetes service account to impersonate the IAM service account. \u0026#x2139;\u0026#xfe0f; ensure you use the proper format serviceAccount:{{ gcp_project }}.svc.id.goog[{{ kubernetes_namespace }}/{{ kubernetes_serviceaccount }}]\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 7 role: roles/iam.workloadIdentityUser 8etag: BwXrGBjt5kQ= 9version: 1 We're ready to create the Kubernetes resources \u0026#x1f4aa;\n\u0026#x1f511; Create the users secrets We need to create the users credentials that will be used during the bootstrap process (more info later on): The superuser and the newly created database owner.\n1kubectl create secret generic cnpg-mydb-superuser --from-literal=username=postgres --from-literal=password=foobar --namespace demo 2secret/cnpg-mydb-superuser created 1kubectl create secret generic cnpg-mydb-user --from-literal=username=smana --from-literal=password=barbaz --namespace demo 2secret/cnpg-mydb-user created \u0026#x1f6e0;\u0026#xfe0f; Deploy the CloudNativePG operator using Helm CloudNativePG is basically a Kubernetes operator which comes with some CRDs. We'll use the Helm chart as follows\n1helm repo add cnpg https://cloudnative-pg.github.io/charts 2 3helm upgrade --install cnpg --namespace cnpg-system \\ 4--create-namespace charts/cloudnative-pg 5 6kubectl get po -n cnpg-system 7NAME READY STATUS RESTARTS AGE 8cnpg-74488f5849-8lhjr 1/1 Running 0 6h17m Here are the Custom Resource Definitions installed along with the operator.\n1kubectl get crds | grep cnpg.io 2backups.postgresql.cnpg.io 2022-10-08T16:15:14Z 3clusters.postgresql.cnpg.io 2022-10-08T16:15:14Z 4poolers.postgresql.cnpg.io 2022-10-08T16:15:14Z 5scheduledbackups.postgresql.cnpg.io 2022-10-08T16:15:14Z For a full list of the available parameters for these CRDs please refer to the API reference.\n\u0026#x1f680; Create a PostgreSQL server Now we can create our first instance using a custom resource Cluster. The following definition is pretty simple: we want to start a PostgreSQL server, automatically create a database named mydb and configure the credentials based on the secrets created previously.\n1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki 5 namespace: demo 6spec: 7 description: \u0026#34;PostgreSQL Demo Ogenki\u0026#34; 8 imageName: ghcr.io/cloudnative-pg/postgresql:14.5 9 instances: 1 10 11 bootstrap: 12 initdb: 13 database: mydb 14 owner: smana 15 secret: 16 name: cnpg-mydb-user 17 18 serviceAccountTemplate: 19 metadata: 20 annotations: 21 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 22 23 superuserSecret: 24 name: cnpg-mydb-superuser 25 26 storage: 27 storageClass: standard 28 size: 10Gi 29 30 backup: 31 barmanObjectStore: 32 destinationPath: \u0026#34;gs://cnpg-ogenki\u0026#34; 33 googleCredentials: 34 gkeEnvironment: true 35 retentionPolicy: \u0026#34;30d\u0026#34; 36 37 resources: 38 requests: 39 memory: \u0026#34;1Gi\u0026#34; 40 cpu: \u0026#34;500m\u0026#34; 41 limits: 42 memory: \u0026#34;1Gi\u0026#34; Create the namespace where our PostgreSQL instance will be deployed\n1kubectl create ns demo 2namespace/demo created Change the above cluster manifest to fit your needs and apply it.\n1kubectl apply -f cluster.yaml 2cluster.postgresql.cnpg.io/ogenki created You'll notice that the cluster will be in initializing phase. Let's use the cnpg plugin for the first time, it will become our best friend to display a neat view of the cluster's status.\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Primary server is initializing 4Name: ogenki 5Namespace: demo 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: (switching to ogenki-1) 8Status: Setting up primary Creating primary instance ogenki-1 9Instances: 1 10Ready instances: 0 11 12Certificates Status 13Certificate Name Expiration Date Days Left Until Expiration 14---------------- --------------- -------------------------- 15ogenki-ca 2023-01-13 20:02:40 +0000 UTC 90.00 16ogenki-replication 2023-01-13 20:02:40 +0000 UTC 90.00 17ogenki-server 2023-01-13 20:02:40 +0000 UTC 90.00 18 19Continuous Backup status 20First Point of Recoverability: Not Available 21No Primary instance found 22Streaming Replication status 23Not configured 24 25Instances status 26Name Database Size Current LSN Replication role Status QoS Manager Version Node 27---- ------------- ----------- ---------------- ------ --- --------------- ---- The first thing that runs under the hood is the bootstrap process. In our example we create a brand new database named mydb with an owner smana and credentials are retrieved from the secret we created previously.\n1[...] 2 bootstrap: 3 initdb: 4 database: mydb 5 owner: smana 6 secret: 7 name: cnpg-mydb-user 8[...] 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 0/1 Running 0 55s 4ogenki-1-initdb-q75cz 0/1 Completed 0 2m32s After a few seconds the cluster becomes ready \u0026#x1f44f;\n1kubectl cnpg status ogenki -n demo 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7154833472216277012 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 1 10Ready instances: 1 11 12[...] 13 14Instances status 15Name Database Size Current LSN Replication role Status QoS Manager Version Node 16---- ------------- ----------- ---------------- ------ --- --------------- ---- 17ogenki-1 33 MB 0/17079F8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xczh Info There are many ways of bootstrapping your cluster. For instance, restoring a backup into a brand new instance, running SQL scripts... more info here.\n🩹 Standby instance and resiliency Info In traditional PostgreSQL architectures we usually find an additional component to handle high availability (e.g. Patroni). A specific aspect of the CloudNativePG operator is that it leverages built-in Kubernetes features and relies on a component named Postgres instance manager.\nAdd a standby instance by setting the number of replicas to 2.\n1kubectl edit cluster -n demo ogenki 2cluster.postgresql.cnpg.io/ogenki edited 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3[...] 4spec: 5 instances: 2 6[...] The operator immediately notices the change, adds a standby instance, and starts the replication process.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Creating a new replica Creating replica ogenki-2-join 9Instances: 2 10Ready instances: 1 11Current Write LSN: 0/1707A30 (Timeline: 1 - WAL File: 000000010000000000000001) 1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 0 3m16s 4ogenki-2-join-xxrwx 0/1 Pending 0 82s After a while (depending on the amount of data to replicate), the standby instance will be up and running and we can see the replication statistics.\n1kubectl cnpg status -n demo ogenki 2Cluster Summary 3Name: ogenki 4Namespace: demo 5System ID: 7155095145869606932 6PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 7Primary instance: ogenki-1 8Status: Cluster in healthy state 9Instances: 2 10Ready instances: 2 11Current Write LSN: 0/3000060 (Timeline: 1 - WAL File: 000000010000000000000003) 12 13[...] 14 15Streaming Replication status 16Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 17---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 18ogenki-2 0/3000060 0/3000060 0/3000060 0/3000060 00:00:00 00:00:00 00:00:00 streaming async 0 19 20Instances status 21Name Database Size Current LSN Replication role Status QoS Manager Version Node 22---- ------------- ----------- ---------------- ------ --- --------------- ---- 23ogenki-1 33 MB 0/3000060 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 24ogenki-2 33 MB 0/3000060 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc Let's promote the standby instance to primary (perform a Switchover).\nThe cnpg plugin allows to do this imperatively by running this command\n1kubectl cnpg promote ogenki ogenki-2 -n demo 2Node ogenki-2 in cluster ogenki will be promoted In my case the switchover was really fast. We can check that the instance ogenki-2 is now the primary and that the replication is done the other way around.\n1kubectl cnpg status -n demo ogenki 2[...] 3Status: Switchover in progress Switching over to ogenki-2 4Instances: 2 5Ready instances: 1 6[...] 7Streaming Replication status 8Name Sent LSN Write LSN Flush LSN Replay LSN Write Lag Flush Lag Replay Lag State Sync State Sync Priority 9---- -------- --------- --------- ---------- --------- --------- ---------- ----- ---------- ------------- 10ogenki-1 0/4004CA0 0/4004CA0 0/4004CA0 0/4004CA0 00:00:00 00:00:00 00:00:00 streaming async 0 11 12Instances status 13Name Database Size Current LSN Replication role Status QoS Manager Version Node 14---- ------------- ----------- ---------------- ------ --- --------------- ---- 15ogenki-2 33 MB 0/4004CA0 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-xszc 16ogenki-1 33 MB 0/4004CA0 Standby (async) OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 Now let's simulate a Failover by deleting the primary pod\n1kubectl delete po -n demo --grace-period 0 --force ogenki-2 2Warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely. 3pod \u0026#34;ogenki-2\u0026#34; force deleted 1Cluster Summary 2Name: ogenki 3Namespace: demo 4System ID: 7155095145869606932 5PostgreSQL Image: ghcr.io/cloudnative-pg/postgresql:14.5 6Primary instance: ogenki-1 7Status: Failing over Failing over from ogenki-2 to ogenki-1 8Instances: 2 9Ready instances: 1 10Current Write LSN: 0/4005D98 (Timeline: 3 - WAL File: 000000030000000000000004) 11 12[...] 13Instances status 14Name Database Size Current LSN Replication role Status QoS Manager Version Node 15---- ------------- ----------- ---------------- ------ --- --------------- ---- 16ogenki-1 33 MB 0/40078D8 Primary OK Burstable 1.18.0 gke-kcdfrance-main-np-0e87115b-76k7 17ogenki-2 - - - pod not available Burstable - gke-kcdfrance-main-np-0e87115b-xszc After a few seconds, the cluster becomes healthy again\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 13m 2 2 Cluster in healthy state ogenki-1 So far so good, we've been able to test the high availability and the experience is pretty smooth 😎.\n👁️ Monitoring We're going to use Prometheus Stack. We won't cover its installation in this article. If you want to see how to install it \u0026quot;the GitOps way\u0026quot; you can check this example.\nTo scrape our instance's metrics, we need to create a PodMonitor.\n1apiVersion: monitoring.coreos.com/v1 2kind: PodMonitor 3metadata: 4 labels: 5 prometheus-instance: main 6 name: cnpg-ogenki 7 namespace: demo 8spec: 9 namespaceSelector: 10 matchNames: 11 - demo 12 podMetricsEndpoints: 13 - port: metrics 14 selector: 15 matchLabels: 16 postgresql: ogenki We can then add the Grafana dashboard available here.\nFinally, you may want to configure alerts and you can create a PrometheusRule using these rules.\n\u0026#x1f525; Performances and benchmark Info Update: It is now possible to use the cnpg plugin. The following method is deprecated, I'll update it asap.\nThis is worth running a performance test in order to know the limits of your current server and keep a baseline for further improvements.\nNote When it comes to performance there are many improvement areas we can work on. It mostly depends on the target we want to achieve. Indeed we don't want to waste time and money for performance we'll likely never need.\nHere are the main things to look at:\nPostgreSQL configuration tuning Compute resources (cpu and memory) Disk type IOPS, local storage (local-volume-provisioner), Dedicated disks for WAL and PG_DATA Connection pooling PGBouncer. The CloudNativePG comes with a CRD Pooler to handle that. Database optimization, analyzing the query plans using explain, use the extension pg_stat_statement ... First of all we'll add labels to the nodes in order to run the pgbench command on different machines than the ones hosting the database.\n1PG_NODE=$(kubectl get po -n demo -l postgresql=ogenki,role=primary -o jsonpath={.items[0].spec.nodeName}) 2kubectl label node ${PG_NODE} workload=postgresql 3node/gke-kcdfrance-main-np-0e87115b-vlzm labeled 4 5 6# Choose any other node different than the ${PG_NODE} 7kubectl label node gke-kcdfrance-main-np-0e87115b-p5d7 workload=pgbench 8node/gke-kcdfrance-main-np-0e87115b-p5d7 labeled And we'll deploy the Helm chart as follows\n1git clone git@github.com:EnterpriseDB/cnp-bench.git 2cd cnp-bench 3 4cat \u0026gt; pgbench-benchmark/myvalues.yaml \u0026lt;\u0026lt;EOF 5cnp: 6 existingCluster: true 7 existingHost: ogenki-rw 8 existingCredentials: cnpg-mydb-superuser 9 existingDatabase: mydb 10 11pgbench: 12 # Node where to run pgbench 13 nodeSelector: 14 workload: pgbench 15 initialize: true 16 scaleFactor: 1 17 time: 600 18 clients: 10 19 jobs: 1 20 skipVacuum: false 21 reportLatencies: false 22EOF 23 24helm upgrade --install -n demo pgbench -f pgbench-benchmark/myvalues.yaml pgbench-benchmark/ Info There are different services depending on wether you want to read and write or read only.\n1kubectl get ep -n demo 2NAME ENDPOINTS AGE 3ogenki-any 10.64.1.136:5432,10.64.1.3:5432 15d 4ogenki-r 10.64.1.136:5432,10.64.1.3:5432 15d 5ogenki-ro 10.64.1.136:5432 15d 6ogenki-rw 10.64.1.3:5432 15d 1kubectl logs -n demo job/pgbench-pgbench-benchmark -f 2Defaulted container \u0026#34;pgbench\u0026#34; out of: pgbench, wait-for-cnp (init), pgbench-init (init) 3pgbench (14.1, server 14.5 (Debian 14.5-2.pgdg110+2)) 4starting vacuum...end. 5transaction type: \u0026lt;builtin: TPC-B (sort of)\u0026gt; 6scaling factor: 1 7query mode: simple 8number of clients: 10 9number of threads: 1 10duration: 600 s 11number of transactions actually processed: 545187 12latency average = 11.004 ms 13initial connection time = 111.585 ms 14tps = 908.782896 (without initial connection time) 💽 Backup and Restore Note Writing backups and WAL files to the GCP bucket is possible because we gave the permissions using an annotation in the pod's serviceaccount\n1 serviceAccountTemplate: 2 metadata: 3 annotations: 4 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com We can first trigger an on-demand backup using the custom resource Backup\n1apiVersion: postgresql.cnpg.io/v1 2kind: Backup 3metadata: 4 name: ogenki-now 5 namespace: demo 6spec: 7 cluster: 8 name: ogenki 1kubectl apply -f backup.yaml 2backup.postgresql.cnpg.io/ogenki-now created 3 4kubectl get backup -n demo 5NAME AGE CLUSTER PHASE ERROR 6ogenki-now 36s ogenki completed If you take a look at the Google Cloud Storage content you'll see an new directory that stores the base backups\n1gcloud storage ls gs://cnpg-ogenki/ogenki/base 2gs://cnpg-ogenki/ogenki/base/20221023T130327/ But most of the time we would want to have a scheduled backup. So let's configure a daily schedule.\n1apiVersion: postgresql.cnpg.io/v1 2kind: ScheduledBackup 3metadata: 4 name: ogenki-daily 5 namespace: demo 6spec: 7 backupOwnerReference: self 8 cluster: 9 name: ogenki 10 schedule: 0 0 0 * * * Recoveries can only be done on new instances. Here we'll use the backup we've created previously to bootstrap a new instance with it.\n1gcloud iam service-accounts add-iam-policy-binding cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com \\ 2--role roles/iam.workloadIdentityUser --member \u0026#34;serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore]\u0026#34; 3Updated IAM policy for serviceAccount [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 4bindings: 5- members: 6 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki-restore] 7 - serviceAccount:{{ gcp_project }}.svc.id.goog[demo/ogenki] 8 role: roles/iam.workloadIdentityUser 9etag: BwXrs755FPA= 10version: 1 1apiVersion: postgresql.cnpg.io/v1 2kind: Cluster 3metadata: 4 name: ogenki-restore 5 namespace: demo 6spec: 7 instances: 1 8 9 serviceAccountTemplate: 10 metadata: 11 annotations: 12 iam.gke.io/gcp-service-account: cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 13 14 storage: 15 storageClass: standard 16 size: 10Gi 17 18 resources: 19 requests: 20 memory: \u0026#34;1Gi\u0026#34; 21 cpu: \u0026#34;500m\u0026#34; 22 limits: 23 memory: \u0026#34;1Gi\u0026#34; 24 25 superuserSecret: 26 name: cnpg-mydb-superuser 27 28 bootstrap: 29 recovery: 30 backup: 31 name: ogenki-now We can notice a first pod that performs the full recovery from the backup.\n1kubectl get po -n demo 2NAME READY STATUS RESTARTS AGE 3ogenki-1 1/1 Running 1 (18h ago) 18h 4ogenki-2 1/1 Running 0 18h 5ogenki-restore-1 0/1 Init:0/1 0 0s 6ogenki-restore-1-full-recovery-5p4ct 0/1 Completed 0 51s Then the new cluster becomes ready.\n1kubectl get cluster -n demo 2NAME AGE INSTANCES READY STATUS PRIMARY 3ogenki 18h 2 2 Cluster in healthy state ogenki-1 4ogenki-restore 80s 1 1 Cluster in healthy state ogenki-restore-1 \u0026#x1f9f9; Cleanup Delete the cluster\n1kubectl delete cluster -n demo ogenki ogenki-restore 2cluster.postgresql.cnpg.io \u0026#34;ogenki\u0026#34; deleted 3cluster.postgresql.cnpg.io \u0026#34;ogenki-restore\u0026#34; deleted Cleanup the IAM serviceaccount\n1gcloud iam service-accounts delete cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com 2You are about to delete service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com]. 3 4Do you want to continue (Y/n)? y 5 6deleted service account [cloudnative-pg@{{ gcp_project }}.iam.gserviceaccount.com] 💭 final thoughts I just discovered CloudNativePG and I only scratched the surface but one thing for sure is that managing PostgreSQL is really made easy. However choosing a database solution is a tough decision. Depending on the use case, the company constraints, the criticity of the application and the ops teams skills, there are plenty of options: Cloud managed databases, traditional bare metal installations, building the architecture with an Infrastructure As Code tool...\nWe may also consider using Crossplane and composition to give an opinionated way of declaring managed databases in cloud providers but that requires more configuration.\nCloudNativePG shines by its simplicity: it is easy to run and easy to understand. Furthermore the documentation is excellent (one of the best I ever seen!), especially for such a young open source project (Hopefuly this will help in the CNCF Sandbox acceptance process 🤞).\nIf you want to learn more about it, there was a presentation on about it at KubeCon NA 2022.\n","link":"https://blog.ogenki.io/post/cnpg/","section":"post","tags":["data"],"title":"`CloudNativePG`: An easy way to run PostgreSQL on Kubernetes"},{"body":"","link":"https://blog.ogenki.io/tags/data/","section":"tags","tags":null,"title":"data"},{"body":"In a previous article, we've seen how to use Crossplane so that we can manage cloud resources the same way as our applications. \u0026#x2764;\u0026#xfe0f; Declarative approach! There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.\nHere we'll achieve exactly the same thing but we'll do that in the GitOps way. According to the OpenGitOps working group there are 4 GitOps principles:\nThe desired state of our system must be expressed declaratively. This state must be stored in a versioning system. Changes are pulled and applied automatically in the target platform whenever the desired state changes. If, for any reason, the current state is modified, it will be automatically reconciled with the desired state. There are several GitOps engine options. The most famous ones are ArgoCD and Flux. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).\nLearn more about GitOps toolkit components here.\n🎯 Our target Here we want to declare our desired infrastructure components only by adding git changes. By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance. We'll discover Flux basics and how to use it in order to build a complete GitOps CD workflow.\n\u0026#x2611;\u0026#xfe0f; Requirements \u0026#x1f4e5; Install required tools First of all we need to install a few tools using asdf\nCreate a local file .tool-versions\n1cd ~/sources/devflux/ 2 3cat \u0026gt; .tool-versions \u0026lt;\u0026lt;EOF 4flux2 0.31.3 5kubectl 1.24.3 6kubeseal 0.18.1 7kustomize 4.5.5 8EOF 1for PLUGIN in $(cat .tool-versions | awk \u0026#39;{print $1}\u0026#39;); do asdf plugin-add $PLUGIN; done 2 3asdf install 4Downloading ... 100.0% 5Copying Binary 6... Check that all the required tools are actually installed.\n1asdf current 2flux2 0.31.3 /home/smana/sources/devflux/.tool-versions 3kubectl 1.24.3 /home/smana/sources/devflux/.tool-versions 4kubeseal 0.18.1 /home/smana/sources/devflux/.tool-versions 5kustomize 4.5.5 /home/smana/sources/devflux/.tool-versions \u0026#x1f511; Create a Github personal access token In this article the git repository is hosted in Github. In order to be able to use the flux bootstrap a personnal access token is required.\nPlease follow this procedure.\nWarning Store the Github token in a safe place for later use\n\u0026#x1f9d1;\u0026zwj;\u0026#x1f4bb; Clone the devflux repository All the files used for the upcoming steps can be retrieved from this repository. You should clone it, that will be easier to copy them into your own repository.\n1git clone https://github.com/Smana/devflux.git \u0026#x1f680; Bootstrap flux in the Crossplane cluster As we will often be using the flux CLI you may want to configure the bash|zsh completion\n1source \u0026lt;(flux completion bash) Warning Here we consider that you already have a local k3d instance. If not you may want to either go through the whole previous article or just run the local cluster creation.\nEnsure that you're working in the right context\n1kubectl config current-context 2k3d-crossplane Run the bootstrap command that will basically deploy all Flux's components in the namespace flux-system. Here I'll create a repository named devflux using my personal Github account.\n1export GITHUB_USER=\u0026lt;YOUR_ACCOUNT\u0026gt; 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/k3d-crossplane 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/k3d-crossplane\u0026#34; for \u0026#34;https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux\u0026#34; 9... 10✔ all components are healthy Check that all the pods are running properly and that the kustomization flux-system has been successfully reconciled.\n1kubectl get po -n flux-system 2NAME READY STATUS RESTARTS AGE 3helm-controller-5985c795f8-gs2pc 1/1 Running 0 86s 4notification-controller-6b7d7485fc-lzlpg 1/1 Running 0 86s 5kustomize-controller-6d4669f847-9x844 1/1 Running 0 86s 6source-controller-5fb4888d8f-wgcqv 1/1 Running 0 86s 7 8flux get kustomizations 9NAME REVISION SUSPENDED READY MESSAGE 10flux-system main/33ebef1 False True Applied revision: main/33ebef1 Running the bootstap command actually creates a github repository if it doesn't exist yet. Clone it now for our upcoming changes. You'll notice that the first commit has been made by Flux.\n1git clone https://github.com/\u0026lt;YOUR_ACCOUNT\u0026gt;/devflux.git 2Cloning into \u0026#39;devflux\u0026#39;... 3 4cd devflux 5 6git log -1 7commit 2beb6aafea67f3386b50cbc706fb34575844040d (HEAD -\u0026gt; main, origin/main, origin/HEAD) 8Author: Flux \u0026lt;\u0026gt; 9Date: Thu Jul 14 17:13:27 2022 +0200 10 11 Add Flux sync manifests 12 13ls clusters/k3d-crossplane/flux-system/ 14gotk-components.yaml gotk-sync.yaml kustomization.yaml \u0026#x1f4c2; Flux repository structure There are several options for organizing your resources in the Flux configuration repository. Here is a proposition for the sake of this article.\n1tree -d -L 2 2. 3├── apps 4│ ├── base 5│ └── dev-cluster 6├── clusters 7│ ├── dev-cluster 8│ └── k3d-crossplane 9├── infrastructure 10│ ├── base 11│ ├── dev-cluster 12│ └── k3d-crossplane 13├── observability 14│ ├── base 15│ ├── dev-cluster 16│ └── k3d-crossplane 17└── security 18 ├── base 19 ├── dev-cluster 20 └── k3d-crossplane Directory Description Example /apps our applications Here we'll deploy a demo application \u0026quot;online-boutique\u0026quot; /infrastructure base infrastructure/network components Crossplane as it will be used to provision cloud resources but we can also find CSI/CNI/EBS drivers... /observability All metrics/apm/logging tools Prometheus of course, Opentelemetry ... /security Any component that enhance our security level SealedSecrets (see below) Info For the upcoming steps please refer to the demo repository here\nLet's use this structure and begin to deploy applications \u0026#x1f680;.\n\u0026#x1f510; SealedSecrets There are plenty of alternatives when it comes to secrets management in Kubernetes. In order to securely store secrets in a git repository the GitOps way we'll make use of SealedSecrets. It uses a custom resource definition named SealedSecrets in order to encrypt the Kubernetes secret at the client side then the controller is in charge of decrypting and generating the expected secret in the cluster.\n\u0026#x1f6e0;\u0026#xfe0f; Deploy the controller using Helm The first thing to do is to declare the kustomization that handles all the security tools.\nclusters/k3d-crossplane/security.yaml\n1apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 2kind: Kustomization 3metadata: 4 name: security 5 namespace: flux-system 6spec: 7 prune: true 8 interval: 4m0s 9 sourceRef: 10 kind: GitRepository 11 name: flux-system 12 path: ./security/k3d-crossplane 13 healthChecks: 14 - apiVersion: helm.toolkit.fluxcd.io/v1beta1 15 kind: HelmRelease 16 name: sealed-secrets 17 namespace: kube-system Info A Kustomization is a custom resource that comes with Flux. It basically points to a set of Kubernetes resources managed with kustomize The above security kustomization points to a local directory where the kustomize resources are.\n1... 2spec: 3 path: ./security/k3d-crossplane 4... Note This is worth noting that there are two types on kustomizations. That can be confusing when you start playing with Flux.\nOne managed by flux's kustomize controller. Its API is kustomization.kustomize.toolkit.fluxcd.io The other kustomization.kustomize.config.k8s.io is for the kustomize overlay The kustomization.yaml file is always used for the kustomize overlay. Flux itself doesn't need this overlay in all cases, but if you want to use features of a Kustomize overlay you will occasionally need to create it in order to access them. It provides instructions to the Kustomize CLI.\nWe will deploy SealedSecrets using the Helm chart. So we need to declare the source of this chart. Using the kustomize overlay system, we'll first create the base files that will be inherited at the cluster level.\nsecurity/base/sealed-secrets/source.yaml\n1apiVersion: source.toolkit.fluxcd.io/v1beta2 2kind: HelmRepository 3metadata: 4 name: sealed-secrets 5 namespace: flux-system 6spec: 7 interval: 30m 8 url: https://bitnami-labs.github.io/sealed-secrets Then we'll define the HelmRelease which references the above source. Put the values you want to apply to the Helm chart under spec.values\nsecurity/base/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 releaseName: sealed-secrets 8 chart: 9 spec: 10 chart: sealed-secrets 11 sourceRef: 12 kind: HelmRepository 13 name: sealed-secrets 14 namespace: flux-system 15 version: \u0026#34;2.4.0\u0026#34; 16 interval: 10m0s 17 install: 18 remediation: 19 retries: 3 20 values: 21 fullnameOverride: sealed-secrets-controller 22 resources: 23 requests: 24 cpu: 80m 25 memory: 100Mi If you're starting your repository from scratch you'll need to generate the kustomization.yaml file (kustomize overlay).\n1kustomize create --autodetect security/base/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4- helmrelease.yaml 5- source.yaml Now we declare the sealed-secret kustomization at the cluster level. Just for the example we'll overwrite a value at the cluster level using kustomize's overlay system.\nsecurity/k3d-crossplane/sealed-secrets/helmrelease.yaml\n1apiVersion: helm.toolkit.fluxcd.io/v2beta1 2kind: HelmRelease 3metadata: 4 name: sealed-secrets 5 namespace: kube-system 6spec: 7 values: 8 resources: 9 requests: 10 cpu: 100m security/k3d-crossplane/sealed-secrets/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3bases: 4 - ../../base 5patches: 6 - helmrelease.yaml Pushing our changes is the only thing to do in order to get sealed-secrets deployed in the target cluster.\n1git commit -m \u0026#34;security: deploy sealed-secrets in k3d-crossplane\u0026#34; 2[security/sealed-secrets 283648e] security: deploy sealed-secrets in k3d-crossplane 3 6 files changed, 66 insertions(+) 4 create mode 100644 clusters/k3d-crossplane/security.yaml 5 create mode 100644 security/base/sealed-secrets/helmrelease.yaml 6 create mode 100644 security/base/sealed-secrets/kustomization.yaml 7 create mode 100644 security/base/sealed-secrets/source.yaml 8 create mode 100644 security/k3d-crossplane/sealed-secrets/helmrelease.yaml 9 create mode 100644 security/k3d-crossplane/sealed-secrets/kustomization.yaml After a few seconds (1 minutes by default) a new kustomization will appear.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3flux-system main/d36a33c False True Applied revision: main/d36a33c 4security main/d36a33c False True Applied revision: main/d36a33c And all the resources that we declared in the flux repository should be available and READY.\n1flux get sources helm 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee False True stored artifact for revision \u0026#39;4c0aa1980e3ec9055dea70abd2b259aad1a2c235325ecf51a25a92a39ac4eeee\u0026#39; 1flux get helmrelease -n kube-system 2NAME REVISION SUSPENDED READY MESSAGE 3sealed-secrets 2.2.0 False True Release reconciliation succeeded 🧪 A first test SealedSecret Let's use the CLI kubeseal to test it out. We'll create a SealedSecret that will be decrypted by the sealed-secrets controller in the cluster and create the expected secret foobar\n1kubectl create secret generic foobar -n default --dry-run=client -o yaml --from-literal=foo=bar \\ 2| kubeseal --namespace default --format yaml | kubectl apply -f - 3sealedsecret.bitnami.com/foobar created 4 5kubectl get secret -n default foobar 6NAME TYPE DATA AGE 7foobar Opaque 1 3m13s 8 9kubectl delete sealedsecrets.bitnami.com foobar 10sealedsecret.bitnami.com \u0026#34;foobar\u0026#34; deleted \u0026#x2601;\u0026#xfe0f; Deploy and configure Crossplane \u0026#x1f511; Create the Google service account secret The first thing we need to do in order to get Crossplane working is to create the GCP serviceaccount. The steps have been covered here in the previous article. We'll create a SealedSecret gcp-creds that contains the serviceaccount file crossplane.json.\ninfrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml\n1kubectl create secret generic gcp-creds --context k3d-crossplane -n crossplane-system --from-file=creds=./crossplane.json --dry-run=client -o yaml \\ 2| kubeseal --format yaml --namespace crossplane-system - \u0026gt; infrastructure/k3d-crossplane/crossplane/configuration/sealedsecrets.yaml 🔄 Crossplane dependencies Now we will deploy Crossplane with Flux. I won't put the manifests here you'll find all of them in this repository. However it's important to understand that, in order to deploy and configure Crossplane properly we need to do that in a specific order. Indeed several CRD's (custom resource definitions) are required:\nFirst of all we'll install the crossplane controller. Then we'll configure the provider because the custom resource is now available thanks to the crossplane controller installation. Finally a provider installation deploys several CRDs that can be used to configure the provider itself and cloud resources. The dependencies between kustomizations can be controlled using the parameters dependsOn. Looking at the file clusters/k3d-crossplane/infrastructure.yaml, we can see for example that the kustomization infrastructure-custom-resources depends on the kustomization crossplane_provider which itself depends on crossplane-configuration....\n1--- 2apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 3kind: Kustomization 4metadata: 5 name: crossplane-provider 6spec: 7... 8 dependsOn: 9 - name: crossplane-core 10--- 11apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 12kind: Kustomization 13metadata: 14 name: crossplane-configuration 15spec: 16... 17 dependsOn: 18 - name: crossplane-provider 19--- 20apiVersion: kustomize.toolkit.fluxcd.io/v1beta2 21kind: Kustomization 22metadata: 23 name: infrastructure-custom-resources 24spec: 25... 26 dependsOn: 27 - name: crossplane-configuration Commit and push the changes for the kustomisations to appear. Note that they'll be reconciled in the defined order.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3infrastructure-custom-resources False False dependency \u0026#39;flux-system/crossplane-configuration\u0026#39; is not ready 4crossplane-configuration False False dependency \u0026#39;flux-system/crossplane-provider\u0026#39; is not ready 5security main/666f85a False True Applied revision: main/666f85a 6flux-system main/666f85a False True Applied revision: main/666f85a 7crossplane-core main/666f85a False True Applied revision: main/666f85a 8crossplane-provider main/666f85a False True Applied revision: main/666f85a Then all Crossplane components will be deployed, we can have a look to the HelmRelease status for instance.\n1kubectl describe helmrelease -n crossplane-system crossplane 2... 3Status: 4 Conditions: 5 Last Transition Time: 2022-07-15T19:12:04Z 6 Message: Release reconciliation succeeded 7 Reason: ReconciliationSucceeded 8 Status: True 9 Type: Ready 10 Last Transition Time: 2022-07-15T19:12:04Z 11 Message: Helm upgrade succeeded 12 Reason: UpgradeSucceeded 13 Status: True 14 Type: Released 15 Helm Chart: crossplane-system/crossplane-system-crossplane 16 Last Applied Revision: 1.9.0 17 Last Attempted Revision: 1.9.0 18 Last Attempted Values Checksum: 056dc1c6029b3a644adc7d6a69a93620afd25b65 19 Last Release Revision: 2 20 Observed Generation: 1 21Events: 22 Type Reason Age From Message 23 ---- ------ ---- ---- ------- 24 Normal info 20m helm-controller HelmChart \u0026#39;crossplane-system/crossplane-system-crossplane\u0026#39; is not ready 25 Normal info 20m helm-controller Helm upgrade has started 26 Normal info 19m helm-controller Helm upgrade succeeded And our GKE cluster should also be created because we defined a bunch of crossplane custom resources in infrastructure/k3d-crossplane/custom-resources/crossplane\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RUNNING 34.x.x.190 europe-west9-a 22m \u0026#x1f680; Bootstrap flux in the dev cluster Our local Crossplane cluster is now ready and it created our dev cluster and we also want it to be managed with Flux. So let's configure Flux for this dev cluster using the same bootstrap command.\nAuthenticate to the newly created cluster. The following command will automatically change your current context.\n1gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project \u0026lt;your_project\u0026gt; 2Fetching cluster endpoint and auth data. 3kubeconfig entry generated for dev-cluster. 4 5kubectl config current-context 6gke_\u0026lt;your_project\u0026gt;_europe-west9-a_dev-cluster Run the bootstrap command for the dev-cluster.\n1export GITHUB_USER=Smana 2export GITHUB_TOKEN=ghp_\u0026lt;REDACTED\u0026gt; # your personal access token 3export GITHUB_REPO=devflux 4 5flux bootstrap github --owner=\u0026#34;${GITHUB_USER}\u0026#34; --repository=\u0026#34;${GITHUB_REPO}\u0026#34; --personal --path=clusters/dev-cluster 6► cloning branch \u0026#34;main\u0026#34; from Git repository \u0026#34;https://github.com/Smana/devflux.git\u0026#34; 7... 8✔ configured deploy key \u0026#34;flux-system-main-flux-system-./clusters/dev-cluster\u0026#34; for \u0026#34;https://github.com/Smana/devflux\u0026#34; 9... 10✔ all components are healthy Note It's worth noting that each Kubernetes cluster generates its own sealing keys. That means that if you recreate the dev-cluster, you must regenerate all the sealedsecrets. In our example we declared a secret in order to set the Grafana credentials. Here's the command you need to run in order to create a new version of the sealedsecret and don't forget to use the proper context \u0026#x1f609;.\n1kubectl create secret generic kube-prometheus-stack-grafana \\ 2--from-literal=admin-user=admin --from-literal=admin-password=\u0026lt;yourpassword\u0026gt; --namespace observability --dry-run=client -o yaml \\ 3| kubeseal --namespace observability --format yaml \u0026gt; observability/dev-cluster/kube-prometheus-stack/sealedsecrets.yaml After a few seconds we'll get the following kustomizations deployed.\n1flux get kustomizations 2NAME REVISION SUSPENDED READY MESSAGE 3apps main/1380eaa False True Applied revision: main/1380eaa 4flux-system main/1380eaa False True Applied revision: main/1380eaa 5observability main/1380eaa False True Applied revision: main/1380eaa 6security main/1380eaa False True Applied revision: main/1380eaa Here we configured the prometheus stack and deployed a demo microservices stack named \u0026quot;online-boutique\u0026quot; This demo application exposes the frontend through a service of type LoadBalancer.\n1kubectl get svc -n demo frontend-external 2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 3frontend-external LoadBalancer 10.140.174.201 34.155.121.2 80:31943/TCP 7m44s Use the EXTERNAL_IP\n\u0026#x1f575;\u0026#xfe0f; Troubleshooting The cheatsheet in Flux's documentation contains many ways for troubleshooting when something goes wrong. Here I'll just give a sample of my favorite command lines.\nObjects that aren't ready\n1flux get all -A --status-selector ready=false Checking the logs of a given kustomization\n1flux logs --kind kustomization --name infrastructure-custom-resources 22022-07-15T19:38:52.996Z info Kustomization/infrastructure-custom-resources.flux-system - server-side apply completed 32022-07-15T19:38:53.016Z info Kustomization/infrastructure-custom-resources.flux-system - Reconciliation finished in 66.12266ms, next run in 4m0s 42022-07-15T19:11:34.697Z info Kustomization/infrastructure-custom-resources.flux-system - Discarding event, no alerts found for the involved object Show how a given pod is managed by Flux.\n1flux trace -n crossplane-system pod/crossplane-5dc8d888d7-g95qx 2 3Object: Pod/crossplane-5dc8d888d7-g95qx 4Namespace: crossplane-system 5Status: Managed by Flux 6--- 7HelmRelease: crossplane 8Namespace: crossplane-system 9Revision: 1.9.0 10Status: Last reconciled at 2022-07-15 21:12:04 +0200 CEST 11Message: Release reconciliation succeeded 12--- 13HelmChart: crossplane-system-crossplane 14Namespace: crossplane-system 15Chart: crossplane 16Version: 1.9.0 17Revision: 1.9.0 18Status: Last reconciled at 2022-07-15 21:11:36 +0200 CEST 19Message: pulled \u0026#39;crossplane\u0026#39; chart with version \u0026#39;1.9.0\u0026#39; 20--- 21HelmRepository: crossplane 22Namespace: crossplane-system 23URL: https://charts.crossplane.io/stable 24Revision: 362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d 25Status: Last reconciled at 2022-07-15 21:11:35 +0200 CEST 26Message: stored artifact for revision \u0026#39;362022f8c7ce215a0bb276887115cb5324b35a3169723900c84456adc3538a8d\u0026#39; If you want to check what would be the changes before pushing your commit. In thi given example I just increased the cpu requests for the sealed-secrets controller.\n1flux diff kustomization security --path security/k3d-crossplane 2✓ Kustomization diffing... 3► HelmRelease/kube-system/sealed-secrets drifted 4 5metadata.generation 6 ± value change 7 - 6 8 + 7 9 10spec.values.resources.requests.cpu 11 ± value change 12 - 100m 13 + 120m 14 15⚠️ identified at least one change, exiting with non-zero exit code \u0026#x1f9f9; Cleanup Don't forget to delete the Cloud resources if you don't want to have a bad suprise \u0026#x1f4b5;! Just comment the file infrastructure/k3d-crossplane/custom-resources/crossplane/kustomization.yaml\n1apiVersion: kustomize.config.k8s.io/v1beta1 2kind: Kustomization 3resources: 4 # - cluster.yaml 5 - network.yaml \u0026#x1f44f; Achievements With our current setup everything is configured using the GitOps approach:\nWe can manage infrastructure resources using Crossplane. Our secrets are securely stored in our git repository. We have a dev-cluster that we can enable or disable just but commenting a yaml file. Our demo application can be deployed from scratch in seconds. 💭 final thoughts Flux is probably the tool I'm using the most on a daily basis. It's really amazing!\nWhen you get familiar with its concepts and the command line it becomes really easy to use and troubleshoot. You can use either Helm when a chart is available or Kustomize.\nHowever we faced a few issues:\nIt's not straightforward to find an efficient structure depending on the company needs. Especially when you have several Kubernetes controllers that depend on other CRDs. The Helm controller doesn't maintain a state of the Kubernetes resources deployed by the Helm chart. That means that if you delete a resource which has been deployed through a Helm chart, it won't be reconciled (It will change soon. Being discussed here) Flux doesn't provide itself a web UI and switching between CLIs (kubectl, flux ...) can be annoying from a developer perspective. (I'm going to test weave-gitops ) I've been using Flux in production for more than a year and we configured it with the image automation so that the only thing a developer has to do is to merge a pull request and the new version of the application is automatically deployed in the target cluster.\nI should probably give another try to ArgoCD in order to be able to compare these precisely 🤔.\n","link":"https://blog.ogenki.io/post/devflux/","section":"post","tags":["gitops","devxp"],"title":"100% `GitOps` using Flux"},{"body":"Who am I? I'm a senior Site Reliability Engineer with a particular interest in Linux containers and cloud technologies. I worked in different companies (small startups and large scale) and I've been working in different areas in order to improve the reliability, availability of the platform as well as the developer experience. I helped several companies in their transition to the Cloud. I was leading SRE/DevOps teams (diverse profiles with developers and SREs) and I really enjoy seeing them engaged in the same direction.\nAs side activities, I am an organizer of the Cloud Native Computing meetup in Paris and the Kubernetes Community Days France.\nHobbies: Reading SF books, Kick Boxing, Surfing/Skating/Inline Roller\n","link":"https://blog.ogenki.io/about/","section":"","tags":null,"title":"About"},{"body":"","link":"https://blog.ogenki.io/tags/devxp/","section":"tags","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/tags/gitops/","section":"tags","tags":null,"title":"gitops"},{"body":"The target of this documentation is to be able to create and manage a GKE cluster using Crossplane.\nCrossplane leverages Kubernetes base principles in order to provision cloud resources and much more: a declarative approach with drift detections and reconciliations using control loops \u0026#x1f92f;. In other words, we declare what cloud resources we want and Crossplane ensures that the target state matches the one applied through the Kubernetes API.\nHere are the steps we'll follow in order to get a Kubernetes cluster for development and experimentations use cases.\n\u0026#x1f433; Create the local k3d cluster for Crossplane's control plane k3d is a lightweight kubernetes cluster that leverages k3s that runs in our local laptop. There are several deployment models for Crossplane, we could for instance deploy the control plane on a management cluster on Kubernetes or a control plane per Kubernetes cluster.\nHere I chose a simple method which is fine for a personal use case: A local Kubernetes instance in which I'll deploy Crossplane.\nLet's install k3d using asdf.\n1asdf plugin-add k3d 2 3asdf install k3d $(asdf latest k3d) 4* Downloading k3d release 5.4.1... 5k3d 5.4.1 installation was successful! Create a single node Kubernetes cluster.\n1k3d cluster create crossplane 2... 3INFO[0043] You can now use it like this: 4kubectl cluster-info 5 6k3d cluster list 7crossplane 1/1 0/0 true Check that the cluster is reachable using the kubectl CLI.\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:40643 3CoreDNS is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:40643/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy We only need a single node for our Crossplane use case.\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-crossplane-server-0 Ready control-plane,master 26h v1.22.7+k3s1 \u0026#x2601;\u0026#xfe0f; Generate the Google Cloud service account Warning Store the downloaded crossplane.json credentials file in a safe place.\nCreate a service account\n1GCP_PROJECT=\u0026lt;your_project\u0026gt; 2gcloud iam service-accounts create crossplane --display-name \u0026#34;Crossplane\u0026#34; --project=${GCP_PROJECT} 3Created service account [crossplane]. Assign the proper permissions to the service account.\nCompute Network Admin Kubernetes Engine Admin Service Account User 1SA_EMAIL=$(gcloud iam service-accounts list --filter=\u0026#34;email ~ ^crossplane\u0026#34; --format=\u0026#39;value(email)\u0026#39;) 2 3gcloud projects add-iam-policy-binding \u0026#34;${GCP_PROJECT}\u0026#34; --member=serviceAccount:\u0026#34;${SA_EMAIL}\u0026#34; \\ 4--role=roles/container.admin --role=roles/compute.networkAdmin --role=roles/iam.serviceAccountUser 5Updated IAM policy for project [\u0026lt;project\u0026gt;]. 6bindings: 7- members: 8 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 9 role: roles/compute.networkAdmin 10- members: 11 - serviceAccount:crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com 12... 13version: 1 Download the service account key (json format)\n1gcloud iam service-accounts keys create crossplane.json --iam-account ${SA_EMAIL} 2created key [ea2eb9ce2939127xxxxxxxxxx] of type [json] as [crossplane.json] for [crossplane@\u0026lt;project\u0026gt;.iam.gserviceaccount.com] \u0026#x1f6a7; Deploy and configure Crossplane Now that we have a credentials file for Google Cloud, we can deploy the Crossplane operator and configure the provider-gcp provider.\nInfo Most of the following steps are issued from the official documentation\nWe'll first use Helm in order to install the operator\n1helm repo add crossplane-master https://charts.crossplane.io/master/ 2\u0026#34;crossplane-master\u0026#34; has been added to your repositories 3 4helm repo update 5...Successfully got an update from the \u0026#34;crossplane-master\u0026#34; chart repository 6 7helm install crossplane --namespace crossplane-system --create-namespace \\ 8--version 1.18.1 crossplane-stable/crossplane 9 10NAME: crossplane 11LAST DEPLOYED: Mon Jun 6 22:00:02 2022 12NAMESPACE: crossplane-system 13STATUS: deployed 14REVISION: 1 15TEST SUITE: None 16NOTES: 17Release: crossplane 18... Check that the operator is running properly.\n1kubectl get po -n crossplane-system 2NAME READY STATUS RESTARTS AGE 3crossplane-rbac-manager-54d96cd559-222hc 1/1 Running 0 3m37s 4crossplane-688c575476-lgklq 1/1 Running 0 3m37s Info All the files used for the upcoming steps are stored within this blog repository. So you should clone and change the current directory:\n1git clone https://github.com/Smana/smana.github.io.git 2 3cd smana.github.io/content/resources/crossplane_k3d Now we'll configure Crossplane so that it will be able to create and manage GCP resources. This is done by configuring the provider provider-gcp as follows.\nprovider.yaml\n1apiVersion: pkg.crossplane.io/v1 2kind: Provider 3metadata: 4 name: crossplane-provider-gcp 5spec: 6 package: crossplane/provider-gcp:v0.21.0 1kubectl apply -f provider.yaml 2provider.pkg.crossplane.io/crossplane-provider-gcp created 3 4kubectl get providers 5NAME INSTALLED HEALTHY PACKAGE AGE 6crossplane-provider-gcp True True crossplane/provider-gcp:v0.21.0 10s Create the Kubernetes secret that holds the GCP credentials file created above\n1kubectl create secret generic gcp-creds -n crossplane-system --from-file=creds=./crossplane.json 2secret/gcp-creds created Then we need to create a resource named ProviderConfig and reference the newly created secret.\nprovider-config.yaml\n1apiVersion: gcp.crossplane.io/v1beta1 2kind: ProviderConfig 3metadata: 4 name: default 5spec: 6 projectID: ${GCP_PROJECT} 7 credentials: 8 source: Secret 9 secretRef: 10 namespace: crossplane-system 11 name: gcp-creds 12 key: creds 1kubectl apply -f provider-config.yaml 2providerconfig.gcp.crossplane.io/default created Info If the serviceaccount has the proper permissions we can create resources in GCP. In order to learn about all the available resources and parameters we can have a look to the provider's API reference.\nThe first resource we'll create is the network that will host our Kubernetes cluster.\nnetwork.yaml\n1apiVersion: compute.gcp.crossplane.io/v1beta1 2kind: Network 3metadata: 4 name: dev-network 5 labels: 6 service: vpc 7 creation: crossplane 8spec: 9 forProvider: 10 autoCreateSubnetworks: false 11 description: \u0026#34;Network used for experimentations and POCs\u0026#34; 12 routingConfig: 13 routingMode: REGIONAL 1kubectl get network 2NAME READY SYNCED 3dev-network True True You can even get more details by describing this resource. For instance if something fails you would see the message returned by the Cloud provider in the events.\n1kubectl describe network dev-network | grep -A 20 \u0026#39;^Status:\u0026#39; 2Status: 3 At Provider: 4 Creation Timestamp: 2022-06-28T09:45:30.703-07:00 5 Id: 3005424280727359173 6 Self Link: https://www.googleapis.com/compute/v1/projects/${GCP_PROJECT}/global/networks/dev-network 7 Conditions: 8 Last Transition Time: 2022-06-28T16:45:31Z 9 Reason: Available 10 Status: True 11 Type: Ready 12 Last Transition Time: 2022-06-30T16:36:59Z 13 Reason: ReconcileSuccess 14 Status: True 15 Type: Synced \u0026#x1f680; Create a GKE cluster Everything is ready so that we can create our GKE cluster. Applying the file cluster.yaml will create a cluster and attach a node group to it.\ncluster.yaml\n1--- 2apiVersion: container.gcp.crossplane.io/v1beta2 3kind: Cluster 4metadata: 5 name: dev-cluster 6spec: 7 forProvider: 8 description: \u0026#34;Kubernetes cluster for experimentations and POCs\u0026#34; 9 initialClusterVersion: \u0026#34;1.24\u0026#34; 10 releaseChannel: 11 channel: \u0026#34;RAPID\u0026#34; 12 location: europe-west9-a 13 addonsConfig: 14 gcePersistentDiskCsiDriverConfig: 15 enabled: true 16 networkPolicyConfig: 17 disabled: false 18 networkRef: 19 name: dev-network 20 ipAllocationPolicy: 21 createSubnetwork: true 22 useIpAliases: true 23 defaultMaxPodsConstraint: 24 maxPodsPerNode: 110 25 networkPolicy: 26 enabled: false 27 writeConnectionSecretToRef: 28 namespace: default 29 name: gke-conn 30--- 31apiVersion: container.gcp.crossplane.io/v1beta1 32kind: NodePool 33metadata: 34 name: main-np 35spec: 36 forProvider: 37 initialNodeCount: 1 38 autoscaling: 39 autoprovisioned: false 40 enabled: true 41 maxNodeCount: 4 42 minNodeCount: 1 43 clusterRef: 44 name: dev-cluster 45 config: 46 machineType: n2-standard-2 47 diskSizeGb: 120 48 diskType: pd-standard 49 imageType: cos_containerd 50 preemptible: true 51 labels: 52 environment: dev 53 managed-by: crossplane 54 oauthScopes: 55 - \u0026#34;https://www.googleapis.com/auth/devstorage.read_only\u0026#34; 56 - \u0026#34;https://www.googleapis.com/auth/logging.write\u0026#34; 57 - \u0026#34;https://www.googleapis.com/auth/monitoring\u0026#34; 58 - \u0026#34;https://www.googleapis.com/auth/servicecontrol\u0026#34; 59 - \u0026#34;https://www.googleapis.com/auth/service.management.readonly\u0026#34; 60 - \u0026#34;https://www.googleapis.com/auth/trace.append\u0026#34; 61 metadata: 62 disable-legacy-endpoints: \u0026#34;true\u0026#34; 63 shieldedInstanceConfig: 64 enableIntegrityMonitoring: true 65 enableSecureBoot: true 66 management: 67 autoRepair: true 68 autoUpgrade: true 69 maxPodsConstraint: 70 maxPodsPerNode: 60 71 locations: 72 - \u0026#34;europe-west9-a\u0026#34; 1kubectl apply -f cluster.yaml 2cluster.container.gcp.crossplane.io/dev-cluster created 3nodepool.container.gcp.crossplane.io/main-np created Note that it takes around 10 minutes for the Kubernetes API and the nodes to be available. The STATE will transition from PROVISIONING to RUNNING and when a change is being applied the cluster status is RECONCILING\n1watch \u0026#39;kubectl get cluster,nodepool\u0026#39; 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3cluster.container.gcp.crossplane.io/dev-cluster False True PROVISIONING 34.155.122.6 europe-west9-a 3m15s 4 5NAME READY SYNCED STATE CLUSTER-REF AGE 6nodepool.container.gcp.crossplane.io/main-np False False dev-cluster 3m15s When the column READY switches to True you can download the cluster's credentials.\n1kubectl get cluster 2NAME READY SYNCED STATE ENDPOINT LOCATION AGE 3dev-cluster True True RECONCILING 34.42.42.42 europe-west9-a 6m23s 4 5gcloud container clusters get-credentials dev-cluster --zone europe-west9-a --project ${GCP_PROJECT} 6Fetching cluster endpoint and auth data. 7kubeconfig entry generated for dev-cluster. For better readability you may want to rename the context id for the newly created cluster\n1kubectl config rename-context gke_${GCP_PROJECT}_europe-west9-a_dev-cluster dev-cluster 2Context \u0026#34;gke_${GCP_PROJECT}_europe-west9-a_dev-cluster\u0026#34; renamed to \u0026#34;dev-cluster\u0026#34;. 3 4kubectl config get-contexts 5CURRENT NAME CLUSTER AUTHINFO NAMESPACE 6* dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster gke_cloud-native-computing-paris_europe-west9-a_dev-cluster 7 k3d-crossplane k3d-crossplane admin@k3d-crossplane Check that you can call our brand new GKE API\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3gke-dev-cluster-main-np-d0d978f9-5fc0 Ready \u0026lt;none\u0026gt; 10m v1.24.1-gke.1400 That's great \u0026#x1f389; we know have a GKE cluster up and running.\n💭 final thoughts I've been using Crossplane for a few months now in a production environment.\nEven if I'm conviced about the declarative approach using the Kubernetes API, we decided to move with caution with it. It clearly doesn't have Terraform's community and maturity. We're still declaring our resources using the deletionPolicy: Orphan so that even if something goes wrong on the controller side the resource won't be deleted.\nFurthermore we limited to a specific list of usual AWS resources requested by our developers. Nevertheless our target has always been to empower developers and we had really positive feedback from them. That's the best indicator for us. As the project matures, we'll move more and more resources from Terraform to Crossplane.\nIMHO the key success of Crossplane depends on the providers maintenance and evolution. The Cloud providers interest and involvement is really important.\nIn our next article we'll see how to use a GitOps engine to run all the above steps.\n","link":"https://blog.ogenki.io/post/crossplane_k3d/","section":"post","tags":["kubernetes","infrastructure"],"title":"My Kubernetes cluster (GKE) with `Crossplane`"},{"body":"","link":"https://blog.ogenki.io/tags/local/","section":"tags","tags":null,"title":"local"},{"body":"In order to install binaries and to be able to switch from a version to another I like to use asdf.\n\u0026#x1f4e5; Installation The recommended installation is to use Git as follows\n1git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.10.0 Then depending on your shell here are the remaining steps to follow\n1. $HOME/.asdf/asdf.sh And you may want to configure the shell completion\n1. $HOME/.asdf/completions/asdf.bash \u0026#x1f680; Let's take an example List all available plugins and look for k3d\n1asdf plugin-list-all | grep k3d 2k3d https://github.com/spencergilbert/asdf-k3d.git Let's install k3d\n1asdf plugin-add k3d Check the versions available\n1asdf list-all k3d| tail -n 3 25.4.0-dev.3 35.4.0 45.4.1 We'll install the latest version\n1asdf install k3d latest 2* Downloading k3d release 5.4.1... 3k3d 5.4.1 installation was successful! Finally we can switch from a version to another. We can set a global version that would be used on all directories.\n1asdf global k3d 5.4.1 or use a local version depending on the current directory\n1cd /tmp 2asdf local k3d 5.4.1 3 4asdf current k3d 5k3d 5.4.1 /tmp/.tool-versions \u0026#x1f9f9; Cleanup Uninstall a given version\n1asdf uninstall k3d 5.4.1 Remove a plugin\n1asdf plugin remove k3d ","link":"https://blog.ogenki.io/post/asdf/asdf/","section":"post","tags":["tooling","local"],"title":"Manage tools versions with `asdf`"},{"body":"","link":"https://blog.ogenki.io/tags/tooling/","section":"tags","tags":null,"title":"tooling"},{"body":"","link":"https://blog.ogenki.io/categories/","section":"categories","tags":null,"title":"Categories"},{"body":"","link":"https://blog.ogenki.io/categories/devxp/","section":"categories","tags":null,"title":"devxp"},{"body":"","link":"https://blog.ogenki.io/tags/helm/","section":"tags","tags":null,"title":"Helm"},{"body":"Template challenge Here you’ll be able to practice in order to get familiar with some of the possibilities offered by a templating language.\n(pro tip: Don't forget the testing)\nThese examples may seem useless but the purpose of this is just playing with templates.\n1 - \u0026quot;Configuration depends on region\u0026quot; Create a secret that contains a key 'secret' and a value \u0026quot;myEuropeanSecret\u0026quot;. Set an environment variable from this secret only if the value global.region is 'eu-west1' So the first step is to add the values into the values.yaml file.\n1global: 2 region: eu-west-1 2 - \u0026quot;Create only if\u0026quot; Create a job that prints the pod's IP on stdout based on a boolean value printIP.enabled. Use the \u0026quot;busybox\u0026quot; image and the command wget -qO - http://ipinfo.io/ip. The job should be created only if the value is True, before every other resource has been created (pre-install and pre-upgrade hooks)\n3 - \u0026quot;Looping\u0026quot; Given the following values, create a loop whether in the deployment or in the helpers.tpl file in order to add the environment variables.\n1envVars: 2 key1: value1 3 key2: value2 4 key3: value3 4 - \u0026quot;Playing with strings and sprigs\u0026quot; Add to the \u0026quot;common labels\u0026quot;, a new label \u0026quot;codename\u0026quot; with a value composed with the release name, the chart name and the date in the form 20061225. The release name must be at most 3 characters long. The whole string has to be in snakecase.\n(you should get something like codename: rel_web_20210215)\n5 - We want to create a list of etcd hosts in the form of \u0026quot;etcd-0,etcd-1,etcd-2\u0026quot; based on a integer that defines the number of etcd hosts 1etcd: 2 count: 5 This list has to be defined in an environment variable ETCD_HOSTS\nProposition of solutions try_first You should try to find a solution by your own to the above exercises before checking these solutions\n1 The following command generates a secrets in the templates directory\n1kubectl create secret generic --dry-run=client eu-secret --from-literal=secret=\u0026#39;myEuropeanSecret\u0026#39; -o yaml | kubectl neat \u0026gt; templates/secret.yaml Then we'll enclose the environment variable definition with a condition depending on the region in the deployment template.\n1 {{- if eq .Values.global.region \u0026#34;eu-west-1\u0026#34; }} 2 - name: eu-secret 3 valueFrom: 4 secretKeyRef: 5 name: eu-secret 6 key: secret 7 {{- end }} 2 First of all we need to add a new value\n1printIP: 2 enabled: True Then this command will generate a job yaml\n1kubectl create job my-ip --dry-run=client --image=busybox -o yaml -- wget -qO - http://ipinfo.io/ip | kubectl neat \u0026gt; templates/job.yaml If we enclose the whole yaml, it won't be created if the boolean is False. With the hook annotation here, the job will be created before any other resource will be applied. We defined a delete policy \u0026quot;hook-failed\u0026quot; in order to keep the job, otherwise it would have been deleted.\n1{{- if .Values.printIP.enabled -}} 2apiVersion: batch/v1 3kind: Job 4metadata: 5 name: my-ip 6 annotations: 7 \u0026#34;helm.sh/hook\u0026#34;: pre-install,pre-upgrade 8 \u0026#34;helm.sh/hook-weight\u0026#34;: \u0026#34;1\u0026#34; 9 \u0026#34;helm.sh/hook-delete-policy\u0026#34;: hook-failed 10... 11{{- end -}} 3 If we want to keep the deployment easy to read, we would prefer adding the code in the _helpers.tpl\n1{{/* 2Environment variables 3*/}} 4{{- define \u0026#34;web.envVars\u0026#34; -}} 5{{- range $key, $value := .Values.envVars }} 6- name: {{ $key }} 7 value: {{ $value }} 8{{- end }} 9{{- end -}} Then this new variable could be used in the deployment as follows\n1 env: 2 {{- include \u0026#34;web.envVars\u0026#34; . | nindent 12 }} 4 The common labels can be changed in the file templates/_helpers.tpl. Here's a proposal This one is tricky, I needed to dig back into the charts available in the stable github repository.\n1codename: {{ printf \u0026#34;%s %s %s\u0026#34; (.Release.Name | trunc 3) .Chart.Name (now | date \u0026#34;20060102\u0026#34;) | snakecase }} 5 Here's an option to achieve the expected results.\n1 env: 2 - name: ETCD_HOSTS 3 value: \u0026#34;{{ range $index, $e := until (.Values.etcd.count|int) }}{{- if $index }},{{end}}etcd-{{ $index }}{{- end }}\u0026#34; ","link":"https://blog.ogenki.io/post/series/workshop_helm/templating/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Templating exercises"},{"body":"","link":"https://blog.ogenki.io/series/","section":"series","tags":null,"title":"Series"},{"body":"","link":"https://blog.ogenki.io/series/workshop-helm/","section":"series","tags":null,"title":"Workshop Helm"},{"body":"Create a simple webserver chart In order to get familiar with a typical chart we will create a simple webserver chart.\n1$ helm create web 2Creating web The above command will create a chart directory named web\nweb/ charts directory that contains the subcharts Chart.yaml metadatas (author, version, description), dependencies and more templates contains all the templates basically kubernetes resources in the form of templated yaml files. (go template) deployment.yaml helpers.tpl helpers, functions that can be used from the templates. hpa.yaml ingress.yaml NOTES.txt This file is used to print information after a release has been successfully installed. serviceaccount.yaml service.yaml tests contains a job that will run a command to check the application after it has been installed. test-connection.yaml values.yaml Maybe the most important file. We’ll play with the values to define how the kubernetes resources will be rendered. Testing the chart Here’s a combo if you want to check properly your chart before actually deploying it:\ntemplate + lint + kubeval + test\nGolang errors When you add templating changes, you should run the command helm template --debug \u0026lt;chart_dir\u0026gt;\n1$ helm template --debug web 2install.go:173: [debug] Original chart version: \u0026#34;\u0026#34; 3install.go:190: [debug] CHART PATH: /tmp/web 4 5Error: parse error at (web/templates/_helpers.tpl:73): unexpected EOF 6helm.go:81: [debug] parse error at (web/templates/_helpers.tpl:73): unexpected EOF Read carefully if there are error messages. Always use the option --debug to see the template rendering.\nChart linting The command helm lint \u0026lt;chart_dir\u0026gt; verifies that the chart is well-formed.\n1$ helm lint web/ 2==\u0026gt; Linting web/ 3[ERROR] Chart.yaml: apiVersion \u0026#39;v3\u0026#39; is not valid. The value must be either \u0026#34;v1\u0026#34; or \u0026#34;v2\u0026#34; 4[INFO] Chart.yaml: icon is recommended 5[ERROR] Chart.yaml: chart type is not valid in apiVersion \u0026#39;v3\u0026#39;. It is valid in apiVersion \u0026#39;v2\u0026#39; 6 7Error: 1 chart(s) linted, 1 chart(s) failed Validate Kubernetes resources In order to validate that the rendered kubernetes objects are well-formed we’ll make use of a tool named kubeval.\nThis is even easier by using the Helm plugin.\nInstall the plugin:\n1$ helm plugin install https://github.com/instrumenta/helm-kubeval 2Installing helm-kubeval v0.13.0 ... 3helm-kubeval 0.13.0 is installed. Then check the chart as follows\n1$ helm kubeval web 2The file web/templates/serviceaccount.yaml contains a valid ServiceAccount 3The file web/templates/secret.yaml contains a valid Secret 4... Now you can safely install the chart\n1$ helm upgrade --install web web 2Release \u0026#34;web\u0026#34; has been upgraded. Happy Helming! 3NAME: web 4LAST DEPLOYED: Mon Feb 15 18:22:23 2021 5NAMESPACE: default 6STATUS: deployed 7REVISION: 1 Check that the application works as expected This is a good practice to add tests under the directory template/tests.\nBasically, this is achieved with a job that you can call when the release is already installed (just after)\nIt returns a code 0 if the command succeeds.\nIn the chart we’ve already generated there’s a job that checks the webserver availability.\nCheck that the release is already installed\n1helm list 2NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION 3web default 1 2021-02-15 15:11:09.036602795 +0100 CET deployed web-0.1.0 1.16.0 1helm test web 2NAME: web 3LAST DEPLOYED: Mon Feb 15 15:11:09 2021 4NAMESPACE: default 5STATUS: deployed 6REVISION: 1 7TEST SUITE: web-test-connection 8Last Started: Mon Feb 15 16:55:17 2021 9Last Completed: Mon Feb 15 16:55:19 2021 10Phase: Succeeded Dependencies Sometimes, the application requires another component to work (caching, database, persistence …).\nThis dependency system has to be used with caution because this is generally recommended to manage the applications lifecycles independently from each other.\nLet’s say that our webserver need to store the information related to the sessions in a Redis server.\nWe’ll add a redis server to our web application by declaring the dependency in the file chart.yaml.\n1dependencies: 2 - name: redis 3 version: \u0026#34;12.6.4\u0026#34; 4 repository: https://charts.bitnami.com/bitnami 5 condition: redis.enabled As you may have noticed, this dependency will be pulled only if the condition redis.enabled is True.\nSo we need to change our values.yaml accordingly:\n1redis: 2 enabled: True 3 master: 4 persistence: 5 enabled: False Check all the available values for this chart here.\nWhenever you add a dependency and you’re using a local chart (on your laptop), you must run the following command to pull it\n1helm dep update web 2Hang tight while we grab the latest from your chart repositories... 3…. 4...Successfully got an update from the \u0026#34;bitnami\u0026#34; chart repository 5Update Complete. ⎈Happy Helming!⎈ 6Saving 1 charts 7Downloading redis from repo https://charts.bitnami.com/bitnami 8Deleting outdated charts The dependencies are stored in the directory charts.\nAfter testing your changes you can install the release with the command\nhelm upgrade --install \u0026lt;release_name\u0026gt; \u0026lt;chart_dir\u0026gt;\n1helm upgrade --install web web 2Release \u0026#34;web\u0026#34; has been upgraded. Happy Helming! 3NAME: web 4LAST DEPLOYED: Mon Feb 15 18:22:23 2021 5NAMESPACE: default 6STATUS: deployed 7REVISION: 2 You can notice that your webserver has been successfully installed along with a HA Redis cluster\n1kubectl get po 2NAME READY STATUS RESTARTS AGE 3web-74bf5c6c66-fjsmb 1/1 Running 0 3h14m 4web-test-connection 0/1 Completed 0 90m 5web-redis-master-0 1/1 Running 0 3m14s 6web-redis-slave-0 1/1 Running 0 3m14s 7web-redis-slave-1 1/1 Running 0 2m42s Hooks Helm comes with a hook system that allows it to run jobs at given times of the lifecycle.\nThe description is crystal clear in the documentation and you’ll have the opportunity to add one later on during this workshop.\nMastering the Golang template The main challenge when you start using Helm is to learn all the tips and tricks of the Golang template\nThe official Helm documentation is very useful for that.\nYour best friends when you write Helm templates are the Sprig functions, you should definitely add this to your bookmarks.\nFurthermore, even if it has been deprecated, you should clone/fork the original stable chart repository. Indeed it has a wide range of examples.\nNote that most of the time, if you want to keep the kubernetes manifests readable, you would put most of the code in what we call helpers files. There’s often at least one named _helpers.tpl.\nNote: Even if you can do pretty advanced things with this templating language, you shouldn’t overuse it in order to keep the kubernetes resources readable and the chart maintainable.\n\u0026#x27a1;\u0026#xfe0f; Next: Application lifecycle using Helm\n","link":"https://blog.ogenki.io/post/series/workshop_helm/build_chart/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Build your first chart"},{"body":"Apply a change, anything. For example we will add a label stage: dev. Edit the file templates/_helpers.tpl\n1{{- define \u0026#34;web.labels\u0026#34; -}} 2stage: \u0026#34;dev\u0026#34; 3... Deploy a new revision with the same command we ran previously\n1helm upgrade --install web web Now we can have a look to the changes we’ve made so far to the release\n1$ helm history web 2REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 31 Mon Feb 15 15:11:09 2021 superseded web-0.1.0 1.16.0 Install complete 4... 54 Mon Feb 15 21:15:25 2021 superseded web-0.1.0 1.16.0 Upgrade complete 65 Mon Feb 15 21:21:21 2021 deployed web-0.1.0 1.16.0 Upgrade complete We can then check what would be the changes if we rollback to the previous revision\n1helm diff rollback web 4 2default, web, Deployment (apps) has changed: 3 # Source: web/templates/deployment.yaml 4 apiVersion: apps/v1 5 kind: Deployment 6 metadata: 7 name: web 8 labels: 9- stage: \u0026#34;dev\u0026#34; 10 helm.sh/chart: web-0.1.0 11... Now that we’re sure we can safely rollback to the previous revision\n1helm rollback web 4 2Rollback was a success! Happy Helming! \u0026#x27a1;\u0026#xfe0f; Next: Helm templating challenge\n","link":"https://blog.ogenki.io/post/series/workshop_helm/lifecycle/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Lifecycle operations"},{"body":"Helm’s configuration is stored in the environment variable $HELM_CONFIG_HOME , by default $HOME/.config/helm\nAll the environment variables are described in the documentation there.\nHere are a few tools (with a wide adoption) that will add capabilities to Helm.\nPlugins There are several plugins in order to extend Helm’s features.\nSome of them are really useful (kubeval, diff, secrets).\nHelmfile Helmfile is a really useful tool that allows you to declare the state of the releases on your cluster.\nIt helps keeping a central view of the releases deployed on a given cluster.\nIt automatically configures repositories, pulls dependencies and it is very helpful to build CI/CD workflows.\nOf course it uses Helm under the hood and a few modules/plugins such as secrets decryption, helm diff\nThese steps are very basic, you should have a look at the documentation for further details.\nInstall the helmdiff plugin (used by helmfile)\n1helm plugin install https://github.com/databus23/helm-diff We’ll make use of some examples provided by CloudPosse\n1git clone git@github.com:cloudposse/helmfiles.git 2cd helmfiles Let’s say we want to install the kubernetes dashboard and the reloader tool.\n1cat \u0026gt; releases/kubernetes-dashboard/dev.yaml \u0026lt;\u0026lt;EOF 2installed: True 3banner: \u0026#34;Workshop cluster\u0026#34; 4EOF Now we’ll create our main helmfile.yaml that describes all the releases we want to install\n1cat \u0026gt; helmfile.yaml \u0026lt;\u0026lt;EOF 2helmfiles: 3 - path: \u0026#34;releases/kubernetes-dashboard/helmfile.yaml\u0026#34; 4 values: 5 - releases/kubernetes-dashboard/dev.yaml 6 - path: \u0026#34;releases/reloader/helmfile.yaml\u0026#34; 7 values: 8 - installed: True 9EOF Now we can see what changes will be applied.\n1helmfile diff 2Adding repo stable https://charts.helm.sh/stable 3\u0026#34;stable\u0026#34; has been added to your repositories 4 5Comparing release=kubernetes-dashboard, chart=stable/kubernetes-dashboard 6******************** 7 8 Release was not present in Helm. Diff will show entire contents as new. 9 10… The command helm sync will install the releases\n1helmfile sync 2Adding repo stable https://charts.helm.sh/stable 3\u0026#34;stable\u0026#34; has been added to your repositories 4 5Affected releases are: 6 kubernetes-dashboard (stable/kubernetes-dashboard) UPDATED 7 8Upgrading release=kubernetes-dashboard, chart=stable/kubernetes-dashboard 9Release \u0026#34;kubernetes-dashboard\u0026#34; does not exist. Installing it now. 10NAME: kubernetes-dashboard 11... You can list all the releases managed by the local helmfile.\n1helmfile list 2NAME NAMESPACE ENABLED LABELS 3kubernetes-dashboard kube-system true chart:kubernetes-dashboard,component:monitoring,namespace:kube-system,repo:stable,vendor:kubernetes 4reloader reloader true chart:stakater/reloader,component:reloader,namespace:reloader,repo:stakater,vendor:stakater Delete all the releases\n1helmfile delete 2Listing releases matching ^reloader$ 3reloader reloader 1 2021-02-16 10:10:35.378800455 +0100 CET deployed reloader-v0.0.68 v0.0.68 4 5Deleting reloader 6release \u0026#34;reloader\u0026#34; uninstalled \u0026#x27a1;\u0026#xfe0f; Next: Build a Helm chart\n","link":"https://blog.ogenki.io/post/series/workshop_helm/ecosystem/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Ecosystem"},{"body":"Looking for a chart Helm works with what is called a “chart”. A chart is basically a package of yaml resources that support a templating language.\nBefore building our own chart we should always have a look of what is available in the community. There are often charts that fits our needs.\nThese charts can be installed from different sources: a Helm chart repository, a local archive or chart directory.\nFirst of all, let’s say that we want to install a Wordpress instance on an empty infrastructure.\nWe’ll need to provision a database as well as the Wordpress application.\nLet’s look for a wordpress chart !\nIf you just installed Helm, your repositories list should be empty\n1helm repo list We’re going to check what are the Wordpress charts available in the artifacthub.\nYou can either browse from the web page or use the command\n1helm search hub wordpress 2URL CHART VERSION APP VERSION DESCRIPTION 3https://artifacthub.io/packages/helm/bitnami/wo... 10.6.4 5.6.1 Web publishing platform for building blogs and ... 4https://artifacthub.io/packages/helm/groundhog2... 0.2.6 5.6.0-apache A Helm chart for Wordpress on Kubernetes 5https://artifacthub.io/packages/helm/seccurecod... 2.4.0 4.0 Insecure \u0026amp; Outdated Wordpress Instance: Never e... 6https://artifacthub.io/packages/helm/presslabs/... 0.10.5 0.10.5 Presslabs WordPress Operator Helm Chart Using the Hub there are a few things that can help to choose the best option.\nFirst of all the number of stars obviously and whether the artifact comes from a verified publisher or signed by the maintainer.\nWe’ll get the one provided by Bitnami. In the chart page you’ll be guided with the commands to add Bitnami’s repository.\n1helm repo add bitnami https://charts.bitnami.com/bitnami 2\u0026#34;bitnami\u0026#34; has been added to your repositories 3 4helm repo update From now on we can install all the charts published by Bitnami:\n1helm search repo bitnami 2NAME CHART VERSION APP VERSION DESCRIPTION 3bitnami/bitnami-common 0.0.9 0.0.9 DEPRECATED Chart with custom templates used in ... 4bitnami/airflow 8.0.3 2.0.1 Apache Airflow is a platform to programmaticall... 5bitnami/apache 8.2.3 2.4.46 Chart for Apache HTTP Server 6bitnami/aspnet-core 1.2.3 3.1.9 ASP.NET Core is an open-source framework create... 7bitnami/cassandra 7.3.2 3.11.10 Apache Cassandra is a free and open-source dist... Inspect the chart OK let’s get back to what we want to achieve: Installing a Wordpress instance.\nNow that we identified the chart, we’re going to check what it actually does. You should always check what will be installed.\nyou can download the chart on your laptop and have a look to its content 1helm pull --untar bitnami/wordpress 2 3tree -L 2 wordpress/ 4wordpress/ 5├── Chart.lock 6├── charts 7│ ├── common 8│ └── mariadb 9├── Chart.yaml 10├── ci 11│ ├── ct-values.yaml 12│ ├── ingress-wildcard-values.yaml 13│ ├── values-hpa-pdb.yaml 14│ └── values-metrics-and-ingress.yaml 15├── README.md 16├── templates 17│ ├── configmap.yaml 18… 19│ ├── tests 20│ └── tls-secrets.yaml 21├── values.schema.json 22└── values.yaml read carefully the readme check what are the dependencies pulled by this chart 1helm show chart bitnami/wordpress 2annotations: 3 category: CMS 4apiVersion: v2 5appVersion: 5.6.1 6dependencies: 7- condition: mariadb.enabled 8 name: mariadb 9 repository: https://charts.bitnami.com/bitnami 10 version: 9.x.x 11- name: common 12 repository: https://charts.bitnami.com/bitnami 13 tags: 14 - bitnami-common 15 version: 1.x.x 16... Note: that the wordpress chart defines the mariadb chart as dependency\nLook at the available values 1helm show values bitnami/wordpress Our first release Our next step will be to set our desired values. Indeed you mentioned that Helm uses a templating language to render the manifests. This will help us to configure our instance according to our environment.\nno persistency at all, this is just a workshop 2 replicas for the wordpress instance a database named “foodb” an owner “foobar” for this database passwords All the charts have a file named values.yaml that contains the default values.\nThese values can be overridden at the command line with --set or we can put them in a yaml file that we’ll use with the -f parameter.\nFor this exercise we’ll create a file named “override-values.yaml” and we’ll use the command line for sensitive information.\n1wordpressUsername: foobar 2wordpressPassword: \u0026#34;\u0026#34; 3wordpressBlogName: Foo\u0026#39;s Blog! 4replicaCount: 2 5persistence: 6 enabled: false 7service: 8 type: ClusterIP 9mariadb: 10 auth: 11 rootPassword: \u0026#34;\u0026#34; 12 database: foodb 13 username: foobar 14 password: \u0026#34;\u0026#34; 15 primary: 16 persistence: 17 enabled: false Note: In order to define the values of a subchart you must put the chart name as the first key. here mariadb.values of the mariadb chart.\nHere we go!\nFirst of all we’ll run it in dry-run mode in order to check the yaml rendering (be careful, the passwords are printed in plain text)\n1helm install foo-blog bitnami/wordpress \\ 2-f override-values.yaml \\ 3--set mariadb.auth.rootPassword=r00tP4ss \\ 4--set mariadb.auth.password=us3rP4ss \\ 5--set wordpressPassword=azerty123 \\ 6--dry-run Another word you need to know is Release.\n“A Release is an instance of a chart running in a Kubernetes cluster”. Our release name here is foo-blog\nIf the output looks OK we can install our wordpress, just remove the --dry-run parameter\n1helm install foo-blog bitnami/wordpress -f override-values.yaml --set mariadb.auth.rootPassword=\u0026#34;r00tP4ss\u0026#34; --set mariadb.auth.password=\u0026#34;us3rP4ss\u0026#34; --set wordpressPassword=\u0026#34;azerty123\u0026#34; 2NAME: foo-blog 3LAST DEPLOYED: Fri Feb 12 16:33:21 2021 4NAMESPACE: default 5STATUS: deployed 6REVISION: 1 7NOTES: 8** Please be patient while the chart is being deployed ** 9 10Your WordPress site can be accessed through the following DNS name from within your cluster: 11 12 foo-blog-wordpress.default.svc.cluster.local (port 80) 13 14To access your WordPress site from outside the cluster follow the steps below: 15 161. Get the WordPress URL by running these commands: 17 18 NOTE: It may take a few minutes for the LoadBalancer IP to be available. 19 Watch the status with: \u0026#39;kubectl get svc --namespace default -w foo-blog-wordpress\u0026#39; 20 21 export SERVICE_IP=$(kubectl get svc --namespace default foo-blog-wordpress --template \u0026#34;{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}\u0026#34;) 22 echo \u0026#34;WordPress URL: http://$SERVICE_IP/\u0026#34; 23 echo \u0026#34;WordPress Admin URL: http://$SERVICE_IP/admin\u0026#34; 24 252. Open a browser and access WordPress using the obtained URL. 26 273. Login with the following credentials below to see your blog: 28 29 echo Username: foobar 30 echo Password: $(kubectl get secret --namespace default foo-blog-wordpress -o jsonpath=\u0026#34;{.data.wordpress-password}\u0026#34; | base64 --decode) When the release has been successfully installed you’ll get the above “NOTES” that are very useful to get access to your application. You just have to copy/paste.\nBut first of all we’re going to check that the pods are actually running\n1kubectl get deploy,sts 2NAME READY UP-TO-DATE AVAILABLE AGE 3deployment.apps/foo-blog-wordpress 2/2 2 2 55m 4 5NAME READY AGEkubectl get deploy,sts 6statefulset.apps/foo-blog-mariadb 1/1 55m We didn’t define an ingress for the purpose of the workshop, therefore we’ll use a port-forward\n1kubectl port-forward svc/foo-blog-wordpress 9090:80 Then open a browser using the URL http://localhost:9090/admin, you’ll be prompted to fill in the credentials you defined above. (wordpressPassword)\nWe’ll check the database credentials too as follows\n1MARIADB=$(kubectl get po -l app.kubernetes.io/name=mariadb -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 1kubectl exec -ti ${MARIADB} -- bash -c \u0026#39;mysql -u foobar -pus3rP4ss\u0026#39; 2Welcome to the MariaDB monitor. Commands end with ; or \\g. 3Your MariaDB connection id is 372 4Server version: 10.5.8-MariaDB Source distribution 5 6Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others. 7 8Type \u0026#39;help;\u0026#39; or \u0026#39;\\h\u0026#39; for help. Type \u0026#39;\\c\u0026#39; to clear the current input statement. 9 10MariaDB [(none)]\u0026gt; SHOW GRANTS; 11+-------------------------------------------------------------------------------------------------------+ 12| Grants for foobar@% | 13+-------------------------------------------------------------------------------------------------------+ 14| GRANT USAGE ON *.* TO `foobar`@`%` IDENTIFIED BY PASSWORD \u0026#39;*CD5BE357349BDA710A444B0BD741E8EB12B8BC2C\u0026#39; | 15| GRANT ALL PRIVILEGES ON `foodb`.* TO `foobar`@`%` | 16+-------------------------------------------------------------------------------------------------------+ 172 rows in set (0.000 sec) Delete the wordpress release\n1helm uninstall foo-blog Deploy a complete monitoring stack with a single command! The purpose of this step is to show that, even if the stack is composed of dozens of manifest, Helm makes things easy.\n1helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 2\u0026#34;prometheus-community\u0026#34; has been added to your repositories 3 4helm repo update 1helm install kube-prometheus prometheus-community/kube-prometheus-stack --create-namespace --namespace monitoring 2NAME: kube-prometheus 3LAST DEPLOYED: Fri Feb 12 18:03:05 2021 4NAMESPACE: monitoring 5STATUS: deployed 6REVISION: 1 7NOTES: 8kube-prometheus-stack has been installed. Check its status by running: 9 kubectl --namespace monitoring get pods -l \u0026#34;release=kube-prometheus\u0026#34; Check that all the pods are running and run a port-forward\n1kubectl port-forward -n monitoring svc/kube-prometheus-grafana 9090:80 Then open a browser using the URL http://localhost:9090/admin\ndefault credentials: admin / prom-operator\nYou should browse a few minutes over all the dashboards available. There is pretty useful info.\nYou can then have a look to the resources that have been applied with a single command line as follows\n1helm get manifest -n monitoring kube-prometheus Well for a production ready prometheus we would have played a bit with the values but you get the point.\nDelete the kube-prometheus stack\n1helm uninstall -n monitoring kube-prometheus \u0026#x27a1;\u0026#xfe0f; Next: Helm ecosystem\n","link":"https://blog.ogenki.io/post/series/workshop_helm/third_party/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop: Third party charts"},{"body":"Requirements docker k3d \u0026gt;5.x.x helm \u0026gt;3.x.x helmfile In order to have an easily provisioned temporary playground we’ll make use of k3d which is a lightweight local Kubernetes instance.\nAfter installing the binary you should enable the completion (bash or zsh) as follows (do the same for both helm and k3d).\n1source \u0026lt;(k3d completion bash) Then create the sandbox cluster named “helm-workshop”\n1k3d cluster create helm-workshop 2INFO[0000] Created network \u0026#39;k3d-helm-workshop\u0026#39; 3INFO[0000] Created volume \u0026#39;k3d-helm-workshop-images\u0026#39; 4INFO[0001] Creating node \u0026#39;k3d-helm-workshop-server-0\u0026#39; 5INFO[0006] Creating LoadBalancer \u0026#39;k3d-helm-workshop-serverlb\u0026#39; 6INFO[0007] (Optional) Trying to get IP of the docker host and inject it into the cluster as \u0026#39;host.k3d.internal\u0026#39; for easy access 7INFO[0010] Successfully added host record to /etc/hosts in 2/2 nodes and to the CoreDNS ConfigMap 8INFO[0010] Cluster \u0026#39;helm-workshop\u0026#39; created successfully! 9INFO[0010] You can now use it like this: 10kubectl cluster-info Note that your current configuration should be automatically switched to the newly created cluster.\n1$ kubectl config current-context 2k3d-helm-workshop Playing with third party charts Environment and ecosystem Build your first chart Application lifecycle Templating challenge Other considerations Hosting and versioning Most of the time we would want to share the charts in order to be used on different systems or to pull the dependencies.\nThere are multiple options for that, here are the ones that are generally used.\nChartmuseum is the official solution. This is a pretty simple webserver that exposes a Rest API. Harbor. Its main purpose is to store images (containers), but it offers many other features such as vulnerability scanning, images signing and integrates chartmuseum. Artifactory can be used to stored Helm charts too An OCI store (container registry). Pushing the charts into a central location requires to manage the versions of the charts. Any changes should trigger a version bump in the Chart.yaml file.\nSecrets management One sensitive topic that we didn’t talk about is how to handle secrets.\nThis is not directly related to Helm but this is a general issue on Kubernetes.\nThere are many options, some of them work great with Helm, some others require managing secrets apart from Helm releases.\nIn the ArgoCD documentation they tried to reference all the options available.\nCleanup Pretty simple we’ll drop the whole k3d cluster\n1k3d cluster delete helm-workshop ","link":"https://blog.ogenki.io/post/series/workshop_helm/intro/","section":"post","tags":["Helm","Kubernetes"],"title":"Helm workshop"},{"body":"","link":"https://blog.ogenki.io/categories/containers/","section":"categories","tags":null,"title":"containers"},{"body":"RBAC is the method used by Kubernetes to authorize access to API resources.\n\u0026#x2139;\u0026#xfe0f; When it makes sense you can use the default roles that are available in all Kubernetes installation instead of having to maintain custom ones.\nFor this lab what we want to achieve is to give permissions the following permissions to an application myapp:\nread the configmaps in the namespace foo List the pods in all the namespaces It is a good practice to configure your pod to make use of a serviceaccount. A serviceaccount are used to identify applications and give them permissions if necessary.\nCreate a service account\n1kubectl create serviceaccount myapp 2serviceaccount/myapp created When a service account is created, a token is automatically generated and stored in a secret.\n1kubectl describe sa myapp | grep -i token 2Mountable secrets: myapp-token-bz2zq 3Tokens: myapp-token-bz2zq 4 5kubectl get secret myapp-token-bz2zq --template={{.data.token}} | base64 -d 6eyJhb...EYxhjI_ckZ74A Using a tool to decode the JWT token you should see the following content\n1kubectl get secret myapp-token-bz2zq --template={{.data.token}} | base64 -d | jwt decode - 2 3Token header 4------------ 5{ 6 \u0026#34;alg\u0026#34;: \u0026#34;RS256\u0026#34;, 7 \u0026#34;kid\u0026#34;: \u0026#34;IdsXYO6E93xozgJg-LY2oETTPEHBJjydTU4vF2wy-wg\u0026#34; 8} 9 10Token claims 11------------ 12{ 13 \u0026#34;iss\u0026#34;: \u0026#34;kubernetes/serviceaccount\u0026#34;, 14 \u0026#34;kubernetes.io/serviceaccount/namespace\u0026#34;: \u0026#34;foo\u0026#34;, 15 \u0026#34;kubernetes.io/serviceaccount/secret.name\u0026#34;: \u0026#34;myapp-token-bz2zq\u0026#34;, 16 \u0026#34;kubernetes.io/serviceaccount/service-account.name\u0026#34;: \u0026#34;myapp\u0026#34;, 17 \u0026#34;kubernetes.io/serviceaccount/service-account.uid\u0026#34;: \u0026#34;eb606bdc-b713-4b7c-8da8-c4f71075995e\u0026#34;, 18 \u0026#34;sub\u0026#34;: \u0026#34;system:serviceaccount:foo:myapp\u0026#34; 19} We're going to create a deployment that will be configured to used this serviceaccount. In the yaml you'll notice that we defined the serviceAccountName.\n1kubectl apply -f content/resources/kubernetes_workshop/rbac/deployment.yaml 2deployment.apps/myapp created As we didn't assigned any permissions to this serviceaccount, our application won't be able to call any of the API endpoints\n1POD_NAME=$(kubectl get po -l app=myapp -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 2 3kubectl exec ${POD_NAME} -- kubectl auth can-i -n foo --list 4Resources Non-Resource URLs Resource Names Verbs 5selfsubjectaccessreviews.authorization.k8s.io [] [] [create] 6selfsubjectrulesreviews.authorization.k8s.io [] [] [create] 7 [/.well-known/openid-configuration] [] [get] 8 [/api/*] [] [get] 9 [/api] [] [get] 10 [/apis/*] [] [get] 11 [/apis] [] [get] 12 [/healthz] [] [get] 13 [/healthz] [] [get] 14 [/livez] [] [get] 15 [/livez] [] [get] 16 [/openapi/*] [] [get] 17 [/openapi] [] [get] 18 [/openid/v1/jwks] [] [get] 19 [/readyz] [] [get] 20 [/readyz] [] [get] 21 [/version/] [] [get] 22 [/version/] [] [get] 23 [/version] [] [get] 24 [/version] [] [get] In order to allow it to read configmaps in the namespace foo, we're going to create 2 resources:\nA role which will describe the permissions and which is bounded to a namespace A rolebinding to assign this role to our application (serviceaccount) Create the role\n1kubectl apply -f content/resources/kubernetes_workshop/rbac/role.yaml 2role.rbac.authorization.k8s.io/read-configmaps created And assign it to the serviceaccount we've created previously\n1kubectl create rolebinding -n foo myapp-configmap --serviceaccount=foo:myapp --role=read-configmaps 2rolebinding.rbac.authorization.k8s.io/myapp-configmap created Note that in the above command the serviceaccount must be specified with the namespace as a prefix and separated by a semicolon.\nYou don't have to restart the pod to get the permissions enabled.\n1kubectl exec ${POD_NAME} -- kubectl auth can-i get configmaps -n foo 2yes 3 4kubectl exec ${POD_NAME} -- kubectl get cm 5NAME DATA AGE 6kube-root-ca.crt 1 3d20h 7helloworld 2 2d2h This is possible thanks to the token mounted within the container\n1kubectl exec -ti ${POD_NAME} -- bash -c \u0026#39;curl -skH \u0026#34;Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)\u0026#34; https://kubernetes.default/api/v1/namespaces/foo/configmaps\u0026#39; 2{ 3 \u0026#34;kind\u0026#34;: \u0026#34;ConfigMapList\u0026#34;, 4 \u0026#34;apiVersion\u0026#34;: \u0026#34;v1\u0026#34;, 5 \u0026#34;metadata\u0026#34;: { 6 \u0026#34;resourceVersion\u0026#34;: \u0026#34;76334\u0026#34; 7 }, 8 \u0026#34;items\u0026#34;: [ 9 { 10 \u0026#34;metadata\u0026#34;: { 11 \u0026#34;name\u0026#34;: \u0026#34;kube-root-ca.crt\u0026#34;, 12 \u0026#34;namespace\u0026#34;: \u0026#34;foo\u0026#34;, 13 \u0026#34;uid\u0026#34;: \u0026#34;c352e4cd-3b88-4400-80a0-cbba318794e4\u0026#34;, 14... Finally we want to list the pods in all the namespaces of our cluster. we need:\nA clusterrole which will describe the permissions that are cluster wide. A clusterrolebinding to assign this clusterrole to our application (serviceaccount) 1$ kubectl apply -f content/resources/kubernetes_workshop/rbac/clusterrole.yaml 2clusterrole.rbac.authorization.k8s.io/list-pods created 1kubectl create clusterrolebinding -n foo myapp-pods --serviceaccount=foo:myapp --clusterrole=list-pods 2clusterrolebinding.rbac.authorization.k8s.io/myapp-pods created Now lets have a look to the permissions our applications has in the namespace foo\n1kubectl exec ${POD_NAME} -- kubectl auth can-i -n foo --list 2Resources Non-Resource URLs Resource Names Verbs 3selfsubjectaccessreviews.authorization.k8s.io [] [] [create] 4selfsubjectrulesreviews.authorization.k8s.io [] [] [create] 5pods [] [] [get list] 6configmaps [] [] [get watch list] 7 [/.well-known/openid-configuration] [] [get] 8 [/api/*] [] [get] 9 [/api] [] [get] 10 [/apis/*] [] [get] 11... ","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/rbac/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Manage permissions in Kubernetes"},{"body":"","link":"https://blog.ogenki.io/series/workshop-kubernetes/","section":"series","tags":null,"title":"Workshop Kubernetes"},{"body":"Events The first source of information when something goes wrong is the event stream. Note that you may want to sort them by creation time\n1kubectl get events -n foo --sort-by=.metadata.creationTimestamp 2... 320m Normal Created pod/web-85575f4476-5pbqv Created container nginx 420m Normal Started pod/web-85575f4476-5pbqv Started container nginx 520m Normal SuccessfulDelete replicaset/web-987f6cf9 Deleted pod: web-987f6cf9-mzsxd 620m Normal ScalingReplicaSet deployment/web Scaled down replica set web-987f6cf9 to 0 Logs Having a look to a pod's logs is just the matter of running\n1kubectl logs -f --tail=7 -c mysql wordpress-mysql-6c597b98bd-4mbbd 22021-06-24 08:27:38 1 [Note] - \u0026#39;::\u0026#39; resolves to \u0026#39;::\u0026#39;; 32021-06-24 08:27:38 1 [Note] Server socket created on IP: \u0026#39;::\u0026#39;. 42021-06-24 08:27:38 1 [Warning] Insecure configuration for --pid-file: Location \u0026#39;/var/run/mysqld\u0026#39; in the path is accessible to all OS users. Consider choosing a different directory. 52021-06-24 08:27:38 1 [Warning] \u0026#39;proxies_priv\u0026#39; entry \u0026#39;@ root@wordpress-mysql-6c597b98bd-4mbbd\u0026#39; ignored in --skip-name-resolve mode. 62021-06-24 08:27:38 1 [Note] Event Scheduler: Loaded 0 events 72021-06-24 08:27:38 1 [Note] mysqld: ready for connections. 8Version: \u0026#39;5.6.51\u0026#39; socket: \u0026#39;/var/run/mysqld/mysqld.sock\u0026#39; port: 3306 MySQL Community Server (GPL) Alternatively you can use a tool made to display logs from multiple pods: stern. A better way to explore logs is to send them to a central location using a tool such as Loki or the well know EFK stack.\nHealth checks Kubernetes self healing system is mostly based on health checks. There are different types of health checks (please have a look to the official documentation).\nWe'll add a new plugin to kubectl which is really useful to export a resource while cleaning useless metadatas: neat\n1kubectl krew install neat 2Updated the local copy of plugin index. 3Installing plugin: neat 4Installed plugin: neat 5... Let's create a new deployment using the image nginx\n1kubectl create deploy web --image=nginx --dry-run=client -o yaml | kubectl neat \u0026gt; /tmp/web.yaml Edit its content and add an HTTP health check on port 80. The endpoint must return a code ranging between 200 and 400 and it has to be a relevant test that shows the actual availability of the service.\n1apiVersion: apps/v1 2kind: Deployment 3metadata: 4 labels: 5 app: web 6 name: web 7spec: 8 replicas: 1 9 selector: 10 matchLabels: 11 app: web 12 template: 13 metadata: 14 creationTimestamp: null 15 labels: 16 app: web 17 spec: 18 containers: 19 - image: nginx 20 name: nginx 21 livenessProbe: 22 httpGet: 23 path: / 24 port: 80 25 initialDelaySeconds: 3 26 periodSeconds: 3 1kubectl apply -f /tmp/web.yaml 2deployment.apps/web created 3 4kubectl describe deploy web | grep Liveness: 5 Liveness: http-get http://:80/ delay=3s timeout=1s period=3s #success=1 #failure=3 The pod should be up without any error\n1kubectl get po -l app=web 2NAME READY STATUS RESTARTS AGE 3web-85575f4476-6qvd5 1/1 Running 0 92s We're going to simulate a service being unavailable, just change the path being checked. Here we'll use another method to modify a resource by creating a patch and applying it.\nCreate a yaml /tmp/patch.yaml file\n1cat \u0026gt; /tmp/patch.yaml \u0026lt;\u0026lt;EOF 2spec: 3 template: 4 spec: 5 containers: 6 - name: nginx 7 livenessProbe: 8 httpGet: 9 path: /foobar 10EOF And we're going to apply our change as follows\n1kubectl patch deployment web --patch \u0026#34;$(cat /tmp/patch.yaml)\u0026#34; --record 2deployment.apps/web patched 3 4kubectl describe deployment web | grep Liveness: 5 Liveness: http-get http://:80/foobar delay=3s timeout=1s period=3s #success=1 #failure=3 Now our pod should start to fail, the number of restarts increases\n1kubectl get po -l app=web 2web-987f6cf9-n4rnb 1/1 Running 4 83s Until the pod enter in a CrashLoopBackOff, meaning that it constantly restarts.\n1kubectl get po -l app=web 2NAME READY STATUS RESTARTS AGE 3web-987f6cf9-n4rnb 0/1 CrashLoopBackOff 5 3m23s Describing the pod will give you a hint on the reason it restarts\n1kubectl describe po web-987f6cf9-n4rnb | tail -n 5 2Normal Created 4m7s (x3 over 4m30s) kubelet Created container nginx 3Normal Started 4m7s (x3 over 4m30s) kubelet Started container nginx 4Warning Unhealthy 3m56s (x9 over 4m26s) kubelet Liveness probe failed: HTTP probe failed with statuscode: 404 5Normal Killing 3m56s (x3 over 4m20s) kubelet Container nginx failed liveness probe, will be restarted 6Normal Pulling 3m56s (x4 over 4m35s) kubelet Pulling image \u0026#34;nginx\u0026#34; Rollback the latest change in order to return to a working state. Note that we used the option --record when we applied the patch. That helps saving changes history.\n1kubectl rollout history deployment web 2deployment.apps/web 3REVISION CHANGE-CAUSE 41 \u0026lt;none\u0026gt; 52 kubectl patch deployment web --patch=spec: 6 template: 7 spec: 8 containers: 9 - name: nginx 10 livenessProbe: 11 httpGet: 12 path: /foobar --record=true 13 14kubectl rollout undo deployment web 15deployment.apps/web rolled back Cleanup 1kubectl delete deploy web 2deployment.apps \u0026#34;web\u0026#34; deleted learnk8s documentation There is a great documentation that contains all the steps that help debugging a deployment: https://learnk8s.io/troubleshooting-deployments\n\u0026#x27a1;\u0026#xfe0f; Next: RBAC\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/troubleshoot/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Troubleshooting"},{"body":"Resources allocation in Kubernetes Resources allocation in Kubernetes is made using requests and limits in the container's definition.\nrequests: What the container is guaranteed to get. These values are used when the scheduler takes a decision on where (what node) to place a given pod. limits: Are values that cannot be exceeded \u0026#x2139;\u0026#xfe0f; You can use explain to have a look to the documentation of resources.\n1kubectl explain --recursive pod.spec.containers.resources.limits 2KIND: Pod 3VERSION: v1 4 5FIELD: limits \u0026lt;map[string]string\u0026gt; 6 7DESCRIPTION: 8 Limits describes the maximum amount of compute resources allowed. More 9... The wordpress we've created in the previous lab doesn't have resources definition. There are different ways to edit its current state (kubectl edit, apply, patch ...)\n1kubectl edit deploy wordpress replace resources: {} with this block\n1... 2 resources: 3 requests: 4 cpu: 100m 5 memory: 100Mi 6 limits: 7 cpu: 1000m 8 memory: 200Mi 9... The pods resources usage can be displayed using (this might take a few seconds)\n1kubectl top pods 2NAME CPU(cores) MEMORY(bytes) 3wordpress-694866c6b7-mqxdd 1m 171Mi 4wordpress-mysql-6c597b98bd-4mbbd 1m 531Mi Configure the autoscaling base on cpu usage. When a pod reaches 50% of its allocated cpu a new pod is created.\n1kubectl autoscale deployment wordpress --cpu-percent=50 --min=1 --max=5 2horizontalpodautoscaler.autoscaling/wordpress autoscaled It takes up to 15 seconds (default configuration) to get the first values\n1kubectl get hpa 2NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 3wordpress Deployment/wordpress \u0026lt;unknown\u0026gt;/50% 1 5 0 10s 4 5kubectl get hpa 6NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 7wordpress Deployment/wordpress 1%/50% 1 5 1 20s Now we'll run an HTTP bench using wrk. Open a new shell and run\n1kubectl run -ti --rm bench --image=jess/wrk -- /bin/sh -c \u0026#39;wrk -t12 -c100 -d180s http://wordpress\u0026#39; During the benchmark above (3 minutes duration) let's have a look to the hpa\n1watch kubectl get hpa 2Every 2.0s: kubectl get hpa 3hostname: Tue Jun 22 11:13:08 2021 4 5NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 6wordpress Deployment/wordpress 1%/50% 1 5 1 8m28s After a few seconds we'll see that the upscaling will be done automatically. Here the number of replicas will reach the maximum we defined (5 pods).\n1Every 2.0s: kubectl get hpa 2hostname: Tue Jun 22 11:14:13 2021 3 4NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE 5wordpress Deployment/wordpress 998%/50% 1 5 5 9m33s That was a pretty simple configuration, basing the autoscaling on CPU usage for a webserver makes sense. You can also base the autoscaling on any other metrics that are reported by your application.\n\u0026#x27a1;\u0026#xfe0f; Next: Troubleshooting\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/autoscaling/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Resources allocation and autoscaling"},{"body":"\u0026#x2139;\u0026#xfe0f; This section is, for a most part, based on the official Kubernetes doc.\nBy the end of this lab we'll create the following components. You may want to come back to this schema from time to time in order to get the whole picture.\nA database with a persistent volume Check that your cluster is up and running and that your context is still configured with the namespace foo\n1kubectl config get-contexts 2CURRENT NAME CLUSTER AUTHINFO NAMESPACE 3* k3d-workshop k3d-workshop admin@k3d-workshop foo Create a persistent volume claim There are several options when it comes to persistent workloads on Kubernetes. For this workshop we'll use our local disks thanks to the local path provisionner.\nCreate a persistentVolumeClaim, it will stay pending until a pod consumes it\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/pvc.yaml 2persistentvolumeclaim/local-path-mysql created 3 4kubectl get pvc 5NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 6local-path-mysql Pending local-path 16s Create the MySQL secret In Kubernetes sensitive data are stored in Secrets. Here we'll create a secret that stores the MySQL root password\n1kubectl create secret generic mysql-pass --from-literal=password=YOUR_PASSWORD 2secret/mysql-pass created Note that a secret is stored in an base64 encoded format and can be easily decoded. (There are best practices to enforce safe access to the secrets that we're not going to cover there)\n1kubectl get secrets mysql-pass -o yaml 2apiVersion: v1 3data: 4 password: WU9VUl9QQVNTV09SRA== 5kind: Secret 6metadata: 7 creationTimestamp: \u0026#34;2021-06-20T09:11:59Z\u0026#34; 8 name: mysql-pass 9 namespace: foo 10 resourceVersion: \u0026#34;2809\u0026#34; 11 uid: c96c58d6-8472-4d68-8554-5dcfb69d834c 12type: Opaque 13 14echo -n \u0026#34;WU9VUl9QQVNTV09SRA==\u0026#34; | base64 -d 15YOUR_PASSWORD Run the MySQL deployment We will now create a MySQL deployment. It will be composed of a single replica as we're accessing to a local volume and it is configured to make use of the secret we've created previously.\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/deployment.yaml 2deployment.apps/wordpress-mysql created 3 4kubectl get po -w 5NAME READY STATUS RESTARTS AGE 6wordpress-mysql-6c597b98bd-vcm62 0/1 ContainerCreating 0 9s 7wordpress-mysql-6c597b98bd-vcm62 1/1 Running 0 13s 8^C Service discovery in Kubernetes In order to be able to call our MySQL deployment we may want to expose it using a service.\n1kubectl apply -f content/resources/kubernetes_workshop/mysql/svc.yaml 2service/wordpress-mysql created 3 4kubectl get svc 5NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 6wordpress-mysql ClusterIP None \u0026lt;none\u0026gt; 3306/TCP 6s Kubernetes's service discovery is based on an internal DNS system. For instance a service A service is accessible using the following nomenclature: \u0026lt;service_name\u0026gt;.\u0026lt;Namespace\u0026gt;.svc.\u0026lt;Cluster_domain_name\u0026gt;\nLet's try to access to the database server using a mysql client pod and create a database named foobar\n1kubectl run -ti --rm mysql-client --restart=Never --image=mysql:5.7 -- /bin/bash 2If you don\u0026#39;t see a command prompt, try pressing enter. 3root@mysql-client:/# apt -qq update \u0026amp;\u0026amp; apt install -yq netcat 4... 5Setting up netcat (1.10-41.1) ... 6 7 8root@mysql-client:/# nc -vz wordpress-mysql.foo.svc.cluster.local 3306 9DNS fwd/rev mismatch: wordpress-mysql.foo.svc.cluster.local != 10-42-1-8.wordpress-mysql.foo.svc.cluster.local 10wordpress-mysql.foo.svc.cluster.local [10.42.1.8] 3306 (?) open 11 12root@mysql-client:/# mysql -u root -h wordpress-mysql -p 13Enter password: 14... 15 16mysql\u0026gt; show databases; 17+--------------------+ 18| Database | 19+--------------------+ 20| information_schema | 21| mysql | 22| performance_schema | 23+--------------------+ 243 rows in set (0.01 sec) 25 26mysql\u0026gt; create database foobar; 27Query OK, 1 row affected (0.00 sec) 28 29mysql\u0026gt; exit 30Bye Note: You can either use the service name wordpress-mysql, or if your source pod is in another namespace use wordpress-mysql.foo\nCheck how the data is persisted with the local-path-provisioner We may want to check how the data is stored. Now that we have a MySQL instance running and consuming the pvc, a persistent volume has been provision\n1kubectl get pvc 2NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE 3local-path-mysql Bound pvc-4bb3c033-2261-4d5c-ba61-41e364769599 500Mi RWO local-path 14m Having a closer look we notice that the volume is actually a directory within a worker node.\n1kubectl describe pv pvc-4bb3c033-2261-4d5c-ba61-41e364769599 2Name: pvc-4bb3c033-2261-4d5c-ba61-41e364769599 3Labels: \u0026lt;none\u0026gt; 4Annotations: pv.kubernetes.io/provisioned-by: rancher.io/local-path 5Finalizers: [kubernetes.io/pv-protection] 6StorageClass: local-path 7Status: Bound 8Claim: foo/local-path-mysql 9Reclaim Policy: Delete 10Access Modes: RWO 11VolumeMode: Filesystem 12Capacity: 500Mi 13Node Affinity: 14 Required Terms: 15 Term 0: kubernetes.io/hostname in [k3d-workshop-agent-0] 16Message: 17Source: 18 Type: HostPath (bare host directory volume) 19 Path: /var/lib/rancher/k3s/storage/pvc-4bb3c033-2261-4d5c-ba61-41e364769599_foo_local-path-mysql 20 HostPathType: DirectoryOrCreate 21Events: \u0026lt;none\u0026gt; 1docker exec k3d-workshop-agent-0 ls /var/lib/rancher/k3s/storage/pvc-4bb3c033-2261-4d5c-ba61-41e364769599_foo_local-path-mysql 2auto.cnf 3foobar 4ib_logfile0 5ib_logfile1 6ibdata1 7mysql 8performance_schema That means that even if you restart your laptop you should retrieve the data (here the database foobar we've created previously)\n1k3d cluster stop workshop 2INFO[0000] Stopping cluster \u0026#39;workshop\u0026#39; 3 4k3d cluster list 5NAME SERVERS AGENTS LOADBALANCER 6workshop 0/1 0/1 true 7 8k3d cluster start workshop 9INFO[0000] Starting cluster \u0026#39;workshop\u0026#39; 10INFO[0000] Starting servers... 11INFO[0000] Starting Node \u0026#39;k3d-workshop-server-0\u0026#39; 12INFO[0006] Starting agents... 13INFO[0006] Starting Node \u0026#39;k3d-workshop-agent-0\u0026#39; 14INFO[0013] Starting helpers... 15INFO[0013] Starting Node \u0026#39;k3d-workshop-serverlb\u0026#39; 16 17kubectl run -ti --rm mysql-client --restart=Never --image=mysql:5.7 -- mysql -u root -h wordpress-mysql --password=\u0026#34;YOUR_PASSWORD\u0026#34; 18If you don\u0026#39;t see a command prompt, try pressing enter. 19 20mysql\u0026gt; show databases; 21+--------------------+ 22| Database | 23+--------------------+ 24| information_schema | 25| foobar | 26| mysql | 27| performance_schema | 28+--------------------+ 294 rows in set (0.00 sec) The Wordpress deployment Now we will deploy the wordpress instance with a persistent volume.\nSo first of all create a pvc as follows\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/pvc.yaml 2persistentvolumeclaim/wp-pv-claim created Then create the deployment. Note that it is configured with our mysql database as backend.\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/deployment.yaml 2deployment.apps/wordpress created 3 4$ kubectl get deploy 5NAME READY UP-TO-DATE AVAILABLE AGE 6wordpress-mysql 1/1 1 1 11h 7wordpress 1/1 1 1 4s Most of the time, when we want to expose an HTTP service to the outside world (outside of the cluster), we would create an ingress\n1kubectl apply -f content/resources/kubernetes_workshop/wordpress/svc.yaml 2service/wordpress created 3 4kubectl apply -f content/resources/kubernetes_workshop/wordpress/ingress.yaml 5ingress.networking.k8s.io/wordpress created With k3d the ingress endpoint has been defined when we've created the cluster. With the parameter -p \u0026quot;8081:80@loadbalancer\u0026quot; Our wordpress should therefore be accessible through http://localhost:8081\nConfigure your pods A ConfigMap is a kubernetes resource that stores non-sensitive data. Its content can be consumed as config files, environment variables or command args.\nLet's consider that we need a configfile to be mounted in our wordpress deployment as well as an environment variable made available.\nCreate a dumb \u0026quot;hello world\u0026quot; config file\n1echo \u0026#34;Hello World!\u0026#34; \u0026gt; /tmp/helloworld.conf Then we'll create a configmap that contains a file and environment variable we want to make use of. Note This following command doesn't actually apply the resource on our Kubernetes cluster. It just generate a local yaml file using --dry-run and -o yaml.\n1kubectl create configmap helloworld --from-file=/tmp/helloworld.conf --from-literal=HELLO=WORLD -o yaml --dry-run=client \u0026gt; /tmp/cm.yaml Check the configmap\n1apiVersion: v1 2data: 3 HELLO: WORLD 4 helloworld.conf: | 5 Hello World! 6kind: ConfigMap 7metadata: 8 creationTimestamp: null 9 name: helloworld And apply it\n1$ kubectl apply -f /tmp/cm.yaml 2configmap/helloworld created Now we're gonna make use of it by changing the wordpress deployment. For this kind of change it is recommended to use an IDE with a Kubernetes plugin that will highlight errors.\nEdit the file located here: content/resources/kubernetes_workshop/wordpress/deployment.yaml\n1... 2 env: 3 - name: WORDPRESS_DB_HOST 4 value: wordpress-mysql 5 - name: WORDPRESS_DB_PASSWORD 6 valueFrom: 7 secretKeyRef: 8 name: mysql-pass 9 key: password 10 - name: HELLO 11 valueFrom: 12 configMapKeyRef: 13 name: helloworld 14 key: HELLO 15 volumeMounts: 16 - name: wordpress-persistent-storage 17 mountPath: /var/www/html 18 - name: helloworld-config 19 mountPath: /config 20 volumes: 21 - name: wordpress-persistent-storage 22 persistentVolumeClaim: 23 claimName: wp-pv-claim 24 - name: helloworld-config 25 configMap: 26 name: helloworld 27 items: 28 - key: helloworld.conf 29 path: helloworld.conf Applying this change will trigger a rolling-update\n1$ kubectl apply -f content/resources/kubernetes_workshop/wordpress/deployment.yaml 2deployment.apps/wordpress configured 3 4$ kubectl get po 5NAME READY STATUS RESTARTS AGE 6wordpress-mysql-6c597b98bd-4mbbd 1/1 Running 2 41h 7wordpress-594f88c9c4-n9qqr 1/1 Running 0 5s And the configuration will be available in the newly created pod\n1$ kubectl exec -ti wordpress-594f88c9c4-n9qqr -- env | grep HELLO 2HELLO=WORLD 3 4$ kubectl exec -ti wordpress-594f88c9c4-n9qqr -- cat /config/helloworld.conf 5Hello World! \u0026#x26a0;\u0026#xfe0f; Do not delete anything, we'll make use of these resources in the next section.\n\u0026#x27a1;\u0026#xfe0f; Next: Resources in Kubernetes\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/application_stack/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Complete application stack"},{"body":"Prepare your local Kubernetes environment Goal: Having a running local Kubernetes environment\nEnsure that you fulfilled the requirements\nUsing k3d to create a cluster In order to have an easily provisioned temporary playground we’ll make use of k3d which is a lightweight local Kubernetes instance. (Note that they are alternatives to run a local Kubernetes cluster such as: kubeadm, microk8s, minikube)\nAfter installing the binary you should enable the completion (bash or zsh) as follows (do the same for both kubectl and k3d).\n1source \u0026lt;(k3d completion bash) Then create the sandbox cluster named \u0026quot;workshop\u0026quot; with an additional worker\n1k3d cluster create workshop -p \u0026#34;8081:80@loadbalancer\u0026#34; --agents 1 2INFO[0000] Prep: Network 3INFO[0000] Created network \u0026#39;k3d-workshop\u0026#39; (ce74508d3fe09d8622f1ae83effd412d754dfdb441aa9d550723805f9b528c6b) 4INFO[0000] Created volume \u0026#39;k3d-workshop-images\u0026#39; 5INFO[0001] Creating node \u0026#39;k3d-workshop-server-0\u0026#39; 6INFO[0001] Creating node \u0026#39;k3d-workshop-agent-0\u0026#39; 7INFO[0001] Creating LoadBalancer \u0026#39;k3d-workshop-serverlb\u0026#39; 8INFO[0001] Starting cluster \u0026#39;workshop\u0026#39; 9INFO[0001] Starting servers... 10INFO[0001] Starting Node \u0026#39;k3d-workshop-server-0\u0026#39; 11INFO[0006] Starting agents... 12INFO[0006] Starting Node \u0026#39;k3d-workshop-agent-0\u0026#39; 13INFO[0018] Starting helpers... 14INFO[0018] Starting Node \u0026#39;k3d-workshop-serverlb\u0026#39; 15INFO[0019] (Optional) Trying to get IP of the docker host and inject it into the cluster as \u0026#39;host.k3d.internal\u0026#39; for easy access 16INFO[0023] Successfully added host record to /etc/hosts in 3/3 nodes and to the CoreDNS ConfigMap 17INFO[0023] Cluster \u0026#39;workshop\u0026#39; created successfully! 18INFO[0023] --kubeconfig-update-default=false --\u0026gt; sets --kubeconfig-switch-context=false 19INFO[0023] You can now use it like this: 20kubectl config use-context k3d-workshop 21kubectl cluster-info As k3d is made to be used on top of docker you can see the status of the running containers. You should have 3 containers, one for the loadbalancing, one for the control-plane and an agent (worker).\n1docker ps 2CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 34b5847b265dd rancher/k3d-proxy:v4.4.6 \u0026#34;/bin/sh -c nginx-pr…\u0026#34; About a minute ago Up About a minute 80/tcp, 0.0.0.0:43903-\u0026gt;6443/tcp k3d-workshop-serverlb 4523a025087b3 rancher/k3s:v1.21.1-k3s1 \u0026#34;/bin/entrypoint.sh …\u0026#34; About a minute ago Up About a minute k3d-workshop-agent-0 5791b8a69bc1f rancher/k3s:v1.21.1-k3s1 \u0026#34;/bin/entrypoint.sh …\u0026#34; About a minute ago Up About a minute k3d-workshop-server-0 With kubectl you'll see 2 running pods: a control-plane and a worker\n1kubectl get nodes 2NAME STATUS ROLES AGE VERSION 3k3d-workshop-agent-0 Ready \u0026lt;none\u0026gt; 2m34s v1.21.1+k3s1 4k3d-workshop-server-0 Ready control-plane,master 2m44s v1.21.1+k3s1 You can also have a look to the default cluster's components that are all located in the namespace kube-system\n1kubectl get pods -n kube-system 2NAME READY STATUS RESTARTS AGE 3helm-install-traefik-crd-h5j7m 0/1 Completed 0 16h 4helm-install-traefik-8mzhk 0/1 Completed 0 16h 5svclb-traefik-gh4rk 2/2 Running 2 16h 6traefik-97b44b794-lcmh4 1/1 Running 1 16h 7coredns-7448499f4d-h7xvn 1/1 Running 2 16h 8local-path-provisioner-5ff76fc89d-qvpf7 1/1 Running 1 16h 9svclb-traefik-cbvmp 2/2 Running 2 16h 10metrics-server-86cbb8457f-5v9ls 1/1 Running 1 16h The CLI configuration The main interface to the Kubernetes API is kubectl. This CLI is configured with what we call a kubeconfig you can have a look at its content wether by having a look at its default location is ~/.kube/config or running the command\n1kubectl config view 2apiVersion: v1 3clusters: 4- cluster: 5 certificate-authority-data: DATA+OMITTED 6 server: https://0.0.... and you can check if the CLI is properly configured by running\n1kubectl cluster-info 2Kubernetes control plane is running at https://0.0.0.0:43903 3CoreDNS is running at https://0.0.0.0:43903/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy 4Metrics-server is running at https://0.0.0.0:43903/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy Kubectl plugins It is really easy to extend the capabilities of he kubectl CLI. Here is a basic \u0026quot;hello-world\u0026quot; example:\nWrite a dumb script, just ensure its name is prefixed with kubectl- and put it in your PATH\n1cat \u0026gt; kubectl-helloworld\u0026lt;\u0026lt;EOF 2#!/bin/bash 3echo \u0026#34;Hello world!\u0026#34; 4EOF 5 6chmod u+x kubectl-helloworld \u0026amp;\u0026amp; sudo mv kubectl-helloworld /usr/local/bin Then it can be used as an argument of kubectl\n1kubectl helloworld 2Hello world! Delete our test\n1sudo rm /usr/local/bin/kubectl-helloworld You can find more information on how to create a kubectl plugin here\nIn order to benefit from the plugins written by the community there's a tool named krew\nUpdate the local index\n1kubectl krew update 2Adding \u0026#34;default\u0026#34; plugin index from https://github.com/kubernetes-sigs/krew-index.git. 3Updated the local copy of plugin index. Browse the available plugins\n1kubectl krew search 2NAME DESCRIPTION INSTALLED 3access-matrix Show an RBAC access matrix for server resources no 4advise-psp Suggests PodSecurityPolicies for cluster. no 5allctx Run commands on contexts in your kubeconfig no 6apparmor-manager Manage AppArmor profiles for cluster. no 7... For the current workshop we'll make use of ctx ns\nctx: Switch between contexts in your kubeconfig (Really helpful when you have multiple clusters to manage) ns: Switch between Kubernetes namespaces (Avoid to specify the namespace for each kubectl commands when working on a given namespace) 1kubectl krew install ctx ns 2Updated the local copy of plugin index. 3Installing plugin: ctx 4... Then you'll be able to switch between contexts (clusters) and namespaces.\n1kubectl ns 2Context \u0026#34;k3d-workshop\u0026#34; modified. 3Active namespace is \u0026#34;kube-system\u0026#34;. \u0026#x27a1;\u0026#xfe0f; Next: Run an application on Kubernetes\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/local/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop: Local environment"},{"body":"Namespaces Namespaces allow to logically distribute your applications, generally based on teams, projects or applications stacks. Resources names are unique within a namespace. That means that you could have a service named webserver on 2 different namespaces.\nThey can be used to isolate applications using network policies, or to define quotas.\nFor this training we'll work on a namespace named foo\n1kubectl create ns foo 2namespace/foo created And we'll make use of the plugin ns installed in the previous section to set the default namespace as follows\n1kubectl ns 2Context \u0026#34;k3d-workshop\u0026#34; modified. 3Active namespace is \u0026#34;foo\u0026#34;. Check that your kubectl is properly configured, here you can see the cluter and the namespace:\n1kubectl config get-contexts 2CURRENT NAME CLUSTER AUTHINFO NAMESPACE 3* k3d-workshop k3d-workshop admin@k3d-workshop foo Create your first pod Creating resources in Kubernetes is often done by applying a yaml/json definition through the API.\nFirst of all we need to clone this repository and change the current path to its root\n1git clone https://github.com/Smana/workshop_kubernetes_2021.git 2 3cd workshop_kubernetes_2021.git Start by creating a pretty simple pod:\n1kubectl apply -f content/resources/kubernetes_workshop/pod.yaml --namespace foo 2pod/web created 3 4kubectl get po 5NAME READY STATUS RESTARTS AGE 6web 1/1 Running 0 98s We can get detailed information about the pod as follows\n1kubectl describe po web 2Name: web 3Namespace: foo 4Priority: 0 5Node: k3d-workshop-agent-0/172.20.0.3 6Start Time: Fri, 18 Jun 2021 17:05:46 +0200 7Labels: run=web 8Annotations: \u0026lt;none\u0026gt; 9Status: Running 10IP: 10.42.1.6 11... Or even get a specific attribute, here is an example to get the pod's IP\n1kubectl get po web --template={{.status.podIP}} 210.42.1.6 This is worth noting that a pod isn't controlled by a replicaset-controller. That means that when it is deleted, it is not restarted automatically.\n1kubectl delete po web 2pod \u0026#34;web\u0026#34; deleted 3 4kubectl get po 5No resources found in foo namespace. A pod with 2 containers Now create a new pod using the manifest content/resources/kubernetes_workshop/pod2containers.yaml. Look at its content, we will be using a shared temporary directory and we'll mount its content on both containers. That way we can share data between 2 containers of a given pod.\n1kubectl apply -f content/resources/kubernetes_workshop/pod2containers.yaml 2pod/web created 3 4kubectl get pod 5NAME READY STATUS RESTARTS AGE 6web 2/2 Running 0 36s We can check that the logs are accessible on the 2 containers\n1kubectl logs web -c logger --tail=6 -f 2Mon Jun 28 21:06:20 2021 3Mon Jun 28 21:06:21 2021 4Mon Jun 28 21:06:22 2021 5Mon Jun 28 21:06:23 2021 6Mon Jun 28 21:06:24 2021 7 8kubectl exec web -c web -- tail -n 5 /log/out.log 9Mon Jun 28 21:07:19 2021 10Mon Jun 28 21:07:20 2021 11Mon Jun 28 21:07:21 2021 12Mon Jun 28 21:07:22 2021 13Mon Jun 28 21:07:23 2021 Delete the pod\n1kubectl delete po web 2pod \u0026#34;web\u0026#34; deleted Create a simple webserver deployment A deployment is a resource that describes the desired state of an application. Kubernetes will ensure that its current status is aligned with the desired one.\nCreating a simple deployment can be done using kubectl\n1kubectl create deployment podinfo --image stefanprodan/podinfo 2deployment.apps/podinfo created After a few seconds the deployment will be up to date, meaning that the a pod is up and running.\n1kubectl get deploy 2NAME READY UP-TO-DATE AVAILABLE AGE 3podinfo 1/1 1 1 14s Replicas and scaling A deployment creates a replicaset under the hood in order to ensure that the number of replicas (pods) matches the desired one.\n1kubectl get replicasets 2NAME DESIRED CURRENT READY AGE 3podinfo-7fbb45ccfc 1 1 1 36s Creating a deployment without specifying the number of replicas will create a single replica. We can scale it on demand using\n1kubectl scale deploy podinfo --replicas 6 2deployment.apps/podinfo scaled 3 4kubectl rollout status deployment podinfo 5Waiting for deployment \u0026#34;podinfo\u0026#34; rollout to finish: 4 of 6 updated replicas are available... 6Waiting for deployment \u0026#34;podinfo\u0026#34; rollout to finish: 5 of 6 updated replicas are available... 7deployment \u0026#34;podinfo\u0026#34; successfully rolled out The default Kubernetes scheduler will try to spread evenly the pods according to the available resources on worker nodes.\n1kubectl get po -o wide 2NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES 3podinfo-7fbb45ccfc-dwxtx 1/1 Running 0 114s 10.42.1.8 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 4podinfo-7fbb45ccfc-p2djv 1/1 Running 0 34s 10.42.1.11 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 5podinfo-7fbb45ccfc-4fk9z 1/1 Running 0 34s 10.42.1.9 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 6podinfo-7fbb45ccfc-gqwz6 1/1 Running 0 34s 10.42.1.10 k3d-workshop-agent-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 7podinfo-7fbb45ccfc-4qgvs 1/1 Running 0 34s 10.42.0.8 k3d-workshop-server-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; 8podinfo-7fbb45ccfc-r6dn5 1/1 Running 0 34s 10.42.0.9 k3d-workshop-server-0 \u0026lt;none\u0026gt; \u0026lt;none\u0026gt; The deployment controller will ensure to start new pods if the number of replicas doesn't match its configuration.\n1kubectl delete po $(kubectl get po -l app=podinfo -o jsonpath=\u0026#39;{.items[0].metadata.name}\u0026#39;) 2pod \u0026#34;podinfo-7fbb45ccfc-r6dn5\u0026#34; deleted 3 4kubectl describe rs podinfo-7fbb45ccfc 5Name: podinfo-7fbb45ccfc 6Namespace: foo 7Selector: app=podinfo,pod-template-hash=7fbb45ccfc 8Labels: app=podinfo 9 pod-template-hash=7fbb45ccfc 10Annotations: deployment.kubernetes.io/desired-replicas: 6 11 deployment.kubernetes.io/max-replicas: 8 12 deployment.kubernetes.io/revision: 5 13 deployment.kubernetes.io/revision-history: 1,3 14Controlled By: Deployment/podinfo 15Replicas: 6 current / 6 desired 16Pods Status: 6 Running / 0 Waiting / 0 Succeeded / 0 Failed 17... 18Events: 19 Type Reason Age From Message 20 ---- ------ ---- ---- ------- 21... 22 Normal SuccessfulCreate 16h (x3 over 16h) replicaset-controller (combined from similar events): Created pod: podinfo-7fbb45ccfc-pkt4r 23 Normal SuccessfulCreate 96s replicaset-controller Created pod: podinfo-7fbb45ccfc-bkm8n 24 25kubectl get deploy 26NAME READY UP-TO-DATE AVAILABLE AGE 27podinfo 6/6 6 6 18m Rolling update Using a deployment allows to manage the application lifecycle. Changing its configuration will trigger a rolling update.\nFirst of all we'll change the image tag of our deployment\n1kubectl set image deployment podinfo podinfo=stefanprodan/podinfo:5.2.1 2deployment.apps/podinfo image updated During a rolling update a new replicaset is created in order to update the application in place without any downtime. New pods (with the current deployment state) will be created in the new replicaset while they will be deleted progressively from the previous replicaset.\n1kubectl get rs -o wide 2NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR 3podinfo-7fbb45ccfc 0 0 0 21m podinfo stefanprodan/podinfo app=podinfo,pod-template-hash=7fbb45ccfc 4podinfo-564b4ddd7c 6 6 6 30s podinfo stefanprodan/podinfo:5.2.1 app=podinfo,pod-template-hash=564b4ddd7c Keeping the old replicaset makes very easy to rollback\n1kubectl rollout undo deployment podinfo 2deployment.apps/podinfo rolled back 3 4kubectl get rs -o wide 5NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR 6podinfo-7fbb45ccfc 6 6 6 22m podinfo stefanprodan/podinfo app=podinfo,pod-template-hash=7fbb45ccfc 7podinfo-564b4ddd7c 0 0 0 77s podinfo stefanprodan/podinfo:5.2.1 app=podinfo,pod-template-hash=564b4ddd7c Expose a deployment Now that we have a running web application we may want to access it. There are several ways to expose an app, here we'll use the easiest way: Create a service and run a port-forward.\nThe following command will create a service which will be in charge of forwarding calls through the tcp port 9898\n1kubectl expose deploy podinfo --port 9898 2service/podinfo exposed We can get more information on the service as follows\n1kubectl get svc -o yaml podinfo 2apiVersion: v1 3kind: Service 4metadata: 5 labels: 6 app: podinfo 7 name: podinfo 8 namespace: foo 9spec: 10 clusterIP: 10.43.47.17 11 clusterIPs: 12 - 10.43.47.17 13 ipFamilies: 14 - IPv4 15 ipFamilyPolicy: SingleStack 16 ports: 17 - port: 9898 18 selector: 19 app: podinfo A service uses the selector above to identify on which pod to forward the traffic and usually creates the endpoints accordingly.\n1kubectl get po -l app=podinfo 2NAME READY STATUS RESTARTS AGE 3podinfo-7fbb45ccfc-bkm8n 1/1 Running 1 146m 4podinfo-7fbb45ccfc-sbqht 1/1 Running 2 18h 5... 6 7kubectl get endpoints 8NAME ENDPOINTS AGE 9podinfo 10.42.0.16:9898,10.42.0.17:9898,10.42.1.18:9898 + 3 more... 92s The service we've created has an IP that's only accessible from within the cluster. Using the port-forward command we're able to forward the traffic from our local machine to the application (through the API server). Note that you can target either a deployment, a service or a single pod\n1 2kubectl port-forward svc/podinfo 9898 \u0026amp; 3Forwarding from 127.0.0.1:9898 -\u0026gt; 9898 4Forwarding from [::1]:9898 -\u0026gt; 9898 5 6curl http://localhost:9898 7Handling connection for 9898 8{ 9 \u0026#34;hostname\u0026#34;: \u0026#34;podinfo-7fbb45ccfc-sbqht\u0026#34;, 10 \u0026#34;version\u0026#34;: \u0026#34;6.0.0\u0026#34;, 11 \u0026#34;revision\u0026#34;: \u0026#34;\u0026#34;, 12 \u0026#34;color\u0026#34;: \u0026#34;#34577c\u0026#34;, 13 \u0026#34;logo\u0026#34;: \u0026#34;https://raw.githubusercontent.com/stefanprodan/podinfo/gh-pages/cuddle_clap.gif\u0026#34;, 14 \u0026#34;message\u0026#34;: \u0026#34;greetings from podinfo v6.0.0\u0026#34;, 15 \u0026#34;goos\u0026#34;: \u0026#34;linux\u0026#34;, 16 \u0026#34;goarch\u0026#34;: \u0026#34;amd64\u0026#34;, 17 \u0026#34;runtime\u0026#34;: \u0026#34;go1.16.5\u0026#34;, 18 \u0026#34;num_goroutine\u0026#34;: \u0026#34;6\u0026#34;, 19 \u0026#34;num_cpu\u0026#34;: \u0026#34;16\u0026#34; 20} Cleanup In this section we created 2 resources: a deployment and a service.\n1fg 2kubectl port-forward svc/podinfo 9898 3^C 4 5kubectl delete svc,deploy podinfo 6service \u0026#34;podinfo\u0026#34; deleted 7deployment.apps \u0026#34;podinfo\u0026#34; deleted \u0026#x27a1;\u0026#xfe0f; Next: Deploy a Wordpress\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/run_app/","section":"post","tags":["Kubernetes"],"title":"Run an application on Kubernetes"},{"body":"This repository aims to quickly learn the basics of Kubernetes.\n\u0026#x26a0;\u0026#xfe0f; None of the examples given here are made for production.\nRequirements docker k3d \u0026gt;5.x.x kubectl krew (optional)fzf Agenda Prepare your local Kubernetes environment Run an application on Kubernetes Deploy a Wordpress Resources and autoscaling Troubleshooting RBAC Cleanup Pretty simple we’ll drop the whole k3d cluster\n1k3d cluster delete workshop 2INFO[0000] Deleting cluster \u0026#39;workshop\u0026#39; 3... 4INFO[0008] Successfully deleted cluster workshop! \u0026#x27a1;\u0026#xfe0f; You may want to continue with the Helm workshop\n","link":"https://blog.ogenki.io/post/series/workshop_kubernetes/intro/","section":"post","tags":["Kubernetes"],"title":"Kubernetes workshop"},{"body":"","link":"https://blog.ogenki.io/tags/index/","section":"tags","tags":null,"title":"index"}]
\ No newline at end of file
diff --git a/post/cilium-gateway-api/index.html b/post/cilium-gateway-api/index.html
index fa6f32a..ac2b87f 100644
--- a/post/cilium-gateway-api/index.html
+++ b/post/cilium-gateway-api/index.html
@@ -8,7 +8,7 @@
 </a><a href=https://blog.ogenki.io/tags/infrastructure/ title=infrastructure class="post_tag button button_translucent">infrastructure
 </a><a href=https://blog.ogenki.io/tags/network/ title=network class="post_tag button button_translucent">network
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Share on:
-<a href="https://twitter.com/intent/tweet?text=%60Gateway%20API%60%3a%20Can%20I%20replace%20my%20Ingress%20Controller%20with%20%60Cilium%60%3f&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fcilium-gateway-api%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fcilium-gateway-api%2f&t=%60Gateway%20API%60%3a%20Can%20I%20replace%20my%20Ingress%20Controller%20with%20%60Cilium%60%3f" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/cilium-gateway-api/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#-introduction-to-gateway-api>☸ Introduction to Gateway API</a></li><li><a href=#ballot_box_with_check-prerequisites>Prerequisites</a></li><li><a href=#-the-entry-point-gatewayclass-and-gateway>🚪 The Entry Point: GatewayClass and Gateway</a></li><li><a href=#arrow_right_hook-routing-rules-httproute>Routing rules: HTTPRoute</a><ul><li><a href=#a-basic-rule>A basic rule</a></li><li><a href=#configure-a-tls-certificate>Configure a TLS certificate</a></li><li><a href=#sharing-a-gateway-accross-multiple-namespaces>Sharing a Gateway accross multiple namespaces</a></li><li><a href=#traffic-splitting>Traffic splitting</a></li><li><a href=#headers-modifications>Headers modifications</a></li></ul></li><li><a href=#-assign-the-proper-permissions>🔒 Assign the proper permissions</a></li><li><a href=#-a-somewhat-unclear-scope-at-first-glance>🤔 A somewhat unclear scope at first glance</a></li><li><a href=#-final-thoughts>💭 Final thoughts</a></li><li><a href=#-references>🔖 References</a></li></ul></nav></div><div class=post_body><p>When deploying an application on Kubernetes, the next step usually involves making it accessible to users. We commonly use <a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/><strong>Ingress controllers</strong></a>, such as Nginx, Haproxy, Traefik, or those from Cloud providers, to direct incoming traffic to the application, manage load balancing, TLS termination, and more.</p><p>Then we have to choose from the plethora of <a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/>available options</a> 🤯. Cilium is, relatively recently, one of them and aims to handle all these networking aspects.</p><p><a href=https://cilium.io/><strong>Cilium</strong></a> is an Open-Source networking and security solution based on eBPF whose adoption is growing rapidly. It's probably the network plugin that provides the most features. We won't cover all of them, but one such feature involves managing incoming traffic using the <a href=https://gateway-api.sigs.k8s.io/><strong>Gateway API</strong></a> (<code>GAPI</code>).</p><h2 id=bullseye-our-target>:bullseye: Our target</h2><ul><li>Understand exactly what the <strong>Gateway API</strong> is and how it represents an evolution from the <code>Ingress</code> API.</li><li>Demonstrations of real-world scenarios deployed the GitOps way.</li><li>Current limitations and upcoming developments.</li></ul><div class="notices tip"><div class=label>Tip</div><p>All the steps carried out in this article come from this <a href=https://github.com/Smana/cilium-gateway-api><strong>git repository</strong></a>.</p><p>I encourage you to explore it, as it goes far beyond the context of this article:</p><ul><li>Installation of an <strong>EKS</strong> cluster with <code>Cilium</code> configured with the kube-proxy replacement enbled and a dedicated Daemonset for <code>Envoy</code>.</li><li>Proposal of a <code>Flux</code> structure with dependency management and a DRY code I think is efficient.</li><li><code>Crossplane</code> and <a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> composition which simplifies the management of IAM permissions for platform components.</li><li>Automated domain names and certificates management with <code>External-DNS</code> and <code>Let's Encrypt</code>.</li></ul><p>The idea being to have everything set up in just a few minutes, with a single command line 🤩.</p></div><h2 id=-introduction-to-gateway-api>☸ Introduction to Gateway API</h2><p>As mentioned previously, there are many <strong>Ingress Controllers</strong> options, and each has its own specificities and particular features, sometimes making their use complex. Furthermore, the traditionnal <code>Ingress</code> API in Kubernetes has very limited parameters. Some solutions have even created their own <a href=https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>CRDs</a> (Kubernetes Custom Resources) while others use <code>annotations</code> to overcome these limitations.</p><p>Here comes the <strong>Gateway API</strong>! This is actually a <strong>standard</strong> that allows declaring advanced networking features without requiring specific extensions to the underlying controller. Moreover, since all controllers use the same API, it is possible to switch from one solution to another <strong>without changing the configuration</strong> (The Kubenetes manifests which describe how the incoming traffic should be routed).</p><p>Among the concepts that we will explore, GAPI brings a granular authorization model which defines <strong>explicit roles with distinct permissions</strong>. (More information on the GAPI security model <a href=https://gateway-api.sigs.k8s.io/concepts/security-model/#roles-and-personas>here</a>).</p><p>This is worth noting that this project is driven by the <a href=https://github.com/kubernetes/community/tree/master/sig-network>sig-network-kubernetes</a> working group, and there's a slack <a href=https://kubernetes.slack.com/archives/CR0H13KGA>channel</a> where you can reach out to them if needed.</p><p>Let's see how GAPI is used in practice with Cilium 🚀!</p><h2 id=ballot_box_with_check-prerequisites>&#9745;&#xfe0f; Prerequisites</h2><p>For the remainder of this article, we assume an EKS cluster has been deployed. If you're not using the <a href=https://github.com/Smana/cilium-gateway-api/tree/main/terraform/eks>method suggested in the demo repo</a> as the basis for this article, there are a few <strong>points to check</strong> for GAPI to be usable.</p><p>ℹ️ The installation method described here is based on <code>Helm</code>, all the <code>values</code> can be viewed <a href=https://github.com/Smana/cilium-gateway-api/blob/main/terraform/eks/helm_values/cilium.yaml>here</a>.</p><ul><li><p><strong>Install</strong> the <code>CRDs</code> available in the <a href=https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd>Gateway API</a> repository.<div class="notices note"><div class=label>Note</div><p>If Cilium is set up with <code>GAPI</code> support (see below) and the CRDs are missing, it won't start.
+<a href="https://twitter.com/intent/tweet?text=%60Gateway%20API%60%3a%20Can%20I%20replace%20my%20Ingress%20Controller%20with%20%60Cilium%60%3f&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fcilium-gateway-api%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fcilium-gateway-api%2f&t=%60Gateway%20API%60%3a%20Can%20I%20replace%20my%20Ingress%20Controller%20with%20%60Cilium%60%3f" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/cilium-gateway-api/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#-introduction-to-gateway-api>☸ Introduction to Gateway API</a></li><li><a href=#ballot_box_with_check-prerequisites>Prerequisites</a></li><li><a href=#-the-entry-point-gatewayclass-and-gateway>🚪 The Entry Point: GatewayClass and Gateway</a></li><li><a href=#arrow_right_hook-routing-rules-httproute>Routing rules: HTTPRoute</a><ul><li><a href=#a-basic-rule>A basic rule</a></li><li><a href=#configure-a-tls-certificate>Configure a TLS certificate</a></li><li><a href=#sharing-a-gateway-accross-multiple-namespaces>Sharing a Gateway accross multiple namespaces</a></li><li><a href=#traffic-splitting>Traffic splitting</a></li><li><a href=#headers-modifications>Headers modifications</a></li></ul></li><li><a href=#-assign-the-proper-permissions>🔒 Assign the proper permissions</a></li><li><a href=#-a-somewhat-unclear-scope-at-first-glance>🤔 A somewhat unclear scope at first glance</a></li><li><a href=#-final-thoughts>💭 Final thoughts</a></li><li><a href=#-references>🔖 References</a></li></ul></nav></div><div class=post_body><p>When deploying an application on Kubernetes, the next step usually involves making it accessible to users. We commonly use <a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/><strong>Ingress controllers</strong></a>, such as Nginx, Haproxy, Traefik, or those from Cloud providers, to direct incoming traffic to the application, manage load balancing, TLS termination, and more.</p><p>Then we have to choose from the plethora of <a href=https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/>available options</a> 🤯. Cilium is, relatively recently, one of them and aims to handle all these networking aspects.</p><p><a href=https://cilium.io/><strong>Cilium</strong></a> is an Open-Source networking and security solution based on eBPF whose adoption is growing rapidly. It's probably the network plugin that provides the most features. We won't cover all of them, but one such feature involves managing incoming traffic using the <a href=https://gateway-api.sigs.k8s.io/><strong>Gateway API</strong></a> (<code>GAPI</code>).</p><h2 id=-our-target>🎯 Our target</h2><ul><li>Understand exactly what the <strong>Gateway API</strong> is and how it represents an evolution from the <code>Ingress</code> API.</li><li>Demonstrations of real-world scenarios deployed the GitOps way.</li><li>Current limitations and upcoming developments.</li></ul><div class="notices tip"><div class=label>Tip</div><p>All the steps carried out in this article come from this <a href=https://github.com/Smana/cilium-gateway-api><strong>git repository</strong></a>.</p><p>I encourage you to explore it, as it goes far beyond the context of this article:</p><ul><li>Installation of an <strong>EKS</strong> cluster with <code>Cilium</code> configured with the kube-proxy replacement enbled and a dedicated Daemonset for <code>Envoy</code>.</li><li>Proposal of a <code>Flux</code> structure with dependency management and a DRY code I think is efficient.</li><li><code>Crossplane</code> and <a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> composition which simplifies the management of IAM permissions for platform components.</li><li>Automated domain names and certificates management with <code>External-DNS</code> and <code>Let's Encrypt</code>.</li></ul><p>The idea being to have everything set up in just a few minutes, with a single command line 🤩.</p></div><h2 id=-introduction-to-gateway-api>☸ Introduction to Gateway API</h2><p>As mentioned previously, there are many <strong>Ingress Controllers</strong> options, and each has its own specificities and particular features, sometimes making their use complex. Furthermore, the traditionnal <code>Ingress</code> API in Kubernetes has very limited parameters. Some solutions have even created their own <a href=https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>CRDs</a> (Kubernetes Custom Resources) while others use <code>annotations</code> to overcome these limitations.</p><p>Here comes the <strong>Gateway API</strong>! This is actually a <strong>standard</strong> that allows declaring advanced networking features without requiring specific extensions to the underlying controller. Moreover, since all controllers use the same API, it is possible to switch from one solution to another <strong>without changing the configuration</strong> (The Kubenetes manifests which describe how the incoming traffic should be routed).</p><p>Among the concepts that we will explore, GAPI brings a granular authorization model which defines <strong>explicit roles with distinct permissions</strong>. (More information on the GAPI security model <a href=https://gateway-api.sigs.k8s.io/concepts/security-model/#roles-and-personas>here</a>).</p><p>This is worth noting that this project is driven by the <a href=https://github.com/kubernetes/community/tree/master/sig-network>sig-network-kubernetes</a> working group, and there's a slack <a href=https://kubernetes.slack.com/archives/CR0H13KGA>channel</a> where you can reach out to them if needed.</p><p>Let's see how GAPI is used in practice with Cilium 🚀!</p><h2 id=ballot_box_with_check-prerequisites>&#9745;&#xfe0f; Prerequisites</h2><p>For the remainder of this article, we assume an EKS cluster has been deployed. If you're not using the <a href=https://github.com/Smana/cilium-gateway-api/tree/main/terraform/eks>method suggested in the demo repo</a> as the basis for this article, there are a few <strong>points to check</strong> for GAPI to be usable.</p><p>ℹ️ The installation method described here is based on <code>Helm</code>, all the <code>values</code> can be viewed <a href=https://github.com/Smana/cilium-gateway-api/blob/main/terraform/eks/helm_values/cilium.yaml>here</a>.</p><ul><li><p><strong>Install</strong> the <code>CRDs</code> available in the <a href=https://github.com/kubernetes-sigs/gateway-api/tree/main/config/crd>Gateway API</a> repository.<div class="notices note"><div class=label>Note</div><p>If Cilium is set up with <code>GAPI</code> support (see below) and the CRDs are missing, it won't start.
 In the <a href=https://github.com/Smana/cilium-gateway-api>demo repo</a>, the <strong>GAPI CRDs</strong> are installed once during the cluster creation so that Cilium can start, and then they are managed by Flux.</p></div></p></li><li><p><strong>Replace</strong> <code>kube-proxy</code> with the network forwarding features provided by Cilium and <code>eBPF</code>.</p></li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=ln>1</span><span class=cl><span class=nt>kubeProxyReplacement</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
 </span></span></span></code></pre></div><ul><li><strong>Enable</strong> Gateway API support.</li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=ln>1</span><span class=cl><span class=nt>gatewayAPI</span><span class=p>:</span><span class=w>
 </span></span></span><span class=line><span class=ln>2</span><span class=cl><span class=w>  </span><span class=nt>enabled</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
diff --git a/post/cnpg/index.html b/post/cnpg/index.html
index 9c3d4d9..3dd626a 100644
--- a/post/cnpg/index.html
+++ b/post/cnpg/index.html
@@ -6,7 +6,7 @@
 <a href=https://blog.ogenki.io/fr/ class="nav_child nav_item">Français</a></div></div><div class=follow><a href=https://github.com/Smana><svg class="icon"><title>github</title><use xlink:href="#github"/></svg></a><a href=https://twitter.com/_smana_><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href=https://www.linkedin.com/in/sma%C3%AFne-kahlouch-44374110/><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><div class=color_mode><input type=checkbox class=color_choice id=mode></div></div></div></nav></header><main><div class="grid-inverse wrap content"><article class=post_content><h1 class=post_title><code>CloudNativePG</code>: An easy way to run PostgreSQL on Kubernetes</h1><div class=post_meta><span><svg class="icon"><title>calendar</title><use xlink:href="#calendar"/></svg></span><span class=post_date>23-10-2022</span>
 <span class=post_time>· 15 min read</span><span>&nbsp;· <a href=https://blog.ogenki.io/tags/data/ title=data class="post_tag button button_translucent">data
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Share on:
-<a href="https://twitter.com/intent/tweet?text=%60CloudNativePG%60%3a%20An%20easy%20way%20to%20run%20PostgreSQL%20on%20Kubernetes&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fcnpg%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fcnpg%2f&t=%60CloudNativePG%60%3a%20An%20easy%20way%20to%20run%20PostgreSQL%20on%20Kubernetes" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/cnpg/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/cnpg/cnpg.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-tooling>Tooling</a></li><li><a href=#-create-the-google-cloud-requirements>☁️ Create the Google cloud requirements</a></li><li><a href=#key-create-the-users-secrets>Create the users secrets</a></li></ul></li><li><a href=#hammer_and_wrench-deploy-the-cloudnativepg-operator-using-helm>Deploy the CloudNativePG operator using Helm</a></li><li><a href=#rocket-create-a-postgresql-server>Create a PostgreSQL server</a></li><li><a href=#-standby-instance-and-resiliency>🩹 Standby instance and resiliency</a></li><li><a href=#-monitoring>👁️ Monitoring</a></li><li><a href=#fire-performances-and-benchmark>Performances and benchmark</a></li><li><a href=#-backup-and-restore>💽 Backup and Restore</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>Kubernetes is now the de facto platform for orchestrating stateless applications. Containers that don't store data can be destroyed and easily recreated elsewhere. On the other hand, running persistent applications in an ephemeral environment can be <strong>quite challenging</strong>. There is an increasing number of mature cloud-native database solutions (like CockroachDB, TiDB, K8ssandra, Strimzi...) and there are a lot of <strong>things to consider</strong> when evaluating them:</p><ul><li>How mature is the operator?</li><li>What do the CRDs look like, which options, settings, and status do they expose?</li><li>Which Kubernetes storage APIs does it leverage? (PV/PVC, CSI, snapshots...)</li><li>Can it differentiate HDD and SSD, local/remote storage?</li><li>What happens when something goes wrong: how resilient is the system?</li><li>Backup and recovery: how easy is it to perform and schedule backups?</li><li>What replication and scaling options are available?</li><li>What about connection and concurrency limits, connection pooling, bouncers?</li><li>Observability: what metrics are exposed and how?</li></ul><p>I was looking for a solution to host a <strong>PostgreSQL database</strong>. This database is a requirement for a ticket reservation software named <a href=https://alf.io/>Alf.io</a> that's being used for an upcoming event: <a href=https://kcdfrance.fr>The Kubernetes Community Days France</a>. (By the way you're welcome to submit a talk 👐, the <a href=https://cfp.kcdfrance.fr>CFP</a> closes soon).</p><p>I was specifically looking for a cloud-agnostic solution, with emphasis on ease of use. I was already familiar with several Kubernetes operators, and I ended up evaluating a fairly new kid on the block: <a href=https://cloudnative-pg.io/><strong>CloudNativePG</strong></a>.</p><blockquote><p>CloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.</p></blockquote><p>It has been created by the company <a href=https://www.enterprisedb.com/>EnterpriseDB</a>, who submitted it to the <strong>CNCF</strong> in order to join the <em>Sandbox</em> projects.</p><h2 id=bullseye-our-target>:bullseye: Our target</h2><p>I'm going to give you an <strong>introduction</strong> to the main CloudNativePG features.
+<a href="https://twitter.com/intent/tweet?text=%60CloudNativePG%60%3a%20An%20easy%20way%20to%20run%20PostgreSQL%20on%20Kubernetes&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fcnpg%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fcnpg%2f&t=%60CloudNativePG%60%3a%20An%20easy%20way%20to%20run%20PostgreSQL%20on%20Kubernetes" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/cnpg/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/cnpg/cnpg.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-tooling>Tooling</a></li><li><a href=#-create-the-google-cloud-requirements>☁️ Create the Google cloud requirements</a></li><li><a href=#key-create-the-users-secrets>Create the users secrets</a></li></ul></li><li><a href=#hammer_and_wrench-deploy-the-cloudnativepg-operator-using-helm>Deploy the CloudNativePG operator using Helm</a></li><li><a href=#rocket-create-a-postgresql-server>Create a PostgreSQL server</a></li><li><a href=#-standby-instance-and-resiliency>🩹 Standby instance and resiliency</a></li><li><a href=#-monitoring>👁️ Monitoring</a></li><li><a href=#fire-performances-and-benchmark>Performances and benchmark</a></li><li><a href=#-backup-and-restore>💽 Backup and Restore</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>Kubernetes is now the de facto platform for orchestrating stateless applications. Containers that don't store data can be destroyed and easily recreated elsewhere. On the other hand, running persistent applications in an ephemeral environment can be <strong>quite challenging</strong>. There is an increasing number of mature cloud-native database solutions (like CockroachDB, TiDB, K8ssandra, Strimzi...) and there are a lot of <strong>things to consider</strong> when evaluating them:</p><ul><li>How mature is the operator?</li><li>What do the CRDs look like, which options, settings, and status do they expose?</li><li>Which Kubernetes storage APIs does it leverage? (PV/PVC, CSI, snapshots...)</li><li>Can it differentiate HDD and SSD, local/remote storage?</li><li>What happens when something goes wrong: how resilient is the system?</li><li>Backup and recovery: how easy is it to perform and schedule backups?</li><li>What replication and scaling options are available?</li><li>What about connection and concurrency limits, connection pooling, bouncers?</li><li>Observability: what metrics are exposed and how?</li></ul><p>I was looking for a solution to host a <strong>PostgreSQL database</strong>. This database is a requirement for a ticket reservation software named <a href=https://alf.io/>Alf.io</a> that's being used for an upcoming event: <a href=https://kcdfrance.fr>The Kubernetes Community Days France</a>. (By the way you're welcome to submit a talk 👐, the <a href=https://cfp.kcdfrance.fr>CFP</a> closes soon).</p><p>I was specifically looking for a cloud-agnostic solution, with emphasis on ease of use. I was already familiar with several Kubernetes operators, and I ended up evaluating a fairly new kid on the block: <a href=https://cloudnative-pg.io/><strong>CloudNativePG</strong></a>.</p><blockquote><p>CloudNativePG is the Kubernetes operator that covers the full lifecycle of a highly available PostgreSQL database cluster with a primary/standby architecture, using native streaming replication.</p></blockquote><p>It has been created by the company <a href=https://www.enterprisedb.com/>EnterpriseDB</a>, who submitted it to the <strong>CNCF</strong> in order to join the <em>Sandbox</em> projects.</p><h2 id=-our-target>🎯 Our target</h2><p>I'm going to give you an <strong>introduction</strong> to the main CloudNativePG features.
 The plan is to:</p><ul><li>create a PostgreSQL database on a GKE cluster,</li><li>add a standby instance,</li><li>run a few resiliency tests.</li></ul><p>We will also see how it behaves in terms of performances and what are the observability tools available.
 Finally we'll have a look to the backup/restore methods.</p><div class="notices info"><div class=label>Info</div><p>In this article, we will create and update everything manually; but in production, we probably should use a <strong>GitOps</strong> engine, for instance Flux (which has been covered in a <a href=/post/devflux/>previous article</a>).</p><p>If you want to see a complete end-to-end <strong>example</strong>, you can look at the <a href=https://github.com/cncfparis/kcdfrance-gitops>KCD France infrastructure repository</a>.</p><p>All the manifests shown in this article can be found in <a href=https://github.com/Smana/smana.github.io/tree/main/content/resources/cnpg>this repository</a>.</p></div><h2 id=ballot_box_with_check-requirements>&#9745;&#xfe0f; Requirements</h2><h3 id=inbox_tray-tooling>&#x1f4e5; Tooling</h3><ul><li><p><strong>gcloud SDK:</strong> we're going to deploy on Google Cloud (specifically, on GKE) and we will need to create a few resources in our GCP project; so we'll need the Google Cloud SDK and CLI. If needed, you can install and configure it using <a href=https://cloud.google.com/sdk/docs/install-sdk>this documentation</a>.</p></li><li><p><strong>kubectl plugin:</strong> to facilitate cluster management, CloudNativePG comes with a handy <code>kubectl</code> plugin that gives insights of your PostgreSQL instance and allows to perform some operations.
 It can be installed using <a href=https://krew.sigs.k8s.io/>krew</a> as follows:</p></li></ul><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>kubectl krew install cnpg
diff --git a/post/devflux/index.html b/post/devflux/index.html
index 8958fb3..b88e9d5 100644
--- a/post/devflux/index.html
+++ b/post/devflux/index.html
@@ -7,9 +7,9 @@
 <span class=post_time>· 14 min read</span><span>&nbsp;· <a href=https://blog.ogenki.io/tags/gitops/ title=gitops class="post_tag button button_translucent">gitops
 </a><a href=https://blog.ogenki.io/tags/devxp/ title=devxp class="post_tag button button_translucent">devxp
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Share on:
-<a href="https://twitter.com/intent/tweet?text=100%25%20%60GitOps%60%20using%20Flux&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fdevflux%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fdevflux%2f&t=100%25%20%60GitOps%60%20using%20Flux" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/devflux/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/devflux/devflux.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-install-required-tools>Install required tools</a></li><li><a href=#key-create-a-github-personal-access-token>Create a Github personal access token</a></li><li><a href=#technologist-clone-the-devflux-repository>Clone the devflux repository</a></li></ul></li><li><a href=#rocket-bootstrap-flux-in-the-crossplane-cluster>Bootstrap flux in the Crossplane cluster</a></li><li><a href=#open_file_folder-flux-repository-structure>Flux repository structure</a></li><li><a href=#closed_lock_with_key-sealedsecrets>SealedSecrets</a><ul><li><a href=#hammer_and_wrench-deploy-the-controller-using-helm>Deploy the controller using Helm</a></li><li><a href=#-a-first-test-sealedsecret>🧪 A first test SealedSecret</a></li></ul></li><li><a href=#cloud-deploy-and-configure-crossplane>Deploy and configure Crossplane</a><ul><li><a href=#key-create-the-google-service-account-secret>Create the Google service account secret</a></li><li><a href=#-crossplane-dependencies>🔄 Crossplane dependencies</a></li><li><a href=#rocket-bootstrap-flux-in-the-dev-cluster>Bootstrap flux in the dev cluster</a></li></ul></li><li><a href=#detective-troubleshooting>Troubleshooting</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#clap-achievements>Achievements</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>In a <a href=/post/crossplane_k3d/>previous article</a>, we've seen how to use <strong>Crossplane</strong> so that we can manage cloud resources the same way as our applications. &#x2764;&#xfe0f; Declarative approach!
+<a href="https://twitter.com/intent/tweet?text=100%25%20%60GitOps%60%20using%20Flux&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fdevflux%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fdevflux%2f&t=100%25%20%60GitOps%60%20using%20Flux" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/devflux/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/devflux/devflux.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#ballot_box_with_check-requirements>Requirements</a><ul><li><a href=#inbox_tray-install-required-tools>Install required tools</a></li><li><a href=#key-create-a-github-personal-access-token>Create a Github personal access token</a></li><li><a href=#technologist-clone-the-devflux-repository>Clone the devflux repository</a></li></ul></li><li><a href=#rocket-bootstrap-flux-in-the-crossplane-cluster>Bootstrap flux in the Crossplane cluster</a></li><li><a href=#open_file_folder-flux-repository-structure>Flux repository structure</a></li><li><a href=#closed_lock_with_key-sealedsecrets>SealedSecrets</a><ul><li><a href=#hammer_and_wrench-deploy-the-controller-using-helm>Deploy the controller using Helm</a></li><li><a href=#-a-first-test-sealedsecret>🧪 A first test SealedSecret</a></li></ul></li><li><a href=#cloud-deploy-and-configure-crossplane>Deploy and configure Crossplane</a><ul><li><a href=#key-create-the-google-service-account-secret>Create the Google service account secret</a></li><li><a href=#-crossplane-dependencies>🔄 Crossplane dependencies</a></li><li><a href=#rocket-bootstrap-flux-in-the-dev-cluster>Bootstrap flux in the dev cluster</a></li></ul></li><li><a href=#detective-troubleshooting>Troubleshooting</a></li><li><a href=#broom-cleanup>Cleanup</a></li><li><a href=#clap-achievements>Achievements</a></li><li><a href=#-final-thoughts>💭 final thoughts</a></li></ul></nav></div><div class=post_body><p>In a <a href=/post/crossplane_k3d/>previous article</a>, we've seen how to use <strong>Crossplane</strong> so that we can manage cloud resources the same way as our applications. &#x2764;&#xfe0f; Declarative approach!
 There were several steps and command lines in order to get everything working and reach our target to provision a dev Kubernetes cluster.</p><p>Here we'll achieve exactly the same thing but we'll do that in the <strong>GitOps</strong> way.
-According to the <a href=https://opengitops.dev/>OpenGitOps</a> working group there are 4 GitOps principles:</p><ul><li>The desired state of our system must be expressed <strong>declaratively</strong>.</li><li>This state must be stored in a <strong>versioning</strong> system.</li><li>Changes are pulled and <strong>applied automatically</strong> in the target platform whenever the desired state changes.</li><li>If, for any reason, the current state is modified, it will be <strong>automatically reconciled</strong> with the desired state.</li></ul><p>There are several GitOps engine options. The most famous ones are <a href=https://argo-cd.readthedocs.io/en/stable/>ArgoCD</a> and <a href=https://fluxcd.io/>Flux</a>. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).</p><p><figure><picture><img loading=lazy decoding=async alt=flux_components class="image_figure image_internal image_processed" width=1352 height=575 src=/post/devflux/flux_components.png></picture></figure></p><p>Learn more about <strong>GitOps toolkit</strong> components <a href=https://fluxcd.io/docs/components/>here</a>.</p><h2 id=bullseye-our-target>:bullseye: Our target</h2><p>Here we want to declare our desired infrastructure components only by adding <strong>git</strong> changes.
+According to the <a href=https://opengitops.dev/>OpenGitOps</a> working group there are 4 GitOps principles:</p><ul><li>The desired state of our system must be expressed <strong>declaratively</strong>.</li><li>This state must be stored in a <strong>versioning</strong> system.</li><li>Changes are pulled and <strong>applied automatically</strong> in the target platform whenever the desired state changes.</li><li>If, for any reason, the current state is modified, it will be <strong>automatically reconciled</strong> with the desired state.</li></ul><p>There are several GitOps engine options. The most famous ones are <a href=https://argo-cd.readthedocs.io/en/stable/>ArgoCD</a> and <a href=https://fluxcd.io/>Flux</a>. We won't compare them here. I chose Flux because I like its composable architecture with different controllers, each one handling a core Flux feature (GitOps toolkit).</p><p><figure><picture><img loading=lazy decoding=async alt=flux_components class="image_figure image_internal image_processed" width=1352 height=575 src=/post/devflux/flux_components.png></picture></figure></p><p>Learn more about <strong>GitOps toolkit</strong> components <a href=https://fluxcd.io/docs/components/>here</a>.</p><h2 id=-our-target>🎯 Our target</h2><p>Here we want to declare our desired infrastructure components only by adding <strong>git</strong> changes.
 By the end of this article you'll get a GKE cluster provisioned using a local Crossplane instance.
 We'll discover <a href=https://fluxcd.io/><strong>Flux</strong></a> basics and how to use it in order to build a complete GitOps CD workflow.</p><br><h2 id=ballot_box_with_check-requirements>&#9745;&#xfe0f; Requirements</h2><h3 id=inbox_tray-install-required-tools>&#x1f4e5; Install required tools</h3><p>First of all we need to install a few tools using <a href=/post/asdf/asdf/>asdf</a></p><p>Create a local file
 .tool-versions</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>cd ~/sources/devflux/
diff --git a/post/tailscale/index.html b/post/tailscale/index.html
index 3c27836..7ebc76d 100644
--- a/post/tailscale/index.html
+++ b/post/tailscale/index.html
@@ -7,7 +7,7 @@
 <span class=post_time>· 13 min read</span><span>&nbsp;· <a href=https://blog.ogenki.io/tags/security/ title=security class="post_tag button button_translucent">security
 </a><a href=https://blog.ogenki.io/tags/network/ title=network class="post_tag button button_translucent">network
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Share on:
-<a href="https://twitter.com/intent/tweet?text=Beyond%20Traditional%20VPNs%3a%20Simplifying%20Cloud%20Access%20with%20%60Tailscale%60&url=https%3a%2f%2fblog.ogenki.io%2fpost%2ftailscale%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2ftailscale%2f&t=Beyond%20Traditional%20VPNs%3a%20Simplifying%20Cloud%20Access%20with%20%60Tailscale%60" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/tailscale/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#-under-the-hood>💡 Under the Hood</a></li><li><a href=#inbox_tray-client-installation>Client Installation</a></li><li><a href=#-accessing-aws-privately>☁️ Accessing AWS Privately</a><ul><li><a href=#-deploying-a-subnet-router>🚀 Deploying a Subnet Router</a></li><li><a href=#-a-different-way-to-ssh>💻 A Different Way to SSH</a></li><li><a href=#-what-about-kubernetes>☸ What About Kubernetes?</a></li></ul></li><li><a href=#-final-thoughts>💭 Final Thoughts</a></li></ul></nav></div><div class=post_body><p>When we talk about securing access to cloud resources, one of the golden rules is to avoid direct exposure to the Internet. This raises a question for Dev/Ops: How can we , for instance, access a database, a Kubernetes cluster, or a server via SSH without compromising security?</br></p><p><strong>Virtual Private Networks (VPN)</strong> offer an answer by establishing a secure link between different elements of a network, regardless of their geographical location. Solutions range from SaaS models to self-hosted platforms, using various protocols, either open-source or proprietary.</p><p>Among these options, I wanted to introduce you to <a href=https://tailscale.com/><strong>Tailscale</strong></a>. This solution uses <code>WireGuard</code> under the hood, known for its simplicity and performance. With Tailscale, you can securely connect devices or servers as if they were on the same local network, even though they are spread across the globe.</p><h2 id=bullseye-our-target>:bullseye: Our target</h2><ul><li>Understanding how Tailscale works</li><li>Implementing a secure connection with AWS in a few minutes</li><li>Interacting with an EKS cluster API via a private network</li><li>Accessing services hosted on Kubernetes using the private network.</li></ul><center><img src=admin_console.png width=400></center><p>Throughout this article, you will obviously need to <strong>create a Tailscale account</strong>. It's worth noting that authentication is delegated to third-party identity providers (e.g., Okta, OneLogin, Google).</p><p>Once the account is created, you'll have instant access to the management console. It allows you to list connected devices, visualize logs, and change most of the settings.</p><h2 id=-under-the-hood>💡 Under the Hood</h2><div class="notices info"><div class=label>Terminology</div><p><strong>Mesh VPN</strong>: A <em>mesh VPN</em> refers to a VPN network configuration in which every node (i.e., each device or machine) connects directly to every other node, creating a mesh-like structure. This stands in contrast to traditional VPN configurations that usually adopt a "hub and spoke" topology, where multiple clients connect to one central server.</p><p><strong>Zero Trust</strong>: This approach operates on the principle that every access request to a network is inherently untrusted. Whether an application or user, identity must be proven and authorization given before accessing a resource. Trust isn't assumed based solely on originating from an internal network or a specific geographic location.</p><p><strong>Tailnet</strong>: When you first use Tailscale, a <em>Tailnet</em> is automatically generated for you, representing your personal private network. Every device within this tailnet is assigned a unique Tailscale IP.</p></div><center><img src=mesh.png width=500></center><p>Tailscale's architecture clearly distinguishes between the Control plane and the Data plane:</p><ul><li><p>The coordination server primarily manages the exchange of metadata and public keys among all Tailnet members. Importantly, by Tailscale's design, only the public key is shared, with the private key securely retained on its originating node.</p></li><li><p>The nodes in the Tailnet establish a mesh network: Instead of passing data through the coordination server, these nodes communicate directly in a peer-to-peer manner. Each node has a unique identity, allowing it to authenticate and become part of the Tailnet.</p></li></ul><h2 id=inbox_tray-client-installation>&#x1f4e5; Client Installation</h2><p>Most platforms are supported, and the installation options are listed <a href=https://tailscale.com/kb/installation/>here</a>.
+<a href="https://twitter.com/intent/tweet?text=Beyond%20Traditional%20VPNs%3a%20Simplifying%20Cloud%20Access%20with%20%60Tailscale%60&url=https%3a%2f%2fblog.ogenki.io%2fpost%2ftailscale%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2ftailscale%2f&t=Beyond%20Traditional%20VPNs%3a%20Simplifying%20Cloud%20Access%20with%20%60Tailscale%60" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/tailscale/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#-under-the-hood>💡 Under the Hood</a></li><li><a href=#inbox_tray-client-installation>Client Installation</a></li><li><a href=#-accessing-aws-privately>☁️ Accessing AWS Privately</a><ul><li><a href=#-deploying-a-subnet-router>🚀 Deploying a Subnet Router</a></li><li><a href=#-a-different-way-to-ssh>💻 A Different Way to SSH</a></li><li><a href=#-what-about-kubernetes>☸ What About Kubernetes?</a></li></ul></li><li><a href=#-final-thoughts>💭 Final Thoughts</a></li></ul></nav></div><div class=post_body><p>When we talk about securing access to cloud resources, one of the golden rules is to avoid direct exposure to the Internet. This raises a question for Dev/Ops: How can we , for instance, access a database, a Kubernetes cluster, or a server via SSH without compromising security?</br></p><p><strong>Virtual Private Networks (VPN)</strong> offer an answer by establishing a secure link between different elements of a network, regardless of their geographical location. Solutions range from SaaS models to self-hosted platforms, using various protocols, either open-source or proprietary.</p><p>Among these options, I wanted to introduce you to <a href=https://tailscale.com/><strong>Tailscale</strong></a>. This solution uses <code>WireGuard</code> under the hood, known for its simplicity and performance. With Tailscale, you can securely connect devices or servers as if they were on the same local network, even though they are spread across the globe.</p><h2 id=-our-target>🎯 Our target</h2><ul><li>Understanding how Tailscale works</li><li>Implementing a secure connection with AWS in a few minutes</li><li>Interacting with an EKS cluster API via a private network</li><li>Accessing services hosted on Kubernetes using the private network.</li></ul><center><img src=admin_console.png width=400></center><p>Throughout this article, you will obviously need to <strong>create a Tailscale account</strong>. It's worth noting that authentication is delegated to third-party identity providers (e.g., Okta, OneLogin, Google).</p><p>Once the account is created, you'll have instant access to the management console. It allows you to list connected devices, visualize logs, and change most of the settings.</p><h2 id=-under-the-hood>💡 Under the Hood</h2><div class="notices info"><div class=label>Terminology</div><p><strong>Mesh VPN</strong>: A <em>mesh VPN</em> refers to a VPN network configuration in which every node (i.e., each device or machine) connects directly to every other node, creating a mesh-like structure. This stands in contrast to traditional VPN configurations that usually adopt a "hub and spoke" topology, where multiple clients connect to one central server.</p><p><strong>Zero Trust</strong>: This approach operates on the principle that every access request to a network is inherently untrusted. Whether an application or user, identity must be proven and authorization given before accessing a resource. Trust isn't assumed based solely on originating from an internal network or a specific geographic location.</p><p><strong>Tailnet</strong>: When you first use Tailscale, a <em>Tailnet</em> is automatically generated for you, representing your personal private network. Every device within this tailnet is assigned a unique Tailscale IP.</p></div><center><img src=mesh.png width=500></center><p>Tailscale's architecture clearly distinguishes between the Control plane and the Data plane:</p><ul><li><p>The coordination server primarily manages the exchange of metadata and public keys among all Tailnet members. Importantly, by Tailscale's design, only the public key is shared, with the private key securely retained on its originating node.</p></li><li><p>The nodes in the Tailnet establish a mesh network: Instead of passing data through the coordination server, these nodes communicate directly in a peer-to-peer manner. Each node has a unique identity, allowing it to authenticate and become part of the Tailnet.</p></li></ul><h2 id=inbox_tray-client-installation>&#x1f4e5; Client Installation</h2><p>Most platforms are supported, and the installation options are listed <a href=https://tailscale.com/kb/installation/>here</a>.
 As for me, I am on Archlinux:</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo pacman -S tailscale
 </span></span></span></code></pre></div><p>It's possible to start the service automatically upon machine startup.</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo systemctl enable --now tailscaled
 </span></span></span></code></pre></div><p>To register your personal computer, run the following command:</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>sudo tailscale up --accept-routes
diff --git a/post/terraform-controller/index.html b/post/terraform-controller/index.html
index 98a5fdd..8cde257 100644
--- a/post/terraform-controller/index.html
+++ b/post/terraform-controller/index.html
@@ -6,10 +6,10 @@
 <a href=https://blog.ogenki.io/fr/ class="nav_child nav_item">Français</a></div></div><div class=follow><a href=https://github.com/Smana><svg class="icon"><title>github</title><use xlink:href="#github"/></svg></a><a href=https://twitter.com/_smana_><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href=https://www.linkedin.com/in/sma%C3%AFne-kahlouch-44374110/><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><div class=color_mode><input type=checkbox class=color_choice id=mode></div></div></div></nav></header><main><div class="grid-inverse wrap content"><article class=post_content><h1 class=post_title>Applying GitOps Principles to Infrastructure: An overview of <code>tf-controller</code></h1><div class=post_meta><span><svg class="icon"><title>calendar</title><use xlink:href="#calendar"/></svg></span><span class=post_date>01-06-2023</span>
 <span class=post_time>· 16 min read</span><span>&nbsp;· <a href=https://blog.ogenki.io/tags/infrastructure/ title=infrastructure class="post_tag button button_translucent">infrastructure
 </a></span><span class=page_only>&nbsp;·<div class=post_share>Share on:
-<a href="https://twitter.com/intent/tweet?text=Applying%20GitOps%20Principles%20to%20Infrastructure%3a%20An%20overview%20of%20%60tf-controller%60&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fterraform-controller%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fterraform-controller%2f&t=Applying%20GitOps%20Principles%20to%20Infrastructure%3a%20An%20overview%20of%20%60tf-controller%60" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/terraform-controller/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/terraform-controller/tf-controller.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#bullseye-our-target>:bullseye: Our target</a></li><li><a href=#-install-the-tf-controller>🛠️ Install the tf-controller</a><ul><li><a href=#-the-control-plane>☸ The control plane</a></li><li><a href=#-the-helm-chart-and-flux>📦 The Helm chart and Flux</a></li></ul></li><li><a href=#-apply-a-change>🚀 Apply a change</a><ul><li><a href=#-drift-detection>🚨 Drift detection</a></li><li><a href=#-manual-execution>🔧 Manual execution</a></li><li><a href=#-automatic-reconciliation>🤖 Automatic reconciliation</a></li><li><a href=#-inputsoutputs-and-modules-dependencies>🔄 Inputs/Outputs and modules dependencies</a></li></ul></li><li><a href=#-backup-and-restore-a-tfstate>💾 Backup and restore a tfstate</a></li><li><a href=#-focus-on-key-features-of-flux>🔍 Focus on Key Features of Flux</a><ul><li><a href=#variable-substitution>Variable Substitution</a></li><li><a href=#web-ui-weave-gitops>Web UI (Weave GitOps)</a></li></ul></li><li><a href=#-final-thoughts>💭 Final thoughts</a></li></ul></nav></div><div class=post_body><p><strong>Terraform</strong> is probably the most used "Infrastructure As Code" tool for building, modifying, and versioning Cloud infrastructure changes.
+<a href="https://twitter.com/intent/tweet?text=Applying%20GitOps%20Principles%20to%20Infrastructure%3a%20An%20overview%20of%20%60tf-controller%60&url=https%3a%2f%2fblog.ogenki.io%2fpost%2fterraform-controller%2f&tw_p=tweetbutton" class=twitter title="Share on Twitter" target=_blank rel=nofollow><svg class="icon"><title>twitter</title><use xlink:href="#twitter"/></svg></a><a href="https://www.facebook.com/sharer.php?u=https%3a%2f%2fblog.ogenki.io%2fpost%2fterraform-controller%2f&t=Applying%20GitOps%20Principles%20to%20Infrastructure%3a%20An%20overview%20of%20%60tf-controller%60" class=facebook title="Share on Facebook" target=_blank rel=nofollow><svg class="icon"><title>facebook</title><use xlink:href="#facebook"/></svg></a><a href=#linkedinshare id=linkedinshare class=linkedin title="Share on LinkedIn" rel=nofollow><svg class="icon"><title>linkedin</title><use xlink:href="#linkedin"/></svg></a><a href=https://blog.ogenki.io/post/terraform-controller/ title="Copy Link" class="link link_yank"><svg class="icon"><title>copy</title><use xlink:href="#copy"/></svg></a></div></span></div><div class=post_featured><figure><picture><img loading=lazy decoding=async alt class="image_figure image_featured image_internal image_unprocessed" src=/post/terraform-controller/tf-controller.png></picture></figure></div><div class=post_toc><h2>Overview</h2><nav id=TableOfContents><ul><li><a href=#-our-target>🎯 Our target</a></li><li><a href=#-install-the-tf-controller>🛠️ Install the tf-controller</a><ul><li><a href=#-the-control-plane>☸ The control plane</a></li><li><a href=#-the-helm-chart-and-flux>📦 The Helm chart and Flux</a></li></ul></li><li><a href=#-apply-a-change>🚀 Apply a change</a><ul><li><a href=#-drift-detection>🚨 Drift detection</a></li><li><a href=#-manual-execution>🔧 Manual execution</a></li><li><a href=#-automatic-reconciliation>🤖 Automatic reconciliation</a></li><li><a href=#-inputsoutputs-and-modules-dependencies>🔄 Inputs/Outputs and modules dependencies</a></li></ul></li><li><a href=#-backup-and-restore-a-tfstate>💾 Backup and restore a tfstate</a></li><li><a href=#-focus-on-key-features-of-flux>🔍 Focus on Key Features of Flux</a><ul><li><a href=#variable-substitution>Variable Substitution</a></li><li><a href=#web-ui-weave-gitops>Web UI (Weave GitOps)</a></li></ul></li><li><a href=#-final-thoughts>💭 Final thoughts</a></li></ul></nav></div><div class=post_body><p><strong>Terraform</strong> is probably the most used "Infrastructure As Code" tool for building, modifying, and versioning Cloud infrastructure changes.
 It is an Open Source project developed by Hashicorp that uses the <a href=https://github.com/hashicorp/hcl>HCL</a> language to declare the desired state of Cloud resources.
 The state of the created resources is stored in a file called terraform state.</p><p>Terraform can be considered a "semi-declarative" tool as there is no built-in <strong>automatic reconciliation</strong> feature. There are several solutions to address this issue, but generally speaking, a modification will be applied using <code>terraform apply</code>. The code is actually written using the HCL configuration files (declarative), but the execution is done imperatively.
-As a result, there can be a drift between the declared and actual state (for example, a colleague who would have changed something directly into the console 😉).</p><p>❓❓ So, how can I ensure that what is committed using Git is really applied. How to be notified if there is a change compared to the desired state and how to automatically apply what is in my code (GitOps)?</p><p>This is the promise of <a href=https://github.com/weaveworks/tf-controller><strong>tf-controller</strong></a>, an Open Source Kubernetes operator from Weaveworks, tightly related to Flux (a GitOps engine from the same company). <a href=https://fluxcd.io/><strong>Flux</strong></a> is one of the solutions I really appreciate, that's why I invite you to have a look on my <a href=https://blog.ogenki.io/post/devflux/>previous article</a></p><div class="notices info"><div class=label>Info</div><p>All the steps described in this article come from this <a href=https://github.com/Smana/demo-tf-controller><strong>Git repo</strong></a></p></div><h2 id=bullseye-our-target>:bullseye: Our target</h2><p>By following the steps in this article, we aim to achieve the following things:</p><ul><li>Deploy a <strong>Control plane</strong> EKS cluster. Long story short, it will host the Terraform controller that will be in charge of managing all the desired infrastructure components.</li><li>Use <strong>Flux</strong> as the GitOps engine for all Kubernetes resources.</li></ul><p>Regarding the Terraform controller, we will see:</p><ul><li>How to define <strong>dependencies</strong> between modules</li><li>Creation of <strong>several AWS resources</strong>: Route53 Zone, ACM Certificate, network, EKS cluster.</li><li>The different <strong>reconciliation</strong> options (automatic, requiring confirmation)</li><li>How to backup and <strong>restore</strong> a Terraform state.</li></ul><h2 id=-install-the-tf-controller>🛠️ Install the tf-controller</h2><h3 id=-the-control-plane>☸ The control plane</h3><p>In order to be able to use the Kubernetes controller <code>tf-controller</code>, we first need a Kubernetes cluster 😆.
+As a result, there can be a drift between the declared and actual state (for example, a colleague who would have changed something directly into the console 😉).</p><p>❓❓ So, how can I ensure that what is committed using Git is really applied. How to be notified if there is a change compared to the desired state and how to automatically apply what is in my code (GitOps)?</p><p>This is the promise of <a href=https://github.com/weaveworks/tf-controller><strong>tf-controller</strong></a>, an Open Source Kubernetes operator from Weaveworks, tightly related to Flux (a GitOps engine from the same company). <a href=https://fluxcd.io/><strong>Flux</strong></a> is one of the solutions I really appreciate, that's why I invite you to have a look on my <a href=https://blog.ogenki.io/post/devflux/>previous article</a></p><div class="notices info"><div class=label>Info</div><p>All the steps described in this article come from this <a href=https://github.com/Smana/demo-tf-controller><strong>Git repo</strong></a></p></div><h2 id=-our-target>🎯 Our target</h2><p>By following the steps in this article, we aim to achieve the following things:</p><ul><li>Deploy a <strong>Control plane</strong> EKS cluster. Long story short, it will host the Terraform controller that will be in charge of managing all the desired infrastructure components.</li><li>Use <strong>Flux</strong> as the GitOps engine for all Kubernetes resources.</li></ul><p>Regarding the Terraform controller, we will see:</p><ul><li>How to define <strong>dependencies</strong> between modules</li><li>Creation of <strong>several AWS resources</strong>: Route53 Zone, ACM Certificate, network, EKS cluster.</li><li>The different <strong>reconciliation</strong> options (automatic, requiring confirmation)</li><li>How to backup and <strong>restore</strong> a Terraform state.</li></ul><h2 id=-install-the-tf-controller>🛠️ Install the tf-controller</h2><h3 id=-the-control-plane>☸ The control plane</h3><p>In order to be able to use the Kubernetes controller <code>tf-controller</code>, we first need a Kubernetes cluster 😆.
 So we are going to create a <strong>control plane</strong> cluster using the <code>terraform</code> command line and EKS best practices.</p><div class="notices warning"><div class=label>Warning</div><p>It is crucial that this cluster is resilient, secure, and supervised as it will be responsible for managing all the AWS resources created subsequently.</p></div><p>Without going into detail, the control plane cluster was created using <a href=https://github.com/Smana/demo-tf-controller/tree/main/terraform/controlplane>this code</a>. That said, it is important to note that all application deployment operations are done using Flux.</p><div class="notices info"><div class=label>Info</div><p>By following the instructions in the <a href=https://github.com/Smana/demo-tf-controller/blob/main/terraform/controlplane/README.md>README</a>, an EKS cluster will be created but not only!</br>Indeed, it is required to give permissions to the Terraform controller so it will able to apply infrastructure changes.
 Furthermore, Flux must be installed and configured to apply the configuration defined <a href=https://github.com/Smana/demo-tf-controller/tree/main/clusters/controlplane-0>here</a>.</p><p>We end up with several components installed and configured:</p><ul><li>The almost unavoidable addons: <code>aws-loadbalancer-controller</code> and <code>external-dns</code></li><li><a href=https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html>IRSA</a> roles for these same components are installed using <code>tf-controller</code></li><li>Prometheus / Grafana monitoring stack.</li><li><code>external-secrets</code> to be able to retrieve sensitive data from AWS secretsmanager.</li></ul><p>To demonstrate all this after a few minutes the web interface for Flux is accessible via the URL <code>gitops-&lt;cluster_name>.&lt;domain_name</code>></p><p>Still you should check that Flux is working properly</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-console data-lang=console><span class=line><span class=ln>1</span><span class=cl><span class=go>aws eks update-kubeconfig --name controlplane-0 --alias controlplane-0
 </span></span></span><span class=line><span class=ln>2</span><span class=cl><span class=go>Updated context controlplane-0 in /home/smana/.kube/config