fix: Allow EKS addons version config

[flamarion]

Given the possibility of running different Kubernetes versions, the add-ons config must also be possible to configure to be compatible.

https://docs.aws.amazon.com/eks/latest/userguide/addon-compat.html

[danielpanzella]

Should we add some documentation, or pointers to relevant EKS docs for determining the correct addon versions

[zacharyblasczyk]

@danielpanzella, if we are going to expose variables for this, what do you think about (psudocode)

variable "eks_addon_versions" {
  description = "Map of EKS add-ons and their versions."
  type = map(string)
  default = {
    "aws-efs-csi-driver" = "v2.0.4-eksbuild.1",
    "aws-ebs-csi-driver" = "v1.31.0-eksbuild.1",
    "coredns"            = "v1.10.1-eksbuild.11",
    "kube-proxy"         = "v1.28.8-eksbuild.5",
    "vpc-cni"            = "v1.18.2-eksbuild.1"
  }
}

resource "aws_eks_addon" "default" {
  for_each               = var.eks_addon_versions
  cluster_name           = var.namespace
  addon_name             = each.key
  addon_version          = each.value
  resolve_conflicts      = "OVERWRITE"

  depends_on = [
    aws_eks_addon.vpc_cni
  ]
}

We could also make a local that has the map for each eks version say 1.26-1.29, and references the local default for the eks version unless you pass in your own map.

Not sure if the

 depends_on = [
    aws_eks_addon.vpc_cni
  ]  

Will be graceful, but I am sure we could just separate that one out if it is an issue.

Then we are only adding one variable that is extensible, and not adding n variables that we have to maintain and might create confusion.

[danielpanzella]

@zacharyblasczyk since the user would or could be overriding the map as a whole, that allows them to accidentally leave out a required addon, no?

[zacharyblasczyk]

We would have a merge like this:

locals {
  eks_addon_versions = merge(local.required_addons, var.eks_addon_versions_override)
}

[zacharyblasczyk]

So nothing should be left out.

[flamarion]

@zacharyblasczyk and @danielpanzella, is it ok to make it simple for now and allow the override?
I fixed what was suggested about the default duplication and docs.

[jsbroks]

This PR is included in version 5.0.1 🎉