Support other CPU architectures such as arm64

[AndiDog]

ARM CPUs seem to deliver on their promise to be more power efficient (Ampere Altra comparative benchmark) and yet competitive regarding performance. Prices are also advantageous. Hetzner Cloud, for example, currently offers the CAX11 ARM cloud server for the same price as the CX11 x64 equivalent, but with double vCPU (1 vs. 2), RAM (2 GB vs. 4 GB) and disk (20 GB vs. 40 GB).

I think the hobby-kube project should support this architecture, and in general more upcoming CPU types. For instance, the open architecture RISC-V may also result in great processors over the next years, and is already very usable.

When trying to add arm64 support with Hetzner Cloud instances, I found two problems:

1. etcd installation hardcoded to amd64. Easy to fix, and I'll post the PR soon.
2. Weave Net does not seem to support other architectures. And given the project had no releases or relevant commits in years, we should think about replacing it. The project is not a CNCF project, and apparently the company behind the open source project has changed priorities towards enterprise products. Compared to Flux CD, which became a very active project outside company control, Weave Net didn't. Even if there are some replies to PRs at the moment, I don't think there will be the same traction behind the project as we see for other projects. There are several alternative CNIs ranging from simple to complex. In my opinion, Cilium is the most promising one in many regards: active development, features, low power consumption (thanks to in-kernel eBPF), performance, extended features such as network policy and service mesh support, CNCF Incubating project, great documentation, observability (e.g. Hubble UI), etc. Let's discuss if it makes sense to switch to this CNI, or if you want to consider some others.

[pstadler]

@mweibel any input on cluster networking from your side? And while we‘re at it, how feasible would it be to introduce support for Windows nodes? 😏

[mweibel]

Cilium is probably the best option right now, yes. Calico is another option which I‘d consider if you‘d really want windows support (which you don’t tbh, it’s too much work to make it work and doesn’t fit into hobbyists projects imo).

[AndiDog]

Calico is quite a beast: complex to understand and install (nowadays usually with an operator), taking up more resources during runtime (if this is still correct as of 2023), incomplete BPF support. It offers Windows support, though.

Maybe we can treat Windows support as separate feature. If someone is really interested in it, a pull request or draft design issue is very welcome. That may include running another CNI software for those nodes specifically, removing our hardcoded choice (currently Weave Net) and making it configurable (such as installing via Helm). I personally don't have a need for running Windows on Kubernetes.

[pstadler]

Thanks for your input! Let's forget about Windows support in this case. If you guys agree that Cilium would be a good replacement for Weave, let's go for it.

[pstadler]

Replaced Weave with Cilium in #92.
It took me a while to figure out how to pass config variables to cilium install, but it seems to work now.

[AndiDog]

I have a fully working arm64 cluster, using Cilium (from #92) 🎉. It was easy to adapt my apps (mostly just skaffold build [...] --platform=linux/amd64,linux/arm64). Once the Cilium PR is done, I think we can close this issue.

[pstadler]

Glad that this is working!

[pstadler]

Before closing this we should briefly mention arm64 support in hobby-kube/guide (probably in the section mentioning pricing) and make sure the architecture can be controlled using variables during provisioning.

[pstadler]

I'd say we're done here. Thanks for your efforts! Please re-open if we're missing something.