ublue-image template repository

[marcoceppi]

This is a conversation that started in our discord that I want to pursue more here for discussion.

Github has the ability to define template repositories. Today the ublue-os/main repo, from https://github.com/orgs/ublue-os/discussions/101 is a template repo. This works well if you're making another ublue-os org - but there is no easy way to make a downstream ublue image. An example of what a downstream image template repo was demonstrated here https://github.com/ublue-os/startingpoint. I think this is a great start and I'd like to discuss the following enhancements.

First, the goal of this repo is to solve the following user stories:

I would like to further customize and publish a ublue-os derived distro image

For this, I think the following skeleton repo structure would best serve that need:

ublue-image
├── .github
│   └── workflows
│       ├── build.yml
│       └── release.yml
├── Containerfile
├── README.md
├── etc
│   └── README.md
└── packages.json

(this structure may change as the main image iterates)

Containerfile will be parameterized with something like:

ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG IMAGE_FLAVOR="${IMAGE_FLAVOR:-main}"
ARG SOURCE_IMAGE="${SOURCE_IMAGE:-$IMAGE_NAME-$IMAGE_FLAVOR}"
ARG BASE_IMAGE="ghcr.io/ublue-os/${SOURCE_IMAGE}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-37}"

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION} AS builder

ARG IMAGE_NAME="${IMAGE_NAME}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION}"

ADD packages.json /tmp/packages.json

RUN /tmp/build.sh \
 && /tmp/post-install.sh \
 && rm -rf /tmp/* /var/* \
 && ostree container commit \
 && mkdir -p /var/tmp && chmod -R 1777 /var/tmp

This sets the image base as the flavor / image from ublue-os (so with not arguments it'd be silverblue-main but that could be changed to silverblue-nvidia or a matrix of silverblue-main and silverblue-nvidia could be built or, like main, a flavor/variant of silverblue, kinoite, vauxite, etc with -nvidia and -main flavors.

This skips quite a few lines from ublue-os/main Containerfile as those scripts will already be present.

From an action point of view, build and release will be similar to the actions in main. Something to allow for PR builds / main builds and a release action which builds and uploads an iso whenever the repo is tagged.

README will be geared towards how to customize the newly created ublue-image.

There will need to be a few tweaks and I'd like to propose we finish 38 builds and install plans before completing this and keep 38 forward in mind for this template repo. I think the ubuntu repo being moved to bluefin is a great opportunity to test this new template when that time comes.

I'm curious thoughts or other considerations while moving to consensus - thanks!

[xynydev]

Good to see the discussion finally being moved here!

First: I'd like to acknowledge some miscommunications that have happened.
In the [graph I made](graph I made), the box "Starting point for custom images" was apparently understood by some to not be a separate repository. I also wasn't aware of the plans to have the main repository serve as a starting point, so I started work on a startingpoint repository.

Here's what I view as the benefits of using a dedicated startingpoint repository instead of just telling people to template off of main:

• A simple codebase designed to be modified to suit many people's purposes instead of being constantly added to
• Overdocumenting all aspects of the codebase to make it even easier to get started
• README tailored to educate the "tinkerer" about the creation of their own image
• A more straightforward path for more people to create more custom images

I also feel that having a separate repository for a starting point makes intuitively more sense (obviously this is subjective, but this serves to further illustrate my viewpoint). When creating your own image templating from main, the first thing you'd probably want to do is to change the base image to be main, so that you'd always get the latest changes from the uBlue project and not have to sync the delightfully frequent changes from upstream. Then, you'd be duplicating the work already done on uBlue.

For someone like me, to just create their own image with a couple packages and configuration files shipped with it, it could be confusing when upon templating from main you would have the following situation; A repository that builds all possible images for all possible desktop environments.
Now, I want only Kinoite builds, so I remove everything else from build.yml. I also want Nvidia drivers, so I change the ARG BASE_IMAGE to create the correct string for the image URL using the ARG SOURCE_IMAGE.

On a truly simplified dedicated startingpoint image, that same process above would be just changing the image URL after the FROM string to the same URL I would use to directly rebase to that image. In the amount of steps, not that much of a difference, but in the clarity, much better.

And not to mention, if you were to just override the FROM line in the main image, you would still get builds named after all supported variants, but all of them would be based on the same image that you specified, with some containing DE-specific packages for a DE that is not installed.

That being said, there are still valid use cases for templating from main instead. One that comes to mind is explicitly removing features or packages. If you templated from main, you can just remove the lines where they get added and do no further changes. If you want to also provide Nvidia images with your slim-ublue-main, however, you also need to fork the Nvidia repo to build them based on your own main.
Having to create and maintain multiple separate repositories, effectively cloning the whole of uBlue, is definitely not something anyone should be pushed towards.

[atimeofday]

Good points, and I think the concrete examples here address a lot of the concerns I had in the prior discussion. I think a lot of the info I was working with was from planning and discussions which didn't translate directly into the actual implementation.

Having to create and maintain multiple separate repositories, effectively cloning the whole of uBlue, is definitely not something anyone should be pushed towards.

I fully agree with this, however that is to be implemented.

That being said, there are still valid use cases for templating from main instead. One that comes to mind is explicitly removing features or packages.

I don't know if the following is a bad idea but right now it seems appealing: "Both. Both is good." If there are genuinely different use cases and advantages to using the main repo and a 'starting point' repo which can't easily be consolidated into one of them, I feel it could be good to explicitly provide both as templates, with descriptions and links between them for those specific use cases.

I hadn't realized or it wasn't well formalized at the time that the main repo would handle building all possible (downstream) images, and that that would be replicated when using it as a template. I thought it was, well, further upstream, and meant to cascade down to what would build those images. I'm sure that would introduce other architectural hurdles and complications, though. With main doing everything all in one, a separate repo as a custom image template for just one selected image makes sense.

I don't really know enough about the internals of the repo structure and image config files to give a solid vote in any direction, but I hope that the extra perspective and note that my concerns were addressed are of some help.

[xynydev]

I think we should also discuss the structure of the starting point repository.
Should we move more configuration into build.yml, making it easier to create multi-building repos, or should we keep as much of it as possible in the Containerfile to make it possible easier to understand and harder to break?
Shouldn't we use a different package configuration than main, both to separate them from each other and to use yaml, so that people wouldn't have to write json by hand? (yq is easily installed and used with the same syntax as jq, and we've already used it in base)

[maisatanel]

Having only a single repo to build both -main and -nvidia images would be excellent.

[xynydev]

Having only a single repo to build both -main and -nvidia images would be excellent.

As in a single repository not based on either nvidia or main which creates builds based on both -main and -nvidia images?

[ribbanya]

One thing I find concerning about the template workflow is that is squashes the git history. So if you create a template off main, modify it, and then ublue updates with things you want, it's harder (but still possible) to pull the changes downstream.

Templating is normally used for like, providing the user a Hello World starting point in a programming language. But a Containerfile is more of a living document.

[xynydev]

If the user needs just one repo and wants the template automatically set as a remote, they could make a fork. We should probably also provide a guide for merging the latest changes from the template with your own repository by setting it as a remote and pulling from it.

[ribbanya]

Oh, can you still fork a template repository? I know you can manually clone and set the remote but that's not ideal.

[xynydev]

Oh, can you still fork a template repository? I know you can manually clone and set the remote but that's not ideal.

Yes, you can fork any repository on Github, and adding the remote manually is trivially easy too.

[ribbanya]

It might be useful to have, in the starting point repo, a shell script that walks you through basic customization (like do you want nvidia, prompts you to create a signing key, etc.) Similar to vue create.

[xynydev]

Yeah, pretty interesting, I agree. I figure this could be a part of a repo-specific justfile, or maybe a curl ... | bash script that clones the repo and asks some basic questions.

[xynydev]

We could look at https://github.com/charmbracelet/gum for a library for the script, it's pretty cool. IIRC in other repo's too we're using custom-built containers for doing actions that require packages, so I figure this could also be a simple alpine container base.

[xynydev]

If it's a small container it could also include github cli and cosign to set up everything automatically, which would be pretty cool.