Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring the CI pipeline and Dockerfiles #1307

Open
rihafilip opened this issue Aug 8, 2024 · 0 comments
Open

Refactoring the CI pipeline and Dockerfiles #1307

rihafilip opened this issue Aug 8, 2024 · 0 comments

Comments

@rihafilip
Copy link
Collaborator

Currently, the structure of Dockerfiles used for the CI/CD pipeline is pretty ineffective. On each run of the pipeline, new container is created, which downloads all of the needed programs, sets up locale, and only after that builds the Ř codebase.

The first step can be easily abstracted - creating a base container with all of the needed programs installed. On each run of the pipeline, we would then create a new container based on this base container, that would only need to build the Ř codebase.

We can check if the image hasn't been build already, for example on manually rerunning the pipeline. By adding to the script before the docker build command the following line:

DOCKER_CLI_EXPERIMENTAL=enabled docker manifest inspect [CONTAINER_TAG] > /dev/null && exit || true

the job will exit (sucessfully) if the [CONTAINER_TAG] container is already present.

The part that takes a large amount of time is also building the GnuR submodule, more importatly running the configure script, which cannot be parallized. We might consider some sort of caching or prebuilding the submodule, since it is not updated that frequently.

A .dockerignore file can be used to filter the needed files and lower the size of containers, altrough this would be a very small optimization.

The gitlab-ci.yml file could be further refactored. The test jobs could be abstracted via the extends keyword and hidden jobs.
Also, in all places where the GitLab container registry is used (registry.gitlab.com/rirvm/rir_mirror), it is hardcoded, whereas the GitLab variable $CI_REGISTRY_NAME should be probably used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant