Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use quay.io as source of docker images #3254

Merged
merged 14 commits into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,15 @@ jobs:
export KUBECONFIG="$HOME/.kube/config"
export HUB_URL=http://localhost:30080
. ./ci/common
pytest --verbose --color=yes ./tests -m 'not netpol'
# Print out logs & definition info from all pods if the tests fail
pytest --verbose --color=yes ./tests -m 'not netpol' || \
kubectl get pod -o name | \
xargs -I {} /bin/bash -c \
"echo Logs for {} && \
kubectl get {} -o yaml && \
kubectl describe {} && \
kubectl logs --all-containers {} && \
echo --------------------------------"
name: Run tests

- run:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
# JupyterHub organization Helm chart repository.
#
# ref: https://github.com/jupyterhub/helm-chart
# ref: https://hub.docker.com/orgs/jupyterhub
# ref: https://quay.io/organization/jupyterhub

publish:
runs-on: ubuntu-22.04
Expand Down Expand Up @@ -108,7 +108,7 @@ jobs:
# https://github.com/jupyterhub/zero-to-jupyterhub-k8s/settings/secrets/actions
if: steps.publishing.outputs.publishing
run: |
docker login -u "${{ secrets.DOCKER_USERNAME }}" -p "${{ secrets.DOCKER_PASSWORD }}"
docker login -u "${{ secrets.QUAY_USERNAME }}" -p "${{ secrets.QUAY_PASSWORD }}" quay.io

- name: Configure a git user
# Having a user.email and user.name configured with git is required to
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ jobs:
- k3s-channel: stable # also test hub-slim, and prePuller.hook
test: install
local-chart-extra-args: >-
--set hub.image.name=jupyterhub/k8s-hub-slim
--set hub.image.name=quay.io/jupyterhub/k8s-hub-slim
--set prePuller.hook.enabled=true
--set prePuller.hook.pullOnlyOnChanges=true
- k3s-channel: v1.26 # also test hub.existingSecret
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ Did you get an error like one of these below?
# while running apt-get install while building a docker image with chartpress
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/r/rtmpdump/librtmp1_2.4+20151223.gitfa8646d.1-1_amd64.deb Could not connect to archive.ubuntu.com:80 (91.189.88.174). - connect (113: No route to host) Could not connect to archive.ubuntu.com:80 (91.189.88.31). - connect (113: No route to host) [IP: 91.189.88.174 80]
# [...]
subprocess.CalledProcessError: Command '['docker', 'build', '-t', 'jupyterhub/k8s-hub:0.9-217f798', 'images/hub', '--build-arg', 'JUPYTERHUB_VERSION=git+https://github.com/jupyterhub/jupyterhub@master']' returned non-zero exit status 100.
subprocess.CalledProcessError: Command '['docker', 'build', '-t', 'quay.io/jupyterhub/k8s-hub:0.9-217f798', 'images/hub', '--build-arg', 'JUPYTERHUB_VERSION=git+https://github.com/jupyterhub/jupyterhub@master']' returned non-zero exit status 100.

# while installing a dependency for our k8s cluster
Unable to connect to the server: dial tcp: lookup docs.projectcalico.org on 127.0.0.53:53: read udp 127.0.0.1:56409->127.0.0.53:53: i/o timeout
Expand Down
2 changes: 1 addition & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Also consider nudging dependent projects in the JupyterHub GitHub organization f
These images version/tags are set in [values.yaml](jupyterhub/values.yaml), consider bumping the version of these as well.

- [ ] [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy)
- [Available image tags](https://hub.docker.com/r/jupyterhub/configurable-http-proxy/tags)
- [Available image tags](https://quay.io/repository/jupyterhub/configurable-http-proxy?tab=tags)
- values.yaml entry: `proxy.chp.image`
- [ ] [traefik/traefik](https://github.com/traefik/traefik)
- [Available image tags](https://hub.docker.com/_/traefik?tab=tags)
Expand Down
2 changes: 1 addition & 1 deletion chartpress.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ charts:
- name: jupyterhub
# Dev: imagePrefix can be useful to override if you want to trial something
# locally developed in a remote k8s cluster.
imagePrefix: jupyterhub/k8s-
imagePrefix: quay.io/jupyterhub/k8s-
# baseVersion should be a -0.dev suffixed version, where the version should
# be the next major, minor, or patch version depending on what we have
# merged so far into the main branch. If for example we have merged a
Expand Down
4 changes: 2 additions & 2 deletions docs/source/administrator/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ changes to your `config.yaml` file:

**NOTE:**

If the proxy service is of type `LoadBalancer`, which it is by default, then a specific static IP address can be requested (if available) instead of a dynamically acquired one.
If the proxy service is of type `LoadBalancer`, which it is by default, then a specific static IP address can be requested (if available) instead of a dynamically acquired one.
Although not essential for HTTPS, using a static IP address is a recommended practice for domain names referencing fixed IPs.
This ensures the same IP address for multiple deployments.
The IP can be provided like:
Expand Down Expand Up @@ -179,7 +179,7 @@ hub:
# when debugging something from the hub pod. To use it, apply this
# configuration.
#
name: jupyterhub/k8s-hub-slim
name: quay.io/jupyterhub/k8s-hub-slim
```

```{note}
Expand Down
7 changes: 4 additions & 3 deletions docs/source/administrator/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Services can be run [externally](https://jupyterhub.readthedocs.io/en/stable/get

## Hub-managed services in z2jh

A Hub-managed service will run in the same container/pod as the Hub itself. First, you'll need to install or copy the appropriate files for the service into your Hub image, either by creating a custom image derived from [`jupyterhub/k8s-hub`](https://hub.docker.com/r/jupyterhub/k8s-hub) or the [hub.extraFiles](schema_hub.extraFiles) configuration. Keep in mind that your Hub container may need to install dependency libraries like flask or fastapi, depending on the service. In those cases, you'll need a custom image.
A Hub-managed service will run in the same container/pod as the Hub itself. First, you'll need to install or copy the appropriate files for the service into your Hub image, either by creating a custom image derived from [`jupyterhub/k8s-hub`](https://quay.io/repository/jupyterhub/k8s-hub) or the [hub.extraFiles](schema_hub.extraFiles) configuration. Keep in mind that your Hub container may need to install dependency libraries like flask or fastapi, depending on the service. In those cases, you'll need a custom image.

In addition to the code for the service, you need to modify the Hub Kubernetes Service object to include [multiple ports](https://kubernetes.io/docs/concepts/services-networking/service/#multi-port-services), and update the Hub Network Policy. If you want to allow access from all sources, you can use [hub.networkPolicy.allowedIngressPorts](schema_hub.networkPolicy.allowedIngressPorts). Otherwise if you want to more precisely control access, you can use [hub.networkPolicy.ingress](schema_hub.networkPolicy.ingress).

Expand All @@ -18,8 +18,9 @@ In the following snippet, I'm using a custom image that copies over the applicat

```Dockerfile
# Dockerfile
# 2.0.0 is latest stable release at the time of this writing
FROM jupyterhub/k8s-hub:2.0.0
# 3.1.0 is latest stable release at the time of this writing
# Find all tags in https://quay.io/repository/jupyterhub/k8s-hub?tab=tags
FROM quay.io/jupyterhub/k8s-hub:3.1.0

# Depending on version, the k8s-hub image may have installed
# pip packages as root, forcing you to install as root as well
Expand Down
36 changes: 36 additions & 0 deletions docs/source/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,42 @@ changes in pull requests], this list should be updated.
[development releases]: https://hub.jupyter.org/helm-chart/#development-releases-jupyterhub
[breaking changes in pull requests]: https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pulls?q=is%3Apr+is%3Aclosed+label%3Abreaking

### Default image registry moved to [quay.io](https://quay.io)

We have moved the registry where we publish our docker images from [Docker Hub](https://hub.docker.com)
to [Quay.io](https://quay.io). This move is to ensure our users are not [throttled by Docker Hub](https://docs.docker.com/docker-hub/download-rate-limit/),
and us maintainers don't have to apply for 'sponsored OSS Project' from docker each year. This
should have no material impact on your experience.

For the benefit of people running older versions of z2jh and are throttled by dockerhub,
we have actually copied all our _released_ images from Docker Hub to Quay.io as well.
So you can opt in to using the images from Quay.io with the following config:

```yaml
hub:
image:
name: quay.io/jupyterhub/k8s-hub
proxy:
chp:
image:
name: quay.io/jupyterhub/configurable-http-proxy
secretSync:
image:
name: quay.io/jupyterhub/k8s-secret-sync
singleuser:
networkTools:
image:
name: quay.io/jupyterhub/k8s-network-tools
prePuller:
hook:
image:
name: quay.io/jupyterhub/k8s-image-awaiter
```

You don't have to explicitly specify the tag, as the existing tags
will work. Note that this **only** works for _released_ versions of
z2jh - if you are using a _dev_ version of z2jh, this will not work.

## 3.1

### 3.1.0 - 2023-09-29
Expand Down
8 changes: 3 additions & 5 deletions docs/source/repo2docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,7 @@ to configure JupyterHub to build off of this image:
4. **Get credentials for a docker repository.**

The image you will build for your JupyterHub must be made available by being
published to some container registry. You could for example use [Docker Hub](https://hub.docker.com/) or [Google Container Registry](https://cloud.google.com/artifact-registry).

<!-- FIXME: We link to "google container registry", but its deprecated and they now redirect and promote artifact registry with small differences -->
published to some container registry. You could for example use [quay.io](https://quay.io) or [Docker Hub](https://hub.docker.com/).

In the next step, you need an image reference for you and others to find your
image with.
Expand All @@ -85,10 +83,10 @@ to configure JupyterHub to build off of this image:
<dockerhub-username>/<image-name>:<image-tag>
```

An image reference on Google Container Registry:
An image reference on quay.io:

```
gcr.io/<cloud-project-name>/<image-name>:<image-tag>
quay.io/<quay-username>/<image-name>:<image-tag>
```

- Your image name can be anything memorable.
Expand Down
2 changes: 1 addition & 1 deletion images/hub/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ RUN --mount=type=cache,target=${PIP_CACHE_DIR} \

# The final stage - slim version
# ------------------------------
# This stage is built and published as jupyterhub/k8s-hub-slim. It is meant to
# This stage is built and published as quay.io/jupyterhub/k8s-hub-slim. It is meant to
# provide no non-essential packages.
#
FROM python:3.11-slim-bullseye as slim-stage
Expand Down
4 changes: 2 additions & 2 deletions images/singleuser-sample/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ RUN adduser \
${NB_USER}

RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
&& apt-get upgrade --yes \
&& apt-get install --yes --no-install-recommends \
ca-certificates \
dnsutils \
iputils-ping \
Expand Down
4 changes: 2 additions & 2 deletions images/singleuser-sample/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@ To quickly try out this Docker image on your computer:

```sh
# with JupyterLab
docker run -it --rm -p 8888:8888 jupyterhub/k8s-singleuser-sample:2.0.0 -- jupyter lab --ip 0.0.0.0
docker run -it --rm -p 8888:8888 quay.io/jupyterhub/k8s-singleuser-sample:2.0.0 -- jupyter lab --ip 0.0.0.0
```

This image available tags can be found [here](https://hub.docker.com/r/jupyterhub/k8s-singleuser-sample/tags/).
This image available tags can be found [here](https://quay.io/repository/jupyterhub/k8s-singleuser-sample?tab=tags).

## In the base-notebook image

Expand Down
12 changes: 6 additions & 6 deletions jupyterhub/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ hub:
extraVolumes: []
extraVolumeMounts: []
image:
name: jupyterhub/k8s-hub
name: quay.io/jupyterhub/k8s-hub
tag: "set-by-chartpress"
pullPolicy:
pullSecrets: []
Expand Down Expand Up @@ -201,7 +201,7 @@ proxy:
runAsGroup: 65534 # nobody group
allowPrivilegeEscalation: false
image:
name: jupyterhub/configurable-http-proxy
name: quay.io/jupyterhub/configurable-http-proxy
# tag is automatically bumped to new patch versions by the
# watch-dependencies.yaml workflow.
#
Expand Down Expand Up @@ -304,7 +304,7 @@ proxy:
runAsGroup: 65534 # nobody group
allowPrivilegeEscalation: false
image:
name: jupyterhub/k8s-secret-sync
name: quay.io/jupyterhub/k8s-secret-sync
tag: "set-by-chartpress"
pullPolicy:
pullSecrets: []
Expand Down Expand Up @@ -344,7 +344,7 @@ singleuser:
preferred: []
networkTools:
image:
name: jupyterhub/k8s-network-tools
name: quay.io/jupyterhub/k8s-network-tools
tag: "set-by-chartpress"
pullPolicy:
pullSecrets: []
Expand Down Expand Up @@ -396,7 +396,7 @@ singleuser:
volumeNameTemplate: volume-{username}{servername}
storageAccessModes: [ReadWriteOnce]
image:
name: jupyterhub/k8s-singleuser-sample
name: quay.io/jupyterhub/k8s-singleuser-sample
tag: "set-by-chartpress"
pullPolicy:
pullSecrets: []
Expand Down Expand Up @@ -605,7 +605,7 @@ prePuller:
pullOnlyOnChanges: true
# image and the configuration below relates to the hook-image-awaiter Job
image:
name: jupyterhub/k8s-image-awaiter
name: quay.io/jupyterhub/k8s-image-awaiter
tag: "set-by-chartpress"
pullPolicy:
pullSecrets: []
Expand Down