Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin jupyter-notebook version #469

Merged
merged 2 commits into from
Jun 20, 2024
Merged

Pin jupyter-notebook version #469

merged 2 commits into from
Jun 20, 2024

Conversation

danielhollas
Copy link
Contributor

@danielhollas danielhollas commented Jun 11, 2024

In aiidalab/aiidalab-qe#640, @superstar54 and I were debugging an issue with a stalled pip installation, likely caused by his quite complex dev environment. One thing I noticed that while the pip was trying hard to solve the dependencies, it started backtracking and looking at older versions of some of the core Jupyter libraries, most notably Jupyter notebook (notebook package). This is definitely not desirable, and if pip was able to finish, it would likely break the container.

We've been having discussions in the past about pinning some jupyter-related packages to prevent this scenario from happening. I'll open a separate issue for that, but for starters I think we should pin at least the notebook package. We definitely don't want users to be able to downgrade the version, and we also don't want them to upgrade it, since we know that v7 will currently not work as it is a completely different architecture.

RUN cat /opt/requirements.txt | xargs -I{} conda config --system --add pinned_packages {}

# Upgrade pip to latest
RUN pip install --upgrade --no-cache-dir pip
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pip is originally installed via conda so I've changed this and use mamba update pip instead.

Copy link
Contributor Author

@danielhollas danielhollas Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should think about updating conda and mamba as well? (for another PR)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mamba is from jupyter image, so I think it is safe to use any version it used, and if I remember correctly they spin up the build process periodically and will try to bring the latest mamba.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but we're not using the latest jupyter image. In fact our current one is quite old. And even when we upgrade, I don't think we'll ever want to use the most recent one, since it would likely produce a lot of churn.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't use aiidalab mamba for quite a while, did you see some annoying warnings ask to update? If so I am totally fine with updating to the latest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not, I just think it is a good practice to keep it updated. I'll take a closer look and prepare a PR at some point.

btw: Thanks for the review, I've added some tests in case you want to take a look. Waiting for the build to finish and will then merge.

@danielhollas danielhollas marked this pull request as ready for review June 11, 2024 23:49
@danielhollas
Copy link
Contributor Author

@unkcpz would you have time to take a look in the next few days? I'd like to publish a new version of the image after we merge this.

@unkcpz
Copy link
Member

unkcpz commented Jun 18, 2024

Yes, it is in my list, I'll have a look asap. Sorry.

@danielhollas
Copy link
Contributor Author

No worries at all, thanks!

Copy link
Member

@unkcpz unkcpz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a safe move. Thanks @danielhollas.

We've been having discussions in the past about pinning some jupyter-related packages to prevent this scenario from happening.

Yep.

stack/base/Dockerfile Show resolved Hide resolved
RUN cat /opt/requirements.txt | xargs -I{} conda config --system --add pinned_packages {}

# Upgrade pip to latest
RUN pip install --upgrade --no-cache-dir pip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mamba is from jupyter image, so I think it is safe to use any version it used, and if I remember correctly they spin up the build process periodically and will try to bring the latest mamba.

@danielhollas
Copy link
Contributor Author

Thanks @unkcpz. I've added some tests. After merging this I'm going to publish a new version.

@danielhollas danielhollas merged commit 178d7c1 into main Jun 20, 2024
14 of 16 checks passed
@danielhollas danielhollas deleted the pin-notebook branch July 16, 2024 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants