Chocolatey Community
(python3) Proposal to publish meta packages for each minor version (e.g. python38, python39, etc.)
#1750
Replies: 13 comments
-
|
I wouldn't do this as this is something that |
Beta Was this translation helpful? Give feedback.
-
|
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? |
Beta Was this translation helpful? Give feedback.
-
|
I still (and increasingly) believe this should follow the homebrew model and there should be a distinct package for each significant version: 3.6, 3.7, 3.8, 3.9, and now 3.10. These should not auto-upgrade between each other. When and if Chocolatey features are expanded to support version constraints and version pinning, this approach could be revised at that time. |
Beta Was this translation helpful? Give feedback.
-
@aaronsteers Can you elaborate on this as it doesn't make sense so I'm assuming I'm misunderstanding it? |
Beta Was this translation helpful? Give feedback.
-
|
My proposal was separate packages per version (3.6,3.7, etc.). Currently, the If separate packages were used, this would ensure python 3.6.4 may auto-upgrade to 3.6.10 but never to 3.7.x or 3.10.x. Does that make sense? |
Beta Was this translation helpful? Give feedback.
-
|
Is there a reason package versions can't be pinned? |
Beta Was this translation helpful? Give feedback.
-
Yes. Or, well they can, but this would cause another issue. The issue is that python is not just a program, it also acts as a runtime, like vcredist or dotnet. It is possible to package up a python script into a single standalone executable (with pyinstaller or similar), but many python scripts or libraries depend on python being installed to be able to run. And not just any version of python, but a specific minor version (e.g. 3.x) or range of minor versions. Python 3 can have breaking changes between minor versions, so you cannot use a script written for 3.n and always expect python 3.n+1 to run it (although it often works). Python minor versions get two-ish years of full support, and five-ish years of security fixes. This means that you could theoretically be running four or five minor versions behind, and still be getting security updates. So, if you had python 3.8.10 (for example) installed and pinned, and python released a security update for 3.8, how would you know that? This negates half the benefit of having a package manager, in that you can check for updates with one command. Technically, yes this could be scripted as @majkinator mentions, but not everyone has the time or knowledge to do that. |
Beta Was this translation helpful? Give feedback.
-
|
I'm gonna jump in again just as an example to drive home the need for this... Python 3.10 released this month, and there are hundreds, perhaps thousands, of pip-installable python libraries and tools that don't yet support it. Some of those pip-installed libraries and tools will get updated within the next 2-4 months, and some are abandoned (although nonetheless useful!) and may never be updated. By not having control to pin to Python 3.9 any user who lets chocolatey upgrade python is likely to immediately start running into failures and incompatibilities. |
Beta Was this translation helpful? Give feedback.
-
|
Taking another tact here, has anyone on this thread tried https://community.chocolatey.org/packages/pyenv-win? I gave it a short amount of effort but the install for some reason was not going through as expected. In absense of version-specific python packages, perhaps the guidance if stability is needed should be:
This is a lot of additional steps, and still doesn't support pulling in Python security updates as @TheCakeIsNaOH also calls out - but it does get around the problems with The guidance then becomes: don't use chocolatey to directly install python if you need stability and/or want to use What do other folks think? I know maintaining multiple python packages in chocolatey would be a lot of work. Is pyenv a good approach? |
Beta Was this translation helpful? Give feedback.
-
|
I think it should be better supported by chocolatey in some way (I don't know whether that means separate packages, something like chocolatey/choco#800, or something entirely different) but for now, I just settled for this script I've written (requires chocolatey 0.11.1+): # Instructions: To use this script, you need to install NuGet.Versioning package using command:
#
# Install-Package NuGet.Versioning -Source https://www.nuget.org/api/v2
#
# This script relies on chocolatey 0.11.1+
$package_path = Get-Package -Name NuGet.Versioning | % Source | Split-Path -parent
$assembly_path = $package_path | Join-Path -ChildPath 'lib\netstandard2.0\NuGet.Versioning.dll'
$null = [System.Reflection.Assembly]::LoadFrom($assembly_path)
# upgrade all packages
choco upgrade all --yes --except=python3
# find currently installed versions of Python
$current_versions = @{}
$local_packages = choco list python3 --exact --local --all-versions --limit-output
$local_packages | % {
$pkgname, $raw_version = $_.Split('|', 2)
$version = [NuGet.Versioning.NuGetVersion]::Parse($raw_version)
$key = "$($version.Major).$($version.Minor)"
# $current_versions[$key] might not exist which means PowerShell will return $null instead.
# Object returned by NuGet.Versioning is considered to be greater than $null.
if ($version -gt $current_versions[$key]) {
# this is used for uninstall so we might skip some version if we had multiple
$current_versions[$key] = $version
}
}
# find and upgrade to latest micro versions of each installed Python version
$latest_versions = @{}
$matching_packages = choco list python3 --exact --all-versions --limit-output
$matching_packages | % {
$pkgname, $raw_version = $_.Split('|', 2)
$version = [NuGet.Versioning.NuGetVersion]::Parse($raw_version)
$key = "$($version.Major).$($version.Minor)"
if ($current_versions[$key] -eq $null) {
return
}
if ($version -gt $current_versions[$key] -and $version -gt $latest_versions[$key]) {
$latest_versions[$key] = $version
}
}
ForEach ($entry in $latest_versions.GetEnumerator()) {
choco install python3 --version $entry.Value.OriginalVersion --yes --allow-multiple
choco uninstall python3 --version $current_versions[$entry.Name] --yes -n --skipautouninstaller
} |
Beta Was this translation helpful? Give feedback.
-
|
How about we pack a script like this with a package, document it in the package notes and call it a day ? Then you could call it to update microversions whenever you like it. |
Beta Was this translation helpful? Give feedback.
-
|
I've moved this over to Discussions. |
Beta Was this translation helpful? Give feedback.
-
|
With the deprecation of side by side installations, I think this is definitely something that should be done.
|
Beta Was this translation helpful? Give feedback.
-
Expected Behavior
This request would be for a new set of community-managed meta packages at each minor python version release, enabling chocolatey users to permanently "pin" a minor version number by explicitly referencing a meta package within that version. The reason I think this is necessary is due to the fact that according the generally accepted rules of backwards compatibility:
There should be zero expectation from a Python user/developer that incrementing their Python "minor" version number will give meaningful backwards compatibility against an otherwise working version. The problem is multiplied exponentially depending upon the number of packages the user has installed.
Current Behavior
Installs:
Currently, the behavior of a Python install is that it will grab the latest version - which basically breaks a well-meaning new-hire page unless the python version is pinned.
Upgrades:
Assuming the version is pinned during install, we still have no version stability across calls to
choco upgrade. Any user who makes the mistake ofchoco upgrade all(or pushing the very inviting matching "Upgrade All" button inChocolatey GUIwill basically wipe out their previously-stable Python environment.Possible Solution
The ideal solution, I believe, is to have meta packages published for each "dot" release, starting with 3.8. At start, this would require two new meta packages, and over time each new version (at least in the 3.x line) would get a new meta-package.
Impact on Chocolatey Community Overall
Since Python is such a useful and commonly installed library in the dev community, the current behavior (after one big "oops" per user) has a negative affect on the communities willingness to
upgrade allin chocolatey at all - since this will almost certainly break their existing Python environment. The entire Windows environment therefor becomes stale over time, since other software isn't getting the security and patch updates they would otherwise be getting if they could liberally runchoco upgrade all- or the point-and-click ChocolateyGUI version of the same. Regular and universal software upgrades are then likely to become the exception and not the rule. (Case in point: I run a set of technical trainings and even if most of my students aren't going to need Python, I specifically avoid mentioningchoco upgrade allbecause I know it'll break anyone who has python installed.)In terms of impact, the Chocolatey
python3andpythonpackages currently show 5.4 million downloads and 4.2 million downloads, respectively. That likely translates into a LOT of users in the community avoiding the "upgrade all" option.Context
I have been tremendously grateful for the presence of the Chocolatey install packages for Python, and I fully recognize that this is a large ask. This one issue, however, creates significant confusion amongst my team members and students whenever there's a new release. If we agree that "minor" version releases of Python should be treated as major versions, the common best practice of having distinct packages per major version seems to hold here as well. (Hopefully the maintenance is not extremely onerous.)
FWIW, Homebrew similarly publishes separate formula for each python version - [email protected], [email protected], etc. and those distinct versioned "3.x" formula each make reference to the central python formula.
See: https://docs.brew.sh/Homebrew-and-Python#python-3x , https://formulae.brew.sh/formula/[email protected], https://formulae.brew.sh/formula/[email protected], etc.
Beta Was this translation helpful? Give feedback.
All reactions