Add support for the .python-version file (#272)

This adds support for configuring the app's Python version using a `.python-version` file. This file is used by several tools in the Python ecosystem (such as pyenv, `actions/setup-python`, uv), whereas the existing `runtime.txt` file is proprietary to Heroku. For now, if both a `runtime.txt` file and a `.python-version` file are present, then the `runtime.txt` file will take precedence. In the future, support for `runtime.txt` will be deprecated (and eventually removed) in favour of the `.python-version` file. We support the following `.python-version` syntax: - Major Python version (e.g. `3.12`, which will then be resolved to the latest Python 3.12). (This form is recommended, since it allows for Python security updates to be pulled in without having to manually bump the version.) - Exact Python version (e.g. `3.12.6`) - Comments (lines starting with `#`) - Blank lines We don't support the following `.python-version` features: - Specifying multiple Python versions - Prefixing versions with `python-` (since this form is undocumented and will likely be deprecated in the future) Since the `.python-version` file (unlike `runtime.txt`) supports specifying just the Python major version, adding support also required: - adding a mapping of major versions to the latest patch releases - explicit handling for EOL/unrecognised major versions - adding the concept of a "requested Python version" vs the resolved Python version (which should hopefully tie in well with use of a manifest in the future) In addition, the "origin" of a Python version now has to be tracked, so that build output can state which file was used, or in the case of invalid version errors, which file needs to be fixed by the user. Closes #6. Closes #9. GUS-W-12151504. GUS-W-11475071.
heroku · Sep 17, 2024 · e3d1f4a · e3d1f4a
1 parent 05aa01e
commit e3d1f4a
Show file tree

Hide file tree

Showing 41 changed files with 996 additions and 321 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,9 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- The Python version can now be configured using a `.python-version` file. Both the `3.X` and `3.X.Y` version forms are supported. ([#272](https://github.com/heroku/buildpacks-python/pull/272))
+
 ### Changed
 
 - pip is now only available during the build, and is longer included in the final app image. ([#264](https://github.com/heroku/buildpacks-python/pull/264))
+- Improved the error messages shown when an end-of-life or unknown Python version is requested. ([#272](https://github.com/heroku/buildpacks-python/pull/272))
 
 ## [0.17.1] - 2024-09-07
 

diff --git a/README.md b/README.md
@@ -39,16 +39,15 @@ A `requirements.txt` or `poetry.lock` file must be present in the root (top-leve
 
 By default, the buildpack will install the latest version of Python 3.12.
 
-To install a different version, add a `runtime.txt` file to your app's root directory that declares the exact version number to use:
+To install a different version, add a `.python-version` file to your app's root directory that declares the version number to use:
 
 ```term
-$ cat runtime.txt
-python-3.12.6
+$ cat .python-version
+3.12
 ```
 
 In the future this buildpack will also support specifying the Python version using:
 
-- A `.python-version` file: [#6](https://github.com/heroku/buildpacks-python/issues/6)
 - `tool.poetry.dependencies.python` in `pyproject.toml`: [#260](https://github.com/heroku/buildpacks-python/issues/260)
 
 ## Contributing

diff --git a/src/errors.rs b/src/errors.rs
@@ -5,8 +5,12 @@ use crate::layers::poetry::PoetryLayerError;
 use crate::layers::poetry_dependencies::PoetryDependenciesLayerError;
 use crate::layers::python::PythonLayerError;
 use crate::package_manager::DeterminePackageManagerError;
-use crate::python_version::{PythonVersion, PythonVersionError, DEFAULT_PYTHON_VERSION};
-use crate::runtime_txt::{ParseRuntimeTxtError, RuntimeTxtError};
+use crate::python_version::{
+    RequestedPythonVersion, RequestedPythonVersionError, ResolvePythonVersionError,
+    DEFAULT_PYTHON_FULL_VERSION, DEFAULT_PYTHON_VERSION,
+};
+use crate::python_version_file::ParsePythonVersionFileError;
+use crate::runtime_txt::ParseRuntimeTxtError;
 use crate::utils::{CapturedCommandError, DownloadUnpackArchiveError, StreamedCommandError};
 use crate::BuildpackError;
 use indoc::{formatdoc, indoc};
@@ -53,7 +57,8 @@ fn on_buildpack_error(error: BuildpackError) {
         BuildpackError::PoetryDependenciesLayer(error) => on_poetry_dependencies_layer_error(error),
         BuildpackError::PoetryLayer(error) => on_poetry_layer_error(error),
         BuildpackError::PythonLayer(error) => on_python_layer_error(error),
-        BuildpackError::PythonVersion(error) => on_python_version_error(error),
+        BuildpackError::RequestedPythonVersion(error) => on_requested_python_version_error(error),
+        BuildpackError::ResolvePythonVersion(error) => on_resolve_python_version_error(error),
     };
 }
 
@@ -117,47 +122,139 @@ fn on_determine_package_manager_error(error: DeterminePackageManagerError) {
     };
 }
 
-fn on_python_version_error(error: PythonVersionError) {
+fn on_requested_python_version_error(error: RequestedPythonVersionError) {
     match error {
-        PythonVersionError::RuntimeTxt(error) => match error {
-            // TODO: (W-12613425) Write the supported Python versions inline, instead of linking out to Dev Center.
-            RuntimeTxtError::Parse(ParseRuntimeTxtError { cleaned_contents }) => {
-                let PythonVersion {
-                    major,
-                    minor,
-                    patch,
-                } = DEFAULT_PYTHON_VERSION;
+        RequestedPythonVersionError::ReadPythonVersionFile(io_error) => log_io_error(
+            "Unable to read .python-version",
+            "reading the .python-version file",
+            &io_error,
+        ),
+        RequestedPythonVersionError::ReadRuntimeTxt(io_error) => log_io_error(
+            "Unable to read runtime.txt",
+            "reading the runtime.txt file",
+            &io_error,
+        ),
+        RequestedPythonVersionError::ParsePythonVersionFile(error) => match error {
+            ParsePythonVersionFileError::InvalidVersion(version) => log_error(
+                "Invalid Python version in .python-version",
+                formatdoc! {"
+                    The Python version specified in '.python-version' is not in the correct format.
+                    
+                    The following version was found:
+                    {version}
+                    
+                    However, the version must be specified as either:
+                    1. '<major>.<minor>' (recommended, for automatic security updates)
+                    2. '<major>.<minor>.<patch>' (to pin to an exact Python version)
+                    
+                    Do not include quotes or a 'python-' prefix. To include comments, add them
+                    on their own line, prefixed with '#'.
+                    
+                    For example, to request the latest version of Python {DEFAULT_PYTHON_VERSION},
+                    update the '.python-version' file so it contains:
+                    {DEFAULT_PYTHON_VERSION}
+                "},
+            ),
+            ParsePythonVersionFileError::MultipleVersions(versions) => {
+                let version_list = versions.join("\n");
                 log_error(
-                    "Invalid Python version in runtime.txt",
+                    "Invalid Python version in .python-version",
                     formatdoc! {"
-                        The Python version specified in 'runtime.txt' is not in the correct format.
-                        
-                        The following file contents were found:
-                        {cleaned_contents}
+                        Multiple Python versions were found in '.python-version':
                         
-                        However, the file contents must begin with a 'python-' prefix, followed by the
-                        version specified as '<major>.<minor>.<patch>'. Comments are not supported.
+                        {version_list}
                         
-                        For example, to request Python {DEFAULT_PYTHON_VERSION}, the correct version format is:
-                        python-{major}.{minor}.{patch}
+                        Update the file so it contains only one Python version.
                         
-                        Please update 'runtime.txt' to use the correct version format, or else remove
-                        the file to instead use the default version (currently Python {DEFAULT_PYTHON_VERSION}).
-                        
-                        For a list of the supported Python versions, see:
-                        https://devcenter.heroku.com/articles/python-support#supported-runtimes
+                        If the additional versions are actually comments, prefix those lines with '#'.
                     "},
                 );
             }
-            RuntimeTxtError::Read(io_error) => log_io_error(
-                "Unable to read runtime.txt",
-                "reading the (optional) runtime.txt file",
-                &io_error,
+            ParsePythonVersionFileError::NoVersion => log_error(
+                "Invalid Python version in .python-version",
+                formatdoc! {"
+                    No Python version was found in the '.python-version' file.
+                    
+                    Update the file so that it contain a valid Python version (such as '{DEFAULT_PYTHON_VERSION}'),
+                    or else delete the file to use the default version (currently Python {DEFAULT_PYTHON_VERSION}).
+
+                    If the file already contains a version, check the line is not prefixed by
+                    a '#', since otherwise it will be treated as a comment.
+                "},
             ),
         },
+        RequestedPythonVersionError::ParseRuntimeTxt(ParseRuntimeTxtError { cleaned_contents }) => {
+            log_error(
+                "Invalid Python version in runtime.txt",
+                formatdoc! {"
+                    The Python version specified in 'runtime.txt' is not in the correct format.
+                    
+                    The following file contents were found:
+                    {cleaned_contents}
+                    
+                    However, the file contents must begin with a 'python-' prefix, followed by the
+                    version specified as '<major>.<minor>.<patch>'. Comments are not supported.
+                    
+                    For example, to request Python {DEFAULT_PYTHON_FULL_VERSION}, update the 'runtime.txt' file so it
+                    contains exactly:
+                    python-{DEFAULT_PYTHON_FULL_VERSION}
+                "},
+            );
+        }
     };
 }
 
+fn on_resolve_python_version_error(error: ResolvePythonVersionError) {
+    match error {
+        ResolvePythonVersionError::EolVersion(requested_python_version) => {
+            let RequestedPythonVersion {
+                major,
+                minor,
+                origin,
+                ..
+            } = requested_python_version;
+            log_error(
+                "Requested Python version has reached end-of-life",
+                formatdoc! {"
+                    The requested Python version {major}.{minor} has reached its upstream end-of-life,
+                    and is therefore no longer receiving security updates:
+                    https://devguide.python.org/versions/#supported-versions
+                    
+                    As such, it is no longer supported by this buildpack.
+                    
+                    Please upgrade to a newer Python version by updating the version
+                    configured via the {origin} file.
+                    
+                    If possible, we recommend upgrading all the way to Python {DEFAULT_PYTHON_VERSION},
+                    since it contains many performance and usability improvements.
+                "},
+            );
+        }
+        ResolvePythonVersionError::UnknownVersion(requested_python_version) => {
+            let RequestedPythonVersion {
+                major,
+                minor,
+                origin,
+                ..
+            } = requested_python_version;
+            log_error(
+                "Requested Python version is not recognised",
+                formatdoc! {"
+                    The requested Python version {major}.{minor} is not recognised.
+                    
+                    Check that this Python version has been officially released:
+                    https://devguide.python.org/versions/#supported-versions
+                    
+                    If it has, make sure that you are using the latest version of this buildpack.
+                    
+                    If it has not, please switch to a supported version (such as Python {DEFAULT_PYTHON_VERSION})
+                    by updating the version configured via the {origin} file.
+                "},
+            );
+        }
+    }
+}
+
 fn on_python_layer_error(error: PythonLayerError) {
     match error {
         PythonLayerError::DownloadUnpackPythonArchive(error) => match error {
@@ -186,8 +283,8 @@ fn on_python_layer_error(error: PythonLayerError) {
             formatdoc! {"
                 The requested Python version ({python_version}) is not available for this builder image.
                 
-                Please update the version in 'runtime.txt' to a supported Python version, or else
-                remove the file to instead use the default version (currently Python {DEFAULT_PYTHON_VERSION}).
+                Please switch to a supported Python version, or else don't specify a version
+                and the buildpack will use a default version (currently Python {DEFAULT_PYTHON_VERSION}).
                 
                 For a list of the supported Python versions, see:
                 https://devcenter.heroku.com/articles/python-support#supported-runtimes

diff --git a/src/layers/python.rs b/src/layers/python.rs
@@ -335,14 +335,7 @@ mod tests {
         base_env.insert("PYTHONHOME", "this-should-be-overridden");
         base_env.insert("PYTHONUNBUFFERED", "this-should-be-overridden");
 
-        let layer_env = generate_layer_env(
-            Path::new("/layer-dir"),
-            &PythonVersion {
-                major: 3,
-                minor: 11,
-                patch: 1,
-            },
-        );
+        let layer_env = generate_layer_env(Path::new("/layer-dir"), &PythonVersion::new(3, 11, 1));
 
         assert_eq!(
             utils::environment_as_sorted_vector(&layer_env.apply(Scope::Build, &base_env)),

diff --git a/src/main.rs b/src/main.rs
@@ -5,6 +5,7 @@ mod layers;
 mod package_manager;
 mod packaging_tool_versions;
 mod python_version;
+mod python_version_file;
 mod runtime_txt;
 mod utils;
 
@@ -16,7 +17,10 @@ use crate::layers::poetry_dependencies::PoetryDependenciesLayerError;
 use crate::layers::python::PythonLayerError;
 use crate::layers::{pip, pip_cache, pip_dependencies, poetry, poetry_dependencies, python};
 use crate::package_manager::{DeterminePackageManagerError, PackageManager};
-use crate::python_version::PythonVersionError;
+use crate::python_version::{
+    PythonVersionOrigin, RequestedPythonVersionError, ResolvePythonVersionError,
+};
+use indoc::formatdoc;
 use libcnb::build::{BuildContext, BuildResult, BuildResultBuilder};
 use libcnb::detect::{DetectContext, DetectResult, DetectResultBuilder};
 use libcnb::generic::{GenericMetadata, GenericPlatform};
@@ -53,8 +57,28 @@ impl Buildpack for PythonBuildpack {
             .map_err(BuildpackError::DeterminePackageManager)?;
 
         log_header("Determining Python version");
-        let python_version = python_version::determine_python_version(&context.app_dir)
-            .map_err(BuildpackError::PythonVersion)?;
+
+        let requested_python_version =
+            python_version::read_requested_python_version(&context.app_dir)
+                .map_err(BuildpackError::RequestedPythonVersion)?;
+        let python_version = python_version::resolve_python_version(&requested_python_version)
+            .map_err(BuildpackError::ResolvePythonVersion)?;
+
+        match requested_python_version.origin {
+            PythonVersionOrigin::BuildpackDefault => log_info(formatdoc! {"
+                No Python version specified, using the current default of Python {requested_python_version}.
+                We recommend setting an explicit version. In the root of your app create
+                a '.python-version' file, containing a Python version like '{requested_python_version}'."
+            }),
+            PythonVersionOrigin::PythonVersionFile => log_info(format!(
+                "Using Python version {requested_python_version} specified in .python-version"
+            )),
+            // TODO: Add a deprecation message for runtime.txt once .python-version support has been
+            // released for both the CNB and the classic buildpack.
+            PythonVersionOrigin::RuntimeTxt => log_info(format!(
+                "Using Python version {requested_python_version} specified in runtime.txt"
+            )),
+        }
 
         // We inherit the current process's env vars, since we want `PATH` and `HOME` from the OS
         // to be set (so that later commands can find tools like Git in the base image), along
@@ -100,13 +124,13 @@ impl Buildpack for PythonBuildpack {
 
 #[derive(Debug)]
 pub(crate) enum BuildpackError {
-    /// IO errors when performing buildpack detection.
+    /// I/O errors when performing buildpack detection.
     BuildpackDetection(io::Error),
     /// Errors determining which Python package manager to use for a project.
     DeterminePackageManager(DeterminePackageManagerError),
     /// Errors running the Django collectstatic command.
     DjangoCollectstatic(DjangoCollectstaticError),
-    /// IO errors when detecting whether Django is installed.
+    /// I/O errors when detecting whether Django is installed.
     DjangoDetection(io::Error),
     /// Errors installing the project's dependencies into a layer using pip.
     PipDependenciesLayer(PipDependenciesLayerError),
@@ -118,8 +142,10 @@ pub(crate) enum BuildpackError {
     PoetryLayer(PoetryLayerError),
     /// Errors installing Python into a layer.
     PythonLayer(PythonLayerError),
-    /// Errors determining which Python version to use for a project.
-    PythonVersion(PythonVersionError),
+    /// Errors determining which Python version was requested for a project.
+    RequestedPythonVersion(RequestedPythonVersionError),
+    /// Errors resolving a requested Python version to a specific Python version.
+    ResolvePythonVersion(ResolvePythonVersionError),
 }
 
 impl From<BuildpackError> for libcnb::Error<BuildpackError> {