Release v180 (#1078 )

Closes @W-8120452@.
Add support for Python 3.8.6 (#1072 )
2026-06-05 23:10:16 +00:00 · 2020-09-24 19:17:18 +01:00 · 2020-09-24 18:55:27 +01:00 · 2020-09-23 14:49:46 +01:00 · 2020-09-23 11:15:54 +01:00 · 2020-09-22 18:48:31 +01:00
30 changed files with 104 additions and 93 deletions
@@ -1,5 +1,5 @@
 language: minimal
-dist: bionic
+dist: focal
 branches:
  only:
  - main
@@ -3,6 +3,25 @@
 ## Unreleased


+## v180 (2020-09-24)
+
+- Python 3.8.6 is now available (CPython) (#1072).
+
+## v179 (2020-09-23)
+
+- Remove duplicate pipenv metric event (#1070).
+- Emit metrics for how the Python version was chosen for an app (#1069).
+- Emit Python version metric events for all builds, not just clean installs (#1066).
+
+## v178 (2020-09-07)
+
+- Python 3.5.10 is now available (CPython) (#1062).
+
+## v177 (2020-08-18)
+
+- Python 3.6.12 and 3.7.9 are now available (CPython) (#1054).
+- The default Python version for new apps is now 3.6.12 (previously 3.6.11) (#1054).
+
 ## v176 (2020-08-12)

 - Rebuild the Python 3.4.10 archives with the correct version of Python (#1048).
@@ -9,7 +9,7 @@ GEM
      excon
      moneta
      multi_json (>= 1.9.2)
-    heroku_hatchet (7.0.0)
+    heroku_hatchet (7.2.0)
      excon (~> 0)
      platform-api (~> 3)
      rrrretry (~> 1)
@@ -26,7 +26,7 @@ remote: Compressing source files... done.
 remote: Building source:
 remote:
 remote: -----> Python app detected
-remote: -----> Installing python-3.7.4
+remote: -----> Installing python
 remote: -----> Installing pip
 remote: -----> Installing SQLite3
 remote: -----> Installing requirements with pip
@@ -44,7 +44,7 @@ A `requirements.txt` must be present at the root of your application's repositor

 To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version.

-Current default Python Runtime: Python 3.6.9
+Current default Python Runtime: Python 3.6.12

 Alternatively, you can provide a `setup.py` file, or a `Pipfile`.
 Using `pipenv` will generate `runtime.txt` at build time if one of the field `python_version` or `python_full_version` is specified in the `requires` section of your `Pipfile`.
@@ -62,9 +62,9 @@ Specify a Python Runtime

 Supported runtime options include:

- `python-3.8.5`
- `python-3.7.8`
- `python-3.6.11`
+- `python-3.8.6`
+- `python-3.7.9`
+- `python-3.6.12`
 - `python-2.7.18`

 ## Tests
@@ -189,7 +189,7 @@ source "$BIN_DIR/steps/hooks/pre_compile"
 # continue to use that version of Python in perpituity (warnings will be raised if
 # they are out–of–date).
 if [ -f "$CACHE_DIR/.heroku/python-version" ]; then
-  DEFAULT_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version")
+  CACHED_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version")
 fi

 # We didn't always record the stack version. This code is in place because of that.
@@ -206,9 +206,14 @@ fi
 # shellcheck source=bin/steps/pipenv-python-version
 source "$BIN_DIR/steps/pipenv-python-version"

-# If no runtime was provided by the user, assume the default Python runtime version.
-if [ ! -f runtime.txt ]; then
-  echo "$DEFAULT_PYTHON_VERSION" > runtime.txt
+if [[ -f runtime.txt ]]; then
+  mcount "version.reason.python.specified"
+elif [[ -n "${CACHED_PYTHON_VERSION:-}" ]]; then
+  mcount "version.reason.python.cached"
+  echo "${CACHED_PYTHON_VERSION}" > runtime.txt
+else
+  mcount "version.reason.python.default"
+  echo "${DEFAULT_PYTHON_VERSION}" > runtime.txt
 fi

 # Create the directory for .profile.d, if it doesn't exist.
@@ -5,11 +5,11 @@
 # the env vars to subprocesses.
 # shellcheck disable=2034

-DEFAULT_PYTHON_VERSION="python-3.6.11"
-LATEST_38="python-3.8.5"
-LATEST_37="python-3.7.8"
-LATEST_36="python-3.6.11"
-LATEST_35="python-3.5.9"
+DEFAULT_PYTHON_VERSION="python-3.6.12"
+LATEST_38="python-3.8.6"
+LATEST_37="python-3.7.9"
+LATEST_36="python-3.6.12"
+LATEST_35="python-3.5.10"
 LATEST_34="python-3.4.10"
 LATEST_27="python-2.7.18"
 LATEST_PYPY_36="pypy3.6-7.3.1"
@@ -9,10 +9,7 @@ set -e
 if [[ -f Pipfile.lock ]]; then
    if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then
        if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then
-            # Measure that we're using Pipenv.
-            mcount "tool.pipenv"
-
-            # Don't skip installation of there are git deps.
+            # Don't skip installation if there are git deps.
            if ! grep -q 'git' Pipfile.lock; then
                echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent

@@ -52,11 +52,9 @@ if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
    fi
  fi
  if [[ "$PYTHON_VERSION" == $PY27* ]]; then
+    puts-warn "$PYTHON_2_EOL_UPDATE"
+    echo "       Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
    # security update note
-    if [[ "$(date "+%Y")" -gt "2019" ]]; then
-      puts-warn "$PYTHON_2_EOL_UPDATE"
-      echo "       Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
-    fi
    if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then
      puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27"
      echo "       Learn More: https://devcenter.heroku.com/articles/python-runtimes"
@@ -82,6 +80,8 @@ else
  exit 1
 fi

+mcount "version.python.${PYTHON_VERSION}"
+
 if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then
    puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache"
    rm -fr .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version
@@ -121,8 +121,6 @@ if [ ! "$SKIP_INSTALL" ]; then
    # Prepare destination directory.
    mkdir -p .heroku/python

-    mcount "version.python.$PYTHON_VERSION"
-
    if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then
      puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
      puts-warn "Aborting.  More info: https://devcenter.heroku.com/articles/python-support"
@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+# Build Path: /app/.heroku/python/
+
+source $(dirname $0)/python3
@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+# Build Path: /app/.heroku/python/
+
+source $(dirname $0)/python3
@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+# Build Path: /app/.heroku/python/
+
+source $(dirname $0)/python3
@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+# Build Path: /app/.heroku/python/
+
+source $(dirname $0)/python3
@@ -1,5 +1,5 @@
 ---
 - - "./repos/python/python-getting-started"
-  - master
+  - main
 - - "./repos/python/python_default"
  - ca947f69027b2a30be5d26f9a42f25e54f4d7a1a
@@ -1 +0,0 @@
-requests
@@ -1 +0,0 @@
-python-2.7.99
@@ -1 +0,0 @@
-flask
@@ -1 +0,0 @@
-python-3.4.99
@@ -1 +1 @@
-python-3.5.9
+python-3.5.10
@@ -1 +0,0 @@
-flask
@@ -1 +0,0 @@
-python-3.5.99
@@ -1 +0,0 @@
-flask
@@ -1 +0,0 @@
-python-3.6.99
@@ -1 +0,0 @@
-flask
@@ -1 +0,0 @@
-python-3.7.99
@@ -1 +0,0 @@
-flask
@@ -19,11 +19,7 @@ testPython2() {
    echo $LATEST_27 > "runtime.txt"
    compile "python2"
    assertCaptured $LATEST_27
-    if [[ $(date "+%Y") > "2019" ]]; then
-      assertCaptured "python-2-7-eol-faq";
-    else
-      assertNotCaptured "python-2-7-eol-faq";
-    fi
+    assertCaptured "python-2-7-eol-faq";
    assertNotCaptured "security update"
    assertCaptured "Installing pip 20.1.1, setuptools 44.1.1 and wheel 0.34.2"
    assertCaptured "Installing SQLite3"
@@ -33,22 +29,12 @@ testPython2() {
 testPython2_warn() {
    compile "python2_warn"
    assertCaptured "python-2.7.15"
-    if [[ $(date "+%Y") > "2019" ]]; then
-      assertCaptured "python-2-7-eol-faq";
-    else
-      assertNotCaptured "python-2-7-eol-faq";
-    fi
+    assertCaptured "python-2-7-eol-faq";
    assertCaptured "Only the latest version"
    assertCaptured "Installing SQLite3"
    assertCapturedSuccess
 }

-testPython2_fail() {
-    compile "python2_fail"
-    assertCaptured "Aborting"
-    assertCapturedError
-}
-
 testPython3_4() {
  compile "python3_4"
  assertCaptured $LATEST_34
@@ -64,12 +50,6 @@ testPython3_4_warn() {
  assertCapturedSuccess
 }

-testPython3_4_fail() {
-    compile "python3_4_fail"
-    assertCaptured "Aborting"
-    assertCapturedError
-}
-
 testPython3_5() {
  compile "python3_5"
  assertCaptured $LATEST_35
@@ -86,12 +66,6 @@ testPython3_5_warn() {
  assertCapturedSuccess
 }

-testPython3_5_fail() {
-    compile "python3_5_fail"
-    assertCaptured "Aborting"
-    assertCapturedError
-}
-
 testPython3_6() {
  updateVersion "python3_6" $LATEST_36
  compile "python3_6"
@@ -110,12 +84,6 @@ testPython3_6_warn() {
  assertCapturedSuccess
 }

-testPython3_6_fail() {
-    compile "python3_6_fail"
-    assertCaptured "Aborting"
-    assertCapturedError
-}
-
 testPython3_7() {
  updateVersion "python3_7" $LATEST_37
  compile "python3_7"
@@ -142,24 +110,6 @@ testPython3_7_warn() {
  fi
 }

-testPython3_7_fail() {
-  compile "python3_7_fail"
-  assertCaptured "Aborting"
-  assertCapturedError
-}
-
-testPython3_7_warn() {
-  compile "python3_8_warn"
-  if [[ $STACK = "cedar-14" ]]; then
-    assertCapturedError
-  else
-    assertCaptured "python-3.8.0"
-    assertCaptured "security update!"
-    assertCaptured "Installing SQLite3"
-    assertCapturedSuccess
-  fi
-}
-
 testPython3_8() {
  updateVersion "python3_8" $LATEST_38
  compile "python3_8"
@@ -174,8 +124,21 @@ testPython3_8() {
  fi
 }

-testPython3_8_fail() {
-  compile "python3_8_fail"
+testPython3_8_warn() {
+  compile "python3_8_warn"
+  if [[ $STACK = "cedar-14" ]]; then
+    assertCapturedError
+  else
+    assertCaptured "python-3.8.0"
+    assertCaptured "security update!"
+    assertCaptured "Installing SQLite3"
+    assertCapturedSuccess
+  fi
+}
+
+testPythonVersionInvalid() {
+  compile "python_version_invalid"
+  assertCaptured "Requested runtime (python-3.8.99) is not available for this stack"
  assertCaptured "Aborting"
  assertCapturedError
 }
@@ -222,6 +185,30 @@ testPypy2_7_warn() {
  fi
 }

+testStickyPythonVersion() {
+  local cache_dir="$(mktmpdir)"
+  compile "python3_6_warn" "$cache_dir"
+  assertCaptured "Installing python-3.6.7"
+  assertCapturedSuccess
+  compile "no-runtime-txt" "$cache_dir"
+  assertCaptured "Installing python-3.6.7"
+  assertCapturedSuccess
+  # Whilst this file seems like an implementation detail (so something that should
+  # not be tested), we must guarantee the filename remains consistent for backwards
+  # compatibility across buildpack versions for already-built apps.
+  assertFile "python-3.6.7" ".heroku/python-version"
+}
+
+testPythonVersionChange() {
+  local cache_dir="$(mktmpdir)"
+  compile "python3_6_warn" "$cache_dir"
+  assertCaptured "Installing python-3.6.7"
+  assertCapturedSuccess
+  compile "python3_6" "$cache_dir"
+  assertCaptured "Found python-3.6.7, removing"
+  assertCapturedSuccess
+}
+
 pushd $(dirname 0) >/dev/null
 popd >/dev/null
Author	SHA1	Message	Date
Ed Morley	6c612a7fb1	Release v180 (#1078 ) Closes @W-8120452@.	2020-09-24 19:17:18 +01:00
Ed Morley	f91f4ee4ce	Add support for Python 3.8.6 (#1072 ) https://www.python.org/downloads/release/python-386/ https://pythoninsider.blogspot.com/2020/09/python-386-is-now-available.html Closes @W-7791243@.	2020-09-24 18:55:27 +01:00
Eric Carmichael	eea9a7fd0f	Fix code comment typo in bin/steps/pipenv (#1076 )	2020-09-23 14:49:46 +01:00
Ed Morley	215a3e3670	Release v179 (#1073 )	2020-09-23 11:15:54 +01:00
Ed Morley	4505968fcb	Tests: Remove redundant Python version tests (#1075 ) There were previously 6 virtually identical tests checking the handling of a non-existent Python version being specified in `runtime.txt`. Only one is necessary - removing the rest will improve CI run time. Closes @W-8110383@. [skip changelog]	2020-09-22 18:48:31 +01:00
Ed Morley	37d1474bee	Tests: Prevent shadowing of the Python 3.7 version warning test (#1074 ) Previously the test for Python 3.8 version warnings was named the same as an earlier test for Python 3.7, meaning the earlier test definition was overwritten and so never run. The later test has now been renamed to the correct version, and the test ordering adjusted for consistency with the rest of the file. Closes @W-8110123@. [skip changelog]	2020-09-22 18:04:11 +01:00
dependabot[bot]	40167d83f6	Bump heroku_hatchet from 7.1.3 to 7.2.0 (#1071 ) Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.3 to 7.2.0. - [Release notes](https://github.com/heroku/hatchet/releases) - [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md) - [Commits](https://github.com/heroku/hatchet/compare/v7.1.3...v7.2.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2020-09-21 12:41:14 +01:00
Ed Morley	64fb396b73	Remove duplicate pipenv metric event (#1070 ) Since the `tool.pipenv` event is being emitted twice per pipenv build, inflating its usage. This whole file could do with a massive refactor (4 levels deep of conditionals is never a good sign), but that can wait until a later PR. In the future it would also be great to have testing of metrics events. Closes @W-8094963@.	2020-09-18 19:04:28 +01:00
Ed Morley	eb6ee49dfe	Emit metrics for how the Python version was chosen (#1069 ) Currently an app's Python version can be set via a few different means: - explicitly by the user (via `runtime.txt` or `Pipfile.lock`) - implicitly via the sticky versions feature (for existing apps) - implicitly via default version for new apps / those with empty cache In order to determine the priority of features like automatic Python patch version upgrades for sticky-versioned apps, it's useful to have metrics for these. There were previously no tests for either the sticky versions feature, or changing the Python version by updating the `runtime.txt` file, so I've added some now (given that I updated the conditional to add the metrics, so useful to have coverage). I've also removed the confusing overwrite of `DEFAULT_PYTHON_VERSION` with the cached version, and kept them as two separate variables. Closes @W-8099632@. Closes @W-8099645@.	2020-09-18 18:48:57 +01:00
Ed Morley	64abfb2978	Emit Python version metric events for all builds (#1066 ) Previously the metric events describing the chosen Python version were only emitted when that Python version was installed, and not when it was being used from the build cache (the common case). Now the version is emitted for all builds, improving visibility into the distribution of Python usage, and helping determine the priority of features like opt-in automatic Python patch updates. Closes @W-8059668@.	2020-09-16 12:28:15 +01:00
Ed Morley	12d1cbb3be	Tests: Switch to branch 'main' of python-getting-started repo (#1065 ) Since the default branch for that repository has been changed from `master` to `main` in W-7903771. Closes @W-8047331@. [skip changelog]	2020-09-16 12:21:53 +01:00
dependabot[bot]	7817aa3fc3	Bump heroku_hatchet from 7.1.1 to 7.1.3 (#1064 ) Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.1 to 7.1.3. - [Release notes](https://github.com/heroku/hatchet/releases) - [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md) - [Commits](https://github.com/heroku/hatchet/compare/v7.1.1...v7.1.3) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2020-09-14 08:19:46 +01:00
Ed Morley	ae56342a81	Release v178 (#1063 )	2020-09-07 13:12:36 +01:00
Ed Morley	3e49aeb940	Add support for Python 3.5.10 (#1062 ) Since it was released over the weekend: https://www.python.org/downloads/release/python-3510/ https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-10 Closes @W-7835961@.	2020-09-07 13:01:09 +01:00
dependabot[bot]	a91a5427de	Bump heroku_hatchet from 7.1.0 to 7.1.1 (#1061 ) Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.0 to 7.1.1. - [Release notes](https://github.com/heroku/hatchet/releases) - [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md) - [Commits](https://github.com/heroku/hatchet/compare/v7.1.0...v7.1.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2020-09-07 10:28:06 +01:00
dependabot[bot]	ab69658efb	Bump heroku_hatchet from 7.0.0 to 7.1.0 (#1057 ) Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.0.0 to 7.1.0. - [Release notes](https://github.com/heroku/hatchet/releases) - [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md) - [Commits](https://github.com/heroku/hatchet/compare/v7.0.0...v7.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> [skip changelog]	2020-08-24 10:24:16 +01:00
Ed Morley	dfbe8ddaf5	Release v177 (#1055 ) Closes @W-7975422@.	2020-08-18 12:50:17 +01:00
Ed Morley	60b9d1a562	Add support for Python 3.6.12 and 3.7.9 (#1054 ) Since they were released yesterday: https://www.python.org/downloads/release/python-3612/ https://www.python.org/downloads/release/python-379/ Closes @W-7975179@. Closes @W-7975181@.	2020-08-18 12:38:54 +01:00
Ed Morley	3fa3f15d35	Remove unnecessary date conditional for Python 2 EOL warning (#1053 ) Since we're past the end of 2019, so the conditional is always true. Closes @W-7952394@. [skip changelog]	2020-08-12 22:49:31 +01:00
Ed Morley	a510c47b26	Make etc/publish.sh executable (#1052 ) Since otherwise: ``` $ ./etc/publish.sh -bash: ./etc/publish.sh: Permission denied ``` Closes @W-7952258@. [skip changelog]	2020-08-12 22:48:59 +01:00
Ed Morley	1972e6094e	Travis: Switch to Ubuntu 20.04 base image (#1051 ) Since it's a smaller, more up to date image, with fewer services starting at boot, improving start times: https://blog.travis-ci.com/2020-08-10-focal-build-environment https://docs.travis-ci.com/user/reference/focal/ Closes @W-7951908@. [skip changelog]	2020-08-12 22:48:48 +01:00