Compare commits

..

109 Commits

Author SHA1 Message Date
Ed Morley d03cfe59b8 Add support for Python 3.8.7
Closes @W-7791264@.
2020-11-20 11:49:34 +00:00
Ed Morley b8e432edf1 Release v186 (#1124)
Closes @W-8440921@.
2020-11-18 13:26:39 +00:00
Ed Morley 74a6c86c4f Add a make target for running a local compile (#1123)
It's often useful to be able to run a compile outside of the unit or
Hatchet tests when developing the buildpack or debugging.

Whilst running the compile locally inside Docker won't fully replicate
a compile on the platform, for many use-cases it's close enough and
importantly gives a faster development feedback loop, since buildpack
changes don't have to be committed/pushed prior to triggering a compile.

Closes @W-8436406@.
2020-11-17 15:19:48 +00:00
Ed Morley 54115fc89b Update the BUILD_WITH_GEO_LIBRARIES error message (#1121)
So that it links to the changelog post now that it exists:
https://devcenter.heroku.com/changelog-items/1947

And to adjust the emphasis towards the migration guide
over the Geo buildpack repository link.

Closes @W-8391415@.
2020-11-13 13:10:52 +00:00
Ed Morley 6a914193b9 Remove env vars used by legacy build systems (#1120)
These variables were used by Anvil and/or the legacy `slug_compiler`
that predated cytokine.

I've not added a changelog entry since none were exported, so wouldn't
have been available to subprocesses anyway.

Closes @W-8387606@.

[skip changelog]
2020-11-12 18:40:31 +00:00
Ed Morley 71aef447a6 Migrate away from sp-grep (#1119)
Switches the last consumers of it to a simpler utility function that
uses `pkgutil.find_loader()`:
https://docs.python.org/3/library/pkgutil.html#pkgutil.find_loader

Both of these consumers are covered by existing tests.

Then removes `sp-grep` and the remaining parts of `pip-pop`.

Closes @W-8208817@.
2020-11-12 17:42:51 +00:00
Ed Morley 768d3fb9e5 Remove vendored pip-diff script (#1118)
Since the last usage was removed in #1117.

Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.

Refs @W-8208817@.
2020-11-12 13:54:14 +00:00
Ed Morley eb44bc03f1 Remove vendored pip-grep script (#1116)
Since the last usage was removed in #1113, and pip-grep is actually
broken on newer pip anyway (though helpfully silently ignores
exceptions, so one wouldn't know).

This is the first of the pip-pop removals, the rest will follow as the
last usages are switched over.

Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.

Refs @W-8208817@.
2020-11-12 13:45:27 +00:00
Ed Morley c112ef81ad Remove unused pip-uninstall step (#1117)
The pip-uninstall step has been unused for pip-using apps since #925,
since the buildpack now invalidates the entire package cache instead.

Whilst the step appears to still be used for pipenv-using apps, the
code is not run, since `SKIP_PIP_INSTALL=1` is set too early.

This bug was introduced in a334672a1a
which landed straight to `master` two days after the feature was
introduced in #650.

Longer term we should likely get pipenv installs to do something similar
to pip (invalidate the whole cache based on checksum of the lockfile),
however for now I'm removing this deadcode since it's the last consumer
of the `pip-diff` script which we want to remove.

Closes @W-8386830@.

[skip changelog]
2020-11-12 13:43:49 +00:00
Ed Morley 5012113d68 Release v185 (#1114)
Closes @W-8381914@.
2020-11-12 07:57:11 +00:00
Ed Morley ce684e4539 Make the BUILD_WITH_GEO_LIBRARIES warning fatal (#1115)
The `BUILD_WITH_GEO_LIBRARIES` feature was removed in the previous PR,
and replaced with a warning that the feature is no longer supported.

After further thought I believe it would be best to make this warning
fatal, to prevent unexpected failures at runtime, if consumers of the
library either aren't invoked during the build, or were previously
installed/cached and were dynamically linked against the library.

Continuation of #1113 / @W-7654424@.
2020-11-11 18:47:45 +00:00
Ed Morley 42076f1bf4 Remove deprecated GDAL/GEOS/PROJ support (#1113)
The standalone Geo buildpack offers more modern GDAL/GEOS/PROJ library
versions, and can be used by apps in all languages, not just Python:
https://github.com/heroku/heroku-geo-buildpack

As such the Python buildpack's undocumented built-in support was
deprecated back in April 2020, with a scheduled removal date of
6th October 2020:
https://devcenter.heroku.com/changelog-items/1759
https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-now-deprecated

Metrics show very few builds continuing to use the built-in support.

Apps with the `BUILD_WITH_GEO_LIBRARIES` env var set will now be shown a
warning directing them to the standalone buildpack, as well as apps that
hit GDAL related pip install errors but aren't using the env var.

This also moves us one step closer to being able to remove
the vendored copy of pip-pop (which is partially broken on
newer pip).

Closes @W-7654424@.
2020-11-11 12:39:30 +00:00
Ed Morley 41f657fbff Remove vendored jq binary (#1112)
Since `jq` is installed in the `-build` stack image for all stacks:
https://github.com/heroku/stack-images/pull/174

Closes @W-7974682@.
2020-11-10 15:34:56 +00:00
Ed Morley 5f6941f04a Remove redundant Mercurial install step (#1111)
Mercurial is installed in the stack image for all stacks, so the
pip install of packages from Mercurial VCS URLs works without the
need for the buildpack to install it itself.

See:
https://github.com/heroku/stack-images/pull/141
https://github.com/heroku/stack-images/search?q=mercurial

Closes @W-7906950@.
2020-11-10 14:33:52 +00:00
Ed Morley 452443d420 Remove support for the Cedar-14 stack (#1110)
Since the stack is end of life and builds have been disabled:
https://devcenter.heroku.com/changelog-items/1943

There are only two temporarily exempted customers using Python, who
can switch to the Cedar-14 support branch if they still need to build
their Python apps (most of which haven't been built recently).

Closes @W-8054727@.
2020-11-10 13:58:33 +00:00
dependabot[bot] c08cad592d Bump rspec from 3.9.0 to 3.10.0 (#1107)
Bumps [rspec](https://github.com/rspec/rspec) from 3.9.0 to 3.10.0.
- [Release notes](https://github.com/rspec/rspec/releases)
- [Commits](https://github.com/rspec/rspec/compare/v3.9.0...v3.10.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-11-10 11:51:30 +00:00
Ed Morley c9504ffd2e Release v184 (#1106) 2020-10-21 14:56:33 +01:00
Ed Morley 96822983ed Add Heroku-20 to deploy-runtimes default stacks list (#1104)
Now `make deploy-runtimes` will build binaries for Heroku-20 by default
too, without the need to pass it in via an explicit `STACKS=...`list.

Closes @W-8233698@.

[skip changelog]
2020-10-19 20:48:34 +01:00
Ed Morley f9d5c0010d Move changelog entries to the correct section (#1103)
The changelog entries for #1099 and #1100 were added to the previous
release section (rather than the "unreleased" section) by mistake.
2020-10-19 13:02:12 +01:00
Ed Morley 9b1a69a1b3 Vendor buildpack-stdlib instead of fetching from S3 (#1100)
Since fetching buildpack-stdlib from S3 has a number of disadvantages:
- it's not possible to grep the repo when trying to work out what
  something from the stdlib is doing
- shellcheck can't fully scan the code, since it similarly doesn't
  have the source
- another compile-time HTTP request that can fail due to transient
  network issues and so reduce reliability
- dependency on the security/release-process of an additional bucket

Since the stdlib is small, doesn't often change, and is not a binary,
it's a great fit for just vendoring in the repo.

The `BIN_DIR` calculation added to `bin/utils` is based on the approach
mentioned here:
https://www.ostricher.com/2014/10/the-right-way-to-get-the-directory-of-a-bash-script/

...rather than copying the version in `bin/compile`, since the latter uses
`$0` so doesn't work with sourced scripts (such as `bin/utils`).

Closes W-8094463.
2020-10-19 12:27:16 +01:00
dependabot[bot] 58dd638fb6 Bump heroku_hatchet from 7.3.1 to 7.3.3 (#1102)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.3.1 to 7.3.3.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.3.1...v7.3.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-19 12:20:27 +01:00
Ed Morley ead59ac7ff Expose BPLOG_PREFIX to sub-shells again (#1099)
In #1011 the number of buildpack variables that are exported (and so
exposed to subprocesses) was reduced, since in general we don't want
to leak buildpack internals into end-user steps such as the pre/post
compile hooks.

However since this change, any buildpack metric emitted from within
a `sub_env` wrapper (which is a buildpack-stdlib utility function)
is missing its `buildpack.python` prefix.

This is because buildpack-stdlib:
* lazy-loads `BPLOG_PREFIX` (rather than doing so when initially
  sourced, which is the approach used for `BUILDPACK_LOG_FILE`)
* doesn't check whether `BPLOG_PREFIX` is set before emitting metrics

See:
https://github.com/heroku/buildpack-stdlib/blob/v8/stdlib.sh

As a stop-gap until we either fix this in buildpack-stdlib (W-8095466),
or remove usages of the `sub_env` wrapper (since I think they are
counter-productive in this buildpack), I've added back the export
for `BPLOG_PREFIX`.

Fixes @W-8095436@.
2020-10-15 15:31:32 +01:00
Ed Morley ac8fd555b8 Release v183 (#1097) 2020-10-12 12:31:44 +01:00
dependabot[bot] f825896c4e Bump heroku_hatchet from 7.3.0 to 7.3.1 (#1095)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.3.0 to 7.3.1.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.3.0...v7.3.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-12 10:01:20 +01:00
Ed Morley fcf696b835 Add support for Heroku-20 (#968)
This adds support for the upcoming Heroku-20 stack.

The Heroku-20 Dockerfile is identical to that for Heroku-18, other
than the base image, and stack-related env var changes.

The initial Python versions made available will be those in:
https://devcenter.heroku.com/articles/python-support#supported-runtimes
https://devcenter.heroku.com/articles/python-support#supported-pypy-runtimes

...minus CPython 2.7, since it's EOL.

Which are:
* `python-3.6.12`
* `python-3.7.9`
* `python-3.8.6`
* `python-3.9.0`
* `pypy2.7-7.3.2`
* `pypy3.6-7.3.2`

Note: Unlike CPython 2.7, the PyPy 2.7 branch is still supported:
https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2

In addition, I've generated binaries for each patch release immediately
prior to the latest versions (with the exception of 3.9.0, since there
isn't one), otherwise it's not possible to run the "out of date Python"
warning tests.

The binaries were generated using the process here:
https://github.com/heroku/heroku-buildpack-python/blob/main/builds/README.md

Specifically:

```
make deploy-runtimes STACKS='heroku-20' \
  RUNTIMES='python-3.6.11 python-3.6.12 python-3.7.8 python-3.7.9 python-3.8.5 python-3.8.6 python-3.9.0 pypy2.7-7.3.1 pypy2.7-7.3.2 pypy3.6-7.3.1 pypy3.6-7.3.2' \
  ENV_FILE=...
```

Binaries for the GDAL/GEOS/PROJ feature have not been generated, since
it's deprecated and due for removal shortly:
https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-now-deprecated

Note: Like the Python 3.9.0 release, this uses the new S3 bucket, so
apps will need to be using a recent version of the buildpack in order
to build on Heroku-20:
https://devcenter.heroku.com/articles/python-support#checking-the-python-buildpack-version

Closes @W-7485877@.
2020-10-07 19:44:33 +01:00
Ed Morley a98ef91566 Tests: Clean up the Python version unit tests (#1092)
* Fixes the "Installing <version>" assertions so that they don't false
  positive against the "please upgrade to <version>" output.
* Removes modification of test fixtures during tests, since it can lead
  to failures depending on test order, and confusion when debugging.
* Updates the PyPy version warning tests to use a slightly newer (but
  still not latest) PyPy version, which means that the test now passes
  on Cedar-14 and can be unskipped.
* Switches to using an empty requirements file for version tests that
  duplicate the main test, to save spending time installing dependencies
  unnecessarily.
* Switches the NLTK test to using the default buildpack Python version,
  rather than an ancient Python 3.6.
* Skips the Python 3.7/3.8 tests on Cedar-14 rather than asserting
  failure, since we know they'll never pass due to Cedar-14's libssl being
  older than required.
* Removes redundant `testSqliteInstall` test since it duplicates the
  Python version install tests.

Longer term I'll be moving many of the unit tests to Hatchet, however
this at least makes the tests more dependable in the meantime.

Closes @W-8060219@.
Closes @W-8176779@.

[skip changelog]
2020-10-07 15:10:20 +01:00
Ed Morley e67235f906 Release v182 (#1091) 2020-10-06 10:25:07 +01:00
Ed Morley b1690e9f47 Add support for Python 3.9.0 (#1090)
https://pythoninsider.blogspot.com/2020/10/python-390-is-now-available-and-you-can.html
https://www.python.org/downloads/release/python-390/
https://docs.python.org/release/3.9.0/whatsnew/3.9.html

Binaries generated using:

```
make deploy-runtimes RUNTIMES='python-3.9.0' STACKS='heroku-16 heroku-18' ENV_FILE=...
```

Closes @W-7791272@.
2020-10-06 09:35:36 +01:00
Ed Morley b250300b74 Migrate to a new S3 bucket (#1089)
Since:
* We want the S3 bucket to be owned by a different AWS account and it's
  not possible to transfer ownership of an existing bucket.
* In the future we want to rebuild some of the Python runtime archives
  (for example to improve the sqlite3 handling, or to tweak the compile
  flags used), and it will be easier to reason about the change if we
  can guarantee only recent buildpack versions are using the assets
  rather than several year old unmaintained forks.

The assets were synced from the old bucket using (minus the `--dryrun`):

```
aws s3 sync s3://lang-python s3://heroku-buildpack-python \
  --dryrun \
  --metadata-directive REPLACE \
  --exclude "*" \
  --include 'common/*' \
  --include 'heroku-*/runtimes/*' \
  --include 'heroku-*/libraries/vendor/gdal.tar.gz' \
  --include 'heroku-*/libraries/vendor/geos.tar.gz' \
  --include 'heroku-*/libraries/vendor/proj.tar.gz' \
  --exclude 'common/pip-20.0.2-py2.py3-none-any.whl' \
  --exclude '*/runtimes/*-opt.tar.gz' \
  --exclude '*/runtimes/sqlite-free/*'
```

The files that were `--exclude`d are those that are no longer used,
or test assets that were not officially released.

The Cedar-14 assets were not migrated since it's EOL next month.

The old S3 bucket will be left untouched for the foreseeable future
(ie: we won't be deleting it), since builds using older versions of this
buildpack (either due to pinning to a tag or via a fork) will still be
using assets from it.

Closes @W-8060097@.
2020-10-06 09:23:38 +01:00
dependabot[bot] 096709dcf7 Bump bob-builder from 0.0.18 to 0.0.19 (#1088)
Bumps [bob-builder](https://github.com/heroku-python/bob-builder) from 0.0.18 to 0.0.19.
- [Release notes](https://github.com/heroku-python/bob-builder/releases)
- [Commits](https://github.com/heroku-python/bob-builder/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-05 08:47:36 +01:00
Ed Morley ff8945c0c2 Tests: Make assertCapturedSuccess check stderr is empty again (#1087)
`assertCapturedSuccess` used to check that `stderr` was empty, until:
https://github.com/heroku/heroku-buildpack-python/commit/797652a75d69a1fece96a26bf4514fe9e9e1c020#diff-65c067a6f0a3aef292fb54ec21a1fe8cR98

Adding back the assertion exposes some new bugs, which I'll fix in
later PRs. In the meantime affected tests have been adjusted to use
a new `assertCapturedSuccessWithStdErr`.

The test harness output for the failing case has also been improved,
to ease debugging.

Closes @W-7918492@.

[skip changelog]
2020-10-01 17:29:13 +01:00
Ed Morley 4e78b5d57c Remove duplicate copy of shunit2 (#1086)
The tests use `test/shunit2` so the copy under `vendor/` is redundant.

I've added to the CHANGELOG since `vendor/` is added by the buildpack to
`PATH` (something long term we should stop doing), increasing the chance
an app was accidentally relying upon `shunit2` (albeit still unlikely).

Closes @W-8145517@.
2020-10-01 16:02:09 +01:00
Ed Morley b74a41395e Refactor S3 asset URL handling (#1085)
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.

In order to:
* reduce this duplication
* simplify this buildpack's S3 bucket migration (where we'll soon be
  needing the vary the bucket name and wouldn't want to have to
  duplicate that logic in multiple places)
* allow overriding of the URL for the pip use-case

...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.

The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see #989).

The unused `USE_STAGING_BINARIES` environment variable has been
removed, since it's a leftover from the project to stand up a staging S3 bucket.
It's redundant given the `BUILDPACK_S3_BASE_URL` variable.

Closes @W-8142401@.
2020-10-01 10:13:26 +01:00
Ed Morley c550143a59 Use 'rm -rf' instead of 'rm -fr' (#1084)
Not super urgent, but seeing as it closes #927, might as well do now.

[skip changelog]
2020-09-29 15:34:13 +01:00
Ed Morley 838f4c125b Simplify binary build process and documentation (#1083)
* Moves all manual build steps to make targets, to simplify
  the commands run, and reduce chance for error.
* Removes the need to remember to rebuild the builder
  image by building it automatically prior to launching.
* Adds a new make target for deploying multiple runtime
  versions at once to speed up the common case.
* Reduces repetition/superfluous content in documentation.
* Removes unused `S3_REGION` from `dockerenv.default`
  (the contents of S3 buckets inherit the region of the bucket).
* Documents build dependencies in `requirements.txt`.

Closes @W-8119717@.

[skip changelog]
2020-09-29 13:40:56 +01:00
Ed Morley 0020aae078 Release v181 (#1082)
Closes @W-8133372@.
2020-09-29 13:14:41 +01:00
Ed Morley 63651e042f Add support for PyPy 2.7 and 3.6 version 7.3.2 (#1081)
Since they were released yesterday:
https://morepypy.blogspot.com/2020/09/pypy-732-triple-release-python-27-36.html

The archive URL had to be updated now that PyPy has migrated from BitBucket.
The new URLs are from:
https://www.pypy.org/download.html

Skipping PyPy 3.7 for now, since it's in alpha.

Closes @W-8128094@.
2020-09-29 09:30:51 +01:00
Ed Morley c0609a881c Clean up binary generation Dockerfiles (#1080)
* Switches from using Python 2 to Python 3 for `bob-builder`
* Adds `--no-install-recommends` to reduce unnecessary packages
* Removes unnecessary pinning of pip for Heroku-18 build
* Moves `DEBIAN_FRONTEND` to RUN level env var, given:
   https://serverfault.com/a/797318

Refs @W-8119717@.

[skip changelog]
2020-09-29 09:29:33 +01:00
dependabot[bot] ef4696a922 Bump heroku_hatchet from 7.2.0 to 7.3.0 (#1079)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.2.0 to 7.3.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.2.0...v7.3.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-28 10:38:03 +01:00
Ed Morley 6c612a7fb1 Release v180 (#1078)
Closes @W-8120452@.
2020-09-24 19:17:18 +01:00
Ed Morley f91f4ee4ce Add support for Python 3.8.6 (#1072)
https://www.python.org/downloads/release/python-386/
https://pythoninsider.blogspot.com/2020/09/python-386-is-now-available.html

Closes @W-7791243@.
2020-09-24 18:55:27 +01:00
Eric Carmichael eea9a7fd0f Fix code comment typo in bin/steps/pipenv (#1076) 2020-09-23 14:49:46 +01:00
Ed Morley 215a3e3670 Release v179 (#1073) 2020-09-23 11:15:54 +01:00
Ed Morley 4505968fcb Tests: Remove redundant Python version tests (#1075)
There were previously 6 virtually identical tests checking the handling of
a non-existent Python version being specified in `runtime.txt`. 

Only one is necessary - removing the rest will improve CI run time.

Closes @W-8110383@.

[skip changelog]
2020-09-22 18:48:31 +01:00
Ed Morley 37d1474bee Tests: Prevent shadowing of the Python 3.7 version warning test (#1074)
Previously the test for Python 3.8 version warnings was named the same
as an earlier test for Python 3.7, meaning the earlier test definition
was overwritten and so never run.

The later test has now been renamed to the correct version, and the
test ordering adjusted for consistency with the rest of the file.

Closes @W-8110123@.

[skip changelog]
2020-09-22 18:04:11 +01:00
dependabot[bot] 40167d83f6 Bump heroku_hatchet from 7.1.3 to 7.2.0 (#1071)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.3 to 7.2.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.3...v7.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-21 12:41:14 +01:00
Ed Morley 64fb396b73 Remove duplicate pipenv metric event (#1070)
Since the `tool.pipenv` event is being emitted twice per pipenv build,
inflating its usage.

This whole file could do with a massive refactor (4 levels deep of
conditionals is never a good sign), but that can wait until a later PR.

In the future it would also be great to have testing of metrics events.

Closes @W-8094963@.
2020-09-18 19:04:28 +01:00
Ed Morley eb6ee49dfe Emit metrics for how the Python version was chosen (#1069)
Currently an app's Python version can be set via a few different means:
- explicitly by the user (via `runtime.txt` or `Pipfile.lock`)
- implicitly via the sticky versions feature (for existing apps)
- implicitly via default version for new apps / those with empty cache

In order to determine the priority of features like automatic Python
patch version upgrades for sticky-versioned apps, it's useful to have
metrics for these.

There were previously no tests for either the sticky versions feature,
or changing the Python version by updating the `runtime.txt` file, so
I've added some now (given that I updated the conditional to add the
metrics, so useful to have coverage).

I've also removed the confusing overwrite of `DEFAULT_PYTHON_VERSION`
with the cached version, and kept them as two separate variables.

Closes @W-8099632@.
Closes @W-8099645@.
2020-09-18 18:48:57 +01:00
Ed Morley 64abfb2978 Emit Python version metric events for all builds (#1066)
Previously the metric events describing the chosen Python version were
only emitted when that Python version was installed, and not when it
was being used from the build cache (the common case).

Now the version is emitted for all builds, improving visibility into
the distribution of Python usage, and helping determine the priority
of features like opt-in automatic Python patch updates.

Closes @W-8059668@.
2020-09-16 12:28:15 +01:00
Ed Morley 12d1cbb3be Tests: Switch to branch 'main' of python-getting-started repo (#1065)
Since the default branch for that repository has been changed from
`master` to `main` in W-7903771.

Closes @W-8047331@.

[skip changelog]
2020-09-16 12:21:53 +01:00
dependabot[bot] 7817aa3fc3 Bump heroku_hatchet from 7.1.1 to 7.1.3 (#1064)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.1 to 7.1.3.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.1...v7.1.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-14 08:19:46 +01:00
Ed Morley ae56342a81 Release v178 (#1063) 2020-09-07 13:12:36 +01:00
Ed Morley 3e49aeb940 Add support for Python 3.5.10 (#1062)
Since it was released over the weekend:
https://www.python.org/downloads/release/python-3510/
https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-10

Closes @W-7835961@.
2020-09-07 13:01:09 +01:00
dependabot[bot] a91a5427de Bump heroku_hatchet from 7.1.0 to 7.1.1 (#1061)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.0 to 7.1.1.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.0...v7.1.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-07 10:28:06 +01:00
dependabot[bot] ab69658efb Bump heroku_hatchet from 7.0.0 to 7.1.0 (#1057)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.0.0...v7.1.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[skip changelog]
2020-08-24 10:24:16 +01:00
Ed Morley dfbe8ddaf5 Release v177 (#1055)
Closes @W-7975422@.
2020-08-18 12:50:17 +01:00
Ed Morley 60b9d1a562 Add support for Python 3.6.12 and 3.7.9 (#1054)
Since they were released yesterday:
https://www.python.org/downloads/release/python-3612/
https://www.python.org/downloads/release/python-379/

Closes @W-7975179@.
Closes @W-7975181@.
2020-08-18 12:38:54 +01:00
Ed Morley 3fa3f15d35 Remove unnecessary date conditional for Python 2 EOL warning (#1053)
Since we're past the end of 2019, so the conditional is always true.

Closes @W-7952394@.

[skip changelog]
2020-08-12 22:49:31 +01:00
Ed Morley a510c47b26 Make etc/publish.sh executable (#1052)
Since otherwise:

```
$ ./etc/publish.sh
-bash: ./etc/publish.sh: Permission denied
```

Closes @W-7952258@.

[skip changelog]
2020-08-12 22:48:59 +01:00
Ed Morley 1972e6094e Travis: Switch to Ubuntu 20.04 base image (#1051)
Since it's a smaller, more up to date image, with fewer services
starting at boot, improving start times:
https://blog.travis-ci.com/2020-08-10-focal-build-environment
https://docs.travis-ci.com/user/reference/focal/

Closes @W-7951908@.

[skip changelog]
2020-08-12 22:48:48 +01:00
Ed Morley 84ac34b1d4 Release v176 (#1050) 2020-08-12 16:17:33 +01:00
Ed Morley 7c4f55bc32 Switch the remaining build formulas to the shared script approach (#1049)
To be consistent with the other build formulas after #799.

Refs @W-7947035@.

[skip changelog]
2020-08-12 15:19:53 +01:00
Ed Morley eabe71d578 Update the Python 3.4.10 build script to use the correct Python version (#1048)
The existing Python 3.4.10 archive actually contained Python 3.7.2,
since the version in the source URL was not updated when the file was
created in #813.

The build formula now uses the shared build script approach like all of
the other build scripts, which ensures the version can never get out of
sync (since it's extracted from the formula filename).

The build for Heroku-18 failed to compile `_ssl` properly (even though
the build exited zero) since Python 3.4.10 is old enough it doesn't work
well with libssl1.1. Installing `libssl1.0-dev` in the build image
locally resolved the issue - however we don't want to use that in the
future for newer Python, so I've not updated the `heroku-18.Dockerfile`.

In addition, with the rebuilt archives the tests now pass on Cedar-14,
so no longer need to be marked as failing.

Closes @W-7947035@.
2020-08-12 15:19:31 +01:00
Ed Morley 95fca53d38 Travis: Correctly skip Hatchet on PRs from forks (#1045)
The Hatchet run requires a valid Heroku login, the credentials for which
are set via Travis secure environment variables, which by design are not
revealed to PRs from forks.

The previous conditional wasn't working as intended - the Hatchet job
was still being triggered for forks from PRs.

The new conditional fixes this, and also means that forks could set
their own credentials via Travis environment variables if they wanted
a way to run the tests in CI on their own repo.

See:
https://docs.travis-ci.com/user/conditions-v1

Fixes #1019.
Closes @W-7918482@.

[skip changelog]
2020-08-12 11:17:38 +01:00
Ed Morley ff4f677435 Travis: Remove Hatchet credentials from .travis.yml (#1047)
Since they are now set via the Travis repository level secrets feature
instead. This both works around the Travis bug seen in #1045, and also
means its easier to set up Travis on forks, since otherwise the
`.travis.yml` secrets would overwrite the global secrets.

As part of this move the test account used has also been changed, and
will be documented here:
https://github.com/heroku/languages-team/blob/main/guides/create_test_users_for_ci.md#known-usernames

Closes @W-7949880@.

[skip changelog]
2020-08-12 10:49:57 +01:00
Ed Morley cba3201ac2 Rename confusing PYPY_* variables (#1044)
Since these variables refer to the latest version of PyPy, compared to
the similarly named `PYPY27` and `PYPY36` variables (ie same name except
without the underscore) which refer to the major/minor version only.

The similar names caused me to use the wrong one locally whilst working
on another PR, which was caught by tests but demonstrates why we should
rename them.

Closes @W-7935256@.

[skip changelog]
2020-08-11 19:31:53 +01:00
Ed Morley ac29db32f8 Remove unused vendor/test-utils (#1043)
Since the unit tests instead use the utilities in this separate file:
https://github.com/heroku/heroku-buildpack-python/blob/419ef479969c4d5945f2c0620292229ef464f4c8/test/utils

A changelog entry has been added since whilst this file is for internal
testing only, the buildpack's `vendor/` directory is put on `PATH`, so
in theory it could have been called outside the buildpack (though this
seems extremely unlikely since the script isn't very useful externally).

Fixes #1027.
Closes @W-7918496@.
2020-08-11 19:31:07 +01:00
Ed Morley e83576f6b4 Remove unused .pre-commit-config.yaml (#1042)
Since we don't use that tool (<https://pre-commit.com>), and there are
better alternatives should we want to expand coverage of these kind of
things.

Closes @W-7923935@.

[skip changelog]
2020-08-11 19:21:40 +01:00
Ed Morley 43600f25a5 Remove unused etc/ci-setup.sh (#1041)
It stopped being used as of #781.

```
$ rg ci-setup.sh --stats
0 matches

$ git-content-search ci-setup.sh
e7da63f update to newer hatchet integration
M      .travis.yml
576def4 fix travis dependency blocker
M      .travis.yml

$ git show e7da63f | rg ci-setup.sh -C 1
-before_install:
- - sudo bash etc/ci-setup.sh
+ - bundle exec hatchet ci:setup
```

Hatchet embeds its own setup script, which is called via the rake task:
https://github.com/heroku/hatchet/blob/v6.0.0/etc/ci_setup.rb

Closes @W-7923930@.

[skip changelog]
2020-08-11 19:21:03 +01:00
Ed Morley f508bd538d Fix the security update version check message for PyPy (#1040)
Previously if an app was using an older version of PyPy, the buildpack
would show a confusing "Could not find that version" message (even
though the version was found), when it really meant to warn about there
being a newer release available.

It looks like the version check messages were perhaps copied and pasted
from something else, but the message wording not updated at the time.

I've also added tests since there were none for this feature.

Fixes #1004.
Closes @W-7918745@.
2020-08-11 19:15:16 +01:00
Richard Schneeman bc7e34dbad Tests: Update Hatchet from 6.0.0 to 7.0.0 (#1046)
- ActiveSupport's Object#blank? and Object#present? are no longer provided by default (https://github.com/heroku/hatchet/pull/107)
- Remove deprecated support for passing a block to `App#run` (https://github.com/heroku/hatchet/pull/105)
- Ignore  403 on app delete due to race condition (https://github.com/heroku/hatchet/pull/101)
- The hatchet.lock file can now be locked to "main" in addition to "master" (https://github.com/heroku/hatchet/pull/86)
- Allow concurrent one-off dyno runs with the `run_multi: true` flag on apps (https://github.com/heroku/hatchet/pull/94)
- Apps are now marked as being "finished" by enabling maintenance mode on them when `teardown!` is called. Finished apps can be reaped immediately (https://github.com/heroku/hatchet/pull/97)
- Applications that are not marked as "finished" will be allowed to live for a HATCHET_ALIVE_TTL_MINUTES duration before they're deleted by the reaper to protect against deleting an app mid-deploy, default is seven minutes (https://github.com/heroku/hatchet/pull/97)
- The HEROKU_APP_LIMIT env var no longer does anything, instead hatchet application reaping is manually executed if an app cannot be created (https://github.com/heroku/hatchet/pull/97)
- App#deploy without a block will no longer run `teardown!` automatically (https://github.com/heroku/hatchet/pull/97)
- Calls to `git push heroku` are now rate throttled (https://github.com/heroku/hatchet/pull/98)
- Calls to `app.run` are now rate throttled (https://github.com/heroku/hatchet/pull/99)
- Deployment now raises and error when the release failed (https://github.com/heroku/hatchet/pull/93)

[skip changelog]
2020-08-11 19:06:18 +01:00
dependabot[bot] 419ef47996 Bump bob-builder from 0.0.17 to 0.0.18 (#1039)
Bumps [bob-builder](https://github.com/heroku-python/bob-builder) from 0.0.17 to 0.0.18.
- [Release notes](https://github.com/heroku-python/bob-builder/releases)
- [Commits](https://github.com/heroku-python/bob-builder/compare/v0.0.17...v0.0.18)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-10 14:25:42 +01:00
dependabot[bot] c1f6679dd7 Bump boto from 2.48.0 to 2.49.0 (#1038)
Bumps [boto](https://github.com/boto/boto) from 2.48.0 to 2.49.0.
- [Release notes](https://github.com/boto/boto/releases)
- [Commits](https://github.com/boto/boto/compare/2.48.0...2.49.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-10 13:11:48 +01:00
Ed Morley de81e41336 Add config file for Dependabot (#1037)
GitHub are gradually integrating Dependabot into GitHub as a native
feature. The native integration now supports updating not only security
updates, but all dependencies - however this must be explicitly enabled
by adding a config file:
https://docs.github.com/en/github/administering-a-repository/enabling-and-disabling-version-updates

The config options are described here:
https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates

Closes @W-7937066@.

[skip changelog]
2020-08-10 12:30:32 +01:00
Ed Morley 99c1027cb2 Tests: Update Hatchet dependencies (#1036)
* `bundler update`
* Ruby 2.6 -> 2.7
* Remove unnecessary bundler version downgrade

Of note this picks up:
https://github.com/grosser/parallel_split_test/pull/20

@W-7930909@

[skip changelog]
2020-08-10 11:29:34 +01:00
Ed Morley 46d3d2b042 Travis: Move hatchet ci:setup to before_script (#1035)
So that any failures during `hatchet ci:setup` cause the build to fail
early, rather than try to proceed with running the Hatchet tests.

@W-7929878@

[skip changelog]
2020-08-07 17:53:33 +01:00
Ed Morley 7b4f16145b Add a configuration file for Git2Gus (#1034)
This will allow for one-way sync of GitHub issues in this repository
into our internal issues tracker, GUS. Issues are only synced when the
specified GitHub label is added.

In the future I may switch the chosen label to just be the standard
`t: bug` type labels, but for now I'm choosing a separate label so that
we have more control over what is synced.

See:
https://lwc-gus-bot.herokuapp.com/#getting-started

@W-7918433@
2020-08-05 10:48:25 +01:00
Ed Morley a165deadfb Release v175 (#1032) 2020-08-05 09:12:16 +01:00
Ed Morley b64897a0b7 Fix inconsistent formatting in CHANGELOG.md (#1031)
- Switches releases to always using H2 (`##`) rather than a mixture of
  H1s (meaning multiple H1s in the same document) and H2s.
- Always refers to the versions as `vNNN` rather than sometimes without
  the `v` prefix.
- Other formatting and typo fixes.

@W-7905079@
2020-08-03 20:18:18 +01:00
Ed Morley fc6698e597 Update pip to 20.1.1 (#1030)
Updates pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+:
https://pip.pypa.io/en/stable/news/#id40

The version used for Python 3.4 remains unchanged at 19.1.1, since it's
the last version of pip that supports it.

Pip has been updated to 20.1.1 rather than the recently released 20.2,
since the latter has a few regressions and even though these will be
fixed shortly in 20.2.1, we should let the changes soak for longer
before picking them up.

The `PIP_NO_PYTHON_VERSION_WARNING` environment variable has been set
(equivalent to passing `--no-python-version-warning`) to prevent the
Python 2.7 EOL warnings added in pip 20.1 from spamming the build log:
https://github.com/pypa/pip/blob/20.1.1/src/pip/_internal/cli/base_command.py#L139-L154

This was set via environment variable rather than CLI flag, since:
* otherwise we'd have to pass it to every pip invocation
* older pip (such as the 19.1.1 used by Python 3.4) doesn't support this
  option and would error out due to an unknown CLI flag being passed,
  unless we added conditional flags throughout.

The new pip wheel was uploaded to S3 using:

```
$ pip download --no-cache pip==20.1.1
Collecting pip==20.1.1
  Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
  Saved ./pip-20.1.1-py2.py3-none-any.whl
Successfully downloaded pip

$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun
(dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl

$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read
upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl
```

Fixes #1005.
@W-7659489@
2020-08-03 19:45:41 +01:00
Ed Morley 6fa6feb75d Update setuptools (#1024)
Upgrades setuptools from 39.0.1 to:
- 44.1.1 for Python 2.7 (since it's the last supported version)
- 43.0.0 for Python 3.4 (since it's the last supported version)
- 47.1.1 for Python 3.5+ (since we can't use 47.2.0+ until #1006 fixed)

https://setuptools.readthedocs.io/en/latest/history.html#v47-1-1

Fixes #949.
Closes #973.
2020-08-03 18:36:01 +01:00
Ed Morley 81874dad47 Replace 'master' branch references with 'main' (#1029)
For #1028.
2020-08-03 17:31:08 +01:00
Ed Morley d74880b322 Release v174 (#1023) 2020-07-30 09:15:44 +01:00
Ed Morley 00e70fffc9 Correctly handle failed pip/setuptools/wheel installs (#1007)
They are now displayed in the build output (instead of being sent to
`/dev/null`) and fail the build early instead of failing later in
`bin/steps/pip-install`.

Fixes #1002.
2020-07-29 19:11:35 +01:00
Ed Morley 60f2fac8e1 Disable pip's version check + cache when installing pip/setuptools/wheel (#1007)
Since the version check is redundant given we control/choose the version.

The pip cache is redundant since we instead cache site-packages. The pip
cache also ends up in `/app` so isn't included in the build cache anyway.
2020-07-29 19:11:35 +01:00
Ed Morley 405c7651ea Install pip using itself rather than get-pip.py (#1007)
`get-pip.py` is no longer used, since:
- It uses `--force-reinstall`, which is unnecessary here and slows down
  repeat builds (given we call pip install every time now). Trying to
  work around this by using `get-pip.py` only for the initial install,
  and real pip for subsequent updates would mean we lose protection
  against cached broken installs, plus significantly increase the
  version combinations test matrix.
- It means downloading pip twice (once embedded in `get-pip.py`, and
  again during the install, since `get-pip.py` can't install the
  embedded version directly).
- We would still have to manage several versions of `get-pip.py`, to
  support older Pythons (once we upgrade to newer pip).

We don't use `ensurepip` since:
- not all of the previously generated Python runtimes on S3 include it.
- we would still have to upgrade pip/setuptools afterwards.
- the versions of pip/setuptools bundled with ensurepip differ greatly
  depending on Python version, and we could easily start using a CLI
  flag for the first pip install before upgrade that isn't supported on
  all versions, without even knowing it (unless we test against hundreds
  of Python archives).

Instead we install pip using itself in wheel form. See:
https://github.com/pypa/pip/issues/2351#issuecomment-69994524

The new pip wheel assets on S3 were generated using:

```
$ pip download --no-cache pip==19.1.1
Collecting pip==19.1.1
  Downloading pip-19.1.1-py2.py3-none-any.whl (1.4 MB)
  Saved ./pip-19.1.1-py2.py3-none-any.whl
Successfully downloaded pip

$ pip download --no-cache pip==20.0.2
Collecting pip==20.0.2
  Downloading pip-20.0.2-py2.py3-none-any.whl (1.4 MB)
  Saved ./pip-20.0.2-py2.py3-none-any.whl
Successfully downloaded pip

$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun
(dryrun) upload: ./pip-19.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-19.1.1-py2.py3-none-any.whl
(dryrun) upload: ./pip-20.0.2-py2.py3-none-any.whl to s3://lang-python/common/pip-20.0.2-py2.py3-none-any.whl

$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read
upload: ./pip-19.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-19.1.1-py2.py3-none-any.whl
upload: ./pip-20.0.2-py2.py3-none-any.whl to s3://lang-python/common/pip-20.0.2-py2.py3-none-any.whl
```
2020-07-29 19:11:35 +01:00
Ed Morley 7279ddded8 Always check/adjust the installed versions of setuptools/wheel (#1007)
Previously the pip/setuptools/wheel install step was skipped so long
as Python hadn't just been clean installed (ie so long as not a new app,
emptied cache, Python upgrade, stack change) and pip was the expected
version.

This meant that setuptool/wheel could be the wrong version (or even just
not installed at all), and this would not be corrected.

Now, we now use pip itself to determine whether the installed packages
are up to date, since parsing pip's output is fragile (eg #1003) and
would be tedious given there would be three packages to check.

Unfortunately `get-pip.py` uses `--force-reinstall` which means
performing this step every time is not the no-op it would otherwise be,
but this will be resolved by switching away from `get-pip.py` in the
next commit.

Fixes #1000.
Fixes #1003.
Closes #999.
2020-07-29 19:11:35 +01:00
Ed Morley 0027f23065 Remove redundant site-packages cleanup steps (#1007)
Since `get-pip.py` / pip will automatically detect and remove old
pip/setuptools versions if needed, so removing them manually is both not
necessary and slows down the build in the case where the pip version
changed, but setuptools remained the same.
2020-07-29 19:11:35 +01:00
Ed Morley 2097eab028 Install an explicit version of wheel rather than latest (#1007)
Before:
- if `wheel` was not already installed, then `get-pip.py` would
  automatically install the latest version on PyPI, which is `0.34.2`
  (or `0.33.6` for Python 3.4).
- if `wheel` was already installed, then it was left unchanged
  regardless of the version installed.

Now:
- if `wheel` is not already installed, then the same versions will be
  installed as before, except these versions are pinned and will now not
  change unexpectedly after future `wheel` releases.
- if `wheel` is already installed, then it's upgraded/downgraded to the
  target version as needed.

Partly addresses #1000, though this change only helps builds where the
pip/setuptools/wheel install flow is triggered (currently only new apps
or ones where Python was purged or pip was not the correct version).

Since the wheel version is now known, it's output to the build log to
ease debugging and for parity with pip/setuptools.

The rest of #1000 will be fixed in later commits.
2020-07-29 19:11:35 +01:00
Ed Morley 46581612fc Install pip and setuptools in the same pip invocation (#1007)
`get-pip.py` installs setuptools itself (if it's not already installed):
https://pip.pypa.io/en/stable/installing/#installing-with-get-pip-py
https://github.com/pypa/get-pip/blob/eff16c878c7fd6b688b9b4c4267695cf1a0bf01b/templates/default.py#L152-L153

This means that previously the latest version of setuptools (currently
`49.2.0`) was being installed from PyPI, and then immediately after the
target version (currently `39.0.1`) installed over it.

This added time to the build unnecessarily.

The version of setuptools installed by `get-pip.py` can be overridden
by passing in a version as a normal requirements specifier.

Fixes #1001.
2020-07-29 19:11:35 +01:00
Ed Morley 31e8f48db8 Install setuptools from PyPI rather than a vendored copy (#1007)
Since:
* we'll be updating setuptools soon, and newer setuptools has dropped
  support for Python versions this buildpack needs to support. As such
  if we continued to vendor setuptools, we would need to vendor at
  least three different versions.
* we want to try and update setuptools more frequently than we have
  in the past, which will mean more repo bloat from binary churn.
* we're still pinning to a specific version, meaning vendoring doesn't
  have determinism benefits.
* setuptools is only fetched from PyPI for new installs (or where
  versions have changed), so this doesn't increase build time, load on
  PyPI, or reliance on PyPI in the common case.
* setuptools is already being inadvertently installed from PyPI prior to
  being installed from the vendored copy (see #1001), so we're in effect
  already using/depending on PyPI here.
* switching to storing setuptools on S3 wouldn't help reliability as
  much as it would appear at first glance, since the later `pip install`
  of customer dependencies will fail if PyPI is down anyway.
2020-07-29 19:11:35 +01:00
Ed Morley 47a8b4b3b9 Output the installed version of setuptools in the build log (#1007)
Since:
* "explicit is better than implicit"
* we'll soon be upgrading setuptools, and debugging breakage caused by
  upgrades will be easier if versions are visible in the build log
2020-07-29 19:11:35 +01:00
Ed Morley 4080587538 Move Pip version handling to bin/steps/python (#1007)
And use the `$PYTHON_VERSION` calculated in `bin/steps/python` instead
of re-implementing the Python version handling.
2020-07-29 19:11:35 +01:00
Ed Morley 157ce25694 Output the installed version of pip in the build log (#1007)
Since:
* "explicit is better than implicit"
* we'll soon be upgrading pip, and debugging breakage caused by upgrades
  will be easier if versions are visible in the build log

Closes #939.
2020-07-29 19:11:35 +01:00
Ed Morley e7c7dfdb26 Reduce the number of env vars exposed to subprocess (#1011)
The following env vars are no longer exposed to subprocesses run by the
buildpack (such as the `bin/pre_compile` and `bin/post_compile` hooks):

* `BPLOG_PREFIX`
* `CACHED_PYTHON_STACK`
* `DEFAULT_PYTHON_STACK`
* `DEFAULT_PYTHON_VERSION`
* `LATEST_27`
* `LATEST_34`
* `LATEST_35`
* `LATEST_36`
* `LATEST_37`
* `LATEST_38`
* `PIP_UPDATE`
* `PY27`
* `PY34`
* `PY35`
* `PY36`
* `PY37`
* `PYPY_27`
* `PYPY_36`
* `RECOMMENDED_PYTHON_VERSION`
* `WARNINGS_LOG`

There were previously no tests at all for the pre/post-compile hooks,
so I've added some now.

Fixes #1010.
2020-07-28 18:12:08 +01:00
Ed Morley aa8a0f43bb Travis: Reduce end to end testing time (#1022)
This change (along with #1021, which skips an unnecessary docker build)
reduces the wall clock time from ~22 minutes to ~6 minutes. Even with
the additional overhead from increased parallelism, the combined job
duration (~50 minutes) has not increased due to the other time savings.

Changes:

- for the unit tests, each stack is now tested in its own job and so
  tested in parallel
- the use of Travis stages has been removed, since by design it blocks
  later tasks on earlier stages having completed - reducing parallelism
  unnecessarily for this use case
- all jobs except for the Hatchet job now use Travis' `minimal` image,
  and no longer install redundant Ruby + bundler
- the `sudo: {required,false}` references have been removed, since
  Travis no longer supports its non-sudo container infrastructure so
  ignores that option

Fixes #1018.

[skip changelog]
2020-07-28 16:18:45 +01:00
Ed Morley 93a5b4021d Tests: Support running only a subset of the test suites (#1021)
Previously `make test` ran all unit test suites against all stacks, which
would take up to an hour locally. This could be sped up by using one of
the stack-specific targets (such as `make test-heroku-18`), however
there was still no way to only run one of the test suites.

Now `make test` can be controlled more precisely using optional `STACK`
and `TEST_CMD` arguments, eg:

`make test STACK=heroku-16 TEST_CMD=test/versions`

Travis has now been made to use this feature, which unblocks future
Travis speedups (such as splitting the jobs up further in #1018) and
means on Travis the correct Docker image is now used (see #958).

The `tests.sh` script has been removed since it's unused after #839 and
redundant given the make targets.

Fixes #958.
Fixes #1020.
2020-07-28 15:08:20 +01:00
Ed Morley f21e538fde Tests: Delete redundant detect(), compile() and release() (#1017)
Since they are all shadowed by functions with the same name later in
the file.

Fixes #1013.

[skip changelog]
2020-07-24 18:19:01 +01:00
Ed Morley a97da6382f Tests: Run detect/compile/release in a clean environment (#1016)
To prevent external environment variables from leaking into the tests,
which otherwise causes problems trying to write tests for #1011.

Several tests which were relying on this leak had to be fixed, so that
the env vars they were using are set using `ENV_DIR`, as happens in
production.

Fixes #1014.
Fixes #1015.
2020-07-24 18:10:11 +01:00
Ed Morley d9b1c73f63 Improve the Check Changelog GitHub action (#1009)
It now:
* uses the PR body rather than title for controlling whether to skip the
  check, to reduce PR title noise
* supports `[skip changelog]` in addition to `[changelog skip]`, since I
  could never remember which way around the words should go
* uses the GitHub `if` syntax with the `github` event context, which
  simplifies the implementation, means the action doesn't run steps like
  git checkout when it's going to be skipped anyway, and gives the
  status check the grey icon when skipped instead of the green check
* renames the inner job from `build` to `check`

The file has also been style-formatted using Prettier.

See:
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idif
https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions
https://docs.github.com/en/developers/webhooks-and-events/webhook-events-and-payloads#pull_request

Example failing (since no skip syntax used):
https://github.com/heroku/heroku-buildpack-python/pull/1009/checks?check_run_id=906015118

Example skipped (since skip syntax used):
https://github.com/heroku/heroku-buildpack-python/pull/1009/checks?check_run_id=906022522
2020-07-24 10:34:27 +01:00
Ed Morley 7209bb4483 Remove get-pip.py reference from NOTICE (#1008)
Since `get-pip.py` is no longer distributed with the buildpack.

Leftover from #840.
2020-07-24 10:10:50 +01:00
Ed Morley 156b07ce2b Release v173 (#998) 2020-07-21 11:31:01 +01:00
Rust Saiargaliev e288ed5a9e Add support for CPython 3.8.5 (#996)
https://www.python.org/downloads/release/python-385/

Release contains a couple of security bugfixes.
Changelog: https://docs.python.org/release/3.8.5/whatsnew/changelog.html#changelog
2020-07-21 11:01:31 +01:00
Ed Morley 181e3395f9 Release v172 (#995) 2020-07-17 12:03:37 +01:00
Denis Cornehl 013ba6b1d9 Add support for Python 3.8.4 (#993) 2020-07-17 10:19:21 +01:00
Ed Morley bce5bf4869 Release v171 (#991) 2020-07-07 19:20:13 +01:00
Denis Cornehl 0fdb62faa9 Add support for Python 3.6.11 and 3.7.8 (#988) 2020-07-07 18:39:44 +01:00
dependabot[bot] 42507a3f9a Bump rake from 12.3.1 to 12.3.3 (#981)
Bumps [rake](https://github.com/ruby/rake) from 12.3.1 to 12.3.3.
- [Release notes](https://github.com/ruby/rake/releases)
- [Changelog](https://github.com/ruby/rake/blob/master/History.rdoc)
- [Commits](https://github.com/ruby/rake/compare/v12.3.1...v12.3.3)
2020-05-27 14:14:43 +01:00
dependabot[bot] f89ee6750e Bump activesupport from 6.0.2.2 to 6.0.3.1 (#980)
Bumps [activesupport](https://github.com/rails/rails) from 6.0.2.2 to 6.0.3.1.
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v6.0.3.1/activesupport/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v6.0.2.2...v6.0.3.1)
2020-05-27 13:42:59 +01:00
119 changed files with 1192 additions and 3310 deletions
+6
View File
@@ -0,0 +1,6 @@
{
"productTag": "a1aB0000000MR0RIAW",
"issueTypeLabels": { "gus: story": "USER STORY", "gus: bug": "BUG P3" },
"defaultBuild": "Heroku Unscheduled",
"statusWhenClosed": "CLOSED"
}
+15
View File
@@ -0,0 +1,15 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
labels:
- "c: dependencies"
- package-ecosystem: "bundler"
directory: "/"
schedule:
interval: "weekly"
labels:
- "c: dependencies"
+12 -9
View File
@@ -1,14 +1,17 @@
name: Check Changelog name: Check Changelog
on: on:
pull_request: pull_request:
types: [opened, reopened, edited, synchronize] types: [opened, reopened, edited, synchronize]
jobs: jobs:
build: check:
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: if: |
- uses: actions/checkout@v1 !contains(github.event.pull_request.body, '[skip changelog]') &&
- name: Check that CHANGELOG is touched !contains(github.event.pull_request.body, '[changelog skip]') &&
run: | !contains(github.event.pull_request.body, '[skip ci]')
cat $GITHUB_EVENT_PATH | jq .pull_request.title | grep -i '\[\(\(changelog skip\)\|\(ci skip\)\)\]' || git diff remotes/origin/${{ github.base_ref }} --name-only | grep CHANGELOG.md steps:
- uses: actions/checkout@v1
- name: Check that CHANGELOG is touched
run: git diff remotes/origin/${{ github.base_ref }} --name-only | grep CHANGELOG.md
-7
View File
@@ -1,7 +0,0 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
+24 -27
View File
@@ -1,44 +1,41 @@
language: ruby language: minimal
dist: bionic dist: focal
sudo: required
branches: branches:
only: only:
- master - main
rvm:
- 2.6.6
before_script:
- gem install bundler -v 1.16.2
script: script:
- docker build --pull --tag travis-build-cedar-14 --file $(pwd)/builds/cedar-14.Dockerfile . - make test STACK="${STACK}" TEST_CMD="${TEST_CMD}"
- docker run --rm -e "STACK=cedar-14" travis-build-cedar-14 bash $TESTFOLDER
- docker build --pull --tag travis-build-heroku-16 --file $(pwd)/builds/heroku-16.Dockerfile .
- docker run --rm -e "STACK=heroku-16" travis-build-heroku-16 bash $TESTFOLDER
- docker build --pull --tag travis-build-heroku-18 --file $(pwd)/builds/heroku-18.Dockerfile .
- docker run --rm -e "STACK=heroku-18" travis-build-heroku-18 bash $TESTFOLDER
jobs: jobs:
include: include:
- stage: Bash linting (shellcheck) - name: Bash linting (shellcheck)
sudo: false
script: make check script: make check
- stage: Hatchet Integration - name: Hatchet integration tests
if: env(TRAVIS_PULL_REQUEST_SLUG) = env(TRAVIS_REPO_SLUG) if: env(HEROKU_API_USER) IS present AND env(HEROKU_API_KEY) IS present
name: Run Hatchet language: ruby
script: rvm:
- 2.7
before_script:
- bundle exec hatchet ci:setup - bundle exec hatchet ci:setup
script:
- PARALLEL_SPLIT_TEST_PROCESSES=11 bundle exec parallel_split_test spec/hatchet/ - PARALLEL_SPLIT_TEST_PROCESSES=11 bundle exec parallel_split_test spec/hatchet/
env: env:
matrix: jobs:
- TESTFOLDER=test/run-deps - STACK=heroku-16 TEST_CMD=test/run-deps
- TESTFOLDER=test/run-versions - STACK=heroku-16 TEST_CMD=test/run-versions
- TESTFOLDER=test/run-features - STACK=heroku-16 TEST_CMD=test/run-features
- STACK=heroku-18 TEST_CMD=test/run-deps
- STACK=heroku-18 TEST_CMD=test/run-versions
- STACK=heroku-18 TEST_CMD=test/run-features
- STACK=heroku-20 TEST_CMD=test/run-deps
- STACK=heroku-20 TEST_CMD=test/run-versions
- STACK=heroku-20 TEST_CMD=test/run-features
global: global:
- HATCHET_RETRIES=3 - HATCHET_RETRIES=3
- IS_RUNNING_ON_CI=true - IS_RUNNING_ON_CI=true
- HATCHET_APP_LIMIT=80 - HATCHET_APP_LIMIT=80
- HATCHET_DEPLOY_STRATEGY=git - HATCHET_DEPLOY_STRATEGY=git
- secure: yjtlPE5FbVxTKnjUy/tZUBgSEf4qADD3QOxtgziuid73S0U/1IEXlMGFULsQzIjtlHKmHeywZqpVVEpthIH4RuT7uoX1Pb7SSM/g0T8fT3VoEFbFK1uYl0oZQbUS4Klxv9tPiumj8if3m6ULEGIz1X0wZcMOC0tMLwVCnwmap0E=
- secure: ZeFTHWwnpIKE9nAqs88ocmiQh7bKce84lilGm5J23nf3N6V4wNyLwqlkvsM008WGBCaOg9AUx7ZunasT0ANsR5gLP3eV2UUg7ILdRgV2Gy13eNRFheC4PHdN92RqQ3aKoqlIv2K999xlhVjod0NzhkQQXB6PddfQINbuU7ks6As=
+185 -90
View File
@@ -1,72 +1,168 @@
# Python Buildpack Changelog # Python Buildpack Changelog
# 170 (2020-05-19) ## Unreleased
- Python 3.8.7 is now available (CPython) (#1122).
## v186 (2020-11-18)
- Update the `BUILD_WITH_GEO_LIBRARIES` error message (#1121).
- Switch NLTK feature detection away from `sp-grep` (#1119).
- Switch Django collectstatic feature detection away from `sp-grep` (#1119).
- Remove vendored `sp-grep` script (#1119).
- Remove vendored `pip-diff` script (#1118).
- Remove vendored `pip-grep` script (#1116).
## v185 (2020-11-12)
- Error if the unsupported `BUILD_WITH_GEO_LIBRARIES` env var is set (#1115).
- Remove deprecated GDAL/GEOS/PROJ support (#1113).
- Remove vendored `jq` binary (#1112).
- Remove redundant Mercurial install step (#1111).
- Remove support for the Cedar-14 stack (#1110).
## v184 (2020-10-21)
- Vendor buildpack-stdlib instead of fetching from S3 (#1100).
- Fix metric names for metrics emitted within `sub_env` (#1099).
## v183 (2020-10-12)
- Add support for Heroku-20 (#968).
## v182 (2020-10-06)
- Python 3.9.0 is now available (CPython) (#1090).
- Migrate from the `lang-python` S3 bucket to `heroku-buildpack-python` (#1089).
- Remove `vendor/shunit2` (#1086).
- Replace `BUILDPACK_VENDOR_URL` and `USE_STAGING_BINARIES` with `BUILDPACK_S3_BASE_URL` (#1085).
## v181 (2020-09-29)
- PyPy 2.7 and 3.6, version 7.3.2 are now available (Note: PyPy support is in beta) (#1081).
## v180 (2020-09-24)
- Python 3.8.6 is now available (CPython) (#1072).
## v179 (2020-09-23)
- Remove duplicate pipenv metric event (#1070).
- Emit metrics for how the Python version was chosen for an app (#1069).
- Emit Python version metric events for all builds, not just clean installs (#1066).
## v178 (2020-09-07)
- Python 3.5.10 is now available (CPython) (#1062).
## v177 (2020-08-18)
- Python 3.6.12 and 3.7.9 are now available (CPython) (#1054).
- The default Python version for new apps is now 3.6.12 (previously 3.6.11) (#1054).
## v176 (2020-08-12)
- Rebuild the Python 3.4.10 archives with the correct version of Python (#1048).
- Fix the security update version check message for apps using PyPy (#1040).
- Remove `vendor/test-utils` (#1043).
## v175 (2020-08-05)
- Update pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+ (#1030).
- Update setuptools from 39.0.1 to: (#1024)
- 44.1.1 for Python 2.7
- 43.0.0 for Python 3.4
- 47.1.1 for Python 3.5+
- Switch the `heroku-buildpack-python` repository default branch from `master` to `main` (#1029).
## v174 (2020-07-30)
- For repeat builds, also manage the installed versions of setuptools/wheel, rather than just that of pip (#1007).
- Install an explicit version of wheel rather than the latest release at the time (#1007).
- Output the installed version of pip, setuptools and wheel in the build log (#1007).
- Errors installing pip/setuptools/wheel are now displayed in the build output and fail the build early (#1007).
- Install pip using itself rather than `get-pip.py` (#1007).
- Disable pip's version check + cache when installing pip/setuptools/wheel (#1007).
- Install setuptools from PyPI rather than a vendored copy (#1007).
- Reduce the number of environment variables exposed to `bin/{pre,post}_compile` and other subprocesses (#1011).
## v173 (2020-07-21)
- Python 3.8.5 is now available (CPython).
## v172 (2020-07-17)
- Python 3.8.4 is now available (CPython).
## v171 (2020-07-07)
- Python 3.6.11 and 3.7.8 are now available (CPython).
## v170 (2020-05-19)
- Python 2.7.18, 3.5.9, 3.7.7 and 3.8.3 are now available (CPython). - Python 2.7.18, 3.5.9, 3.7.7 and 3.8.3 are now available (CPython).
- PyPy 2.7 and 3.6, version 7.3.1 are now available (Note: PyPy support is in beta) - PyPy 2.7 and 3.6, version 7.3.1 are now available (Note: PyPy support is in beta).
- Docs: Fix explanation of runtime.txt generation when using pipenv - Docs: Fix explanation of runtime.txt generation when using pipenv.
- Bugfix: Correctly detect Python version when using a `python_version` of `3.8` in `Pipfile.lock` - Bugfix: Correctly detect Python version when using a `python_version` of `3.8` in `Pipfile.lock`.
-------------------------------------------------------------------------------- ## v169 (2020-04-22)
# 169 (2020-04-22)
- Add a Hatchet test for python 3.8.2 - Add a Hatchet test for python 3.8.2
- Set Code Owners to @heroku/langauges - Set Code Owners to @heroku/langauges
- Bugfix: Caching on subsequent redeploys - Bugfix: Caching on subsequent redeploys
- Update tests to support latest version of Python - Update tests to support latest version of Python
# 168 (2020-04-06) ## v168 (2020-04-06)
- Doc: Update Readme with version numbers - Doc: Update Readme with version numbers
- update Code Owners to include the Heroku Buildpack Maintainers team - update Code Owners to include the Heroku Buildpack Maintainers team
- Deprecation warning: BUILD_WITH_GEO_LIBRARIES is now deprecated. See warning for details. - Deprecation warning: `BUILD_WITH_GEO_LIBRARIES` is now deprecated. See warning for details.
- Clean up build log output - Clean up build log output
- Update Python versions in README to match docs - Update Python versions in README to match docs
- Django version detection fixed, link updated - Django version detection fixed, link updated
# 167 (2020-03-26) ## v167 (2020-03-26)
- Add failcase for cache busting - Add failcase for cache busting
- Bugfix: Clearing pip dependencies - Bugfix: Clearing pip dependencies
# 166 (2020-03-05) ## v166 (2020-03-05)
- Correct ftp to https in vendored file - Correct ftp to https in vendored file
- Warn for Django 1.11 approaching EOL, provide link to roadmap - Warn for Django 1.11 approaching EOL, provide link to roadmap
# 165 (2020-02-27) ## v165 (2020-02-27)
- Python 3.8.2 now available. - Python 3.8.2 now available.
# 164 (2020-02-20) ## v164 (2020-02-20)
- Update requirements.txt builds to use Pip 20.0.2 - Update requirements.txt builds to use Pip 20.0.2
- Download get-pip.py to tmpdir instead of root dir - Download get-pip.py to tmpdir instead of root dir
# 163 (2019-12-23) ## v163 (2019-12-23)
- New pythons released: - New pythons released:
Python 3.8.1, 3.7.6, 3.6.10 (CPython) Python 3.8.1, 3.7.6, 3.6.10 (CPython)
Beta Release: Pypy 2.7 and 3.6, version 7.2.0 Beta Release: Pypy 2.7 and 3.6, version 7.2.0
# 162 (2019-12-06) ## v162 (2019-12-06)
- Bug fix: fragile sqlite3 install - Bug fix: fragile sqlite3 install
# 161 (2019-12-2) ## v161 (2019-12-2)
- Bug fix: Sqlite3 version bump - Bug fix: Sqlite3 version bump
# 160 (2019-10-23) ## v160 (2019-10-23)
- Bugfix: Pipenv no longer installs twice in CI - Bugfix: Pipenv no longer installs twice in CI
# 159 (2019-10-22) ## v159 (2019-10-22)
- Python 2.7.17 now available on Heroku 18 and 16. - Python 2.7.17 now available on Heroku 18 and 16.
# 158 (2019-10-21) ## v158 (2019-10-21)
- Python 3.7.5 and 3.8.0 now available on Heroku 18 and 16. - Python 3.7.5 and 3.8.0 now available on Heroku 18 and 16.
- Add support for Python 3.8 branch - Add support for Python 3.8 branch
@@ -74,11 +170,11 @@
- Test Improvements - Test Improvements
- Add support for staging binary testing - Add support for staging binary testing
# 157 (2019-09-18) ## v157 (2019-09-18)
- Typo fixes - Typo fixes
# 156 (2019-09-12) ## v156 (2019-09-12)
- Python 3.6.9 and 3.7.4 now available. - Python 3.6.9 and 3.7.4 now available.
@@ -86,125 +182,125 @@
- Build utility and documentation updates - Build utility and documentation updates
- Bump Hatchet tests to point at new default python version. - Bump Hatchet tests to point at new default python version.
# 155 (2019-08-22) ## v155 (2019-08-22)
add docs and make target for heroku-18 bob builds add docs and make target for heroku-18 bob builds
# 154 (2019-07-17) ## v154 (2019-07-17)
Fix python 3.5.7 formula actually building 3.7.2 Fix python 3.5.7 formula actually building 3.7.2
# 153 (2019-06-21) ## v153 (2019-06-21)
Hotfix for broken heroku-16 deploys Hotfix for broken heroku-16 deploys
# 152 (2019-04-04) ## v152 (2019-04-04)
Python 3.7.3 now available. Python 3.7.3 now available.
# 151 (2019-03-21) ## v151 (2019-03-21)
Python 3.5.7 and 3.4.10 now available on all Heroku stacks. Python 3.5.7 and 3.4.10 now available on all Heroku stacks.
# 150 (2019-03-13) ## v150 (2019-03-13)
Python 2.7.16 now available on all Heroku stacks. Python 2.7.16 now available on all Heroku stacks.
# 149 (2019-03-04) ## v149 (2019-03-04)
Hotfix for broken Cedar 14 deploys Hotfix for broken Cedar 14 deploys
# 148 (2019-02-21) ## v148 (2019-02-21)
No user facing changes, improving internal metrics No user facing changes, improving internal metrics
# 147 (2019-02-07) ## v147 (2019-02-07)
Python 3.7.2 and 3.6.8 now available on all Heroku stacks. Python 3.7.2 and 3.6.8 now available on all Heroku stacks.
# 146 (2018-11-11) ## v146 (2018-11-11)
Python 3.7.1, 3.6.7, 3.5.6 and 3.4.9 now available on all Heroku stacks. Python 3.7.1, 3.6.7, 3.5.6 and 3.4.9 now available on all Heroku stacks.
# 145 (2018-11-08) ## v145 (2018-11-08)
Testing and tooling expanded to better support new runtimes Testing and tooling expanded to better support new runtimes
# 144 (2018-10-10) ## v144 (2018-10-10)
Switch to cautious upgrade for Pipenv install to ensure the pinned pip version Switch to cautious upgrade for Pipenv install to ensure the pinned pip version
is used with Pipenv is used with Pipenv
# 143 (2018-10-09) ## v143 (2018-10-09)
Add support for detecting SLUGIFY_USES_TEXT_UNIDECODE, which is required to Add support for detecting `SLUGIFY_USES_TEXT_UNIDECODE`, which is required to
install Apache Airflow version 1.10 or higher. install Apache Airflow version 1.10 or higher.
# 142 (2018-10-08) ## v142 (2018-10-08)
Improvements to Python install messaging Improvements to Python install messaging
# 139, 140, 141 ## v139, 140, 141
No user-facing changes, documenting for version clarity No user-facing changes, documenting for version clarity
# 138 (2018-08-01) ## v138 (2018-08-01)
Use stack image SQLite3 instead of vendoring Use stack image SQLite3 instead of vendoring
# 137 (2018-07-17) ## v137 (2018-07-17)
Prevent 3.7.0 from appearing as unsupported in buildpack messaging. Prevent 3.7.0 from appearing as unsupported in buildpack messaging.
# 136 (2018-06-28) ## v136 (2018-06-28)
Upgrade to 3.6.6 and support 3.7.0 on all runtimes. Upgrade to 3.6.6 and support 3.7.0 on all runtimes.
# 135 (2018-05-29) ## v135 (2018-05-29)
Upgrade Pipenv to v2018.5.18. Upgrade Pipenv to v2018.5.18.
# 134 (2018-05-02) ## v134 (2018-05-02)
Default to 3.6.5, bugfixes. Default to 3.6.5, bugfixes.
# 133 ## v133
Fixes for Pip 10 release. Fixes for Pip 10 release.
# 132 ## v132
Improve pip installation, with the release of v9.0.2. Improve pip installation, with the release of v9.0.2.
# 131 ## v131
Fix bug with pip. Fix bug with pip.
# 130 ## v130
Better upgrade strategy for pip. Better upgrade strategy for pip.
# 129 ## v129
Don't upgrade pip (from v128). Don't upgrade pip (from v128).
# 128 ## v128
Upgrade pip, pin to Pipenv v11.8.2. Upgrade pip, pin to Pipenv v11.8.2.
# 127 ## v127
Pin to Pipenv v11.7.1. Pin to Pipenv v11.7.1.
# 126 ## v126
Bugfixes. Bugfixes.
# 125 ## v125
Bugfixes. Bugfixes.
# 124 ## v124
Update buildpack to automatically install `[dev-packages]` during Heroku CI Pipenv builds. Update buildpack to automatically install `[dev-packages]` during Heroku CI Pipenv builds.
@@ -213,101 +309,101 @@ Update buildpack to automatically install `[dev-packages]` during Heroku CI Pipe
- No longer warn if there is no `Procfile`. - No longer warn if there is no `Procfile`.
- Update Pipenv's "3.6" runtime specifier to point to "3.6.4". - Update Pipenv's "3.6" runtime specifier to point to "3.6.4".
# 123 ## v123
Update gunicorn `init.d` script to allow overrides. Update gunicorn `init.d` script to allow overrides.
# 122 ## v122
Update default Python to v3.6.4. Update default Python to v3.6.4.
# 121 ## v121
Update default Python to v3.6.3. Update default Python to v3.6.3.
# 120 ## v120
Use `$ pipenv --deploy`. Use `$ pipenv --deploy`.
# 119 ## v119
Improvements to Pipenv support, warning on unsupported Python versions. Improvements to Pipenv support, warning on unsupported Python versions.
- We now warn when a user is not using latest 2.x or 3.x Python. - We now warn when a user is not using latest 2.x or 3.x Python.
- Heroku now supports `[requires]` `python_full_version` in addition to `python_version`. - Heroku now supports `[requires]` `python_full_version` in addition to `python_version`.
# 118 ## v118
Improvements to Pipenv support. Improvements to Pipenv support.
# 117 ## v117
Bug fix. Bug fix.
# 116 ## v116
Vendoring improvements. Vendoring improvements.
- Geos libraries should work on Heroku-16 now. - Geos libraries should work on Heroku-16 now.
- The libffi/libmemcached vendoring step is now skipped on Heroku-16 (since they are installed in the base image). - The libffi/libmemcached vendoring step is now skipped on Heroku-16 (since they are installed in the base image).
# 115 ## v115
Revert a pull request. Revert a pull request.
- No longer using `sub_env` for `pip install` step. - No longer using `sub_env` for `pip install` step.
# 114 ## v114
- Bugfixes. - Bugfixes.
Blacklisting `PYTHONHOME` and `PYTHONPATH` for older apps. Upgrades to nltk support. Blacklisting `PYTHONHOME` and `PYTHONPATH` for older apps. Upgrades to nltk support.
# 113 ## v113
Updates to Pipenv support. Updates to Pipenv support.
# 112 ## v112
Bugfix. Bugfix.
- Fixed grep output bug. - Fixed grep output bug.
# 111 ## v111
Linting, bugfixes. Linting, bugfixes.
# 110 ## v110
Update default Python to 3.6.2. Update default Python to 3.6.2.
# 109 ## v109
Update Default Python to 3.6.1, bugfixes. Update Default Python to 3.6.1, bugfixes.
- Fixed automatic pip uninstall of dependencies removed from requirements.txt. - Fixed automatic pip uninstall of dependencies removed from requirements.txt.
# 108 ## v108
Fix output for collectstatic step. Fix output for collectstatic step.
# 107 ## v107
Bugfix for C dependency installation. Bugfix for C dependency installation.
# 106 ## v106
Don't install packages that could mess up packaging. Don't install packages that could mess up packaging.
- The Python buildpack will automatically remove `six`, `pyparsing`, `appdirs`, - The Python buildpack will automatically remove `six`, `pyparsing`, `appdirs`,
`setuptools`, and `distribute` from a `requirements.txt` file now, as these `setuptools`, and `distribute` from a `requirements.txt` file now, as these
packages are provided by the Python buildpack. packages are provided by the Python buildpack.
# 105 ## v105
Improvements to output messaging. Improvements to output messaging.
# 104 ## v104
General improvements. General improvements.
@@ -317,52 +413,52 @@ General improvements.
[395]: https://github.com/heroku/heroku-buildpack-python/pull/395 [395]: https://github.com/heroku/heroku-buildpack-python/pull/395
## 103 ## v103
Bug fixes and improvements. Bug fixes and improvements.
- Fix for Pipenv. - Fix for Pipenv.
- Fix for Heroku CI. - Fix for Heroku CI.
- Improve handling of WEB_CONCURRENCY when using multiple buildpacks. - Improve handling of `WEB_CONCURRENCY` when using multiple buildpacks.
- Adjust environment variables set during the build to more closely match those in the dyno environment (DYNO is now available, STACK is not). - Adjust environment variables set during the build to more closely match those in the dyno environment (`DYNO` is now available, `STACK` is not).
- Restore the build cache prior to running bin/pre_compile. - Restore the build cache prior to running bin/pre_compile.
## 102 ## v102
Buildpack code cleanup. Buildpack code cleanup.
- Improved messaging around NLTK. - Improved messaging around NLTK.
## 101 ## v101
Updated setuptools installation method. Updated setuptools installation method.
- Improved pipenv support. - Improved pipenv support.
## 100 ## v100
Preliminary pipenv support. Preliminary pipenv support.
## 99 ## v99
Cleanup. Cleanup.
## 98 ## v98
Official NLTK support and other improvements. Official NLTK support and other improvements.
- Support for `nltk.txt` file for declaring corpora to be downloaded. - Support for `nltk.txt` file for declaring corpora to be downloaded.
- Leading zeros for auto-set WEB_CONCURRENCY. - Leading zeros for auto-set `WEB_CONCURRENCY`.
## 97 ## v97
Improved egg-link functionality. Improved egg-link functionality.
## 96 ## v96
Bugfix. Bugfix.
## 95 ## v95
Improved output support. Improved output support.
@@ -398,7 +494,7 @@ Fixed bug with editable pip installations.
Updated default Python 2.7.13. Updated default Python 2.7.13.
- Python 2.7.13 uses UCS-4 build, more compatibile with linux wheels. - Python 2.7.13 uses UCS-4 build, more compatible with linux wheels.
- Updated setuptools to v32.1.0. - Updated setuptools to v32.1.0.
## v86 ## v86
@@ -424,7 +520,7 @@ Support for Heroku CI.
## v82 (2016-08-22) ## v82 (2016-08-22)
Update to library detection mechnisms (pip-pop). Update to library detection mechanisms (pip-pop).
- Updated setuptools to v25.5.0 - Updated setuptools to v25.5.0
@@ -498,7 +594,7 @@ Revert to v66.
## v68 (2015-10-12) ## v68 (2015-10-12)
Fixed .heroku/venv error with modern apps. Fixed `.heroku/venv` error with modern apps.
## v67 (2015-10-12) ## v67 (2015-10-12)
@@ -523,7 +619,6 @@ Updated Pip and Setuptools.
- Setuptools updated to v18.3.2 - Setuptools updated to v18.3.2
- Pip updated to v7.1.2 - Pip updated to v7.1.2
## v62 (2015-08-07) ## v62 (2015-08-07)
Updated Pip and Setuptools. Updated Pip and Setuptools.
+27 -45
View File
@@ -1,68 +1,50 @@
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
activesupport (6.0.2.2) diff-lcs (1.4.4)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
tzinfo (~> 1.1)
zeitwerk (~> 2.2)
concurrent-ruby (1.1.6)
diff-lcs (1.3)
erubis (2.7.0) erubis (2.7.0)
excon (0.73.0) excon (0.78.0)
heroics (0.0.25) heroics (0.1.1)
erubis (~> 2.0) erubis (~> 2.0)
excon excon
moneta moneta
multi_json (>= 1.9.2) multi_json (>= 1.9.2)
heroku_hatchet (5.0.3) heroku_hatchet (7.3.3)
excon (~> 0) excon (~> 0)
minitest-retry (~> 0.1.9) platform-api (~> 3)
platform-api (~> 2)
repl_runner (~> 0.0.3)
rrrretry (~> 1) rrrretry (~> 1)
thor (~> 0) thor (~> 1)
threaded (~> 0) threaded (~> 0)
i18n (1.8.2)
concurrent-ruby (~> 1.0)
minitest (5.14.0)
minitest-retry (0.1.9)
minitest (>= 5.0)
moneta (1.0.0) moneta (1.0.0)
multi_json (1.14.1) multi_json (1.15.0)
parallel (1.19.1) parallel (1.19.2)
parallel_split_test (0.7.0) parallel_split_test (0.8.0)
parallel (>= 0.5.13) parallel (>= 0.5.13)
rspec (>= 3.1.0) rspec (>= 3.1.0)
platform-api (2.2.0) platform-api (3.0.0)
heroics (~> 0.0.25) heroics (~> 0.1.1)
moneta (~> 1.0.0) moneta (~> 1.0.0)
rake (12.3.1) rate_throttle_client (~> 0.1.0)
repl_runner (0.0.3) rake (13.0.1)
activesupport rate_throttle_client (0.1.2)
rrrretry (1.0.0) rrrretry (1.0.0)
rspec (3.8.0) rspec (3.10.0)
rspec-core (~> 3.8.0) rspec-core (~> 3.10.0)
rspec-expectations (~> 3.8.0) rspec-expectations (~> 3.10.0)
rspec-mocks (~> 3.8.0) rspec-mocks (~> 3.10.0)
rspec-core (3.8.0) rspec-core (3.10.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-expectations (3.8.1) rspec-expectations (3.10.0)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-mocks (3.8.0) rspec-mocks (3.10.0)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-retry (0.6.1) rspec-retry (0.6.2)
rspec-core (> 3.3) rspec-core (> 3.3)
rspec-support (3.8.0) rspec-support (3.10.0)
thor (0.20.3) thor (1.0.1)
thread_safe (0.3.6)
threaded (0.0.4) threaded (0.0.4)
tzinfo (1.2.7)
thread_safe (~> 0.1)
zeitwerk (2.3.0)
PLATFORMS PLATFORMS
ruby ruby
+47 -41
View File
@@ -1,55 +1,61 @@
# These targets are not files # These targets are not files
.PHONY: tests .PHONY: check test compile builder-image buildenv deploy-runtimes tools
test: test-heroku-18 test-heroku-16 test-cedar-14 STACK ?= heroku-18
STACKS ?= heroku-16 heroku-18 heroku-20
TEST_CMD ?= test/run-versions && test/run-features && test/run-deps
FIXTURE ?= test/fixtures/requirements-standard
ENV_FILE ?= builds/dockerenv.default
BUILDER_IMAGE_PREFIX := heroku-python-build
# Converts a stack name of `heroku-NN` to its build Docker image tag of `heroku/heroku:NN-build`.
STACK_IMAGE_TAG := heroku/$(subst -,:,$(STACK))-build
check: check:
@shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings bin/default_pythons @shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings bin/default_pythons
@shellcheck -x bin/steps/collectstatic bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/gdal bin/steps/geo-libs bin/steps/mercurial bin/steps/nltk bin/steps/pip-install bin/steps/pip-uninstall bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/pylibmc bin/steps/python @shellcheck -x bin/steps/collectstatic bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/nltk bin/steps/pip-install bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/python
@shellcheck -x bin/steps/hooks/* @shellcheck -x bin/steps/hooks/*
test-cedar-14: test:
@echo "Running tests in docker (cedar-14)..." @echo "Running tests using: STACK=$(STACK) TEST_CMD='$(TEST_CMD)'"
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=cedar-14" heroku/cedar:14 bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@echo ""
test-heroku-16:
@echo "Running tests in docker (heroku-16)..."
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=heroku-16" heroku/heroku:16-build bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@echo ""
test-heroku-18:
@echo "Running tests in docker (heroku-18)..."
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=heroku-18" heroku/heroku:18-build bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@echo ""
buildenv-heroku-16:
@echo "Creating build environment (heroku-16)..."
@echo @echo
@docker build --pull -f $(shell pwd)/builds/heroku-16.Dockerfile -t python-buildenv-heroku-16 . @docker run --rm -it -v $(PWD):/buildpack:ro -e "STACK=$(STACK)" "$(STACK_IMAGE_TAG)" bash -c 'cp -r /buildpack /buildpack_test && cd /buildpack_test && $(TEST_CMD)'
@echo
compile:
@echo "Running compile using: STACK=$(STACK) FIXTURE=$(FIXTURE)"
@echo
@docker run --rm -it -v $(PWD):/src:ro -e "STACK=$(STACK)" -w /buildpack "$(STACK_IMAGE_TAG)" \
bash -c 'cp -r /src/{bin,vendor} /buildpack && cp -r /src/$(FIXTURE) /build && mkdir /cache /env && bin/compile /build /cache /env'
@echo
builder-image:
@echo "Generating binary builder image for $(STACK)..."
@echo
@docker build --pull -f builds/$(STACK).Dockerfile -t "$(BUILDER_IMAGE_PREFIX)-$(STACK)" .
@echo
buildenv: builder-image
@echo "Starting build environment for $(STACK)..."
@echo @echo
@echo "Usage..." @echo "Usage..."
@echo @echo
@echo " $$ export AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=bar # Optional unless deploying" @echo " $$ bob build runtimes/python-X.Y.Z"
@echo " $$ bob build runtimes/python-2.7.13"
@echo " $$ bob deploy runtimes/python-2.7.13"
@echo @echo
@docker run -it --rm python-buildenv-heroku-16 @docker run --rm -it --env-file="$(ENV_FILE)" -v $(PWD)/builds:/app/builds "$(BUILDER_IMAGE_PREFIX)-$(STACK)" bash
buildenv-heroku-18: deploy-runtimes:
@echo "Creating build environment (heroku-18)..." ifndef RUNTIMES
$(error No runtimes specified! Use: "make deploy-runtimes RUNTIMES='python-X.Y.Z ...' [STACKS='heroku-18 ...'] [ENV_FILE=...]")
endif
@echo "Using: RUNTIMES='$(RUNTIMES)' STACKS='$(STACKS)' ENV_FILE='$(ENV_FILE)'"
@echo @echo
@docker build --pull -f $(shell pwd)/builds/heroku-18.Dockerfile -t python-buildenv-heroku-18 . @set -eu; for stack in $(STACKS); do \
@echo $(MAKE) builder-image STACK=$${stack}; \
@echo "Usage..." for runtime in $(RUNTIMES); do \
@echo echo "Generating/deploying $${runtime} for $${stack}..."; \
@echo " $$ export AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=bar # Optional unless deploying" echo; \
@echo " $$ bob build runtimes/python-2.7.13" docker run --rm -it --env-file="$(ENV_FILE)" "$(BUILDER_IMAGE_PREFIX)-$${stack}" bob deploy "runtimes/$${runtime}"; \
@echo " $$ bob deploy runtimes/python-2.7.13" echo; \
@echo done; \
@docker run -it --rm python-buildenv-heroku-18 done
tools:
git clone https://github.com/kennethreitz/pip-pop.git
mv pip-pop/bin/* vendor/pip-pop/
rm -fr pip-pop
-72
View File
@@ -1,27 +1,5 @@
This buildpack includes some vendorized packages to ease installation. This buildpack includes some vendorized packages to ease installation.
jq license
----------
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
shunit2 license shunit2 license
--------------- ---------------
@@ -85,53 +63,3 @@ The Free Software Foundation may publish revised and/or new versions of the GNU
Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.
get-pip.py license
------------------
Copyright (c) 2008-2016 The pip developers (see AUTHORS.txt file)
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
pip-pop license
---------------
The MIT License (MIT)
Copyright (c) 2014 Kenneth Reitz.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+17 -11
View File
@@ -2,7 +2,7 @@
# Heroku Buildpack: Python # Heroku Buildpack: Python
[![Build Status](https://travis-ci.org/heroku/heroku-buildpack-python.svg?branch=master)](https://travis-ci.org/heroku/heroku-buildpack-python) [![Build Status](https://travis-ci.com/heroku/heroku-buildpack-python.svg?branch=main)](https://travis-ci.com/heroku/heroku-buildpack-python)
This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps. This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps.
@@ -16,7 +16,7 @@ See it in Action
$ ls $ ls
my-application requirements.txt runtime.txt my-application requirements.txt runtime.txt
$ git push heroku master $ git push heroku main
Counting objects: 4, done. Counting objects: 4, done.
Delta compression using up to 8 threads. Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done. Compressing objects: 100% (2/2), done.
@@ -26,7 +26,7 @@ remote: Compressing source files... done.
remote: Building source: remote: Building source:
remote: remote:
remote: -----> Python app detected remote: -----> Python app detected
remote: -----> Installing python-3.7.4 remote: -----> Installing python
remote: -----> Installing pip remote: -----> Installing pip
remote: -----> Installing SQLite3 remote: -----> Installing SQLite3
remote: -----> Installing requirements with pip remote: -----> Installing requirements with pip
@@ -44,7 +44,7 @@ A `requirements.txt` must be present at the root of your application's repositor
To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version. To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version.
Current default Python Runtime: Python 3.6.9 Current default Python Runtime: Python 3.6.12
Alternatively, you can provide a `setup.py` file, or a `Pipfile`. Alternatively, you can provide a `setup.py` file, or a `Pipfile`.
Using `pipenv` will generate `runtime.txt` at build time if one of the field `python_version` or `python_full_version` is specified in the `requires` section of your `Pipfile`. Using `pipenv` will generate `runtime.txt` at build time if one of the field `python_version` or `python_full_version` is specified in the `requires` section of your `Pipfile`.
@@ -62,9 +62,10 @@ Specify a Python Runtime
Supported runtime options include: Supported runtime options include:
- `python-3.8.3` - `python-3.9.0`
- `python-3.7.7` - `python-3.8.7`
- `python-3.6.10` - `python-3.7.9`
- `python-3.6.12`
- `python-2.7.18` - `python-2.7.18`
## Tests ## Tests
@@ -72,17 +73,22 @@ Supported runtime options include:
The buildpack tests use [Docker](https://www.docker.com/) to simulate The buildpack tests use [Docker](https://www.docker.com/) to simulate
Heroku's [stack images.](https://devcenter.heroku.com/articles/stack) Heroku's [stack images.](https://devcenter.heroku.com/articles/stack)
To run the test suite: To run the test suite against the default stack:
``` ```
make test make test
``` ```
Or to test in a particular stack: Or to test against a particular stack:
``` ```
make test-heroku-18 make test STACK=heroku-16
make test-heroku-16 ```
To run only a subset of the tests:
```
make test TEST_CMD=tests/versions
``` ```
The tests are run via the vendored The tests are run via the vendored
+55 -77
View File
@@ -15,7 +15,7 @@
# Fail fast and fail hard. # Fail fast and fail hard.
set -eo pipefail set -eo pipefail
# Boostrap the Buildpack Standard Library. # Used by buildpack-stdlib's metrics features.
export BPLOG_PREFIX="buildpack.python" export BPLOG_PREFIX="buildpack.python"
export BUILDPACK_LOG_FILE=${BUILDPACK_LOG_FILE:-/dev/null} export BUILDPACK_LOG_FILE=${BUILDPACK_LOG_FILE:-/dev/null}
@@ -35,23 +35,19 @@ ENV_DIR=$3
# Export Path variables, for use in sub-scripts. # Export Path variables, for use in sub-scripts.
export BUILD_DIR CACHE_DIR ENV_DIR export BUILD_DIR CACHE_DIR ENV_DIR
# Set the Buildpack's internet target for downloading Python distributions. # Set the base URL for downloading buildpack assets like Python runtimes.
# The user can provide BUILDPACK_VENDOR_URL to specify a custom target. # The user can provide BUILDPACK_S3_BASE_URL to specify a custom target.
# Note: this is designed for non-Heroku use, as it does not use the user-provided # Note: this is designed for non-Heroku use, as it does not use the user-provided
# environment variable mechanism (the ENV_DIR). # environment variable mechanism (the ENV_DIR).
VENDOR_URL="https://lang-python.s3.amazonaws.com/$STACK" S3_BASE_URL="${BUILDPACK_S3_BASE_URL:-"https://heroku-buildpack-python.s3.amazonaws.com"}"
if [[ -n ${BUILDPACK_VENDOR_URL:-} ]]; then # This has to be exported since it's used by the geo-libs step which is run in a subshell.
VENDOR_URL="$BUILDPACK_VENDOR_URL"
elif [[ -n ${USE_STAGING_BINARIES} ]]; then
VENDOR_URL="$USE_STAGING_BINARIES/$STACK"
fi
export VENDOR_URL
# Default Python Versions # Default Python Versions
# shellcheck source=bin/default_pythons # shellcheck source=bin/default_pythons
source "$BIN_DIR/default_pythons" source "$BIN_DIR/default_pythons"
# Supported Python Branches # Supported Python Branches
PY39="python-3.9"
PY38="python-3.8" PY38="python-3.8"
PY37="python-3.7" PY37="python-3.7"
PY36="python-3.6" PY36="python-3.6"
@@ -62,57 +58,28 @@ PYPY27="pypy2.7"
PYPY36="pypy3.6" PYPY36="pypy3.6"
# Which stack is used (for binary downloading), if none is provided (e.g. outside of Heroku)? # Which stack is used (for binary downloading), if none is provided (e.g. outside of Heroku)?
DEFAULT_PYTHON_STACK="cedar-14" # TODO: Remove this and require that STACK be set explicitly.
# If pip doesn't match this version (the version we install), run the installer. DEFAULT_PYTHON_STACK="heroku-18"
PIP_UPDATE="20.0.2"
for file in "$BUILD_DIR/runtime.txt" "$CACHE_DIR/.heroku/python-version" ; do
[ -f "$file" ] || continue
version=$(tr -d '[:space:]' < "$file")
case "$version" in "$PY34"*)
# Python 3.4 support was dropped in pip >= 19.2.
PIP_UPDATE="19.1.1"
break
;;
esac
done
if [[ -f "$BUILD_DIR/Pipfile" ]]; then
# Do not force pipenv users to re-install pipenv locally.
PIP_UPDATE="9.0.2"
fi
export DEFAULT_PYTHON_STACK PIP_UPDATE
export PY37 PY36 PY35 PY27 PY34
# Common Problem Warnings: # Common Problem Warnings:
# This section creates a temporary file in which to stick the output of `pip install`. # This section creates a temporary file in which to stick the output of `pip install`.
# The `warnings` subscript then greps through this for common problems and guides # The `warnings` subscript then greps through this for common problems and guides
# the user towards resolution of known issues. # the user towards resolution of known issues.
WARNINGS_LOG=$(mktemp) WARNINGS_LOG=$(mktemp)
export WARNINGS_LOG RECOMMENDED_PYTHON_VERSION=$DEFAULT_PYTHON_VERSION
export RECOMMENDED_PYTHON_VERSION=$DEFAULT_PYTHON_VERSION
# The buildpack ships with a few executable tools (e.g. pip-grep, etc). # The buildpack ships with a few executable tools.
# This installs them into the path, so we can execute them directly. # This installs them into the path, so we can execute them directly.
export PATH=$PATH:$ROOT_DIR/vendor/:$ROOT_DIR/vendor/pip-pop export PATH=$PATH:$ROOT_DIR/vendor/
# Set environment variables if they weren't set by the platform. # Set environment variables if they weren't set by the platform.
# Note: this is legacy, for a deprecated build system known as Anvil.
# This can likely be removed, with caution.
[ ! "$SLUG_ID" ] && SLUG_ID="defaultslug"
[ ! "$REQUEST_ID" ] && REQUEST_ID=$SLUG_ID
[ ! "$STACK" ] && STACK=$DEFAULT_PYTHON_STACK [ ! "$STACK" ] && STACK=$DEFAULT_PYTHON_STACK
# Sanitize externally-provided environment variables: # Sanitize externally-provided environment variables:
# The following environment variables are either problematic or simply unneccessary # The following environment variables are either problematic or simply unneccessary
# for the buildpack to have knowledge of, so we unset them, to keep the environment # for the buildpack to have knowledge of, so we unset them, to keep the environment
# as clean and pristine as possible. # as clean and pristine as possible.
unset GIT_DIR PYTHONHOME PYTHONPATH unset PYTHONHOME PYTHONPATH
unset RECEIVE_DATA RUN_KEY BUILD_INFO DEPLOY LOG_TOKEN
unset CYTOKINE_LOG_FILE GEM_PATH
# Import the utils script, which contains helper functions used throughout the buildpack. # Import the utils script, which contains helper functions used throughout the buildpack.
# shellcheck source=bin/utils # shellcheck source=bin/utils
@@ -123,6 +90,33 @@ source "$BIN_DIR/utils"
# shellcheck source=bin/warnings # shellcheck source=bin/warnings
source "$BIN_DIR/warnings" source "$BIN_DIR/warnings"
if [[ "${STACK}" == "cedar-14" ]]; then
mcount "failure.unsupported.cedar-14"
puts-warn "The Cedar-14 stack is no longer supported by the latest release of this buildpack."
puts-warn
puts-warn "Please switch to the Cedar-14 support branch by using this buildpack URL:"
puts-warn "https://github.com/heroku/heroku-buildpack-python#cedar-14"
puts-warn
puts-warn "For instructions on how to change the buildpacks used by an app, see:"
puts-warn "https://devcenter.heroku.com/articles/buildpacks#setting-a-buildpack-on-an-application"
exit 1
fi
if [[ -f "${ENV_DIR}/BUILD_WITH_GEO_LIBRARIES" ]]; then
mcount "failure.unsupported.BUILD_WITH_GEO_LIBRARIES"
puts-warn "The Python buildpack's legacy BUILD_WITH_GEO_LIBRARIES functonality is"
puts-warn "no longer supported:"
puts-warn "https://devcenter.heroku.com/changelog-items/1947"
puts-warn
puts-warn "To continue to use GDAL, GEOS or PROJ support, see the migration guide:"
puts-warn "https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-no-longer-supported"
puts-warn
puts-warn "Or if you no longer need those libraries, this message can be hidden by"
puts-warn "unsetting the BUILD_WITH_GEO_LIBRARIES environment variable, using:"
puts-warn "heroku config:unset BUILD_WITH_GEO_LIBRARIES"
exit 1
fi
# Make the directory in which we will create symlinks from the temporary build directory # Make the directory in which we will create symlinks from the temporary build directory
# to `/app`. # to `/app`.
# Symlinks are required, since Python is not a portable installation. # Symlinks are required, since Python is not a portable installation.
@@ -151,14 +145,19 @@ export PYTHONUNBUFFERED=1
# Set the locale to a well-known and expected standard. # Set the locale to a well-known and expected standard.
export LANG=en_US.UTF-8 export LANG=en_US.UTF-8
# `~/.heroku/vendor` is an place where the buildpack may stick pre-build binaries for known # `~/.heroku/vendor` is an place where the buildpack may stick pre-build binaries for known
# C dependencies (e.g. libmemcached on cedar-14). This section configures Python (GCC, more specifically) # C dependencies. This section configures Python (GCC, more specifically)
# and pip to automatically include these paths when building binaries. # and pip to automatically include these paths when building binaries.
# TODO: Stop adding .heroku/vendor here now that the buildpack no longer vendors anything.
export C_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$C_INCLUDE_PATH export C_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$CPLUS_INCLUDE_PATH export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LIBRARY_PATH export LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=/app/.heroku/vendor/lib/pkg-config:/app/.heroku/python/lib/pkg-config:$PKG_CONFIG_PATH export PKG_CONFIG_PATH=/app/.heroku/vendor/lib/pkg-config:/app/.heroku/python/lib/pkg-config:$PKG_CONFIG_PATH
# Global pip options (https://pip.pypa.io/en/stable/user_guide/#environment-variables).
# Disable pip's warnings about EOL Python since we show our own.
export PIP_NO_PYTHON_VERSION_WARNING=1
# The Application Code # The Application Code
# -------------------- # --------------------
@@ -188,8 +187,6 @@ cp -R "$CACHE_DIR/.heroku/python-stack" .heroku/ &> /dev/null || true
cp -R "$CACHE_DIR/.heroku/python-version" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/python-version" .heroku/ &> /dev/null || true
# A plain text file which contains the current sqlite3 version being used (used for cache busting). # A plain text file which contains the current sqlite3 version being used (used for cache busting).
cp -R "$CACHE_DIR/.heroku/python-sqlite3-version" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/python-sqlite3-version" .heroku/ &> /dev/null || true
# Any pre-compiled binaries, provided by the buildpack.
cp -R "$CACHE_DIR/.heroku/vendor" .heroku/ &> /dev/null || true
# "editable" installations of code repositories, via pip or pipenv. # "editable" installations of code repositories, via pip or pipenv.
if [[ -d "$CACHE_DIR/.heroku/src" ]]; then if [[ -d "$CACHE_DIR/.heroku/src" ]]; then
cp -R "$CACHE_DIR/.heroku/src" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/src" .heroku/ &> /dev/null || true
@@ -207,7 +204,7 @@ source "$BIN_DIR/steps/hooks/pre_compile"
# continue to use that version of Python in perpituity (warnings will be raised if # continue to use that version of Python in perpituity (warnings will be raised if
# they are outofdate). # they are outofdate).
if [ -f "$CACHE_DIR/.heroku/python-version" ]; then if [ -f "$CACHE_DIR/.heroku/python-version" ]; then
DEFAULT_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version") CACHED_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version")
fi fi
# We didn't always record the stack version. This code is in place because of that. # We didn't always record the stack version. This code is in place because of that.
@@ -217,8 +214,6 @@ else
CACHED_PYTHON_STACK=$STACK CACHED_PYTHON_STACK=$STACK
fi fi
export CACHED_PYTHON_STACK
# Pipenv Python version support. # Pipenv Python version support.
# Detect the version of Python requested from a Pipfile (e.g. python_version or python_full_version). # Detect the version of Python requested from a Pipfile (e.g. python_version or python_full_version).
# Convert it to a runtime.txt file. # Convert it to a runtime.txt file.
@@ -226,9 +221,14 @@ export CACHED_PYTHON_STACK
# shellcheck source=bin/steps/pipenv-python-version # shellcheck source=bin/steps/pipenv-python-version
source "$BIN_DIR/steps/pipenv-python-version" source "$BIN_DIR/steps/pipenv-python-version"
# If no runtime was provided by the user, assume the default Python runtime version. if [[ -f runtime.txt ]]; then
if [ ! -f runtime.txt ]; then mcount "version.reason.python.specified"
echo "$DEFAULT_PYTHON_VERSION" > runtime.txt elif [[ -n "${CACHED_PYTHON_VERSION:-}" ]]; then
mcount "version.reason.python.cached"
echo "${CACHED_PYTHON_VERSION}" > runtime.txt
else
mcount "version.reason.python.default"
echo "${DEFAULT_PYTHON_VERSION}" > runtime.txt
fi fi
# Create the directory for .profile.d, if it doesn't exist. # Create the directory for .profile.d, if it doesn't exist.
@@ -272,27 +272,6 @@ fi
# shellcheck source=bin/steps/eggpath-fix # shellcheck source=bin/steps/eggpath-fix
source "$BIN_DIR/steps/eggpath-fix" source "$BIN_DIR/steps/eggpath-fix"
# Mercurial support.
# If a customer appears to be using mercurial for dependency resolution, we install it first.
# Note: this only applies to pip, not pipenv. This can likely be removed, over time. Measure it first.
# shellcheck source=bin/steps/mercurial
source "$BIN_DIR/steps/mercurial"
# Pylibmc support.
# On cedar-14, libmemcached was not available. The buildpack provides its own version, instead.
# shellcheck source=bin/steps/pylibmc
source "$BIN_DIR/steps/pylibmc"
# Support for Geo libraries. This is deprecated functionality, only functional on cedar-14.
# It is undocumented.
# shellcheck source=bin/steps/geo-libs
sub_env "$BIN_DIR/steps/geo-libs"
# GDAL support.
# This is part of the Geo support.
# shellcheck source=bin/steps/gdal
source "$BIN_DIR/steps/gdal"
# SQLite3 support. # SQLite3 support.
# This sets up and installs sqlite3 dev headers and the sqlite3 binary but not the # This sets up and installs sqlite3 dev headers and the sqlite3 binary but not the
# libsqlite3-0 library since that exists on the stack image. # libsqlite3-0 library since that exists on the stack image.
@@ -324,7 +303,7 @@ mtime "nltk.download.time" "${start}"
# and copying it into the proper place (the logical place to do this was early, but it must be done here). # and copying it into the proper place (the logical place to do this was early, but it must be done here).
# In CI, $BUILD_DIR is /app. # In CI, $BUILD_DIR is /app.
if [[ ! "$BUILD_DIR" == "/app" ]]; then if [[ ! "$BUILD_DIR" == "/app" ]]; then
rm -fr "$BUILD_DIR/.heroku/src" rm -rf "$BUILD_DIR/.heroku/src"
deep-cp /app/.heroku/src "$BUILD_DIR/.heroku/src" deep-cp /app/.heroku/src "$BUILD_DIR/.heroku/src"
fi fi
@@ -390,7 +369,6 @@ mkdir -p "$CACHE_DIR/.heroku"
cp -R .heroku/python "$CACHE_DIR/.heroku/" cp -R .heroku/python "$CACHE_DIR/.heroku/"
cp -R .heroku/python-version "$CACHE_DIR/.heroku/" cp -R .heroku/python-version "$CACHE_DIR/.heroku/"
cp -R .heroku/python-stack "$CACHE_DIR/.heroku/" &> /dev/null || true cp -R .heroku/python-stack "$CACHE_DIR/.heroku/" &> /dev/null || true
cp -R .heroku/vendor "$CACHE_DIR/.heroku/" &> /dev/null || true
if [[ -d .heroku/src ]]; then if [[ -d .heroku/src ]]; then
cp -R .heroku/src "$CACHE_DIR/.heroku/" &> /dev/null || true cp -R .heroku/src "$CACHE_DIR/.heroku/" &> /dev/null || true
fi fi
+13 -17
View File
@@ -1,21 +1,17 @@
#!/usr/bin/env bash #!/usr/bin/env bash
DEFAULT_PYTHON_VERSION="python-3.6.10" # Disable unused env var warning, since shellcheck doesn't take into account
LATEST_38="python-3.8.3" # that this file is sourced. We don't want to use export since it exposes
LATEST_37="python-3.7.7" # the env vars to subprocesses.
LATEST_36="python-3.6.10" # shellcheck disable=2034
LATEST_35="python-3.5.9"
DEFAULT_PYTHON_VERSION="python-3.6.12"
LATEST_39="python-3.9.0"
LATEST_38="python-3.8.7"
LATEST_37="python-3.7.9"
LATEST_36="python-3.6.12"
LATEST_35="python-3.5.10"
LATEST_34="python-3.4.10" LATEST_34="python-3.4.10"
LATEST_27="python-2.7.18" LATEST_27="python-2.7.18"
PYPY_36="pypy3.6-7.3.1" LATEST_PYPY_36="pypy3.6-7.3.2"
PYPY_27="pypy2.7-7.3.1" LATEST_PYPY_27="pypy2.7-7.3.2"
export DEFAULT_PYTHON_VERSION \
LATEST_38 \
LATEST_37 \
LATEST_36 \
LATEST_35 \
LATEST_34 \
LATEST_27 \
PYPY_36 \
PYPY_27
-7
View File
@@ -2,13 +2,6 @@
TODO: Add context on Python install steps, such as why symlinking vs copying TODO: Add context on Python install steps, such as why symlinking vs copying
## Installing the Pip tool
The Python Buildpack uses a tool called `get-pip` to install the pip tool. This
is done in the `python` script.
This is in part because Python historically did not come with pip by default.
## Installing Python packages using Pip ## Installing Python packages using Pip
### Convention: Use `python` process to invoke Pip ### Convention: Use `python` process to invoke Pip
+2 -2
View File
@@ -20,8 +20,8 @@ MANAGE_FILE=${MANAGE_FILE:-fakepath}
# Legacy file-based support for $DISABLE_COLLECTSTATIC # Legacy file-based support for $DISABLE_COLLECTSTATIC
[ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1 [ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1
# Ensure that Django is explicitly specified in requirements.txt # Ensure that Django is actually installed.
sp-grep -s django && DJANGO_INSTALLED=1 is_module_available 'django' && DJANGO_INSTALLED=1
if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALLED" ]; then if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALLED" ]; then
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# This script serves as the GDAL build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If GDAL exists within requirements, use vendored gdal.
if (pip-grep -s requirements.txt GDAL gdal pygdal &> /dev/null) then
if [ ! -f ".heroku/vendor/bin/gdalserver" ]; then
puts-warn "The vendored GDAL package in the Heroku Python Buildpack now deprecated."
puts-warn "To enable GDAL use an alternative buildpack is available here - https://github.com/heroku/heroku-geo-buildpack"
echo "-----> Noticed GDAL. Bootstrapping gdal."
mkdir -p .heroku/vendor
# Download and extract cryptography into target vendor directory.
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.gdal"
fi
GDAL=$(pwd)/vendor
export GDAL
fi
-55
View File
@@ -1,55 +0,0 @@
#!/usr/bin/env bash
# This script serves as the GDAL build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
VENDORED_GEOS="${VENDOR_URL}/libraries/vendor/geos.tar.gz"
VENDORED_PROJ="${VENDOR_URL}/libraries/vendor/proj.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If GDAL exists within requirements, use vendored gdal.
if [[ "$BUILD_WITH_GEO_LIBRARIES" ]]; then
mcount "buildvar.BUILD_WITH_GEO_LIBRARIES"
puts-warn "The GDAL, GEOS and PROJ binaries and BUILD_WITH_GEO_LIBRARIES functonality are now deprecated."
puts-warn "An alternative buildpack to enable GDAL, GEOS and PROJ use is available here - https://github.com/heroku/heroku-geo-buildpack"
if [ ! -f ".heroku/vendor/bin/proj" ]; then
echo "-----> Bootstrapping gdal, geos, proj."
mkdir -p .heroku/vendor
# Download and extract cryptography into target vendor directory.
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_GEOS" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_PROJ" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.geo_libs"
# Copy libjasper from build image to slug.
if [[ "$STACK" == "heroku-16" ]]; then
cp /usr/lib/x86_64-linux-gnu/libjasper.so* ".heroku/vendor/lib/."
mcount "steps.vendor.libjasper"
fi
fi
GDAL=$(pwd)/vendor
export GDAL
# set path for post_compile hooks
export GDAL_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgdal.so"
export GEOS_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgeos_c.so"
# set path for runtime environmeht
set_env GDAL_LIBRARY_PATH "/app/.heroku/vendor/lib/libgdal.so"
set_env GEOS_LIBRARY_PATH "/app/.heroku/vendor/lib/libgeos_c.so"
fi
-9
View File
@@ -1,9 +0,0 @@
#!/usr/bin/env bash
# Install Mercurial if it appears to be required.
if [[ -f "requirements.txt" ]]; then
if (grep -Fiq "hg+" requirements.txt) then
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
mcount "steps.mercurial"
fi
fi
+1 -1
View File
@@ -14,7 +14,7 @@
source "$BIN_DIR/utils" source "$BIN_DIR/utils"
# Check that nltk was installed by pip, otherwise obviously not needed # Check that nltk was installed by pip, otherwise obviously not needed
if sp-grep -s nltk; then if is_module_available 'nltk'; then
puts-step "Downloading NLTK corpora…" puts-step "Downloading NLTK corpora…"
nltk_packages_definition="$BUILD_DIR/nltk.txt" nltk_packages_definition="$BUILD_DIR/nltk.txt"
-28
View File
@@ -1,28 +0,0 @@
#!/usr/bin/env bash
set +e
# Install dependencies with Pip.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
if [ ! "$SKIP_PIP_INSTALL" ]; then
if [[ -f .heroku/python/requirements-declared.txt ]]; then
cp .heroku/python/requirements-declared.txt requirements-declared.txt
if ! pip-diff --stale requirements-declared.txt requirements.txt --exclude setuptools pip wheel > .heroku/python/requirements-stale.txt; then
mcount "failure.bad-requirements"
fi
rm -fr requirements-declared.txt
if [[ -s .heroku/python/requirements-stale.txt ]]; then
puts-step "Uninstalling stale dependencies"
/app/.heroku/python/bin/pip uninstall -r .heroku/python/requirements-stale.txt -y --exists-action=w --disable-pip-version-check | cleanup | indent
fi
fi
fi
set -e
+1 -5
View File
@@ -9,10 +9,7 @@ set -e
if [[ -f Pipfile.lock ]]; then if [[ -f Pipfile.lock ]]; then
if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then
if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then
# Measure that we're using Pipenv. # Don't skip installation if there are git deps.
mcount "tool.pipenv"
# Don't skip installation of there are git deps.
if ! grep -q 'git' Pipfile.lock; then if ! grep -q 'git' Pipfile.lock; then
echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent
@@ -72,7 +69,6 @@ if [ ! "$SKIP_PIPENV_INSTALL" ]; then
else else
pipenv-to-pip Pipfile.lock > requirements.txt pipenv-to-pip Pipfile.lock > requirements.txt
"$BIN_DIR/steps/pip-uninstall"
cp requirements.txt .heroku/python/requirements-declared.txt cp requirements.txt .heroku/python/requirements-declared.txt
openssl dgst -sha256 Pipfile.lock > .heroku/python/Pipfile.lock.sha256 openssl dgst -sha256 Pipfile.lock > .heroku/python/Pipfile.lock.sha256
+3
View File
@@ -30,6 +30,9 @@ if [[ -f $BUILD_DIR/Pipfile ]]; then
if [ "$PYTHON" = 3.8 ]; then if [ "$PYTHON" = 3.8 ]; then
echo "$LATEST_38" > "$BUILD_DIR/runtime.txt" echo "$LATEST_38" > "$BUILD_DIR/runtime.txt"
fi fi
if [ "$PYTHON" = 3.9 ]; then
echo "$LATEST_39" > "$BUILD_DIR/runtime.txt"
fi
fi fi
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env bash
# This script serves as the Pylibmc build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
if [[ "$STACK" != "cedar-14" ]]; then
# libmemcached is pre-installed in the stack image so there is no need to vendor it.
return 0
fi
# The location of the pre-compiled libmemcached binary.
VENDORED_MEMCACHED="${VENDOR_URL}/libraries/vendor/libmemcache.tar.gz"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If pylibmc exists within requirements, use vendored libmemcached.
if (pip-grep -s requirements.txt pylibmc &> /dev/null) then
if [ ! -d ".heroku/vendor/lib/sasl2" ]; then
echo "-----> Noticed pylibmc. Bootstrapping libmemcached."
mkdir -p .heroku/vendor
# Download and extract libmemcached into target vendor directory.
curl "$VENDORED_MEMCACHED" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.pylibmc"
fi
LIBMEMCACHED=$(pwd)/vendor
export LIBMEMCACHED
fi
+67 -38
View File
@@ -5,9 +5,10 @@ runtime-fixer runtime.txt
PYTHON_VERSION=$(cat runtime.txt) PYTHON_VERSION=$(cat runtime.txt)
# The location of the pre-compiled python binary. # The location of the pre-compiled python binary.
VENDORED_PYTHON="${VENDOR_URL}/runtimes/$PYTHON_VERSION.tar.gz" VENDORED_PYTHON="${S3_BASE_URL}/${STACK}/runtimes/${PYTHON_VERSION}.tar.gz"
SECURITY_UPDATE="Python has released a security update! Please consider upgrading to" SECURITY_UPDATE="Python has released a security update! Please consider upgrading to"
SECURITY_UPDATE_PYPY="The PyPy project has released a security update! Please consider upgrading to"
ONLY_SUPPORTED_2_VERSION="Only the latest version of Python 2 is supported on the platform. Please consider upgrading to" ONLY_SUPPORTED_2_VERSION="Only the latest version of Python 2 is supported on the platform. Please consider upgrading to"
@@ -15,6 +16,12 @@ PYTHON_2_EOL_UPDATE="Python 2 has reached it's community EOL. Upgrade your Pytho
# check if runtime exists # check if runtime exists
if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
if [[ "$PYTHON_VERSION" == $PY39* ]]; then
if [ "$PYTHON_VERSION" != "$LATEST_39" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_39"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY38* ]]; then if [[ "$PYTHON_VERSION" == $PY38* ]]; then
# do things to alert the user of security release available # do things to alert the user of security release available
if [ "$PYTHON_VERSION" != "$LATEST_38" ]; then if [ "$PYTHON_VERSION" != "$LATEST_38" ]; then
@@ -51,11 +58,9 @@ if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
fi fi
fi fi
if [[ "$PYTHON_VERSION" == $PY27* ]]; then if [[ "$PYTHON_VERSION" == $PY27* ]]; then
puts-warn "$PYTHON_2_EOL_UPDATE"
echo " Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
# security update note # security update note
if [[ "$(date "+%Y")" -gt "2019" ]]; then
puts-warn "$PYTHON_2_EOL_UPDATE"
echo " Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
fi
if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then
puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27" puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
@@ -63,15 +68,15 @@ if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
fi fi
if [[ "$PYTHON_VERSION" == $PYPY27* ]]; then if [[ "$PYTHON_VERSION" == $PYPY27* ]]; then
# security update note # security update note
if [ "$PYTHON_VERSION" != "$PYPY_27" ]; then if [ "$PYTHON_VERSION" != "$LATEST_PYPY_27" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_27}?" puts-warn "$SECURITY_UPDATE_PYPY" "$LATEST_PYPY_27"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi fi
fi fi
if [[ "$PYTHON_VERSION" == $PYPY36* ]]; then if [[ "$PYTHON_VERSION" == $PYPY36* ]]; then
# security update note # security update note
if [ "$PYTHON_VERSION" != "$PYPY_36" ]; then if [ "$PYTHON_VERSION" != "$LATEST_PYPY_36" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_36}?" puts-warn "$SECURITY_UPDATE_PYPY" "$LATEST_PYPY_36"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi fi
fi fi
@@ -81,15 +86,17 @@ else
exit 1 exit 1
fi fi
mcount "version.python.${PYTHON_VERSION}"
if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then
puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache" puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache"
rm -fr .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version rm -rf .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version
fi fi
if [ -f .heroku/python-version ]; then if [ -f .heroku/python-version ]; then
if [ ! "$(cat .heroku/python-version)" = "$PYTHON_VERSION" ]; then if [ ! "$(cat .heroku/python-version)" = "$PYTHON_VERSION" ]; then
puts-step "Found $(cat .heroku/python-version), removing" puts-step "Found $(cat .heroku/python-version), removing"
rm -fr .heroku/python rm -rf .heroku/python
else else
SKIP_INSTALL=1 SKIP_INSTALL=1
fi fi
@@ -120,8 +127,6 @@ if [ ! "$SKIP_INSTALL" ]; then
# Prepare destination directory. # Prepare destination directory.
mkdir -p .heroku/python mkdir -p .heroku/python
mcount "version.python.$PYTHON_VERSION"
if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)." puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support" puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
@@ -131,34 +136,58 @@ if [ ! "$SKIP_INSTALL" ]; then
# Record for future reference. # Record for future reference.
echo "$PYTHON_VERSION" > .heroku/python-version echo "$PYTHON_VERSION" > .heroku/python-version
echo "$STACK" > .heroku/python-stack echo "$STACK" > .heroku/python-stack
FRESH_PYTHON=true
hash -r hash -r
fi fi
# Heroku uses the get-pip utility maintained by the Python community to vendor Pip.
# https://github.com/pypa/get-pip
GETPIP_URL="https://lang-python.s3.amazonaws.com/etc/get-pip.py"
GETPIP_PY="${TMPDIR:-/tmp}/get-pip.py"
if ! curl -s "${GETPIP_URL}" -o "$GETPIP_PY" &> /dev/null; then
mcount "failure.python.get-pip"
echo "Failed to pull down get-pip"
exit 1
fi
# If a new Python has been installed or Pip isn't up to date:
if [ "$FRESH_PYTHON" ] || [[ ! $(pip --version) == *$PIP_UPDATE* ]]; then
puts-step "Installing pip"
# Remove old installations.
rm -fr /app/.heroku/python/lib/python*/site-packages/pip-*
rm -fr /app/.heroku/python/lib/python*/site-packages/setuptools-*
/app/.heroku/python/bin/python "$GETPIP_PY" pip=="$PIP_UPDATE" &> /dev/null
/app/.heroku/python/bin/pip install "$ROOT_DIR/vendor/setuptools-39.0.1-py2.py3-none-any.whl" &> /dev/null
fi
set -e set -e
PIP_VERSION='20.1.1'
# Must use setuptools <47.2.0 until we fix:
# https://github.com/heroku/heroku-buildpack-python/issues/1006
SETUPTOOLS_VERSION='47.1.1'
WHEEL_VERSION='0.34.2'
if [[ "${PYTHON_VERSION}" == ${PY34}* ]]; then
# Python 3.4 support was dropped in pip 19.2+, setuptools 44+ and wheel 0.34+.
PIP_VERSION='19.1.1'
SETUPTOOLS_VERSION='43.0.0'
WHEEL_VERSION='0.33.6'
elif [[ "${PYTHON_VERSION}" == ${PY27}* || "${PYTHON_VERSION}" == ${PYPY27}* ]]; then
# Python 2.7 support was dropped in setuptools 45+.
SETUPTOOLS_VERSION='44.1.1'
fi
# We don't use get-pip.py, since:
# - it uses `--force-reinstall`, which is unnecessary here and slows down repeat builds
# - it means downloading pip twice (once embedded in get-pip.py, and again during
# the install, since get-pip.py can't install the embedded version directly)
# - we would still have to manage several versions of get-pip.py, to support older Pythons.
# Instead, we use the pip wheel to install itself, using the method described here:
# https://github.com/pypa/pip/issues/2351#issuecomment-69994524
PIP_WHEEL_FILENAME="pip-${PIP_VERSION}-py2.py3-none-any.whl"
PIP_WHEEL_URL="${S3_BASE_URL}/common/${PIP_WHEEL_FILENAME}"
PIP_WHEEL="${TMPDIR:-/tmp}/${PIP_WHEEL_FILENAME}"
if ! curl -sSf "${PIP_WHEEL_URL}" -o "$PIP_WHEEL"; then
mcount "failure.python.download-pip"
puts-warn "Failed to download pip"
exit 1
fi
if [[ -f "$BUILD_DIR/Pipfile" ]]; then
# The buildpack is pinned to old pipenv, which requires older pip.
# Pip 9.0.2 doesn't support installing itself from a wheel, so we have to use split
# versions here (ie: installer pip version different from target pip version).
PIP_VERSION='9.0.2'
PIP_TO_INSTALL="pip==${PIP_VERSION}"
else
PIP_TO_INSTALL="${PIP_WHEEL}"
fi
puts-step "Installing pip ${PIP_VERSION}, setuptools ${SETUPTOOLS_VERSION} and wheel ${WHEEL_VERSION}"
/app/.heroku/python/bin/python "${PIP_WHEEL}/pip" install --quiet --disable-pip-version-check --no-cache \
"${PIP_TO_INSTALL}" "setuptools==${SETUPTOOLS_VERSION}" "wheel==${WHEEL_VERSION}"
hash -r hash -r
+15 -6
View File
@@ -2,12 +2,13 @@
shopt -s extglob shopt -s extglob
shopt -s nullglob shopt -s nullglob
# The standard library. # This is necessary since this script is sometimes sourced from
if [[ ! -f /tmp/stdlib.sh ]]; then # subshells that don't have the variables from bin/compile.
curl --retry 3 -s https://lang-common.s3.amazonaws.com/buildpack-stdlib/v8/stdlib.sh > /tmp/stdlib.sh # Remove this once we no longer wrap all the things in `sub_env`.
fi BIN_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=/dev/null ROOT_DIR=$(dirname "${BIN_DIR}")
source /tmp/stdlib.sh # shellcheck source=vendor/buildpack-stdlib_v8.sh
source "${ROOT_DIR}/vendor/buildpack-stdlib_v8.sh"
if [ "$(uname)" == Darwin ]; then if [ "$(uname)" == Darwin ]; then
sed() { command sed -l "$@"; } sed() { command sed -l "$@"; }
@@ -95,3 +96,11 @@ python_sqlite3_check() {
( python2_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_2" ) \ ( python2_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_2" ) \
|| ( python3_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_3" ) || ( python3_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_3" )
} }
is_module_available() {
# Returns 0 is the specified module exists, otherwise returns 1.
# Uses pkgutil rather than pkg_resources or pip's CLI, since pkgutil exists
# in the stdlib, and doesn't depend on the choice of package manager.
local module_name="${1}"
python -c "import sys, pkgutil; sys.exit(0 if pkgutil.find_loader('${module_name}') else 1)"
}
+12 -11
View File
@@ -12,16 +12,6 @@ old-platform() {
fi fi
} }
pylibmc-missing() {
if grep -qi 'fatal error: libmemcached/memcached.h: No such file or directory' "$WARNINGS_LOG"; then
echo
puts-warn "Hello! There was a problem with your build related to libmemcache."
puts-warn "The Python library 'pylibmc' must be explicitly specified in 'requirements.txt' in order to build correctly."
puts-warn "Once you do that, everything should work as expected. -- Much Love, Heroku."
mcount 'warnings.libmemcache'
fi
}
scipy-included() { scipy-included() {
if grep -qi 'running setup.py install for scipy' "$WARNINGS_LOG"; then if grep -qi 'running setup.py install for scipy' "$WARNINGS_LOG"; then
echo echo
@@ -56,11 +46,22 @@ six-included() {
fi fi
} }
gdal-missing() {
if grep -qi 'Could not find gdal-config' "$WARNINGS_LOG"; then
mcount 'warnings.gdal'
echo
puts-warn "Hello! Package installation failed since the GDAL library was not found."
puts-warn "For GDAL, GEOS and PROJ support, use the Geo buildpack alongside the Python buildpack:"
puts-warn "https://github.com/heroku/heroku-geo-buildpack"
puts-warn " -- Much Love, Heroku."
fi
}
show-warnings() { show-warnings() {
old-platform old-platform
pylibmc-missing
scipy-included scipy-included
distribute-included distribute-included
six-included six-included
gdal-missing
} }
+61 -38
View File
@@ -1,57 +1,80 @@
# Python Buildpack Binaries # Python Buildpack Binaries
## Building the Docker Images The binaries for this buildpack are built in Docker containers based on the Heroku stack image.
**After every change to your formulae, perform the following** from the root of the Git repository (not from `builds/`) to rebuild the images for each stack:
$ docker build --pull --tag heroku-python-build-cedar-14 --file $(pwd)/builds/cedar-14.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-16 --file $(pwd)/builds/heroku-16.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-18 --file $(pwd)/builds/heroku-18.Dockerfile .
## Using the Image
You can e.g. `bash` into each of the images you built using their tag:
docker run --rm -ti heroku-python-build-cedar-14 bash
docker run --rm -ti heroku-python-build-heroku-16 bash
docker run --rm -ti heroku-python-build-heroku-18 bash
You then have a shell where you can run `bob build`, `bob deploy`, and so forth. You can of course also invoke these programs directly with `docker run`:
docker run --rm -ti heroku-python-build-heroku-18 bob build runtimes/python-2.7.15
In order to `bob deploy`, AWS credentials must be set up, as well as name and prefix of your custom S3 bucket (unless you're deploying to the Heroku production buckets that are pre-defined in each `Dockerfile`); see next section for details.
## Configuration ## Configuration
File `dockerenv.default` contains a list of required env vars; most of these have default values defined in `Dockerfile`. You can copy this file to a location outside the buildpack and modify it with the values you desire and pass its location with `--env-file`, or pass the env vars to `docker run` using `--env`. In order to publish binaries AWS credentials must be passed to the build container.
If you are testing only the build (ie: `bob build`), these are optional.
Out of the box, each `Dockerfile` has the correct values predefined for `S3_BUCKET`, `S3_PREFIX`, and `S3_REGION`. If you're building your own packages, you'll likely want to change `S3_BUCKET` and `S3_PREFIX` to match your info. Instead of setting `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` into that file, you may also pass them to `docker run` through the environment, or explicitly using `--env`, in order to prevent accidental commits of credentials. In addition, unless you are building the official binaries for Heroku (which use the defaults
specified in each `Dockerfile`), you will need to override `S3_BUCKET` and `S3_PREFIX` to
match your own S3 bucket/use case.
### Passing AWS credentials to the container If you only need to set AWS credentials, you can do so by setting the environment variables
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` before calling the make commands.
If you want to deploy packages and thus need to pass `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, you can either pass them explicitly, through your environment, or through an env file. For example:
#### Passing credentials explicitly ```bash
set +o history # Disable bash history
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
set -o history # Re-enable bash history
make ...
```
docker run --rm -ti -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... heroku-python-build-heroku-18 bash If you need to override the default S3 bucket, or would prefer not to use credentials via
environment variables, then you need to instead use a Docker env file like so:
#### Passing credentials through the environment 1. Copy the `builds/dockerenv.default` env file to a location outside the buildpack repository.
2. Edit the new file, adding at a minimum the values for the variables
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (see Docker
[env-file documentation](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file)).
3. Pass the path of the file to the make commands using `ENV_FILE`. For example:
The two environment variables `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY` are defined in `builds/dockerenv.default`, without values. This will cause Docker to "forward" values for these variables from the current environment, so you can pass them in: ```bash
make ... ENV_FILE=~/.dockerenv.python-buildpack
```
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash ## Launching an interactive build environment
or To start an interactive version of the build environment (ideal for development) use the
`buildenv` make target, passing in the desired `STACK` name. For example:
export AWS_ACCESS_KEY_ID=... ```bash
export AWS_SECRET_ACCESS_KEY=... make buildenv STACK=heroku-18
docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash ```
#### Passing credentials through a separate env file This will create the builder docker image based on the latest image for that stack, and
then start a bash shell where you can run `bob build`, `bob deploy`, and so forth.
This method is the easiest for users who want to build packages in their own S3 bucket, as they will have to adjust the `S3_BUCKET` and `S3_PREFIX` environment variable values anyway from their default values. The `builds/` directory is bind-mounted into the running container, so local build formula
changes will appear there immediately without the need to rebuild the image.
For this method, it is important to keep the credentials file in a location outside the buildpack, so that your credentials aren't accidentally committed. Copy `builds/dockerenv.default` **to a safe location outside the buildpack directory**, and insert your values for `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY`. ## Bulk deploying runtimes
docker run --rm -ti --env-file=../SOMEPATHOUTSIDE/s3.env heroku-python-build-heroku-18 bash When a new Python version is released, binaries have to be generated for multiple stacks.
To automate this, use the `deploy-runtimes` make target, which will ensure the builder
image is up to date, and then run `bob deploy` for each runtime-stack combination.
The build formula name(s) are passed using `RUNTIMES`, like so:
```bash
make deploy-runtimes RUNTIMES='python-X.Y.Z'
```
By default this will deploy to all supported stacks (see `STACKS` in `Makefile`),
but this can be overridden using `STACKS`:
```bash
make deploy-runtimes RUNTIMES='python-X.Y.Z' STACKS='heroku-16 heroku-18'
```
Multiple runtimes can also be specified (useful for when adding a new stack), like so:
```bash
make deploy-runtimes RUNTIMES='python-A.B.C python-X.Y.Z' STACKS='heroku-20'
```
Note: Both `RUNTIMES` and `STACKS` are space delimited.
-15
View File
@@ -1,15 +0,0 @@
FROM heroku/cedar:14
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \
S3_PREFIX="cedar-14/" \
DEBIAN_FRONTEND=noninteractive \
STACK="cedar-14"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev realpath && rm -rf /var/lib/apt/lists/*
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt
COPY . /app
+6 -3
View File
@@ -1,5 +1,8 @@
# Since no values are specified here, these variables will be read from the environment at run time:
# https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
AWS_ACCESS_KEY_ID AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY AWS_SECRET_ACCESS_KEY
S3_BUCKET
S3_PREFIX # Uncomment these if you need to override the default S3 bucket and/or path prefixes.
S3_REGION # S3_BUCKET
# S3_PREFIX
+10 -5
View File
@@ -1,15 +1,20 @@
FROM heroku/heroku:16-build FROM heroku/heroku:16-build
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \ ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \ S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-16/" \ S3_PREFIX="heroku-16/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-16" STACK="heroku-16"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev && rm -rf /var/lib/apt/lists/* RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/ COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app COPY . /app
+10 -5
View File
@@ -1,15 +1,20 @@
FROM heroku/heroku:18-build FROM heroku/heroku:18-build
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \ ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \ S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-18/" \ S3_PREFIX="heroku-18/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-18" STACK="heroku-18"
RUN apt-get update && apt-get install --no-install-recommends -y python-pip-whl=9.0.1-2 python-pip=9.0.1-2 python-setuptools python-wheel libsqlite3-dev && rm -rf /var/lib/apt/lists/* RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/ COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app COPY . /app
+20
View File
@@ -0,0 +1,20 @@
FROM heroku/heroku:20-build
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-20/" \
STACK="heroku-20"
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/
RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app
+1 -1
View File
@@ -19,4 +19,4 @@ make install
# Cleanup # Cleanup
cd .. cd ..
rm -fr sqlite rm -rf sqlite
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal…"
VERSION="2.2.1"
SOURCE_TARBALL="http://download.osgeo.org/gdal/${VERSION}/gdal-${VERSION}.tar.gz"
curl -L $SOURCE_TARBALL | tar zx
pushd "gdal-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building geos…"
VERSION=3.6.2
SOURCE_TARBALL="http://download.osgeo.org/geos/geos-${VERSION}.tar.bz2"
curl -L $SOURCE_TARBALL | tar xj
pushd "geos-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
-45
View File
@@ -1,45 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
if [[ $S3_PREFIX != "cedar-14" ]]; then
echo "libmemcached only needs to be built for cedar-14, since newer stacks include it in the base image"
exit 1
fi
# fail hard
set -o pipefail
# fail harder
set -eux
DEFAULT_VERSION="1.0.18"
dep_version=${VERSION:-$DEFAULT_VERSION}
dep_dirname=libmemcached-${dep_version}
dep_archive_name=${dep_dirname}.tar.gz
dep_url=https://launchpad.net/libmemcached/1.0/${dep_version}/+download/${dep_archive_name}
# SASL Support.
echo "-----> Building cyrus-sasl 2.1.26…"
curl -LO ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.26.tar.gz
# FTP doesn't play well with piping into tar xz
tar xzf cyrus-sasl-2.1.26.tar.gz
pushd cyrus-sasl-2.1.26
./configure --prefix=${OUT_PREFIX} --with-plugindir=${OUT_PREFIX}lib/sasl2 --with-configdir=${OUT_PREFIX}lib/sasl2
make -s -j 9
make install -s
popd
echo "-----> Building libmemcached ${dep_version}…"
curl -L ${dep_url} | tar xz
pushd ${dep_dirname}
CPPFLAGS=-I${OUT_PREFIX}/include LDFLAGS=-L${OUT_PREFIX}/lib ./configure --prefix=${OUT_PREFIX} --without-memcached
make -s -j 9
make install -s
popd
echo "-----> Done."
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal…"
VERSION=4.9.3
SOURCE_TARBALL="http://download.osgeo.org/proj/proj-${VERSION}.tar.gz"
curl -L $SOURCE_TARBALL | tar zx
pushd "proj-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no
make
make install
# Cleanup
popd
+2 -2
View File
@@ -13,11 +13,11 @@ dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full ver
dep_package=${dep_name}-v${dep_version} # it's always "pypy2-…" dep_package=${dep_name}-v${dep_version} # it's always "pypy2-…"
dep_dirname=${dep_package}-linux64 dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2 dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name} dep_url="https://downloads.python.org/pypy/${dep_archive_name}"
echo "Building PyPy…" echo "Building PyPy…"
echo "${dep_url}" echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name curl -fL "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python" ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy2.7
+2 -2
View File
@@ -13,11 +13,11 @@ dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full ver
dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-} dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-}
dep_dirname=${dep_package}-linux64 dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2 dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name} dep_url="https://downloads.python.org/pypy/${dep_archive_name}"
echo "Building PyPy3…" echo "Building PyPy3…"
echo "${dep_url}" echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name curl -fL "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python" ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy3.6
+1 -24
View File
@@ -1,27 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python2
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/2.7.17/Python-2.7.17.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.17 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
+1 -24
View File
@@ -1,27 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python2
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/2.7.18/Python-2.7.18.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.18 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
+1 -29
View File
@@ -1,32 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python3
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.7.2/Python-3.7.2.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.7.2 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
+1 -29
View File
@@ -1,32 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python3
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.4.9/Python-3.4.9.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.4.9 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-5
View File
@@ -1,5 +0,0 @@
#!/usr/bin/env bash
sudo apt-get -qq update
sudo apt-get install software-properties-common
curl --fail --retry 3 --retry-delay 1 --connect-timeout 3 --max-time 30 https://cli-assets.heroku.com/install-ubuntu.sh | sh
Regular → Executable
+4 -4
View File
@@ -14,9 +14,9 @@ case "$choice" in
* ) exit 1;; * ) exit 1;;
esac esac
originMaster=$(git rev-parse origin/master) originMain=$(git rev-parse origin/main)
echo "Tagging commit $originMaster with $newVersion... " echo "Tagging commit $originMain with $newVersion... "
git tag "$newVersion" "${originMaster:?}" git tag "$newVersion" "${originMain:?}"
git push origin refs/tags/$newVersion git push origin refs/tags/$newVersion
heroku buildpacks:publish "$BP_NAME" "$newVersion" heroku buildpacks:publish "$BP_NAME" "$newVersion"
@@ -31,7 +31,7 @@ if [ $(git tag | grep -q latest-version) ]; then
echo "Updating latest-version tag" echo "Updating latest-version tag"
git tag -d latest-version git tag -d latest-version
git push origin :latest-version git push origin :latest-version
git tag latest-version "${originMaster:?}" git tag latest-version "${originMain:?}"
git push --tags git push --tags
fi fi
+1 -1
View File
@@ -1,5 +1,5 @@
--- ---
- - "./repos/python/python-getting-started" - - "./repos/python/python-getting-started"
- master - main
- - "./repos/python/python_default" - - "./repos/python/python_default"
- ca947f69027b2a30be5d26f9a42f25e54f4d7a1a - ca947f69027b2a30be5d26f9a42f25e54f4d7a1a
+5 -2
View File
@@ -1,3 +1,6 @@
# Dependencies for generating/publishing Python binaries.
bob-builder==0.0.19
# Sub-dependencies of bob-builder.
boto==2.49.0
docopt==0.6.2 docopt==0.6.2
bob-builder==0.0.17
boto==2.48.0
+1
View File
@@ -0,0 +1 @@
gdal
-1
View File
@@ -1 +0,0 @@
django
+4
View File
@@ -0,0 +1,4 @@
set -euo pipefail
echo "post_compile ran!"
echo "post_compile env: $(printenv | cut -d '=' -f 1 | sort | xargs)."
+4
View File
@@ -0,0 +1,4 @@
set -euo pipefail
echo "pre_compile ran!"
echo "pre_compile env: $(printenv | cut -d '=' -f 1 | sort | xargs)."
View File
-1
View File
@@ -1 +0,0 @@
python-3.6.6
+1 -1
View File
@@ -6,4 +6,4 @@ verify_ssl = true
requests = "*" requests = "*"
[requires] [requires]
python_full_version = "3.6.3" python_full_version = "3.7.8"
+2 -15
View File
@@ -1,24 +1,11 @@
{ {
"_meta": { "_meta": {
"hash": { "hash": {
"sha256": "22a052f4d1cfe6518b2f236fe45c3208c587a9ab1323bdd390632e27278b541e" "sha256": "8a36860f0f9cb55716222098062cea5c5e0f8127cafb9d0c694de327bac9fbc0"
},
"host-environment-markers": {
"implementation_name": "cpython",
"implementation_version": "3.6.3",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_python_implementation": "CPython",
"platform_release": "16.7.0",
"platform_system": "Darwin",
"platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
"python_full_version": "3.6.3",
"python_version": "3.6",
"sys_platform": "darwin"
}, },
"pipfile-spec": 6, "pipfile-spec": 6,
"requires": { "requires": {
"python_full_version": "3.6.3" "python_full_version": "3.7.8"
}, },
"sources": [ "sources": [
{ {
+1 -1
View File
@@ -1 +1 @@
pypy2.7-7.2.0 pypy2.7-7.3.2
View File
+1
View File
@@ -0,0 +1 @@
pypy2.7-7.3.1
+1 -1
View File
@@ -1 +1 @@
pypy3.6-7.2.0 pypy3.6-7.3.2
View File
+1
View File
@@ -0,0 +1 @@
pypy3.6-7.3.1
-1
View File
@@ -1 +0,0 @@
requests
-1
View File
@@ -1 +0,0 @@
python-2.7.99
-1
View File
@@ -1 +0,0 @@
requests
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.4.99
-1
View File
@@ -1 +0,0 @@
flask
+1 -1
View File
@@ -1 +1 @@
python-3.5.9 python-3.5.10
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.5.99
-1
View File
@@ -1 +0,0 @@
flask
+1 -1
View File
@@ -1 +1 @@
python-3.6.8 python-3.6.12
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.6.99
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.6.7 python-3.6.11
+1 -1
View File
@@ -1 +1 @@
python-3.7.2 python-3.7.9
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.7.99
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.7.1 python-3.7.8
+1 -1
View File
@@ -1 +1 @@
python-3.8.2 python-3.8.7
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.8.0 python-3.8.5
View File
+1
View File
@@ -0,0 +1 @@
python-3.9.0
View File
+1
View File
@@ -0,0 +1 @@
python-3.9.0

Some files were not shown because too many files have changed in this diff Show More