Compare commits

...

77 Commits

Author SHA1 Message Date
Ed Morley d03cfe59b8 Add support for Python 3.8.7
Closes @W-7791264@.
2020-11-20 11:49:34 +00:00
Ed Morley b8e432edf1 Release v186 (#1124)
Closes @W-8440921@.
2020-11-18 13:26:39 +00:00
Ed Morley 74a6c86c4f Add a make target for running a local compile (#1123)
It's often useful to be able to run a compile outside of the unit or
Hatchet tests when developing the buildpack or debugging.

Whilst running the compile locally inside Docker won't fully replicate
a compile on the platform, for many use-cases it's close enough and
importantly gives a faster development feedback loop, since buildpack
changes don't have to be committed/pushed prior to triggering a compile.

Closes @W-8436406@.
2020-11-17 15:19:48 +00:00
Ed Morley 54115fc89b Update the BUILD_WITH_GEO_LIBRARIES error message (#1121)
So that it links to the changelog post now that it exists:
https://devcenter.heroku.com/changelog-items/1947

And to adjust the emphasis towards the migration guide
over the Geo buildpack repository link.

Closes @W-8391415@.
2020-11-13 13:10:52 +00:00
Ed Morley 6a914193b9 Remove env vars used by legacy build systems (#1120)
These variables were used by Anvil and/or the legacy `slug_compiler`
that predated cytokine.

I've not added a changelog entry since none were exported, so wouldn't
have been available to subprocesses anyway.

Closes @W-8387606@.

[skip changelog]
2020-11-12 18:40:31 +00:00
Ed Morley 71aef447a6 Migrate away from sp-grep (#1119)
Switches the last consumers of it to a simpler utility function that
uses `pkgutil.find_loader()`:
https://docs.python.org/3/library/pkgutil.html#pkgutil.find_loader

Both of these consumers are covered by existing tests.

Then removes `sp-grep` and the remaining parts of `pip-pop`.

Closes @W-8208817@.
2020-11-12 17:42:51 +00:00
Ed Morley 768d3fb9e5 Remove vendored pip-diff script (#1118)
Since the last usage was removed in #1117.

Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.

Refs @W-8208817@.
2020-11-12 13:54:14 +00:00
Ed Morley eb44bc03f1 Remove vendored pip-grep script (#1116)
Since the last usage was removed in #1113, and pip-grep is actually
broken on newer pip anyway (though helpfully silently ignores
exceptions, so one wouldn't know).

This is the first of the pip-pop removals, the rest will follow as the
last usages are switched over.

Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.

Refs @W-8208817@.
2020-11-12 13:45:27 +00:00
Ed Morley c112ef81ad Remove unused pip-uninstall step (#1117)
The pip-uninstall step has been unused for pip-using apps since #925,
since the buildpack now invalidates the entire package cache instead.

Whilst the step appears to still be used for pipenv-using apps, the
code is not run, since `SKIP_PIP_INSTALL=1` is set too early.

This bug was introduced in a334672a1a
which landed straight to `master` two days after the feature was
introduced in #650.

Longer term we should likely get pipenv installs to do something similar
to pip (invalidate the whole cache based on checksum of the lockfile),
however for now I'm removing this deadcode since it's the last consumer
of the `pip-diff` script which we want to remove.

Closes @W-8386830@.

[skip changelog]
2020-11-12 13:43:49 +00:00
Ed Morley 5012113d68 Release v185 (#1114)
Closes @W-8381914@.
2020-11-12 07:57:11 +00:00
Ed Morley ce684e4539 Make the BUILD_WITH_GEO_LIBRARIES warning fatal (#1115)
The `BUILD_WITH_GEO_LIBRARIES` feature was removed in the previous PR,
and replaced with a warning that the feature is no longer supported.

After further thought I believe it would be best to make this warning
fatal, to prevent unexpected failures at runtime, if consumers of the
library either aren't invoked during the build, or were previously
installed/cached and were dynamically linked against the library.

Continuation of #1113 / @W-7654424@.
2020-11-11 18:47:45 +00:00
Ed Morley 42076f1bf4 Remove deprecated GDAL/GEOS/PROJ support (#1113)
The standalone Geo buildpack offers more modern GDAL/GEOS/PROJ library
versions, and can be used by apps in all languages, not just Python:
https://github.com/heroku/heroku-geo-buildpack

As such the Python buildpack's undocumented built-in support was
deprecated back in April 2020, with a scheduled removal date of
6th October 2020:
https://devcenter.heroku.com/changelog-items/1759
https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-now-deprecated

Metrics show very few builds continuing to use the built-in support.

Apps with the `BUILD_WITH_GEO_LIBRARIES` env var set will now be shown a
warning directing them to the standalone buildpack, as well as apps that
hit GDAL related pip install errors but aren't using the env var.

This also moves us one step closer to being able to remove
the vendored copy of pip-pop (which is partially broken on
newer pip).

Closes @W-7654424@.
2020-11-11 12:39:30 +00:00
Ed Morley 41f657fbff Remove vendored jq binary (#1112)
Since `jq` is installed in the `-build` stack image for all stacks:
https://github.com/heroku/stack-images/pull/174

Closes @W-7974682@.
2020-11-10 15:34:56 +00:00
Ed Morley 5f6941f04a Remove redundant Mercurial install step (#1111)
Mercurial is installed in the stack image for all stacks, so the
pip install of packages from Mercurial VCS URLs works without the
need for the buildpack to install it itself.

See:
https://github.com/heroku/stack-images/pull/141
https://github.com/heroku/stack-images/search?q=mercurial

Closes @W-7906950@.
2020-11-10 14:33:52 +00:00
Ed Morley 452443d420 Remove support for the Cedar-14 stack (#1110)
Since the stack is end of life and builds have been disabled:
https://devcenter.heroku.com/changelog-items/1943

There are only two temporarily exempted customers using Python, who
can switch to the Cedar-14 support branch if they still need to build
their Python apps (most of which haven't been built recently).

Closes @W-8054727@.
2020-11-10 13:58:33 +00:00
dependabot[bot] c08cad592d Bump rspec from 3.9.0 to 3.10.0 (#1107)
Bumps [rspec](https://github.com/rspec/rspec) from 3.9.0 to 3.10.0.
- [Release notes](https://github.com/rspec/rspec/releases)
- [Commits](https://github.com/rspec/rspec/compare/v3.9.0...v3.10.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-11-10 11:51:30 +00:00
Ed Morley c9504ffd2e Release v184 (#1106) 2020-10-21 14:56:33 +01:00
Ed Morley 96822983ed Add Heroku-20 to deploy-runtimes default stacks list (#1104)
Now `make deploy-runtimes` will build binaries for Heroku-20 by default
too, without the need to pass it in via an explicit `STACKS=...`list.

Closes @W-8233698@.

[skip changelog]
2020-10-19 20:48:34 +01:00
Ed Morley f9d5c0010d Move changelog entries to the correct section (#1103)
The changelog entries for #1099 and #1100 were added to the previous
release section (rather than the "unreleased" section) by mistake.
2020-10-19 13:02:12 +01:00
Ed Morley 9b1a69a1b3 Vendor buildpack-stdlib instead of fetching from S3 (#1100)
Since fetching buildpack-stdlib from S3 has a number of disadvantages:
- it's not possible to grep the repo when trying to work out what
  something from the stdlib is doing
- shellcheck can't fully scan the code, since it similarly doesn't
  have the source
- another compile-time HTTP request that can fail due to transient
  network issues and so reduce reliability
- dependency on the security/release-process of an additional bucket

Since the stdlib is small, doesn't often change, and is not a binary,
it's a great fit for just vendoring in the repo.

The `BIN_DIR` calculation added to `bin/utils` is based on the approach
mentioned here:
https://www.ostricher.com/2014/10/the-right-way-to-get-the-directory-of-a-bash-script/

...rather than copying the version in `bin/compile`, since the latter uses
`$0` so doesn't work with sourced scripts (such as `bin/utils`).

Closes W-8094463.
2020-10-19 12:27:16 +01:00
dependabot[bot] 58dd638fb6 Bump heroku_hatchet from 7.3.1 to 7.3.3 (#1102)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.3.1 to 7.3.3.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.3.1...v7.3.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-19 12:20:27 +01:00
Ed Morley ead59ac7ff Expose BPLOG_PREFIX to sub-shells again (#1099)
In #1011 the number of buildpack variables that are exported (and so
exposed to subprocesses) was reduced, since in general we don't want
to leak buildpack internals into end-user steps such as the pre/post
compile hooks.

However since this change, any buildpack metric emitted from within
a `sub_env` wrapper (which is a buildpack-stdlib utility function)
is missing its `buildpack.python` prefix.

This is because buildpack-stdlib:
* lazy-loads `BPLOG_PREFIX` (rather than doing so when initially
  sourced, which is the approach used for `BUILDPACK_LOG_FILE`)
* doesn't check whether `BPLOG_PREFIX` is set before emitting metrics

See:
https://github.com/heroku/buildpack-stdlib/blob/v8/stdlib.sh

As a stop-gap until we either fix this in buildpack-stdlib (W-8095466),
or remove usages of the `sub_env` wrapper (since I think they are
counter-productive in this buildpack), I've added back the export
for `BPLOG_PREFIX`.

Fixes @W-8095436@.
2020-10-15 15:31:32 +01:00
Ed Morley ac8fd555b8 Release v183 (#1097) 2020-10-12 12:31:44 +01:00
dependabot[bot] f825896c4e Bump heroku_hatchet from 7.3.0 to 7.3.1 (#1095)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.3.0 to 7.3.1.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.3.0...v7.3.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-12 10:01:20 +01:00
Ed Morley fcf696b835 Add support for Heroku-20 (#968)
This adds support for the upcoming Heroku-20 stack.

The Heroku-20 Dockerfile is identical to that for Heroku-18, other
than the base image, and stack-related env var changes.

The initial Python versions made available will be those in:
https://devcenter.heroku.com/articles/python-support#supported-runtimes
https://devcenter.heroku.com/articles/python-support#supported-pypy-runtimes

...minus CPython 2.7, since it's EOL.

Which are:
* `python-3.6.12`
* `python-3.7.9`
* `python-3.8.6`
* `python-3.9.0`
* `pypy2.7-7.3.2`
* `pypy3.6-7.3.2`

Note: Unlike CPython 2.7, the PyPy 2.7 branch is still supported:
https://doc.pypy.org/en/latest/faq.html#how-long-will-pypy-support-python2

In addition, I've generated binaries for each patch release immediately
prior to the latest versions (with the exception of 3.9.0, since there
isn't one), otherwise it's not possible to run the "out of date Python"
warning tests.

The binaries were generated using the process here:
https://github.com/heroku/heroku-buildpack-python/blob/main/builds/README.md

Specifically:

```
make deploy-runtimes STACKS='heroku-20' \
  RUNTIMES='python-3.6.11 python-3.6.12 python-3.7.8 python-3.7.9 python-3.8.5 python-3.8.6 python-3.9.0 pypy2.7-7.3.1 pypy2.7-7.3.2 pypy3.6-7.3.1 pypy3.6-7.3.2' \
  ENV_FILE=...
```

Binaries for the GDAL/GEOS/PROJ feature have not been generated, since
it's deprecated and due for removal shortly:
https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-now-deprecated

Note: Like the Python 3.9.0 release, this uses the new S3 bucket, so
apps will need to be using a recent version of the buildpack in order
to build on Heroku-20:
https://devcenter.heroku.com/articles/python-support#checking-the-python-buildpack-version

Closes @W-7485877@.
2020-10-07 19:44:33 +01:00
Ed Morley a98ef91566 Tests: Clean up the Python version unit tests (#1092)
* Fixes the "Installing <version>" assertions so that they don't false
  positive against the "please upgrade to <version>" output.
* Removes modification of test fixtures during tests, since it can lead
  to failures depending on test order, and confusion when debugging.
* Updates the PyPy version warning tests to use a slightly newer (but
  still not latest) PyPy version, which means that the test now passes
  on Cedar-14 and can be unskipped.
* Switches to using an empty requirements file for version tests that
  duplicate the main test, to save spending time installing dependencies
  unnecessarily.
* Switches the NLTK test to using the default buildpack Python version,
  rather than an ancient Python 3.6.
* Skips the Python 3.7/3.8 tests on Cedar-14 rather than asserting
  failure, since we know they'll never pass due to Cedar-14's libssl being
  older than required.
* Removes redundant `testSqliteInstall` test since it duplicates the
  Python version install tests.

Longer term I'll be moving many of the unit tests to Hatchet, however
this at least makes the tests more dependable in the meantime.

Closes @W-8060219@.
Closes @W-8176779@.

[skip changelog]
2020-10-07 15:10:20 +01:00
Ed Morley e67235f906 Release v182 (#1091) 2020-10-06 10:25:07 +01:00
Ed Morley b1690e9f47 Add support for Python 3.9.0 (#1090)
https://pythoninsider.blogspot.com/2020/10/python-390-is-now-available-and-you-can.html
https://www.python.org/downloads/release/python-390/
https://docs.python.org/release/3.9.0/whatsnew/3.9.html

Binaries generated using:

```
make deploy-runtimes RUNTIMES='python-3.9.0' STACKS='heroku-16 heroku-18' ENV_FILE=...
```

Closes @W-7791272@.
2020-10-06 09:35:36 +01:00
Ed Morley b250300b74 Migrate to a new S3 bucket (#1089)
Since:
* We want the S3 bucket to be owned by a different AWS account and it's
  not possible to transfer ownership of an existing bucket.
* In the future we want to rebuild some of the Python runtime archives
  (for example to improve the sqlite3 handling, or to tweak the compile
  flags used), and it will be easier to reason about the change if we
  can guarantee only recent buildpack versions are using the assets
  rather than several year old unmaintained forks.

The assets were synced from the old bucket using (minus the `--dryrun`):

```
aws s3 sync s3://lang-python s3://heroku-buildpack-python \
  --dryrun \
  --metadata-directive REPLACE \
  --exclude "*" \
  --include 'common/*' \
  --include 'heroku-*/runtimes/*' \
  --include 'heroku-*/libraries/vendor/gdal.tar.gz' \
  --include 'heroku-*/libraries/vendor/geos.tar.gz' \
  --include 'heroku-*/libraries/vendor/proj.tar.gz' \
  --exclude 'common/pip-20.0.2-py2.py3-none-any.whl' \
  --exclude '*/runtimes/*-opt.tar.gz' \
  --exclude '*/runtimes/sqlite-free/*'
```

The files that were `--exclude`d are those that are no longer used,
or test assets that were not officially released.

The Cedar-14 assets were not migrated since it's EOL next month.

The old S3 bucket will be left untouched for the foreseeable future
(ie: we won't be deleting it), since builds using older versions of this
buildpack (either due to pinning to a tag or via a fork) will still be
using assets from it.

Closes @W-8060097@.
2020-10-06 09:23:38 +01:00
dependabot[bot] 096709dcf7 Bump bob-builder from 0.0.18 to 0.0.19 (#1088)
Bumps [bob-builder](https://github.com/heroku-python/bob-builder) from 0.0.18 to 0.0.19.
- [Release notes](https://github.com/heroku-python/bob-builder/releases)
- [Commits](https://github.com/heroku-python/bob-builder/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-10-05 08:47:36 +01:00
Ed Morley ff8945c0c2 Tests: Make assertCapturedSuccess check stderr is empty again (#1087)
`assertCapturedSuccess` used to check that `stderr` was empty, until:
https://github.com/heroku/heroku-buildpack-python/commit/797652a75d69a1fece96a26bf4514fe9e9e1c020#diff-65c067a6f0a3aef292fb54ec21a1fe8cR98

Adding back the assertion exposes some new bugs, which I'll fix in
later PRs. In the meantime affected tests have been adjusted to use
a new `assertCapturedSuccessWithStdErr`.

The test harness output for the failing case has also been improved,
to ease debugging.

Closes @W-7918492@.

[skip changelog]
2020-10-01 17:29:13 +01:00
Ed Morley 4e78b5d57c Remove duplicate copy of shunit2 (#1086)
The tests use `test/shunit2` so the copy under `vendor/` is redundant.

I've added to the CHANGELOG since `vendor/` is added by the buildpack to
`PATH` (something long term we should stop doing), increasing the chance
an app was accidentally relying upon `shunit2` (albeit still unlikely).

Closes @W-8145517@.
2020-10-01 16:02:09 +01:00
Ed Morley b74a41395e Refactor S3 asset URL handling (#1085)
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.

In order to:
* reduce this duplication
* simplify this buildpack's S3 bucket migration (where we'll soon be
  needing the vary the bucket name and wouldn't want to have to
  duplicate that logic in multiple places)
* allow overriding of the URL for the pip use-case

...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.

The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see #989).

The unused `USE_STAGING_BINARIES` environment variable has been
removed, since it's a leftover from the project to stand up a staging S3 bucket.
It's redundant given the `BUILDPACK_S3_BASE_URL` variable.

Closes @W-8142401@.
2020-10-01 10:13:26 +01:00
Ed Morley c550143a59 Use 'rm -rf' instead of 'rm -fr' (#1084)
Not super urgent, but seeing as it closes #927, might as well do now.

[skip changelog]
2020-09-29 15:34:13 +01:00
Ed Morley 838f4c125b Simplify binary build process and documentation (#1083)
* Moves all manual build steps to make targets, to simplify
  the commands run, and reduce chance for error.
* Removes the need to remember to rebuild the builder
  image by building it automatically prior to launching.
* Adds a new make target for deploying multiple runtime
  versions at once to speed up the common case.
* Reduces repetition/superfluous content in documentation.
* Removes unused `S3_REGION` from `dockerenv.default`
  (the contents of S3 buckets inherit the region of the bucket).
* Documents build dependencies in `requirements.txt`.

Closes @W-8119717@.

[skip changelog]
2020-09-29 13:40:56 +01:00
Ed Morley 0020aae078 Release v181 (#1082)
Closes @W-8133372@.
2020-09-29 13:14:41 +01:00
Ed Morley 63651e042f Add support for PyPy 2.7 and 3.6 version 7.3.2 (#1081)
Since they were released yesterday:
https://morepypy.blogspot.com/2020/09/pypy-732-triple-release-python-27-36.html

The archive URL had to be updated now that PyPy has migrated from BitBucket.
The new URLs are from:
https://www.pypy.org/download.html

Skipping PyPy 3.7 for now, since it's in alpha.

Closes @W-8128094@.
2020-09-29 09:30:51 +01:00
Ed Morley c0609a881c Clean up binary generation Dockerfiles (#1080)
* Switches from using Python 2 to Python 3 for `bob-builder`
* Adds `--no-install-recommends` to reduce unnecessary packages
* Removes unnecessary pinning of pip for Heroku-18 build
* Moves `DEBIAN_FRONTEND` to RUN level env var, given:
   https://serverfault.com/a/797318

Refs @W-8119717@.

[skip changelog]
2020-09-29 09:29:33 +01:00
dependabot[bot] ef4696a922 Bump heroku_hatchet from 7.2.0 to 7.3.0 (#1079)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.2.0 to 7.3.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.2.0...v7.3.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-28 10:38:03 +01:00
Ed Morley 6c612a7fb1 Release v180 (#1078)
Closes @W-8120452@.
2020-09-24 19:17:18 +01:00
Ed Morley f91f4ee4ce Add support for Python 3.8.6 (#1072)
https://www.python.org/downloads/release/python-386/
https://pythoninsider.blogspot.com/2020/09/python-386-is-now-available.html

Closes @W-7791243@.
2020-09-24 18:55:27 +01:00
Eric Carmichael eea9a7fd0f Fix code comment typo in bin/steps/pipenv (#1076) 2020-09-23 14:49:46 +01:00
Ed Morley 215a3e3670 Release v179 (#1073) 2020-09-23 11:15:54 +01:00
Ed Morley 4505968fcb Tests: Remove redundant Python version tests (#1075)
There were previously 6 virtually identical tests checking the handling of
a non-existent Python version being specified in `runtime.txt`. 

Only one is necessary - removing the rest will improve CI run time.

Closes @W-8110383@.

[skip changelog]
2020-09-22 18:48:31 +01:00
Ed Morley 37d1474bee Tests: Prevent shadowing of the Python 3.7 version warning test (#1074)
Previously the test for Python 3.8 version warnings was named the same
as an earlier test for Python 3.7, meaning the earlier test definition
was overwritten and so never run.

The later test has now been renamed to the correct version, and the
test ordering adjusted for consistency with the rest of the file.

Closes @W-8110123@.

[skip changelog]
2020-09-22 18:04:11 +01:00
dependabot[bot] 40167d83f6 Bump heroku_hatchet from 7.1.3 to 7.2.0 (#1071)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.3 to 7.2.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.3...v7.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-21 12:41:14 +01:00
Ed Morley 64fb396b73 Remove duplicate pipenv metric event (#1070)
Since the `tool.pipenv` event is being emitted twice per pipenv build,
inflating its usage.

This whole file could do with a massive refactor (4 levels deep of
conditionals is never a good sign), but that can wait until a later PR.

In the future it would also be great to have testing of metrics events.

Closes @W-8094963@.
2020-09-18 19:04:28 +01:00
Ed Morley eb6ee49dfe Emit metrics for how the Python version was chosen (#1069)
Currently an app's Python version can be set via a few different means:
- explicitly by the user (via `runtime.txt` or `Pipfile.lock`)
- implicitly via the sticky versions feature (for existing apps)
- implicitly via default version for new apps / those with empty cache

In order to determine the priority of features like automatic Python
patch version upgrades for sticky-versioned apps, it's useful to have
metrics for these.

There were previously no tests for either the sticky versions feature,
or changing the Python version by updating the `runtime.txt` file, so
I've added some now (given that I updated the conditional to add the
metrics, so useful to have coverage).

I've also removed the confusing overwrite of `DEFAULT_PYTHON_VERSION`
with the cached version, and kept them as two separate variables.

Closes @W-8099632@.
Closes @W-8099645@.
2020-09-18 18:48:57 +01:00
Ed Morley 64abfb2978 Emit Python version metric events for all builds (#1066)
Previously the metric events describing the chosen Python version were
only emitted when that Python version was installed, and not when it
was being used from the build cache (the common case).

Now the version is emitted for all builds, improving visibility into
the distribution of Python usage, and helping determine the priority
of features like opt-in automatic Python patch updates.

Closes @W-8059668@.
2020-09-16 12:28:15 +01:00
Ed Morley 12d1cbb3be Tests: Switch to branch 'main' of python-getting-started repo (#1065)
Since the default branch for that repository has been changed from
`master` to `main` in W-7903771.

Closes @W-8047331@.

[skip changelog]
2020-09-16 12:21:53 +01:00
dependabot[bot] 7817aa3fc3 Bump heroku_hatchet from 7.1.1 to 7.1.3 (#1064)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.1 to 7.1.3.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.1...v7.1.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-14 08:19:46 +01:00
Ed Morley ae56342a81 Release v178 (#1063) 2020-09-07 13:12:36 +01:00
Ed Morley 3e49aeb940 Add support for Python 3.5.10 (#1062)
Since it was released over the weekend:
https://www.python.org/downloads/release/python-3510/
https://docs.python.org/3.5/whatsnew/changelog.html#python-3-5-10

Closes @W-7835961@.
2020-09-07 13:01:09 +01:00
dependabot[bot] a91a5427de Bump heroku_hatchet from 7.1.0 to 7.1.1 (#1061)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.1.0 to 7.1.1.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.1.0...v7.1.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-09-07 10:28:06 +01:00
dependabot[bot] ab69658efb Bump heroku_hatchet from 7.0.0 to 7.1.0 (#1057)
Bumps [heroku_hatchet](https://github.com/heroku/hatchet) from 7.0.0 to 7.1.0.
- [Release notes](https://github.com/heroku/hatchet/releases)
- [Changelog](https://github.com/heroku/hatchet/blob/main/CHANGELOG.md)
- [Commits](https://github.com/heroku/hatchet/compare/v7.0.0...v7.1.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

[skip changelog]
2020-08-24 10:24:16 +01:00
Ed Morley dfbe8ddaf5 Release v177 (#1055)
Closes @W-7975422@.
2020-08-18 12:50:17 +01:00
Ed Morley 60b9d1a562 Add support for Python 3.6.12 and 3.7.9 (#1054)
Since they were released yesterday:
https://www.python.org/downloads/release/python-3612/
https://www.python.org/downloads/release/python-379/

Closes @W-7975179@.
Closes @W-7975181@.
2020-08-18 12:38:54 +01:00
Ed Morley 3fa3f15d35 Remove unnecessary date conditional for Python 2 EOL warning (#1053)
Since we're past the end of 2019, so the conditional is always true.

Closes @W-7952394@.

[skip changelog]
2020-08-12 22:49:31 +01:00
Ed Morley a510c47b26 Make etc/publish.sh executable (#1052)
Since otherwise:

```
$ ./etc/publish.sh
-bash: ./etc/publish.sh: Permission denied
```

Closes @W-7952258@.

[skip changelog]
2020-08-12 22:48:59 +01:00
Ed Morley 1972e6094e Travis: Switch to Ubuntu 20.04 base image (#1051)
Since it's a smaller, more up to date image, with fewer services
starting at boot, improving start times:
https://blog.travis-ci.com/2020-08-10-focal-build-environment
https://docs.travis-ci.com/user/reference/focal/

Closes @W-7951908@.

[skip changelog]
2020-08-12 22:48:48 +01:00
Ed Morley 84ac34b1d4 Release v176 (#1050) 2020-08-12 16:17:33 +01:00
Ed Morley 7c4f55bc32 Switch the remaining build formulas to the shared script approach (#1049)
To be consistent with the other build formulas after #799.

Refs @W-7947035@.

[skip changelog]
2020-08-12 15:19:53 +01:00
Ed Morley eabe71d578 Update the Python 3.4.10 build script to use the correct Python version (#1048)
The existing Python 3.4.10 archive actually contained Python 3.7.2,
since the version in the source URL was not updated when the file was
created in #813.

The build formula now uses the shared build script approach like all of
the other build scripts, which ensures the version can never get out of
sync (since it's extracted from the formula filename).

The build for Heroku-18 failed to compile `_ssl` properly (even though
the build exited zero) since Python 3.4.10 is old enough it doesn't work
well with libssl1.1. Installing `libssl1.0-dev` in the build image
locally resolved the issue - however we don't want to use that in the
future for newer Python, so I've not updated the `heroku-18.Dockerfile`.

In addition, with the rebuilt archives the tests now pass on Cedar-14,
so no longer need to be marked as failing.

Closes @W-7947035@.
2020-08-12 15:19:31 +01:00
Ed Morley 95fca53d38 Travis: Correctly skip Hatchet on PRs from forks (#1045)
The Hatchet run requires a valid Heroku login, the credentials for which
are set via Travis secure environment variables, which by design are not
revealed to PRs from forks.

The previous conditional wasn't working as intended - the Hatchet job
was still being triggered for forks from PRs.

The new conditional fixes this, and also means that forks could set
their own credentials via Travis environment variables if they wanted
a way to run the tests in CI on their own repo.

See:
https://docs.travis-ci.com/user/conditions-v1

Fixes #1019.
Closes @W-7918482@.

[skip changelog]
2020-08-12 11:17:38 +01:00
Ed Morley ff4f677435 Travis: Remove Hatchet credentials from .travis.yml (#1047)
Since they are now set via the Travis repository level secrets feature
instead. This both works around the Travis bug seen in #1045, and also
means its easier to set up Travis on forks, since otherwise the
`.travis.yml` secrets would overwrite the global secrets.

As part of this move the test account used has also been changed, and
will be documented here:
https://github.com/heroku/languages-team/blob/main/guides/create_test_users_for_ci.md#known-usernames

Closes @W-7949880@.

[skip changelog]
2020-08-12 10:49:57 +01:00
Ed Morley cba3201ac2 Rename confusing PYPY_* variables (#1044)
Since these variables refer to the latest version of PyPy, compared to
the similarly named `PYPY27` and `PYPY36` variables (ie same name except
without the underscore) which refer to the major/minor version only.

The similar names caused me to use the wrong one locally whilst working
on another PR, which was caught by tests but demonstrates why we should
rename them.

Closes @W-7935256@.

[skip changelog]
2020-08-11 19:31:53 +01:00
Ed Morley ac29db32f8 Remove unused vendor/test-utils (#1043)
Since the unit tests instead use the utilities in this separate file:
https://github.com/heroku/heroku-buildpack-python/blob/419ef479969c4d5945f2c0620292229ef464f4c8/test/utils

A changelog entry has been added since whilst this file is for internal
testing only, the buildpack's `vendor/` directory is put on `PATH`, so
in theory it could have been called outside the buildpack (though this
seems extremely unlikely since the script isn't very useful externally).

Fixes #1027.
Closes @W-7918496@.
2020-08-11 19:31:07 +01:00
Ed Morley e83576f6b4 Remove unused .pre-commit-config.yaml (#1042)
Since we don't use that tool (<https://pre-commit.com>), and there are
better alternatives should we want to expand coverage of these kind of
things.

Closes @W-7923935@.

[skip changelog]
2020-08-11 19:21:40 +01:00
Ed Morley 43600f25a5 Remove unused etc/ci-setup.sh (#1041)
It stopped being used as of #781.

```
$ rg ci-setup.sh --stats
0 matches

$ git-content-search ci-setup.sh
e7da63f update to newer hatchet integration
M      .travis.yml
576def4 fix travis dependency blocker
M      .travis.yml

$ git show e7da63f | rg ci-setup.sh -C 1
-before_install:
- - sudo bash etc/ci-setup.sh
+ - bundle exec hatchet ci:setup
```

Hatchet embeds its own setup script, which is called via the rake task:
https://github.com/heroku/hatchet/blob/v6.0.0/etc/ci_setup.rb

Closes @W-7923930@.

[skip changelog]
2020-08-11 19:21:03 +01:00
Ed Morley f508bd538d Fix the security update version check message for PyPy (#1040)
Previously if an app was using an older version of PyPy, the buildpack
would show a confusing "Could not find that version" message (even
though the version was found), when it really meant to warn about there
being a newer release available.

It looks like the version check messages were perhaps copied and pasted
from something else, but the message wording not updated at the time.

I've also added tests since there were none for this feature.

Fixes #1004.
Closes @W-7918745@.
2020-08-11 19:15:16 +01:00
Richard Schneeman bc7e34dbad Tests: Update Hatchet from 6.0.0 to 7.0.0 (#1046)
- ActiveSupport's Object#blank? and Object#present? are no longer provided by default (https://github.com/heroku/hatchet/pull/107)
- Remove deprecated support for passing a block to `App#run` (https://github.com/heroku/hatchet/pull/105)
- Ignore  403 on app delete due to race condition (https://github.com/heroku/hatchet/pull/101)
- The hatchet.lock file can now be locked to "main" in addition to "master" (https://github.com/heroku/hatchet/pull/86)
- Allow concurrent one-off dyno runs with the `run_multi: true` flag on apps (https://github.com/heroku/hatchet/pull/94)
- Apps are now marked as being "finished" by enabling maintenance mode on them when `teardown!` is called. Finished apps can be reaped immediately (https://github.com/heroku/hatchet/pull/97)
- Applications that are not marked as "finished" will be allowed to live for a HATCHET_ALIVE_TTL_MINUTES duration before they're deleted by the reaper to protect against deleting an app mid-deploy, default is seven minutes (https://github.com/heroku/hatchet/pull/97)
- The HEROKU_APP_LIMIT env var no longer does anything, instead hatchet application reaping is manually executed if an app cannot be created (https://github.com/heroku/hatchet/pull/97)
- App#deploy without a block will no longer run `teardown!` automatically (https://github.com/heroku/hatchet/pull/97)
- Calls to `git push heroku` are now rate throttled (https://github.com/heroku/hatchet/pull/98)
- Calls to `app.run` are now rate throttled (https://github.com/heroku/hatchet/pull/99)
- Deployment now raises and error when the release failed (https://github.com/heroku/hatchet/pull/93)

[skip changelog]
2020-08-11 19:06:18 +01:00
dependabot[bot] 419ef47996 Bump bob-builder from 0.0.17 to 0.0.18 (#1039)
Bumps [bob-builder](https://github.com/heroku-python/bob-builder) from 0.0.17 to 0.0.18.
- [Release notes](https://github.com/heroku-python/bob-builder/releases)
- [Commits](https://github.com/heroku-python/bob-builder/compare/v0.0.17...v0.0.18)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-10 14:25:42 +01:00
dependabot[bot] c1f6679dd7 Bump boto from 2.48.0 to 2.49.0 (#1038)
Bumps [boto](https://github.com/boto/boto) from 2.48.0 to 2.49.0.
- [Release notes](https://github.com/boto/boto/releases)
- [Commits](https://github.com/boto/boto/compare/2.48.0...2.49.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-08-10 13:11:48 +01:00
Ed Morley de81e41336 Add config file for Dependabot (#1037)
GitHub are gradually integrating Dependabot into GitHub as a native
feature. The native integration now supports updating not only security
updates, but all dependencies - however this must be explicitly enabled
by adding a config file:
https://docs.github.com/en/github/administering-a-repository/enabling-and-disabling-version-updates

The config options are described here:
https://docs.github.com/en/github/administering-a-repository/configuration-options-for-dependency-updates

Closes @W-7937066@.

[skip changelog]
2020-08-10 12:30:32 +01:00
Ed Morley 99c1027cb2 Tests: Update Hatchet dependencies (#1036)
* `bundler update`
* Ruby 2.6 -> 2.7
* Remove unnecessary bundler version downgrade

Of note this picks up:
https://github.com/grosser/parallel_split_test/pull/20

@W-7930909@

[skip changelog]
2020-08-10 11:29:34 +01:00
Ed Morley 46d3d2b042 Travis: Move hatchet ci:setup to before_script (#1035)
So that any failures during `hatchet ci:setup` cause the build to fail
early, rather than try to proceed with running the Hatchet tests.

@W-7929878@

[skip changelog]
2020-08-07 17:53:33 +01:00
Ed Morley 7b4f16145b Add a configuration file for Git2Gus (#1034)
This will allow for one-way sync of GitHub issues in this repository
into our internal issues tracker, GUS. Issues are only synced when the
specified GitHub label is added.

In the future I may switch the chosen label to just be the standard
`t: bug` type labels, but for now I'm choosing a separate label so that
we have more control over what is synced.

See:
https://lwc-gus-bot.herokuapp.com/#getting-started

@W-7918433@
2020-08-05 10:48:25 +01:00
108 changed files with 877 additions and 3053 deletions
+6
View File
@@ -0,0 +1,6 @@
{
"productTag": "a1aB0000000MR0RIAW",
"issueTypeLabels": { "gus: story": "USER STORY", "gus: bug": "BUG P3" },
"defaultBuild": "Heroku Unscheduled",
"statusWhenClosed": "CLOSED"
}
+15
View File
@@ -0,0 +1,15 @@
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
labels:
- "c: dependencies"
- package-ecosystem: "bundler"
directory: "/"
schedule:
interval: "weekly"
labels:
- "c: dependencies"
-7
View File
@@ -1,7 +0,0 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
+8 -11
View File
@@ -1,5 +1,5 @@
language: minimal language: minimal
dist: bionic dist: focal
branches: branches:
only: only:
- main - main
@@ -12,22 +12,17 @@ jobs:
- name: Bash linting (shellcheck) - name: Bash linting (shellcheck)
script: make check script: make check
- name: Hatchet integration tests - name: Hatchet integration tests
if: env(TRAVIS_PULL_REQUEST_SLUG) = env(TRAVIS_REPO_SLUG) if: env(HEROKU_API_USER) IS present AND env(HEROKU_API_KEY) IS present
language: ruby language: ruby
rvm: rvm:
- 2.6.6 - 2.7
before_script: before_script:
- gem install bundler -v 1.16.2
script:
- bundle exec hatchet ci:setup - bundle exec hatchet ci:setup
script:
- PARALLEL_SPLIT_TEST_PROCESSES=11 bundle exec parallel_split_test spec/hatchet/ - PARALLEL_SPLIT_TEST_PROCESSES=11 bundle exec parallel_split_test spec/hatchet/
env: env:
jobs: jobs:
- STACK=cedar-14 TEST_CMD=test/run-deps
- STACK=cedar-14 TEST_CMD=test/run-versions
- STACK=cedar-14 TEST_CMD=test/run-features
- STACK=heroku-16 TEST_CMD=test/run-deps - STACK=heroku-16 TEST_CMD=test/run-deps
- STACK=heroku-16 TEST_CMD=test/run-versions - STACK=heroku-16 TEST_CMD=test/run-versions
- STACK=heroku-16 TEST_CMD=test/run-features - STACK=heroku-16 TEST_CMD=test/run-features
@@ -35,10 +30,12 @@ env:
- STACK=heroku-18 TEST_CMD=test/run-deps - STACK=heroku-18 TEST_CMD=test/run-deps
- STACK=heroku-18 TEST_CMD=test/run-versions - STACK=heroku-18 TEST_CMD=test/run-versions
- STACK=heroku-18 TEST_CMD=test/run-features - STACK=heroku-18 TEST_CMD=test/run-features
- STACK=heroku-20 TEST_CMD=test/run-deps
- STACK=heroku-20 TEST_CMD=test/run-versions
- STACK=heroku-20 TEST_CMD=test/run-features
global: global:
- HATCHET_RETRIES=3 - HATCHET_RETRIES=3
- IS_RUNNING_ON_CI=true - IS_RUNNING_ON_CI=true
- HATCHET_APP_LIMIT=80 - HATCHET_APP_LIMIT=80
- HATCHET_DEPLOY_STRATEGY=git - HATCHET_DEPLOY_STRATEGY=git
- secure: yjtlPE5FbVxTKnjUy/tZUBgSEf4qADD3QOxtgziuid73S0U/1IEXlMGFULsQzIjtlHKmHeywZqpVVEpthIH4RuT7uoX1Pb7SSM/g0T8fT3VoEFbFK1uYl0oZQbUS4Klxv9tPiumj8if3m6ULEGIz1X0wZcMOC0tMLwVCnwmap0E=
- secure: ZeFTHWwnpIKE9nAqs88ocmiQh7bKce84lilGm5J23nf3N6V4wNyLwqlkvsM008WGBCaOg9AUx7ZunasT0ANsR5gLP3eV2UUg7ILdRgV2Gy13eNRFheC4PHdN92RqQ3aKoqlIv2K999xlhVjod0NzhkQQXB6PddfQINbuU7ks6As=
+63
View File
@@ -2,6 +2,69 @@
## Unreleased ## Unreleased
- Python 3.8.7 is now available (CPython) (#1122).
## v186 (2020-11-18)
- Update the `BUILD_WITH_GEO_LIBRARIES` error message (#1121).
- Switch NLTK feature detection away from `sp-grep` (#1119).
- Switch Django collectstatic feature detection away from `sp-grep` (#1119).
- Remove vendored `sp-grep` script (#1119).
- Remove vendored `pip-diff` script (#1118).
- Remove vendored `pip-grep` script (#1116).
## v185 (2020-11-12)
- Error if the unsupported `BUILD_WITH_GEO_LIBRARIES` env var is set (#1115).
- Remove deprecated GDAL/GEOS/PROJ support (#1113).
- Remove vendored `jq` binary (#1112).
- Remove redundant Mercurial install step (#1111).
- Remove support for the Cedar-14 stack (#1110).
## v184 (2020-10-21)
- Vendor buildpack-stdlib instead of fetching from S3 (#1100).
- Fix metric names for metrics emitted within `sub_env` (#1099).
## v183 (2020-10-12)
- Add support for Heroku-20 (#968).
## v182 (2020-10-06)
- Python 3.9.0 is now available (CPython) (#1090).
- Migrate from the `lang-python` S3 bucket to `heroku-buildpack-python` (#1089).
- Remove `vendor/shunit2` (#1086).
- Replace `BUILDPACK_VENDOR_URL` and `USE_STAGING_BINARIES` with `BUILDPACK_S3_BASE_URL` (#1085).
## v181 (2020-09-29)
- PyPy 2.7 and 3.6, version 7.3.2 are now available (Note: PyPy support is in beta) (#1081).
## v180 (2020-09-24)
- Python 3.8.6 is now available (CPython) (#1072).
## v179 (2020-09-23)
- Remove duplicate pipenv metric event (#1070).
- Emit metrics for how the Python version was chosen for an app (#1069).
- Emit Python version metric events for all builds, not just clean installs (#1066).
## v178 (2020-09-07)
- Python 3.5.10 is now available (CPython) (#1062).
## v177 (2020-08-18)
- Python 3.6.12 and 3.7.9 are now available (CPython) (#1054).
- The default Python version for new apps is now 3.6.12 (previously 3.6.11) (#1054).
## v176 (2020-08-12)
- Rebuild the Python 3.4.10 archives with the correct version of Python (#1048).
- Fix the security update version check message for apps using PyPy (#1040).
- Remove `vendor/test-utils` (#1043).
## v175 (2020-08-05) ## v175 (2020-08-05)
+27 -45
View File
@@ -1,68 +1,50 @@
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
activesupport (6.0.3.1) diff-lcs (1.4.4)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
tzinfo (~> 1.1)
zeitwerk (~> 2.2, >= 2.2.2)
concurrent-ruby (1.1.6)
diff-lcs (1.3)
erubis (2.7.0) erubis (2.7.0)
excon (0.73.0) excon (0.78.0)
heroics (0.0.25) heroics (0.1.1)
erubis (~> 2.0) erubis (~> 2.0)
excon excon
moneta moneta
multi_json (>= 1.9.2) multi_json (>= 1.9.2)
heroku_hatchet (5.0.3) heroku_hatchet (7.3.3)
excon (~> 0) excon (~> 0)
minitest-retry (~> 0.1.9) platform-api (~> 3)
platform-api (~> 2)
repl_runner (~> 0.0.3)
rrrretry (~> 1) rrrretry (~> 1)
thor (~> 0) thor (~> 1)
threaded (~> 0) threaded (~> 0)
i18n (1.8.2)
concurrent-ruby (~> 1.0)
minitest (5.14.1)
minitest-retry (0.1.9)
minitest (>= 5.0)
moneta (1.0.0) moneta (1.0.0)
multi_json (1.14.1) multi_json (1.15.0)
parallel (1.19.1) parallel (1.19.2)
parallel_split_test (0.7.0) parallel_split_test (0.8.0)
parallel (>= 0.5.13) parallel (>= 0.5.13)
rspec (>= 3.1.0) rspec (>= 3.1.0)
platform-api (2.2.0) platform-api (3.0.0)
heroics (~> 0.0.25) heroics (~> 0.1.1)
moneta (~> 1.0.0) moneta (~> 1.0.0)
rake (12.3.3) rate_throttle_client (~> 0.1.0)
repl_runner (0.0.3) rake (13.0.1)
activesupport rate_throttle_client (0.1.2)
rrrretry (1.0.0) rrrretry (1.0.0)
rspec (3.8.0) rspec (3.10.0)
rspec-core (~> 3.8.0) rspec-core (~> 3.10.0)
rspec-expectations (~> 3.8.0) rspec-expectations (~> 3.10.0)
rspec-mocks (~> 3.8.0) rspec-mocks (~> 3.10.0)
rspec-core (3.8.0) rspec-core (3.10.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-expectations (3.8.1) rspec-expectations (3.10.0)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-mocks (3.8.0) rspec-mocks (3.10.0)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0) rspec-support (~> 3.10.0)
rspec-retry (0.6.1) rspec-retry (0.6.2)
rspec-core (> 3.3) rspec-core (> 3.3)
rspec-support (3.8.0) rspec-support (3.10.0)
thor (0.20.3) thor (1.0.1)
thread_safe (0.3.6)
threaded (0.0.4) threaded (0.0.4)
tzinfo (1.2.7)
thread_safe (~> 0.1)
zeitwerk (2.3.0)
PLATFORMS PLATFORMS
ruby ruby
+42 -36
View File
@@ -1,55 +1,61 @@
# These targets are not files # These targets are not files
.PHONY: check test buildenv-heroku-16 buildenv-heroku-18 tools .PHONY: check test compile builder-image buildenv deploy-runtimes tools
STACK ?= heroku-18 STACK ?= heroku-18
STACKS ?= heroku-16 heroku-18 heroku-20
TEST_CMD ?= test/run-versions && test/run-features && test/run-deps TEST_CMD ?= test/run-versions && test/run-features && test/run-deps
FIXTURE ?= test/fixtures/requirements-standard
ENV_FILE ?= builds/dockerenv.default
BUILDER_IMAGE_PREFIX := heroku-python-build
ifeq ($(STACK),cedar-14) # Converts a stack name of `heroku-NN` to its build Docker image tag of `heroku/heroku:NN-build`.
# Cedar-14 doesn't have a build image varient. STACK_IMAGE_TAG := heroku/$(subst -,:,$(STACK))-build
IMAGE_TAG := heroku/cedar:14
else
# Converts a stack name of `heroku-NN` to its build Docker image tag of `heroku/heroku:NN-build`.
IMAGE_TAG := heroku/$(subst -,:,$(STACK))-build
endif
check: check:
@shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings bin/default_pythons @shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings bin/default_pythons
@shellcheck -x bin/steps/collectstatic bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/gdal bin/steps/geo-libs bin/steps/mercurial bin/steps/nltk bin/steps/pip-install bin/steps/pip-uninstall bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/pylibmc bin/steps/python @shellcheck -x bin/steps/collectstatic bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/nltk bin/steps/pip-install bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/python
@shellcheck -x bin/steps/hooks/* @shellcheck -x bin/steps/hooks/*
test: test:
@echo "Running tests using: STACK=$(STACK) TEST_CMD='$(TEST_CMD)'" @echo "Running tests using: STACK=$(STACK) TEST_CMD='$(TEST_CMD)'"
@echo ""
@docker run --rm -it -v $(PWD):/buildpack:ro -e "STACK=$(STACK)" "$(IMAGE_TAG)" bash -c 'cp -r /buildpack /buildpack_test && cd /buildpack_test && $(TEST_CMD)'
@echo ""
buildenv-heroku-16:
@echo "Creating build environment (heroku-16)..."
@echo @echo
@docker build --pull -f $(shell pwd)/builds/heroku-16.Dockerfile -t python-buildenv-heroku-16 . @docker run --rm -it -v $(PWD):/buildpack:ro -e "STACK=$(STACK)" "$(STACK_IMAGE_TAG)" bash -c 'cp -r /buildpack /buildpack_test && cd /buildpack_test && $(TEST_CMD)'
@echo
compile:
@echo "Running compile using: STACK=$(STACK) FIXTURE=$(FIXTURE)"
@echo
@docker run --rm -it -v $(PWD):/src:ro -e "STACK=$(STACK)" -w /buildpack "$(STACK_IMAGE_TAG)" \
bash -c 'cp -r /src/{bin,vendor} /buildpack && cp -r /src/$(FIXTURE) /build && mkdir /cache /env && bin/compile /build /cache /env'
@echo
builder-image:
@echo "Generating binary builder image for $(STACK)..."
@echo
@docker build --pull -f builds/$(STACK).Dockerfile -t "$(BUILDER_IMAGE_PREFIX)-$(STACK)" .
@echo
buildenv: builder-image
@echo "Starting build environment for $(STACK)..."
@echo @echo
@echo "Usage..." @echo "Usage..."
@echo @echo
@echo " $$ export AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=bar # Optional unless deploying" @echo " $$ bob build runtimes/python-X.Y.Z"
@echo " $$ bob build runtimes/python-2.7.13"
@echo " $$ bob deploy runtimes/python-2.7.13"
@echo @echo
@docker run -it --rm python-buildenv-heroku-16 @docker run --rm -it --env-file="$(ENV_FILE)" -v $(PWD)/builds:/app/builds "$(BUILDER_IMAGE_PREFIX)-$(STACK)" bash
buildenv-heroku-18: deploy-runtimes:
@echo "Creating build environment (heroku-18)..." ifndef RUNTIMES
$(error No runtimes specified! Use: "make deploy-runtimes RUNTIMES='python-X.Y.Z ...' [STACKS='heroku-18 ...'] [ENV_FILE=...]")
endif
@echo "Using: RUNTIMES='$(RUNTIMES)' STACKS='$(STACKS)' ENV_FILE='$(ENV_FILE)'"
@echo @echo
@docker build --pull -f $(shell pwd)/builds/heroku-18.Dockerfile -t python-buildenv-heroku-18 . @set -eu; for stack in $(STACKS); do \
@echo $(MAKE) builder-image STACK=$${stack}; \
@echo "Usage..." for runtime in $(RUNTIMES); do \
@echo echo "Generating/deploying $${runtime} for $${stack}..."; \
@echo " $$ export AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=bar # Optional unless deploying" echo; \
@echo " $$ bob build runtimes/python-2.7.13" docker run --rm -it --env-file="$(ENV_FILE)" "$(BUILDER_IMAGE_PREFIX)-$${stack}" bob deploy "runtimes/$${runtime}"; \
@echo " $$ bob deploy runtimes/python-2.7.13" echo; \
@echo done; \
@docker run -it --rm python-buildenv-heroku-18 done
tools:
git clone https://github.com/kennethreitz/pip-pop.git
mv pip-pop/bin/* vendor/pip-pop/
rm -fr pip-pop
-47
View File
@@ -1,27 +1,5 @@
This buildpack includes some vendorized packages to ease installation. This buildpack includes some vendorized packages to ease installation.
jq license
----------
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
shunit2 license shunit2 license
--------------- ---------------
@@ -85,28 +63,3 @@ The Free Software Foundation may publish revised and/or new versions of the GNU
Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation. Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library. If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.
pip-pop license
---------------
The MIT License (MIT)
Copyright (c) 2014 Kenneth Reitz.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+6 -5
View File
@@ -26,7 +26,7 @@ remote: Compressing source files... done.
remote: Building source: remote: Building source:
remote: remote:
remote: -----> Python app detected remote: -----> Python app detected
remote: -----> Installing python-3.7.4 remote: -----> Installing python
remote: -----> Installing pip remote: -----> Installing pip
remote: -----> Installing SQLite3 remote: -----> Installing SQLite3
remote: -----> Installing requirements with pip remote: -----> Installing requirements with pip
@@ -44,7 +44,7 @@ A `requirements.txt` must be present at the root of your application's repositor
To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version. To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version.
Current default Python Runtime: Python 3.6.9 Current default Python Runtime: Python 3.6.12
Alternatively, you can provide a `setup.py` file, or a `Pipfile`. Alternatively, you can provide a `setup.py` file, or a `Pipfile`.
Using `pipenv` will generate `runtime.txt` at build time if one of the field `python_version` or `python_full_version` is specified in the `requires` section of your `Pipfile`. Using `pipenv` will generate `runtime.txt` at build time if one of the field `python_version` or `python_full_version` is specified in the `requires` section of your `Pipfile`.
@@ -62,9 +62,10 @@ Specify a Python Runtime
Supported runtime options include: Supported runtime options include:
- `python-3.8.5` - `python-3.9.0`
- `python-3.7.8` - `python-3.8.7`
- `python-3.6.11` - `python-3.7.9`
- `python-3.6.12`
- `python-2.7.18` - `python-2.7.18`
## Tests ## Tests
+51 -53
View File
@@ -15,10 +15,8 @@
# Fail fast and fail hard. # Fail fast and fail hard.
set -eo pipefail set -eo pipefail
# Boostrap the Buildpack Standard Library. # Used by buildpack-stdlib's metrics features.
# Disable unused env var warning since shellcheck doesn't know about the stdlib. export BPLOG_PREFIX="buildpack.python"
# shellcheck disable=2034
BPLOG_PREFIX="buildpack.python"
export BUILDPACK_LOG_FILE=${BUILDPACK_LOG_FILE:-/dev/null} export BUILDPACK_LOG_FILE=${BUILDPACK_LOG_FILE:-/dev/null}
[ "$BUILDPACK_XTRACE" ] && set -o xtrace [ "$BUILDPACK_XTRACE" ] && set -o xtrace
@@ -37,23 +35,19 @@ ENV_DIR=$3
# Export Path variables, for use in sub-scripts. # Export Path variables, for use in sub-scripts.
export BUILD_DIR CACHE_DIR ENV_DIR export BUILD_DIR CACHE_DIR ENV_DIR
# Set the Buildpack's internet target for downloading Python distributions. # Set the base URL for downloading buildpack assets like Python runtimes.
# The user can provide BUILDPACK_VENDOR_URL to specify a custom target. # The user can provide BUILDPACK_S3_BASE_URL to specify a custom target.
# Note: this is designed for non-Heroku use, as it does not use the user-provided # Note: this is designed for non-Heroku use, as it does not use the user-provided
# environment variable mechanism (the ENV_DIR). # environment variable mechanism (the ENV_DIR).
VENDOR_URL="https://lang-python.s3.amazonaws.com/$STACK" S3_BASE_URL="${BUILDPACK_S3_BASE_URL:-"https://heroku-buildpack-python.s3.amazonaws.com"}"
if [[ -n ${BUILDPACK_VENDOR_URL:-} ]]; then # This has to be exported since it's used by the geo-libs step which is run in a subshell.
VENDOR_URL="$BUILDPACK_VENDOR_URL"
elif [[ -n ${USE_STAGING_BINARIES} ]]; then
VENDOR_URL="$USE_STAGING_BINARIES/$STACK"
fi
export VENDOR_URL
# Default Python Versions # Default Python Versions
# shellcheck source=bin/default_pythons # shellcheck source=bin/default_pythons
source "$BIN_DIR/default_pythons" source "$BIN_DIR/default_pythons"
# Supported Python Branches # Supported Python Branches
PY39="python-3.9"
PY38="python-3.8" PY38="python-3.8"
PY37="python-3.7" PY37="python-3.7"
PY36="python-3.6" PY36="python-3.6"
@@ -64,7 +58,8 @@ PYPY27="pypy2.7"
PYPY36="pypy3.6" PYPY36="pypy3.6"
# Which stack is used (for binary downloading), if none is provided (e.g. outside of Heroku)? # Which stack is used (for binary downloading), if none is provided (e.g. outside of Heroku)?
DEFAULT_PYTHON_STACK="cedar-14" # TODO: Remove this and require that STACK be set explicitly.
DEFAULT_PYTHON_STACK="heroku-18"
# Common Problem Warnings: # Common Problem Warnings:
# This section creates a temporary file in which to stick the output of `pip install`. # This section creates a temporary file in which to stick the output of `pip install`.
@@ -73,24 +68,18 @@ DEFAULT_PYTHON_STACK="cedar-14"
WARNINGS_LOG=$(mktemp) WARNINGS_LOG=$(mktemp)
RECOMMENDED_PYTHON_VERSION=$DEFAULT_PYTHON_VERSION RECOMMENDED_PYTHON_VERSION=$DEFAULT_PYTHON_VERSION
# The buildpack ships with a few executable tools (e.g. pip-grep, etc). # The buildpack ships with a few executable tools.
# This installs them into the path, so we can execute them directly. # This installs them into the path, so we can execute them directly.
export PATH=$PATH:$ROOT_DIR/vendor/:$ROOT_DIR/vendor/pip-pop export PATH=$PATH:$ROOT_DIR/vendor/
# Set environment variables if they weren't set by the platform. # Set environment variables if they weren't set by the platform.
# Note: this is legacy, for a deprecated build system known as Anvil.
# This can likely be removed, with caution.
[ ! "$SLUG_ID" ] && SLUG_ID="defaultslug"
[ ! "$REQUEST_ID" ] && REQUEST_ID=$SLUG_ID
[ ! "$STACK" ] && STACK=$DEFAULT_PYTHON_STACK [ ! "$STACK" ] && STACK=$DEFAULT_PYTHON_STACK
# Sanitize externally-provided environment variables: # Sanitize externally-provided environment variables:
# The following environment variables are either problematic or simply unneccessary # The following environment variables are either problematic or simply unneccessary
# for the buildpack to have knowledge of, so we unset them, to keep the environment # for the buildpack to have knowledge of, so we unset them, to keep the environment
# as clean and pristine as possible. # as clean and pristine as possible.
unset GIT_DIR PYTHONHOME PYTHONPATH unset PYTHONHOME PYTHONPATH
unset RECEIVE_DATA RUN_KEY BUILD_INFO DEPLOY LOG_TOKEN
unset CYTOKINE_LOG_FILE GEM_PATH
# Import the utils script, which contains helper functions used throughout the buildpack. # Import the utils script, which contains helper functions used throughout the buildpack.
# shellcheck source=bin/utils # shellcheck source=bin/utils
@@ -101,6 +90,33 @@ source "$BIN_DIR/utils"
# shellcheck source=bin/warnings # shellcheck source=bin/warnings
source "$BIN_DIR/warnings" source "$BIN_DIR/warnings"
if [[ "${STACK}" == "cedar-14" ]]; then
mcount "failure.unsupported.cedar-14"
puts-warn "The Cedar-14 stack is no longer supported by the latest release of this buildpack."
puts-warn
puts-warn "Please switch to the Cedar-14 support branch by using this buildpack URL:"
puts-warn "https://github.com/heroku/heroku-buildpack-python#cedar-14"
puts-warn
puts-warn "For instructions on how to change the buildpacks used by an app, see:"
puts-warn "https://devcenter.heroku.com/articles/buildpacks#setting-a-buildpack-on-an-application"
exit 1
fi
if [[ -f "${ENV_DIR}/BUILD_WITH_GEO_LIBRARIES" ]]; then
mcount "failure.unsupported.BUILD_WITH_GEO_LIBRARIES"
puts-warn "The Python buildpack's legacy BUILD_WITH_GEO_LIBRARIES functonality is"
puts-warn "no longer supported:"
puts-warn "https://devcenter.heroku.com/changelog-items/1947"
puts-warn
puts-warn "To continue to use GDAL, GEOS or PROJ support, see the migration guide:"
puts-warn "https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-no-longer-supported"
puts-warn
puts-warn "Or if you no longer need those libraries, this message can be hidden by"
puts-warn "unsetting the BUILD_WITH_GEO_LIBRARIES environment variable, using:"
puts-warn "heroku config:unset BUILD_WITH_GEO_LIBRARIES"
exit 1
fi
# Make the directory in which we will create symlinks from the temporary build directory # Make the directory in which we will create symlinks from the temporary build directory
# to `/app`. # to `/app`.
# Symlinks are required, since Python is not a portable installation. # Symlinks are required, since Python is not a portable installation.
@@ -129,8 +145,9 @@ export PYTHONUNBUFFERED=1
# Set the locale to a well-known and expected standard. # Set the locale to a well-known and expected standard.
export LANG=en_US.UTF-8 export LANG=en_US.UTF-8
# `~/.heroku/vendor` is an place where the buildpack may stick pre-build binaries for known # `~/.heroku/vendor` is an place where the buildpack may stick pre-build binaries for known
# C dependencies (e.g. libmemcached on cedar-14). This section configures Python (GCC, more specifically) # C dependencies. This section configures Python (GCC, more specifically)
# and pip to automatically include these paths when building binaries. # and pip to automatically include these paths when building binaries.
# TODO: Stop adding .heroku/vendor here now that the buildpack no longer vendors anything.
export C_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$C_INCLUDE_PATH export C_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$CPLUS_INCLUDE_PATH export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LIBRARY_PATH export LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LIBRARY_PATH
@@ -170,8 +187,6 @@ cp -R "$CACHE_DIR/.heroku/python-stack" .heroku/ &> /dev/null || true
cp -R "$CACHE_DIR/.heroku/python-version" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/python-version" .heroku/ &> /dev/null || true
# A plain text file which contains the current sqlite3 version being used (used for cache busting). # A plain text file which contains the current sqlite3 version being used (used for cache busting).
cp -R "$CACHE_DIR/.heroku/python-sqlite3-version" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/python-sqlite3-version" .heroku/ &> /dev/null || true
# Any pre-compiled binaries, provided by the buildpack.
cp -R "$CACHE_DIR/.heroku/vendor" .heroku/ &> /dev/null || true
# "editable" installations of code repositories, via pip or pipenv. # "editable" installations of code repositories, via pip or pipenv.
if [[ -d "$CACHE_DIR/.heroku/src" ]]; then if [[ -d "$CACHE_DIR/.heroku/src" ]]; then
cp -R "$CACHE_DIR/.heroku/src" .heroku/ &> /dev/null || true cp -R "$CACHE_DIR/.heroku/src" .heroku/ &> /dev/null || true
@@ -189,7 +204,7 @@ source "$BIN_DIR/steps/hooks/pre_compile"
# continue to use that version of Python in perpituity (warnings will be raised if # continue to use that version of Python in perpituity (warnings will be raised if
# they are outofdate). # they are outofdate).
if [ -f "$CACHE_DIR/.heroku/python-version" ]; then if [ -f "$CACHE_DIR/.heroku/python-version" ]; then
DEFAULT_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version") CACHED_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version")
fi fi
# We didn't always record the stack version. This code is in place because of that. # We didn't always record the stack version. This code is in place because of that.
@@ -206,9 +221,14 @@ fi
# shellcheck source=bin/steps/pipenv-python-version # shellcheck source=bin/steps/pipenv-python-version
source "$BIN_DIR/steps/pipenv-python-version" source "$BIN_DIR/steps/pipenv-python-version"
# If no runtime was provided by the user, assume the default Python runtime version. if [[ -f runtime.txt ]]; then
if [ ! -f runtime.txt ]; then mcount "version.reason.python.specified"
echo "$DEFAULT_PYTHON_VERSION" > runtime.txt elif [[ -n "${CACHED_PYTHON_VERSION:-}" ]]; then
mcount "version.reason.python.cached"
echo "${CACHED_PYTHON_VERSION}" > runtime.txt
else
mcount "version.reason.python.default"
echo "${DEFAULT_PYTHON_VERSION}" > runtime.txt
fi fi
# Create the directory for .profile.d, if it doesn't exist. # Create the directory for .profile.d, if it doesn't exist.
@@ -252,27 +272,6 @@ fi
# shellcheck source=bin/steps/eggpath-fix # shellcheck source=bin/steps/eggpath-fix
source "$BIN_DIR/steps/eggpath-fix" source "$BIN_DIR/steps/eggpath-fix"
# Mercurial support.
# If a customer appears to be using mercurial for dependency resolution, we install it first.
# Note: this only applies to pip, not pipenv. This can likely be removed, over time. Measure it first.
# shellcheck source=bin/steps/mercurial
source "$BIN_DIR/steps/mercurial"
# Pylibmc support.
# On cedar-14, libmemcached was not available. The buildpack provides its own version, instead.
# shellcheck source=bin/steps/pylibmc
source "$BIN_DIR/steps/pylibmc"
# Support for Geo libraries. This is deprecated functionality, only functional on cedar-14.
# It is undocumented.
# shellcheck source=bin/steps/geo-libs
sub_env "$BIN_DIR/steps/geo-libs"
# GDAL support.
# This is part of the Geo support.
# shellcheck source=bin/steps/gdal
source "$BIN_DIR/steps/gdal"
# SQLite3 support. # SQLite3 support.
# This sets up and installs sqlite3 dev headers and the sqlite3 binary but not the # This sets up and installs sqlite3 dev headers and the sqlite3 binary but not the
# libsqlite3-0 library since that exists on the stack image. # libsqlite3-0 library since that exists on the stack image.
@@ -304,7 +303,7 @@ mtime "nltk.download.time" "${start}"
# and copying it into the proper place (the logical place to do this was early, but it must be done here). # and copying it into the proper place (the logical place to do this was early, but it must be done here).
# In CI, $BUILD_DIR is /app. # In CI, $BUILD_DIR is /app.
if [[ ! "$BUILD_DIR" == "/app" ]]; then if [[ ! "$BUILD_DIR" == "/app" ]]; then
rm -fr "$BUILD_DIR/.heroku/src" rm -rf "$BUILD_DIR/.heroku/src"
deep-cp /app/.heroku/src "$BUILD_DIR/.heroku/src" deep-cp /app/.heroku/src "$BUILD_DIR/.heroku/src"
fi fi
@@ -370,7 +369,6 @@ mkdir -p "$CACHE_DIR/.heroku"
cp -R .heroku/python "$CACHE_DIR/.heroku/" cp -R .heroku/python "$CACHE_DIR/.heroku/"
cp -R .heroku/python-version "$CACHE_DIR/.heroku/" cp -R .heroku/python-version "$CACHE_DIR/.heroku/"
cp -R .heroku/python-stack "$CACHE_DIR/.heroku/" &> /dev/null || true cp -R .heroku/python-stack "$CACHE_DIR/.heroku/" &> /dev/null || true
cp -R .heroku/vendor "$CACHE_DIR/.heroku/" &> /dev/null || true
if [[ -d .heroku/src ]]; then if [[ -d .heroku/src ]]; then
cp -R .heroku/src "$CACHE_DIR/.heroku/" &> /dev/null || true cp -R .heroku/src "$CACHE_DIR/.heroku/" &> /dev/null || true
fi fi
+8 -7
View File
@@ -5,12 +5,13 @@
# the env vars to subprocesses. # the env vars to subprocesses.
# shellcheck disable=2034 # shellcheck disable=2034
DEFAULT_PYTHON_VERSION="python-3.6.11" DEFAULT_PYTHON_VERSION="python-3.6.12"
LATEST_38="python-3.8.5" LATEST_39="python-3.9.0"
LATEST_37="python-3.7.8" LATEST_38="python-3.8.7"
LATEST_36="python-3.6.11" LATEST_37="python-3.7.9"
LATEST_35="python-3.5.9" LATEST_36="python-3.6.12"
LATEST_35="python-3.5.10"
LATEST_34="python-3.4.10" LATEST_34="python-3.4.10"
LATEST_27="python-2.7.18" LATEST_27="python-2.7.18"
PYPY_36="pypy3.6-7.3.1" LATEST_PYPY_36="pypy3.6-7.3.2"
PYPY_27="pypy2.7-7.3.1" LATEST_PYPY_27="pypy2.7-7.3.2"
+2 -2
View File
@@ -20,8 +20,8 @@ MANAGE_FILE=${MANAGE_FILE:-fakepath}
# Legacy file-based support for $DISABLE_COLLECTSTATIC # Legacy file-based support for $DISABLE_COLLECTSTATIC
[ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1 [ -f .heroku/collectstatic_disabled ] && DISABLE_COLLECTSTATIC=1
# Ensure that Django is explicitly specified in requirements.txt # Ensure that Django is actually installed.
sp-grep -s django && DJANGO_INSTALLED=1 is_module_available 'django' && DJANGO_INSTALLED=1
if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALLED" ]; then if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALLED" ]; then
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# This script serves as the GDAL build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If GDAL exists within requirements, use vendored gdal.
if (pip-grep -s requirements.txt GDAL gdal pygdal &> /dev/null) then
if [ ! -f ".heroku/vendor/bin/gdalserver" ]; then
puts-warn "The vendored GDAL package in the Heroku Python Buildpack now deprecated."
puts-warn "To enable GDAL use an alternative buildpack is available here - https://github.com/heroku/heroku-geo-buildpack"
echo "-----> Noticed GDAL. Bootstrapping gdal."
mkdir -p .heroku/vendor
# Download and extract cryptography into target vendor directory.
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.gdal"
fi
GDAL=$(pwd)/vendor
export GDAL
fi
-55
View File
@@ -1,55 +0,0 @@
#!/usr/bin/env bash
# This script serves as the GDAL build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
VENDORED_GEOS="${VENDOR_URL}/libraries/vendor/geos.tar.gz"
VENDORED_PROJ="${VENDOR_URL}/libraries/vendor/proj.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If GDAL exists within requirements, use vendored gdal.
if [[ "$BUILD_WITH_GEO_LIBRARIES" ]]; then
mcount "buildvar.BUILD_WITH_GEO_LIBRARIES"
puts-warn "The GDAL, GEOS and PROJ binaries and BUILD_WITH_GEO_LIBRARIES functonality are now deprecated."
puts-warn "An alternative buildpack to enable GDAL, GEOS and PROJ use is available here - https://github.com/heroku/heroku-geo-buildpack"
if [ ! -f ".heroku/vendor/bin/proj" ]; then
echo "-----> Bootstrapping gdal, geos, proj."
mkdir -p .heroku/vendor
# Download and extract cryptography into target vendor directory.
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_GEOS" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_PROJ" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.geo_libs"
# Copy libjasper from build image to slug.
if [[ "$STACK" == "heroku-16" ]]; then
cp /usr/lib/x86_64-linux-gnu/libjasper.so* ".heroku/vendor/lib/."
mcount "steps.vendor.libjasper"
fi
fi
GDAL=$(pwd)/vendor
export GDAL
# set path for post_compile hooks
export GDAL_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgdal.so"
export GEOS_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgeos_c.so"
# set path for runtime environmeht
set_env GDAL_LIBRARY_PATH "/app/.heroku/vendor/lib/libgdal.so"
set_env GEOS_LIBRARY_PATH "/app/.heroku/vendor/lib/libgeos_c.so"
fi
-9
View File
@@ -1,9 +0,0 @@
#!/usr/bin/env bash
# Install Mercurial if it appears to be required.
if [[ -f "requirements.txt" ]]; then
if (grep -Fiq "hg+" requirements.txt) then
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
mcount "steps.mercurial"
fi
fi
+1 -1
View File
@@ -14,7 +14,7 @@
source "$BIN_DIR/utils" source "$BIN_DIR/utils"
# Check that nltk was installed by pip, otherwise obviously not needed # Check that nltk was installed by pip, otherwise obviously not needed
if sp-grep -s nltk; then if is_module_available 'nltk'; then
puts-step "Downloading NLTK corpora…" puts-step "Downloading NLTK corpora…"
nltk_packages_definition="$BUILD_DIR/nltk.txt" nltk_packages_definition="$BUILD_DIR/nltk.txt"
-28
View File
@@ -1,28 +0,0 @@
#!/usr/bin/env bash
set +e
# Install dependencies with Pip.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
if [ ! "$SKIP_PIP_INSTALL" ]; then
if [[ -f .heroku/python/requirements-declared.txt ]]; then
cp .heroku/python/requirements-declared.txt requirements-declared.txt
if ! pip-diff --stale requirements-declared.txt requirements.txt --exclude setuptools pip wheel > .heroku/python/requirements-stale.txt; then
mcount "failure.bad-requirements"
fi
rm -fr requirements-declared.txt
if [[ -s .heroku/python/requirements-stale.txt ]]; then
puts-step "Uninstalling stale dependencies"
/app/.heroku/python/bin/pip uninstall -r .heroku/python/requirements-stale.txt -y --exists-action=w --disable-pip-version-check | cleanup | indent
fi
fi
fi
set -e
+1 -5
View File
@@ -9,10 +9,7 @@ set -e
if [[ -f Pipfile.lock ]]; then if [[ -f Pipfile.lock ]]; then
if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then
if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then
# Measure that we're using Pipenv. # Don't skip installation if there are git deps.
mcount "tool.pipenv"
# Don't skip installation of there are git deps.
if ! grep -q 'git' Pipfile.lock; then if ! grep -q 'git' Pipfile.lock; then
echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent
@@ -72,7 +69,6 @@ if [ ! "$SKIP_PIPENV_INSTALL" ]; then
else else
pipenv-to-pip Pipfile.lock > requirements.txt pipenv-to-pip Pipfile.lock > requirements.txt
"$BIN_DIR/steps/pip-uninstall"
cp requirements.txt .heroku/python/requirements-declared.txt cp requirements.txt .heroku/python/requirements-declared.txt
openssl dgst -sha256 Pipfile.lock > .heroku/python/Pipfile.lock.sha256 openssl dgst -sha256 Pipfile.lock > .heroku/python/Pipfile.lock.sha256
+3
View File
@@ -30,6 +30,9 @@ if [[ -f $BUILD_DIR/Pipfile ]]; then
if [ "$PYTHON" = 3.8 ]; then if [ "$PYTHON" = 3.8 ]; then
echo "$LATEST_38" > "$BUILD_DIR/runtime.txt" echo "$LATEST_38" > "$BUILD_DIR/runtime.txt"
fi fi
if [ "$PYTHON" = 3.9 ]; then
echo "$LATEST_39" > "$BUILD_DIR/runtime.txt"
fi
fi fi
-37
View File
@@ -1,37 +0,0 @@
#!/usr/bin/env bash
# This script serves as the Pylibmc build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
if [[ "$STACK" != "cedar-14" ]]; then
# libmemcached is pre-installed in the stack image so there is no need to vendor it.
return 0
fi
# The location of the pre-compiled libmemcached binary.
VENDORED_MEMCACHED="${VENDOR_URL}/libraries/vendor/libmemcache.tar.gz"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If pylibmc exists within requirements, use vendored libmemcached.
if (pip-grep -s requirements.txt pylibmc &> /dev/null) then
if [ ! -d ".heroku/vendor/lib/sasl2" ]; then
echo "-----> Noticed pylibmc. Bootstrapping libmemcached."
mkdir -p .heroku/vendor
# Download and extract libmemcached into target vendor directory.
curl "$VENDORED_MEMCACHED" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.pylibmc"
fi
LIBMEMCACHED=$(pwd)/vendor
export LIBMEMCACHED
fi
+19 -14
View File
@@ -5,9 +5,10 @@ runtime-fixer runtime.txt
PYTHON_VERSION=$(cat runtime.txt) PYTHON_VERSION=$(cat runtime.txt)
# The location of the pre-compiled python binary. # The location of the pre-compiled python binary.
VENDORED_PYTHON="${VENDOR_URL}/runtimes/$PYTHON_VERSION.tar.gz" VENDORED_PYTHON="${S3_BASE_URL}/${STACK}/runtimes/${PYTHON_VERSION}.tar.gz"
SECURITY_UPDATE="Python has released a security update! Please consider upgrading to" SECURITY_UPDATE="Python has released a security update! Please consider upgrading to"
SECURITY_UPDATE_PYPY="The PyPy project has released a security update! Please consider upgrading to"
ONLY_SUPPORTED_2_VERSION="Only the latest version of Python 2 is supported on the platform. Please consider upgrading to" ONLY_SUPPORTED_2_VERSION="Only the latest version of Python 2 is supported on the platform. Please consider upgrading to"
@@ -15,6 +16,12 @@ PYTHON_2_EOL_UPDATE="Python 2 has reached it's community EOL. Upgrade your Pytho
# check if runtime exists # check if runtime exists
if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
if [[ "$PYTHON_VERSION" == $PY39* ]]; then
if [ "$PYTHON_VERSION" != "$LATEST_39" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_39"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY38* ]]; then if [[ "$PYTHON_VERSION" == $PY38* ]]; then
# do things to alert the user of security release available # do things to alert the user of security release available
if [ "$PYTHON_VERSION" != "$LATEST_38" ]; then if [ "$PYTHON_VERSION" != "$LATEST_38" ]; then
@@ -51,11 +58,9 @@ if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
fi fi
fi fi
if [[ "$PYTHON_VERSION" == $PY27* ]]; then if [[ "$PYTHON_VERSION" == $PY27* ]]; then
puts-warn "$PYTHON_2_EOL_UPDATE"
echo " Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
# security update note # security update note
if [[ "$(date "+%Y")" -gt "2019" ]]; then
puts-warn "$PYTHON_2_EOL_UPDATE"
echo " Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
fi
if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then
puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27" puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
@@ -63,15 +68,15 @@ if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
fi fi
if [[ "$PYTHON_VERSION" == $PYPY27* ]]; then if [[ "$PYTHON_VERSION" == $PYPY27* ]]; then
# security update note # security update note
if [ "$PYTHON_VERSION" != "$PYPY_27" ]; then if [ "$PYTHON_VERSION" != "$LATEST_PYPY_27" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_27}?" puts-warn "$SECURITY_UPDATE_PYPY" "$LATEST_PYPY_27"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi fi
fi fi
if [[ "$PYTHON_VERSION" == $PYPY36* ]]; then if [[ "$PYTHON_VERSION" == $PYPY36* ]]; then
# security update note # security update note
if [ "$PYTHON_VERSION" != "$PYPY_36" ]; then if [ "$PYTHON_VERSION" != "$LATEST_PYPY_36" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_36}?" puts-warn "$SECURITY_UPDATE_PYPY" "$LATEST_PYPY_36"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes" echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi fi
fi fi
@@ -81,15 +86,17 @@ else
exit 1 exit 1
fi fi
mcount "version.python.${PYTHON_VERSION}"
if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then
puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache" puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache"
rm -fr .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version rm -rf .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version
fi fi
if [ -f .heroku/python-version ]; then if [ -f .heroku/python-version ]; then
if [ ! "$(cat .heroku/python-version)" = "$PYTHON_VERSION" ]; then if [ ! "$(cat .heroku/python-version)" = "$PYTHON_VERSION" ]; then
puts-step "Found $(cat .heroku/python-version), removing" puts-step "Found $(cat .heroku/python-version), removing"
rm -fr .heroku/python rm -rf .heroku/python
else else
SKIP_INSTALL=1 SKIP_INSTALL=1
fi fi
@@ -120,8 +127,6 @@ if [ ! "$SKIP_INSTALL" ]; then
# Prepare destination directory. # Prepare destination directory.
mkdir -p .heroku/python mkdir -p .heroku/python
mcount "version.python.$PYTHON_VERSION"
if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)." puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support" puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
@@ -161,7 +166,7 @@ fi
# Instead, we use the pip wheel to install itself, using the method described here: # Instead, we use the pip wheel to install itself, using the method described here:
# https://github.com/pypa/pip/issues/2351#issuecomment-69994524 # https://github.com/pypa/pip/issues/2351#issuecomment-69994524
PIP_WHEEL_FILENAME="pip-${PIP_VERSION}-py2.py3-none-any.whl" PIP_WHEEL_FILENAME="pip-${PIP_VERSION}-py2.py3-none-any.whl"
PIP_WHEEL_URL="https://lang-python.s3.amazonaws.com/common/${PIP_WHEEL_FILENAME}" PIP_WHEEL_URL="${S3_BASE_URL}/common/${PIP_WHEEL_FILENAME}"
PIP_WHEEL="${TMPDIR:-/tmp}/${PIP_WHEEL_FILENAME}" PIP_WHEEL="${TMPDIR:-/tmp}/${PIP_WHEEL_FILENAME}"
if ! curl -sSf "${PIP_WHEEL_URL}" -o "$PIP_WHEEL"; then if ! curl -sSf "${PIP_WHEEL_URL}" -o "$PIP_WHEEL"; then
+15 -6
View File
@@ -2,12 +2,13 @@
shopt -s extglob shopt -s extglob
shopt -s nullglob shopt -s nullglob
# The standard library. # This is necessary since this script is sometimes sourced from
if [[ ! -f /tmp/stdlib.sh ]]; then # subshells that don't have the variables from bin/compile.
curl --retry 3 -s https://lang-common.s3.amazonaws.com/buildpack-stdlib/v8/stdlib.sh > /tmp/stdlib.sh # Remove this once we no longer wrap all the things in `sub_env`.
fi BIN_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
# shellcheck source=/dev/null ROOT_DIR=$(dirname "${BIN_DIR}")
source /tmp/stdlib.sh # shellcheck source=vendor/buildpack-stdlib_v8.sh
source "${ROOT_DIR}/vendor/buildpack-stdlib_v8.sh"
if [ "$(uname)" == Darwin ]; then if [ "$(uname)" == Darwin ]; then
sed() { command sed -l "$@"; } sed() { command sed -l "$@"; }
@@ -95,3 +96,11 @@ python_sqlite3_check() {
( python2_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_2" ) \ ( python2_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_2" ) \
|| ( python3_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_3" ) || ( python3_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_3" )
} }
is_module_available() {
# Returns 0 is the specified module exists, otherwise returns 1.
# Uses pkgutil rather than pkg_resources or pip's CLI, since pkgutil exists
# in the stdlib, and doesn't depend on the choice of package manager.
local module_name="${1}"
python -c "import sys, pkgutil; sys.exit(0 if pkgutil.find_loader('${module_name}') else 1)"
}
+12 -11
View File
@@ -12,16 +12,6 @@ old-platform() {
fi fi
} }
pylibmc-missing() {
if grep -qi 'fatal error: libmemcached/memcached.h: No such file or directory' "$WARNINGS_LOG"; then
echo
puts-warn "Hello! There was a problem with your build related to libmemcache."
puts-warn "The Python library 'pylibmc' must be explicitly specified in 'requirements.txt' in order to build correctly."
puts-warn "Once you do that, everything should work as expected. -- Much Love, Heroku."
mcount 'warnings.libmemcache'
fi
}
scipy-included() { scipy-included() {
if grep -qi 'running setup.py install for scipy' "$WARNINGS_LOG"; then if grep -qi 'running setup.py install for scipy' "$WARNINGS_LOG"; then
echo echo
@@ -56,11 +46,22 @@ six-included() {
fi fi
} }
gdal-missing() {
if grep -qi 'Could not find gdal-config' "$WARNINGS_LOG"; then
mcount 'warnings.gdal'
echo
puts-warn "Hello! Package installation failed since the GDAL library was not found."
puts-warn "For GDAL, GEOS and PROJ support, use the Geo buildpack alongside the Python buildpack:"
puts-warn "https://github.com/heroku/heroku-geo-buildpack"
puts-warn " -- Much Love, Heroku."
fi
}
show-warnings() { show-warnings() {
old-platform old-platform
pylibmc-missing
scipy-included scipy-included
distribute-included distribute-included
six-included six-included
gdal-missing
} }
+61 -38
View File
@@ -1,57 +1,80 @@
# Python Buildpack Binaries # Python Buildpack Binaries
## Building the Docker Images The binaries for this buildpack are built in Docker containers based on the Heroku stack image.
**After every change to your formulae, perform the following** from the root of the Git repository (not from `builds/`) to rebuild the images for each stack:
$ docker build --pull --tag heroku-python-build-cedar-14 --file $(pwd)/builds/cedar-14.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-16 --file $(pwd)/builds/heroku-16.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-18 --file $(pwd)/builds/heroku-18.Dockerfile .
## Using the Image
You can e.g. `bash` into each of the images you built using their tag:
docker run --rm -ti heroku-python-build-cedar-14 bash
docker run --rm -ti heroku-python-build-heroku-16 bash
docker run --rm -ti heroku-python-build-heroku-18 bash
You then have a shell where you can run `bob build`, `bob deploy`, and so forth. You can of course also invoke these programs directly with `docker run`:
docker run --rm -ti heroku-python-build-heroku-18 bob build runtimes/python-2.7.15
In order to `bob deploy`, AWS credentials must be set up, as well as name and prefix of your custom S3 bucket (unless you're deploying to the Heroku production buckets that are pre-defined in each `Dockerfile`); see next section for details.
## Configuration ## Configuration
File `dockerenv.default` contains a list of required env vars; most of these have default values defined in `Dockerfile`. You can copy this file to a location outside the buildpack and modify it with the values you desire and pass its location with `--env-file`, or pass the env vars to `docker run` using `--env`. In order to publish binaries AWS credentials must be passed to the build container.
If you are testing only the build (ie: `bob build`), these are optional.
Out of the box, each `Dockerfile` has the correct values predefined for `S3_BUCKET`, `S3_PREFIX`, and `S3_REGION`. If you're building your own packages, you'll likely want to change `S3_BUCKET` and `S3_PREFIX` to match your info. Instead of setting `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` into that file, you may also pass them to `docker run` through the environment, or explicitly using `--env`, in order to prevent accidental commits of credentials. In addition, unless you are building the official binaries for Heroku (which use the defaults
specified in each `Dockerfile`), you will need to override `S3_BUCKET` and `S3_PREFIX` to
match your own S3 bucket/use case.
### Passing AWS credentials to the container If you only need to set AWS credentials, you can do so by setting the environment variables
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` before calling the make commands.
If you want to deploy packages and thus need to pass `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, you can either pass them explicitly, through your environment, or through an env file. For example:
#### Passing credentials explicitly ```bash
set +o history # Disable bash history
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
set -o history # Re-enable bash history
make ...
```
docker run --rm -ti -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... heroku-python-build-heroku-18 bash If you need to override the default S3 bucket, or would prefer not to use credentials via
environment variables, then you need to instead use a Docker env file like so:
#### Passing credentials through the environment 1. Copy the `builds/dockerenv.default` env file to a location outside the buildpack repository.
2. Edit the new file, adding at a minimum the values for the variables
`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` (see Docker
[env-file documentation](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file)).
3. Pass the path of the file to the make commands using `ENV_FILE`. For example:
The two environment variables `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY` are defined in `builds/dockerenv.default`, without values. This will cause Docker to "forward" values for these variables from the current environment, so you can pass them in: ```bash
make ... ENV_FILE=~/.dockerenv.python-buildpack
```
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash ## Launching an interactive build environment
or To start an interactive version of the build environment (ideal for development) use the
`buildenv` make target, passing in the desired `STACK` name. For example:
export AWS_ACCESS_KEY_ID=... ```bash
export AWS_SECRET_ACCESS_KEY=... make buildenv STACK=heroku-18
docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash ```
#### Passing credentials through a separate env file This will create the builder docker image based on the latest image for that stack, and
then start a bash shell where you can run `bob build`, `bob deploy`, and so forth.
This method is the easiest for users who want to build packages in their own S3 bucket, as they will have to adjust the `S3_BUCKET` and `S3_PREFIX` environment variable values anyway from their default values. The `builds/` directory is bind-mounted into the running container, so local build formula
changes will appear there immediately without the need to rebuild the image.
For this method, it is important to keep the credentials file in a location outside the buildpack, so that your credentials aren't accidentally committed. Copy `builds/dockerenv.default` **to a safe location outside the buildpack directory**, and insert your values for `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY`. ## Bulk deploying runtimes
docker run --rm -ti --env-file=../SOMEPATHOUTSIDE/s3.env heroku-python-build-heroku-18 bash When a new Python version is released, binaries have to be generated for multiple stacks.
To automate this, use the `deploy-runtimes` make target, which will ensure the builder
image is up to date, and then run `bob deploy` for each runtime-stack combination.
The build formula name(s) are passed using `RUNTIMES`, like so:
```bash
make deploy-runtimes RUNTIMES='python-X.Y.Z'
```
By default this will deploy to all supported stacks (see `STACKS` in `Makefile`),
but this can be overridden using `STACKS`:
```bash
make deploy-runtimes RUNTIMES='python-X.Y.Z' STACKS='heroku-16 heroku-18'
```
Multiple runtimes can also be specified (useful for when adding a new stack), like so:
```bash
make deploy-runtimes RUNTIMES='python-A.B.C python-X.Y.Z' STACKS='heroku-20'
```
Note: Both `RUNTIMES` and `STACKS` are space delimited.
-15
View File
@@ -1,15 +0,0 @@
FROM heroku/cedar:14
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \
S3_PREFIX="cedar-14/" \
DEBIAN_FRONTEND=noninteractive \
STACK="cedar-14"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev realpath && rm -rf /var/lib/apt/lists/*
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt
COPY . /app
+6 -3
View File
@@ -1,5 +1,8 @@
# Since no values are specified here, these variables will be read from the environment at run time:
# https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file
AWS_ACCESS_KEY_ID AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY AWS_SECRET_ACCESS_KEY
S3_BUCKET
S3_PREFIX # Uncomment these if you need to override the default S3 bucket and/or path prefixes.
S3_REGION # S3_BUCKET
# S3_PREFIX
+10 -5
View File
@@ -1,15 +1,20 @@
FROM heroku/heroku:16-build FROM heroku/heroku:16-build
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \ ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \ S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-16/" \ S3_PREFIX="heroku-16/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-16" STACK="heroku-16"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev && rm -rf /var/lib/apt/lists/* RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/ COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app COPY . /app
+10 -5
View File
@@ -1,15 +1,20 @@
FROM heroku/heroku:18-build FROM heroku/heroku:18-build
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \ ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \ S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-18/" \ S3_PREFIX="heroku-18/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-18" STACK="heroku-18"
RUN apt-get update && apt-get install --no-install-recommends -y python-pip-whl=9.0.1-2 python-pip=9.0.1-2 python-setuptools python-wheel libsqlite3-dev && rm -rf /var/lib/apt/lists/* RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/ COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app COPY . /app
+20
View File
@@ -0,0 +1,20 @@
FROM heroku/heroku:20-build
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="heroku-buildpack-python" \
S3_PREFIX="heroku-20/" \
STACK="heroku-20"
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
libsqlite3-dev \
python3-pip \
python3-setuptools \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt /app/
RUN pip3 install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app
+1 -1
View File
@@ -19,4 +19,4 @@ make install
# Cleanup # Cleanup
cd .. cd ..
rm -fr sqlite rm -rf sqlite
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal…"
VERSION="2.2.1"
SOURCE_TARBALL="http://download.osgeo.org/gdal/${VERSION}/gdal-${VERSION}.tar.gz"
curl -L $SOURCE_TARBALL | tar zx
pushd "gdal-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building geos…"
VERSION=3.6.2
SOURCE_TARBALL="http://download.osgeo.org/geos/geos-${VERSION}.tar.bz2"
curl -L $SOURCE_TARBALL | tar xj
pushd "geos-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
-45
View File
@@ -1,45 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
if [[ $S3_PREFIX != "cedar-14" ]]; then
echo "libmemcached only needs to be built for cedar-14, since newer stacks include it in the base image"
exit 1
fi
# fail hard
set -o pipefail
# fail harder
set -eux
DEFAULT_VERSION="1.0.18"
dep_version=${VERSION:-$DEFAULT_VERSION}
dep_dirname=libmemcached-${dep_version}
dep_archive_name=${dep_dirname}.tar.gz
dep_url=https://launchpad.net/libmemcached/1.0/${dep_version}/+download/${dep_archive_name}
# SASL Support.
echo "-----> Building cyrus-sasl 2.1.26…"
curl -LO ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.26.tar.gz
# FTP doesn't play well with piping into tar xz
tar xzf cyrus-sasl-2.1.26.tar.gz
pushd cyrus-sasl-2.1.26
./configure --prefix=${OUT_PREFIX} --with-plugindir=${OUT_PREFIX}lib/sasl2 --with-configdir=${OUT_PREFIX}lib/sasl2
make -s -j 9
make install -s
popd
echo "-----> Building libmemcached ${dep_version}…"
curl -L ${dep_url} | tar xz
pushd ${dep_dirname}
CPPFLAGS=-I${OUT_PREFIX}/include LDFLAGS=-L${OUT_PREFIX}/lib ./configure --prefix=${OUT_PREFIX} --without-memcached
make -s -j 9
make install -s
popd
echo "-----> Done."
-24
View File
@@ -1,24 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/vendor/
OUT_PREFIX=$1
# Use new path, containing autoconf.
export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal…"
VERSION=4.9.3
SOURCE_TARBALL="http://download.osgeo.org/proj/proj-${VERSION}.tar.gz"
curl -L $SOURCE_TARBALL | tar zx
pushd "proj-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no
make
make install
# Cleanup
popd
+2 -2
View File
@@ -13,11 +13,11 @@ dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full ver
dep_package=${dep_name}-v${dep_version} # it's always "pypy2-…" dep_package=${dep_name}-v${dep_version} # it's always "pypy2-…"
dep_dirname=${dep_package}-linux64 dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2 dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name} dep_url="https://downloads.python.org/pypy/${dep_archive_name}"
echo "Building PyPy…" echo "Building PyPy…"
echo "${dep_url}" echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name curl -fL "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python" ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy2.7
+2 -2
View File
@@ -13,11 +13,11 @@ dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full ver
dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-} dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-}
dep_dirname=${dep_package}-linux64 dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2 dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name} dep_url="https://downloads.python.org/pypy/${dep_archive_name}"
echo "Building PyPy3…" echo "Building PyPy3…"
echo "${dep_url}" echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name curl -fL "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python" ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python"
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy3.6
+1 -24
View File
@@ -1,27 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python2
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/2.7.17/Python-2.7.17.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.17 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
+1 -24
View File
@@ -1,27 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python2
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/2.7.18/Python-2.7.18.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.18 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
+1 -29
View File
@@ -1,32 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python3
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.7.2/Python-3.7.2.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.7.2 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
+1 -29
View File
@@ -1,32 +1,4 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Build Path: /app/.heroku/python/ # Build Path: /app/.heroku/python/
OUT_PREFIX=$1 source $(dirname $0)/python3
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.4.9/Python-3.4.9.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.4.9 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-5
View File
@@ -1,5 +0,0 @@
#!/usr/bin/env bash
sudo apt-get -qq update
sudo apt-get install software-properties-common
curl --fail --retry 3 --retry-delay 1 --connect-timeout 3 --max-time 30 https://cli-assets.heroku.com/install-ubuntu.sh | sh
Regular → Executable
View File
+1 -1
View File
@@ -1,5 +1,5 @@
--- ---
- - "./repos/python/python-getting-started" - - "./repos/python/python-getting-started"
- master - main
- - "./repos/python/python_default" - - "./repos/python/python_default"
- ca947f69027b2a30be5d26f9a42f25e54f4d7a1a - ca947f69027b2a30be5d26f9a42f25e54f4d7a1a
+5 -2
View File
@@ -1,3 +1,6 @@
# Dependencies for generating/publishing Python binaries.
bob-builder==0.0.19
# Sub-dependencies of bob-builder.
boto==2.49.0
docopt==0.6.2 docopt==0.6.2
bob-builder==0.0.17
boto==2.48.0
+1
View File
@@ -0,0 +1 @@
gdal
-1
View File
@@ -1 +0,0 @@
django
-1
View File
@@ -1 +0,0 @@
python-3.6.6
+1 -1
View File
@@ -6,4 +6,4 @@ verify_ssl = true
requests = "*" requests = "*"
[requires] [requires]
python_full_version = "3.6.3" python_full_version = "3.7.8"
+2 -15
View File
@@ -1,24 +1,11 @@
{ {
"_meta": { "_meta": {
"hash": { "hash": {
"sha256": "22a052f4d1cfe6518b2f236fe45c3208c587a9ab1323bdd390632e27278b541e" "sha256": "8a36860f0f9cb55716222098062cea5c5e0f8127cafb9d0c694de327bac9fbc0"
},
"host-environment-markers": {
"implementation_name": "cpython",
"implementation_version": "3.6.3",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_python_implementation": "CPython",
"platform_release": "16.7.0",
"platform_system": "Darwin",
"platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
"python_full_version": "3.6.3",
"python_version": "3.6",
"sys_platform": "darwin"
}, },
"pipfile-spec": 6, "pipfile-spec": 6,
"requires": { "requires": {
"python_full_version": "3.6.3" "python_full_version": "3.7.8"
}, },
"sources": [ "sources": [
{ {
+1 -1
View File
@@ -1 +1 @@
pypy2.7-7.2.0 pypy2.7-7.3.2
View File
+1
View File
@@ -0,0 +1 @@
pypy2.7-7.3.1
+1 -1
View File
@@ -1 +1 @@
pypy3.6-7.2.0 pypy3.6-7.3.2
View File
+1
View File
@@ -0,0 +1 @@
pypy3.6-7.3.1
-1
View File
@@ -1 +0,0 @@
requests
-1
View File
@@ -1 +0,0 @@
python-2.7.99
-1
View File
@@ -1 +0,0 @@
requests
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.4.99
-1
View File
@@ -1 +0,0 @@
flask
+1 -1
View File
@@ -1 +1 @@
python-3.5.9 python-3.5.10
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.5.99
-1
View File
@@ -1 +0,0 @@
flask
+1 -1
View File
@@ -1 +1 @@
python-3.6.8 python-3.6.12
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.6.99
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.6.7 python-3.6.11
+1 -1
View File
@@ -1 +1 @@
python-3.7.2 python-3.7.9
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
python-3.7.99
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.7.1 python-3.7.8
+1 -1
View File
@@ -1 +1 @@
python-3.8.2 python-3.8.7
-1
View File
@@ -1 +0,0 @@
flask
-1
View File
@@ -1 +0,0 @@
requests
+1 -1
View File
@@ -1 +1 @@
python-3.8.0 python-3.8.5
View File
+1
View File
@@ -0,0 +1 @@
python-3.9.0
View File
+1
View File
@@ -0,0 +1 @@
python-3.9.0
-1
View File
@@ -1 +0,0 @@
flask
+1
View File
@@ -0,0 +1 @@
hg+https://www.mercurial-scm.org/repo/python-hglib/#egg=python-hglib
+22 -18
View File
@@ -17,21 +17,18 @@ testCollectstatic() {
assertCaptured "collectstatic" assertCaptured "collectstatic"
} }
testGEOS() { testBuildWithGeoLibrariesWarning() {
local env_dir="$(mktmpdir)" local env_dir="$(mktmpdir)"
echo '1' > "${env_dir}/BUILD_WITH_GEO_LIBRARIES" echo '1' > "${env_dir}/BUILD_WITH_GEO_LIBRARIES"
compile 'geos' '' "${env_dir}" compile 'gdal' '' "${env_dir}"
assertCaptured "geos" assertCaptured " ! The Python buildpack's legacy BUILD_WITH_GEO_LIBRARIES functonality"
assertCapturedSuccess assertCapturedError
} }
testGEOSDeprecation() { testGDALWarning() {
local env_dir="$(mktmpdir)" compile 'gdal'
echo '1' > "${env_dir}/BUILD_WITH_GEO_LIBRARIES" assertCaptured " ! Hello! Package installation failed since the GDAL library was not found."
compile 'geos' '' "${env_dir}" assertCapturedError
assertCaptured " ! The GDAL, GEOS and PROJ binaries and BUILD_WITH_GEO_LIBRARIES functonality are now deprecated.
! An alternative buildpack to enable GDAL, GEOS and PROJ use is available here - https://github.com/heroku/heroku-geo-buildpack"
assertCapturedSuccess
} }
testNLTK() { testNLTK() {
@@ -43,7 +40,9 @@ testNLTK() {
echo 'ignore::RuntimeWarning' > "${env_dir}/PYTHONWARNINGS" echo 'ignore::RuntimeWarning' > "${env_dir}/PYTHONWARNINGS"
compile 'nltk' '' "${env_dir}" compile 'nltk' '' "${env_dir}"
assertCaptured "[nltk_data] Downloading package city_database" "STD_ERR" assertCaptured "[nltk_data] Downloading package city_database" "STD_ERR"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since the NLTK downloader outputs all
# progress/status messages to stderr (W-8146040).
assertCapturedSuccessWithStdErr
} }
testPsycopg2() { testPsycopg2() {
@@ -53,17 +52,16 @@ testPsycopg2() {
} }
testPysqlite() { testPysqlite() {
# pysqlite does not support Python 3 (since the sqlite3 stdlib can be used there),
# so we have to test with Python 2, which we've not made available for Heroku-20.
if [[ $STACK == "heroku-20" ]]; then
return
fi
compile "pysqlite" compile "pysqlite"
assertCaptured "pysqlite" assertCaptured "pysqlite"
assertCapturedSuccess assertCapturedSuccess
} }
testSqliteInstall() {
compile "pythonDefault"
assertNotCaptured "Sqlite3 failed to install."
assertCapturedSuccess
}
testCffi() { testCffi() {
compile "cffi" compile "cffi"
assertCaptured "cffi" assertCaptured "cffi"
@@ -76,6 +74,12 @@ testPylibmc() {
assertCapturedSuccess assertCapturedSuccess
} }
testMercurial() {
compile "requirements-mercurial"
assertCaptured "Cloning hg"
assertCapturedSuccess
}
pushd $(dirname 0) >/dev/null pushd $(dirname 0) >/dev/null
popd >/dev/null popd >/dev/null
+31 -14
View File
@@ -33,7 +33,9 @@ testStackChange() {
testSetupPy() { testSetupPy() {
compile "setup-py" compile "setup-py"
assertCaptured "maya" assertCaptured "maya"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testStandardRequirements() { testStandardRequirements() {
@@ -45,29 +47,44 @@ testStandardRequirements() {
testPipenv() { testPipenv() {
compile "pipenv" compile "pipenv"
assertCaptured "Installing pip 9.0.2, setuptools 47.1.1 and wheel 0.34.2" assertCaptured "Installing pip 9.0.2, setuptools 47.1.1 and wheel 0.34.2"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testPipenvLock() { testPipenvLock() {
compile "pipenv-lock" compile "pipenv-lock"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testPipenvVersion() { testPipenvPythonVersion3_6() {
compile "pipenv-version" compile "pipenv-version"
assertCaptured $DEFAULT_PYTHON_VERSION assertCaptured "Installing ${LATEST_36}"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testPipenvVersion2() { testPipenvPythonVersion2_7() {
# Python 2.7 is EOL, so it has not been built for Heroku-20.
if [[ $STACK == "heroku-20" ]]; then
return
fi
compile "pipenv-version2" compile "pipenv-version2"
assertCaptured $LATEST_27 assertCaptured "Installing ${LATEST_27}"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testPipenvFullVersion() {
testPipenvPythonFullVersion() {
compile "pipenv-full-version" compile "pipenv-full-version"
assertCaptured "3.6.3" assertCaptured "3.7.8"
assertCapturedSuccess # Can't use `assertCapturedSuccess` since stderr contains:
# "cp: cannot stat '/tmp/build_*/requirements.txt': No such file or directory" (W-7924941)
assertCapturedSuccessWithStdErr
} }
testNoRequirements() { testNoRequirements() {
@@ -95,6 +112,7 @@ testHooks() {
local expected_env_vars=( local expected_env_vars=(
_ _
BIN_DIR BIN_DIR
BPLOG_PREFIX
BUILD_DIR BUILD_DIR
BUILDPACK_LOG_FILE BUILDPACK_LOG_FILE
CACHE_DIR CACHE_DIR
@@ -116,9 +134,8 @@ testHooks() {
SHLVL SHLVL
SOME_APP_CONFIG_VAR SOME_APP_CONFIG_VAR
STACK STACK
VENDOR_URL
) )
if [[ "${STACK}" == "cedar-14" || "${STACK}" == "heroku-16" ]]; then if [[ "${STACK}" == "heroku-16" ]]; then
# Remove "OLDPWD" from expected_env_vars since for bash <4.4 it's not exported to subshells: # Remove "OLDPWD" from expected_env_vars since for bash <4.4 it's not exported to subshells:
# https://github.com/heroku/heroku-buildpack-python/pull/1011#issuecomment-665117835 # https://github.com/heroku/heroku-buildpack-python/pull/1011#issuecomment-665117835
read -ra expected_env_vars <<< "${expected_env_vars[@]/OLDPWD/}" read -ra expected_env_vars <<< "${expected_env_vars[@]/OLDPWD/}"
+157 -146
View File
@@ -4,90 +4,78 @@
# shellcheck source=bin/default_pythons # shellcheck source=bin/default_pythons
source "bin/default_pythons" source "bin/default_pythons"
testPythonDefault() { testPythonVersionUnspecified() {
updateVersion "pythonDefault" $DEFAULT_PYTHON_VERSION compile "python_version_unspecified"
compile "pythonDefault" assertCaptured "Installing ${DEFAULT_PYTHON_VERSION}"
assertCaptured $DEFAULT_PYTHON_VERSION
assertNotCaptured "security update" assertNotCaptured "security update"
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2" assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3" assertCaptured "Installing SQLite3"
assertCapturedSuccess assertCapturedSuccess
} }
testPython2() { testPython2_7() {
updateVersion "python2" $LATEST_27 # Python 2.7 is EOL, so it has not been built for Heroku-20.
echo $LATEST_27 > "runtime.txt" if [[ $STACK == "heroku-20" ]]; then
compile "python2" return
assertCaptured $LATEST_27 fi
if [[ $(date "+%Y") > "2019" ]]; then compile "python2"
assertCaptured "python-2-7-eol-faq"; assertCaptured "Installing ${LATEST_27}"
else assertCaptured "python-2-7-eol-faq";
assertNotCaptured "python-2-7-eol-faq"; assertNotCaptured "security update"
fi assertCaptured "Installing pip 20.1.1, setuptools 44.1.1 and wheel 0.34.2"
assertNotCaptured "security update" assertCaptured "Installing SQLite3"
assertCaptured "Installing pip 20.1.1, setuptools 44.1.1 and wheel 0.34.2" assertCapturedSuccess
assertCaptured "Installing SQLite3"
assertCapturedSuccess
} }
testPython2_warn() { testPython2_7_warn() {
compile "python2_warn" # Python 2.7 is EOL, so it has not been built for Heroku-20.
assertCaptured "python-2.7.15" if [[ $STACK == "heroku-20" ]]; then
if [[ $(date "+%Y") > "2019" ]]; then return
assertCaptured "python-2-7-eol-faq"; fi
else compile "python2_warn"
assertNotCaptured "python-2-7-eol-faq"; assertCaptured "Installing python-2.7.15"
fi assertCaptured "python-2-7-eol-faq";
assertCaptured "Only the latest version" assertCaptured "Only the latest version"
assertCaptured "Installing SQLite3" assertCaptured "${LATEST_27}"
assertCapturedSuccess assertCapturedSuccess
}
testPython2_fail() {
compile "python2_fail"
assertCaptured "Aborting"
assertCapturedError
} }
testPython3_4() { testPython3_4() {
# Python 3.4 is EOL, so it has not been built for Heroku-20.
if [[ $STACK == "heroku-20" ]]; then
return
fi
compile "python3_4" compile "python3_4"
assertCaptured $LATEST_34 assertCaptured "Installing ${LATEST_34}"
assertNotCaptured "security update" assertNotCaptured "security update"
assertCaptured "Installing pip 19.1.1, setuptools 43.0.0 and wheel 0.33.6" assertCaptured "Installing pip 19.1.1, setuptools 43.0.0 and wheel 0.33.6"
# if cedar 14 and legacy binaries, fail. if cedar 14 and staging, succeed. assertCaptured "Installing SQLite3"
if [[ ! -n $USE_STAGING_BINARIES ]] && [[ $STACK == "cedar-14" ]]; then # Can't use `assertCapturedSuccess` since Pip outputs a Python 3.4 EOL warning to stderr,
assertCapturedError # and the newest Pip that works on Python 3.4 doesn't support `PIP_NO_PYTHON_VERSION_WARNING`.
# if heroku 18 and legacy binaries, succeed. if heroku 18 and staging, fail. assertCapturedSuccessWithStdErr
elif [[ -n $USE_STAGING_BINARIES ]] && [[ $STACK == "heroku-18" ]]; then
assertCapturedError
else
# all else succeed
assertCapturedSuccess
fi
} }
testPython3_4_warn() { testPython3_4_warn() {
compile "python3_4_warn" # Python 3.4 is EOL, so it has not been built for Heroku-20.
assertCaptured "python-3.4.9" if [[ $STACK == "heroku-20" ]]; then
assertCaptured "security update!" return
# if heroku 18 and legacy binaries, succeed. if heroku 18 and staging, fail.
if [[ -n $USE_STAGING_BINARIES ]] && [[ $STACK == "heroku-18" ]]; then
assertCapturedError
else
# all else succeed
assertCapturedSuccess
fi fi
} compile "python3_4_warn"
assertCaptured "Installing python-3.4.9"
testPython3_4_fail() { assertCaptured "security update!"
compile "python3_4_fail" assertCaptured "${LATEST_34}"
assertCaptured "Aborting" # Can't use `assertCapturedSuccess` since Pip outputs a Python 3.4 EOL warning to stderr,
assertCapturedError # and the newest Pip that works on Python 3.4 doesn't support `PIP_NO_PYTHON_VERSION_WARNING`.
assertCapturedSuccessWithStdErr
} }
testPython3_5() { testPython3_5() {
# Python 3.5 is EOL, so it has not been built for Heroku-20.
if [[ $STACK == "heroku-20" ]]; then
return
fi
compile "python3_5" compile "python3_5"
assertCaptured $LATEST_35 assertCaptured "Installing ${LATEST_35}"
assertNotCaptured "security update" assertNotCaptured "security update"
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2" assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3" assertCaptured "Installing SQLite3"
@@ -95,22 +83,20 @@ testPython3_5() {
} }
testPython3_5_warn() { testPython3_5_warn() {
# Python 3.5 is EOL, so it has not been built for Heroku-20.
if [[ $STACK == "heroku-20" ]]; then
return
fi
compile "python3_5_warn" compile "python3_5_warn"
assertCaptured "python-3.5.6" assertCaptured "Installing python-3.5.6"
assertCaptured "security update!" assertCaptured "security update!"
assertCaptured "${LATEST_35}"
assertCapturedSuccess assertCapturedSuccess
} }
testPython3_5_fail() {
compile "python3_5_fail"
assertCaptured "Aborting"
assertCapturedError
}
testPython3_6() { testPython3_6() {
updateVersion "python3_6" $LATEST_36
compile "python3_6" compile "python3_6"
assertCaptured $LATEST_36 assertCaptured "Installing ${LATEST_36}"
assertNotCaptured "security update" assertNotCaptured "security update"
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2" assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3" assertCaptured "Installing SQLite3"
@@ -119,104 +105,129 @@ testPython3_6() {
testPython3_6_warn() { testPython3_6_warn() {
compile "python3_6_warn" compile "python3_6_warn"
assertCaptured "python-3.6.7" assertCaptured "Installing python-3.6.11"
assertCaptured "security update!" assertCaptured "security update!"
assertCaptured "${LATEST_36}"
assertCapturedSuccess
}
testPython3_7() {
compile "python3_7"
assertCaptured "Installing ${LATEST_37}"
assertNotCaptured "security update"
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3" assertCaptured "Installing SQLite3"
assertCapturedSuccess assertCapturedSuccess
} }
testPython3_6_fail() {
compile "python3_6_fail"
assertCaptured "Aborting"
assertCapturedError
}
testPython3_7() {
updateVersion "python3_7" $LATEST_37
compile "python3_7"
if [[ $STACK = "cedar-14" ]]; then
assertCapturedError
else
assertNotCaptured "security update"
assertCaptured $LATEST_37
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3"
assertCapturedSuccess
fi
}
testPython3_7_warn() { testPython3_7_warn() {
compile "python3_7_warn" compile "python3_7_warn"
if [[ $STACK = "cedar-14" ]]; then assertCaptured "Installing python-3.7.8"
assertCapturedError assertCaptured "security update!"
else assertCaptured "${LATEST_37}"
assertCaptured "python-3.7.1" assertCapturedSuccess
assertCaptured "security update!"
assertCaptured "Installing SQLite3"
assertCapturedSuccess
fi
}
testPython3_7_fail() {
compile "python3_7_fail"
assertCaptured "Aborting"
assertCapturedError
}
testPython3_7_warn() {
compile "python3_8_warn"
if [[ $STACK = "cedar-14" ]]; then
assertCapturedError
else
assertCaptured "python-3.8.0"
assertCaptured "security update!"
assertCaptured "Installing SQLite3"
assertCapturedSuccess
fi
} }
testPython3_8() { testPython3_8() {
updateVersion "python3_8" $LATEST_38
compile "python3_8" compile "python3_8"
if [[ $STACK = "cedar-14" ]]; then assertCaptured "Installing ${LATEST_38}"
assertCapturedError assertNotCaptured "security update"
else assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertNotCaptured "security update" assertCaptured "Installing SQLite3"
assertCaptured $LATEST_38 assertCapturedSuccess
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3"
assertCapturedSuccess
fi
} }
testPython3_8_fail() { testPython3_8_warn() {
compile "python3_8_fail" compile "python3_8_warn"
assertCaptured "Installing python-3.8.5"
assertCaptured "security update!"
assertCaptured "${LATEST_38}"
assertCapturedSuccess
}
testPython3_9() {
compile "python3_9"
assertCaptured "Installing ${LATEST_39}"
assertNotCaptured "security update"
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing SQLite3"
assertCapturedSuccess
}
testPython3_9_warn() {
# Can't test the version warning until there is at least one old version of Python 3.9.
if [[ "${LATEST_39}" = "python-3.9.0" ]]; then
return
fi
compile "python3_9_warn"
assertCaptured "Installing python-3.9.0"
assertCaptured "security update!"
assertCaptured "${LATEST_39}"
assertCapturedSuccess
}
testPythonVersionInvalid() {
compile "python_version_invalid"
assertCaptured "Requested runtime (python-3.8.99) is not available for this stack"
assertCaptured "Aborting" assertCaptured "Aborting"
assertCapturedError assertCapturedError
} }
testPypy3_6() { testPypy3_6() {
compile "pypy3_6" compile "pypy3_6"
if [[ $STACK = "cedar-14" ]]; then assertCaptured "Installing ${LATEST_PYPY_36}"
assertCapturedError assertNotCaptured "security update"
else assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCaptured "Installing pypy" assertCapturedSuccess
assertCaptured "$PYPY_36" }
assertCaptured "Installing pip 20.1.1, setuptools 47.1.1 and wheel 0.34.2"
assertCapturedSuccess testPypy3_6_warn() {
fi compile "pypy3_6_warn"
assertCaptured "Installing pypy3.6-7.3.1"
assertCaptured "security update!"
assertCaptured "${LATEST_PYPY_36}"
assertCapturedSuccess
} }
testPypy2_7() { testPypy2_7() {
compile "pypy2_7" compile "pypy2_7"
if [[ $STACK = "cedar-14" ]]; then assertCaptured "Installing ${LATEST_PYPY_27}"
assertCapturedError assertNotCaptured "security update"
else assertCaptured "Installing pip 20.1.1, setuptools 44.1.1 and wheel 0.34.2"
assertCaptured "Installing pypy" assertCapturedSuccess
assertCaptured "$PYPY_27" }
assertCaptured "Installing pip 20.1.1, setuptools 44.1.1 and wheel 0.34.2"
assertCapturedSuccess testPypy2_7_warn() {
fi compile "pypy2_7_warn"
assertCaptured "Installing pypy2.7-7.3.1"
assertCaptured "security update!"
assertCaptured "${LATEST_PYPY_27}"
assertCapturedSuccess
}
testStickyPythonVersion() {
local cache_dir="$(mktmpdir)"
compile "python3_6_warn" "$cache_dir"
assertCaptured "Installing python-3.6.11"
assertCapturedSuccess
compile "python_version_unspecified" "$cache_dir"
assertNotCaptured "Installing python"
assertCaptured "security update!"
assertCapturedSuccess
# Whilst this file seems like an implementation detail (so something that should
# not be tested), we must guarantee the filename remains consistent for backwards
# compatibility across buildpack versions for already-built apps.
assertFile "python-3.6.11" ".heroku/python-version"
}
testPythonVersionChange() {
local cache_dir="$(mktmpdir)"
compile "python3_6_warn" "$cache_dir"
assertCaptured "Installing python-3.6.11"
assertCapturedSuccess
compile "python3_6" "$cache_dir"
assertCaptured "Found python-3.6.11, removing"
assertCapturedSuccess
} }
pushd $(dirname 0) >/dev/null pushd $(dirname 0) >/dev/null
+21 -13
View File
@@ -56,11 +56,6 @@ resetCapture()
unset rtrn # deprecated unset rtrn # deprecated
} }
updateVersion()
{
echo "$2" > "test/fixtures/${1}/runtime.txt"
}
assertCapturedEquals() assertCapturedEquals()
{ {
assertEquals "$@" "$(cat ${STD_OUT})" assertEquals "$@" "$(cat ${STD_OUT})"
@@ -85,12 +80,20 @@ assertNotCaptured()
assertCapturedSuccess() assertCapturedSuccess()
{ {
assertEquals "Captured exit code -" "0" "${RETURN}" assertEquals "Captured exit code -" "0" "${RETURN}"
# assertEquals "STD_ERR -" "" "$(cat ${STD_ERR})" assertEquals "STD_ERR -" "" "$(cat ${STD_ERR})"
if [ $RETURN -ne 0 -a -z "$(cat ${STD_ERR})" ]; then if [ $RETURN -ne 0 -o -n "$(cat ${STD_ERR})" ]; then
# Failing exit code but stderr was empty. Display stdout to help debugging. debug
cat $STD_OUT fi
echo }
assertCapturedSuccessWithStdErr()
{
assertEquals "Captured exit code -" "0" "${RETURN}"
assertNotEquals "STD_ERR -" "" "$(cat ${STD_ERR})"
if [ ${RETURN} -ne 0 ]; then
debug
fi fi
} }
@@ -151,9 +154,14 @@ _assertContains()
debug() debug()
{ {
cat $STD_OUT echo
echo '^^^^^^' echo '### STD_OUT ###'
cat $STD_ERR cat "${STD_OUT}"
echo
echo '### STD_ERR ###'
cat "${STD_ERR}"
echo
echo
} }
assertContains() assertContains()

Some files were not shown because too many files have changed in this diff Show More