The standalone Geo buildpack offers more modern GDAL/GEOS/PROJ library
versions, and can be used by apps in all languages, not just Python:
https://github.com/heroku/heroku-geo-buildpack
As such the Python buildpack's undocumented built-in support was
deprecated back in April 2020, with a scheduled removal date of
6th October 2020:
https://devcenter.heroku.com/changelog-items/1759https://help.heroku.com/D5INLB1A/python-s-build_with_geo_libraries-legacy-feature-is-now-deprecated
Metrics show very few builds continuing to use the built-in support.
Apps with the `BUILD_WITH_GEO_LIBRARIES` env var set will now be shown a
warning directing them to the standalone buildpack, as well as apps that
hit GDAL related pip install errors but aren't using the env var.
This also moves us one step closer to being able to remove
the vendored copy of pip-pop (which is partially broken on
newer pip).
Closes @W-7654424@.
Since the stack is end of life and builds have been disabled:
https://devcenter.heroku.com/changelog-items/1943
There are only two temporarily exempted customers using Python, who
can switch to the Cedar-14 support branch if they still need to build
their Python apps (most of which haven't been built recently).
Closes @W-8054727@.
In #1011 the number of buildpack variables that are exported (and so
exposed to subprocesses) was reduced, since in general we don't want
to leak buildpack internals into end-user steps such as the pre/post
compile hooks.
However since this change, any buildpack metric emitted from within
a `sub_env` wrapper (which is a buildpack-stdlib utility function)
is missing its `buildpack.python` prefix.
This is because buildpack-stdlib:
* lazy-loads `BPLOG_PREFIX` (rather than doing so when initially
sourced, which is the approach used for `BUILDPACK_LOG_FILE`)
* doesn't check whether `BPLOG_PREFIX` is set before emitting metrics
See:
https://github.com/heroku/buildpack-stdlib/blob/v8/stdlib.sh
As a stop-gap until we either fix this in buildpack-stdlib (W-8095466),
or remove usages of the `sub_env` wrapper (since I think they are
counter-productive in this buildpack), I've added back the export
for `BPLOG_PREFIX`.
Fixes @W-8095436@.
* Fixes the "Installing <version>" assertions so that they don't false
positive against the "please upgrade to <version>" output.
* Removes modification of test fixtures during tests, since it can lead
to failures depending on test order, and confusion when debugging.
* Updates the PyPy version warning tests to use a slightly newer (but
still not latest) PyPy version, which means that the test now passes
on Cedar-14 and can be unskipped.
* Switches to using an empty requirements file for version tests that
duplicate the main test, to save spending time installing dependencies
unnecessarily.
* Switches the NLTK test to using the default buildpack Python version,
rather than an ancient Python 3.6.
* Skips the Python 3.7/3.8 tests on Cedar-14 rather than asserting
failure, since we know they'll never pass due to Cedar-14's libssl being
older than required.
* Removes redundant `testSqliteInstall` test since it duplicates the
Python version install tests.
Longer term I'll be moving many of the unit tests to Hatchet, however
this at least makes the tests more dependable in the meantime.
Closes @W-8060219@.
Closes @W-8176779@.
[skip changelog]
Previously the buildpack's S3 bucket was defined in two places - once
in `VENDOR_URL` and again during the pip installation step. This
duplication was necessary since `VENDOR_URL` also contained the stack's
name, whereas the pip use-case used a non-stack-specific S3 key prefix.
In order to:
* reduce this duplication
* simplify this buildpack's S3 bucket migration (where we'll soon be
needing the vary the bucket name and wouldn't want to have to
duplicate that logic in multiple places)
* allow overriding of the URL for the pip use-case
...the `VENDOR_URL` variable has been replaced with `S3_BASE_URL` which
no longer contains the stack name.
The user-configurable override has similarly been renamed from
`BUILDPACK_VENDOR_URL` to `BUILDPACK_S3_BASE_URL`. Note: As before,
this override cannot be set via standard app variables (see #989).
The unused `USE_STAGING_BINARIES` environment variable has been
removed, since it's a leftover from the project to stand up a staging S3 bucket.
It's redundant given the `BUILDPACK_S3_BASE_URL` variable.
Closes @W-8142401@.
Updates pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+:
https://pip.pypa.io/en/stable/news/#id40
The version used for Python 3.4 remains unchanged at 19.1.1, since it's
the last version of pip that supports it.
Pip has been updated to 20.1.1 rather than the recently released 20.2,
since the latter has a few regressions and even though these will be
fixed shortly in 20.2.1, we should let the changes soak for longer
before picking them up.
The `PIP_NO_PYTHON_VERSION_WARNING` environment variable has been set
(equivalent to passing `--no-python-version-warning`) to prevent the
Python 2.7 EOL warnings added in pip 20.1 from spamming the build log:
https://github.com/pypa/pip/blob/20.1.1/src/pip/_internal/cli/base_command.py#L139-L154
This was set via environment variable rather than CLI flag, since:
* otherwise we'd have to pass it to every pip invocation
* older pip (such as the 19.1.1 used by Python 3.4) doesn't support this
option and would error out due to an unknown CLI flag being passed,
unless we added conditional flags throughout.
The new pip wheel was uploaded to S3 using:
```
$ pip download --no-cache pip==20.1.1
Collecting pip==20.1.1
Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
Saved ./pip-20.1.1-py2.py3-none-any.whl
Successfully downloaded pip
$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun
(dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl
$ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read
upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl
```
Fixes#1005.
@W-7659489@
Upgrades setuptools from 39.0.1 to:
- 44.1.1 for Python 2.7 (since it's the last supported version)
- 43.0.0 for Python 3.4 (since it's the last supported version)
- 47.1.1 for Python 3.5+ (since we can't use 47.2.0+ until #1006 fixed)
https://setuptools.readthedocs.io/en/latest/history.html#v47-1-1Fixes#949.
Closes#973.
Before:
- if `wheel` was not already installed, then `get-pip.py` would
automatically install the latest version on PyPI, which is `0.34.2`
(or `0.33.6` for Python 3.4).
- if `wheel` was already installed, then it was left unchanged
regardless of the version installed.
Now:
- if `wheel` is not already installed, then the same versions will be
installed as before, except these versions are pinned and will now not
change unexpectedly after future `wheel` releases.
- if `wheel` is already installed, then it's upgraded/downgraded to the
target version as needed.
Partly addresses #1000, though this change only helps builds where the
pip/setuptools/wheel install flow is triggered (currently only new apps
or ones where Python was purged or pip was not the correct version).
Since the wheel version is now known, it's output to the build log to
ease debugging and for parity with pip/setuptools.
The rest of #1000 will be fixed in later commits.
Since:
* "explicit is better than implicit"
* we'll soon be upgrading setuptools, and debugging breakage caused by
upgrades will be easier if versions are visible in the build log
Since:
* "explicit is better than implicit"
* we'll soon be upgrading pip, and debugging breakage caused by upgrades
will be easier if versions are visible in the build log
Closes#939.
The following env vars are no longer exposed to subprocesses run by the
buildpack (such as the `bin/pre_compile` and `bin/post_compile` hooks):
* `BPLOG_PREFIX`
* `CACHED_PYTHON_STACK`
* `DEFAULT_PYTHON_STACK`
* `DEFAULT_PYTHON_VERSION`
* `LATEST_27`
* `LATEST_34`
* `LATEST_35`
* `LATEST_36`
* `LATEST_37`
* `LATEST_38`
* `PIP_UPDATE`
* `PY27`
* `PY34`
* `PY35`
* `PY36`
* `PY37`
* `PYPY_27`
* `PYPY_36`
* `RECOMMENDED_PYTHON_VERSION`
* `WARNINGS_LOG`
There were previously no tests at all for the pre/post-compile hooks,
so I've added some now.
Fixes#1010.
Ed Morley