Switches the last consumers of it to a simpler utility function that
uses `pkgutil.find_loader()`:
https://docs.python.org/3/library/pkgutil.html#pkgutil.find_loader
Both of these consumers are covered by existing tests.
Then removes `sp-grep` and the remaining parts of `pip-pop`.
Closes @W-8208817@.
Since the last usage was removed in #1117.
Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.
Refs @W-8208817@.
Since the last usage was removed in #1113, and pip-grep is actually
broken on newer pip anyway (though helpfully silently ignores
exceptions, so one wouldn't know).
This is the first of the pip-pop removals, the rest will follow as the
last usages are switched over.
Whilst this tool was intended for internal buildpack usage only, the
vendor directory is on `PATH`, so I've documented the removal in
CHANGELOG just in case.
Refs @W-8208817@.
Since fetching buildpack-stdlib from S3 has a number of disadvantages:
- it's not possible to grep the repo when trying to work out what
something from the stdlib is doing
- shellcheck can't fully scan the code, since it similarly doesn't
have the source
- another compile-time HTTP request that can fail due to transient
network issues and so reduce reliability
- dependency on the security/release-process of an additional bucket
Since the stdlib is small, doesn't often change, and is not a binary,
it's a great fit for just vendoring in the repo.
The `BIN_DIR` calculation added to `bin/utils` is based on the approach
mentioned here:
https://www.ostricher.com/2014/10/the-right-way-to-get-the-directory-of-a-bash-script/
...rather than copying the version in `bin/compile`, since the latter uses
`$0` so doesn't work with sourced scripts (such as `bin/utils`).
Closes W-8094463.
The tests use `test/shunit2` so the copy under `vendor/` is redundant.
I've added to the CHANGELOG since `vendor/` is added by the buildpack to
`PATH` (something long term we should stop doing), increasing the chance
an app was accidentally relying upon `shunit2` (albeit still unlikely).
Closes @W-8145517@.
Since the unit tests instead use the utilities in this separate file:
https://github.com/heroku/heroku-buildpack-python/blob/419ef479969c4d5945f2c0620292229ef464f4c8/test/utils
A changelog entry has been added since whilst this file is for internal
testing only, the buildpack's `vendor/` directory is put on `PATH`, so
in theory it could have been called outside the buildpack (though this
seems extremely unlikely since the script isn't very useful externally).
Fixes#1027.
Closes @W-7918496@.
Since:
* we'll be updating setuptools soon, and newer setuptools has dropped
support for Python versions this buildpack needs to support. As such
if we continued to vendor setuptools, we would need to vendor at
least three different versions.
* we want to try and update setuptools more frequently than we have
in the past, which will mean more repo bloat from binary churn.
* we're still pinning to a specific version, meaning vendoring doesn't
have determinism benefits.
* setuptools is only fetched from PyPI for new installs (or where
versions have changed), so this doesn't increase build time, load on
PyPI, or reliance on PyPI in the common case.
* setuptools is already being inadvertently installed from PyPI prior to
being installed from the vendored copy (see #1001), so we're in effect
already using/depending on PyPI here.
* switching to storing setuptools on S3 wouldn't help reliability as
much as it would appear at first glance, since the later `pip install`
of customer dependencies will fail if PyPI is down anyway.
The pip-diff and pip-grep tools from the vendorized `pip-pop` package
import internal modules from pip. In pip >= 10, internal modules were
moved under `pip._internal`, breaking the imports. Use `try...except
ImportError` to handle both import paths.
Also, the interface of the `PackageFinder` class from one of these
modules changed. Provide a wrapper function to allow creating objects of
this type using the old interface.
* Revert "Fix pyyaml (#402)"
This reverts commit ff94908505.
* Revert "Don't install packages that could mess packaging up (#397)"
This reverts commit 5496c02f9f.
* Revert "use pkg_resources to check for distributions (#395)"
This reverts commit 9b185f99d5.
* use sp-grep for django detection
* sp-grep
* fix setuptools script
* update python.gunicorn.sh to WEB_CONCURRENCY.sh
* don't remove WEB_CONCURRENCY for prefixed 0s
* split gunicorn functionality into its own file
* cleanup comments
* fix var ordering
* add retry to curl
* use proper url for buildpack stdlib
The Node buildpack now exports a leading zero in its numbers. This lets us detect that it (or another buildpack) set the value, and overwrite it accordingly.
Fixes the issue where adding the node buildpack to a Python app would cause only one gunicorn worker to be spawned for a 1X dyno, and not two.
We also need to again produce leading zeroes in the value, so that e.g. the PHP buildpack can do the same on boot.
This means pip-pop will use the buildpack-installed pip 8, which doesn't
fail when encountering requirements files that use the new hash syntax.
Fixes#302.
The extraneous `format` call is harmless in itself, but it's possible for `requirement.req` to be `None` here, which results in `AttributeError: 'NoneType' object has no attribute 'project_name'`.
Ed Morley
kennethreitz