Compare commits

..

7 Commits

206 changed files with 20742 additions and 3257 deletions
-1
View File
@@ -1 +0,0 @@
* @heroku/languages
-8
View File
@@ -1,8 +0,0 @@
<!-- Hi and welcome to the Heroku Python buildpack repository!
If you meant to open a PR against a fork instead of upstream, please adjust the base branch:
https://help.github.com/articles/changing-the-base-branch-of-a-pull-request/
Otherwise thank you in advance for your Pull Request - just remember to
include as much information as possible to help the reviewers :-)
-->
-12
View File
@@ -1,12 +0,0 @@
name: Check Changelog
on: [pull_request]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- name: Check that CHANGELOG is touched
run: |
cat $GITHUB_EVENT_PATH | jq .pull_request.title | grep -i '\[\(\(changelog skip\)\|\(ci skip\)\)\]' || git diff remotes/origin/${{ github.base_ref }} --name-only | grep CHANGELOG.md
-9
View File
@@ -3,12 +3,3 @@ site
.DS_Store
/.envrc
repos/*
#Venv
buildpack/*
builds/dockerenv.staging*
builds/dockerenv.production
test/scratch
-7
View File
@@ -1,7 +0,0 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
+20 -36
View File
@@ -1,39 +1,23 @@
language: ruby
language: bash
dist: trusty
sudo: required
rvm:
- 2.4.4
before_script:
- gem install bundler -v 1.16.2
- bundle exec hatchet ci:setup
script:
- docker build --pull --tag travis-build-cedar-14 --file $(pwd)/builds/cedar-14.Dockerfile .
- docker run --rm -e "STACK=cedar-14" travis-build-cedar-14 bash $TESTFOLDER
- docker build --pull --tag travis-build-heroku-16 --file $(pwd)/builds/heroku-16.Dockerfile .
- docker run --rm -e "STACK=heroku-16" travis-build-heroku-16 bash $TESTFOLDER
- docker build --pull --tag travis-build-heroku-18 --file $(pwd)/builds/heroku-18.Dockerfile .
- docker run --rm -e "STACK=heroku-18" travis-build-heroku-18 bash $TESTFOLDER
jobs:
include:
- stage: Bash linting (shellcheck)
sudo: false
script: make check
- stage: Hatchet Integration
if: branch = master
name: Run Hatchet
script: "bundle exec rspec"
env:
matrix:
- TESTFOLDER=test/run-deps
- TESTFOLDER=test/run-versions
- TESTFOLDER=test/run-features
global:
- HATCHET_RETRIES=3
- IS_RUNNING_ON_CI=true
- HATCHET_APP_LIMIT=5
- HATCHET_DEPLOY_STRATEGY=git
- secure: yjtlPE5FbVxTKnjUy/tZUBgSEf4qADD3QOxtgziuid73S0U/1IEXlMGFULsQzIjtlHKmHeywZqpVVEpthIH4RuT7uoX1Pb7SSM/g0T8fT3VoEFbFK1uYl0oZQbUS4Klxv9tPiumj8if3m6ULEGIz1X0wZcMOC0tMLwVCnwmap0E=
- secure: ZeFTHWwnpIKE9nAqs88ocmiQh7bKce84lilGm5J23nf3N6V4wNyLwqlkvsM008WGBCaOg9AUx7ZunasT0ANsR5gLP3eV2UUg7ILdRgV2Gy13eNRFheC4PHdN92RqQ3aKoqlIv2K999xlhVjod0NzhkQQXB6PddfQINbuU7ks6As=
- stage: "Bash linting (shellcheck)"
sudo: false
addons:
apt:
sources:
- debian-sid # Grab shellcheck from the Debian repo (o_O)
packages:
- shellcheck
script: make check
- stage: "Stack Tests"
services: docker
env: STACK=heroku-16
script: ./tests.sh
- stage: "Stack Tests"
services: docker
env: STACK=cedar-14
script: ./tests.sh
+6 -271
View File
@@ -1,288 +1,23 @@
# Python Buildpack Changelog
# Master
- Set Code Owners to @heroku/langauges
- Bugfix: Caching on subsequent redeploys
- Update tests to support latest version of Python
--------------------------------------------------------------------------------
# 168 (2020-04-06)
- Doc: Update Readme with version numbers
- update Code Owners to include the Heroku Buildpack Maintainers team
- Deprecation warning: BUILD_WITH_GEO_LIBRARIES is now deprecated. See warning for details.
- Clean up build log output
- Update Python versions in README to match docs
- Django version detection fixed, link updated
# 167 (2020-03-26)
- Add failcase for cache busting
- Bugfix: Clearing pip dependencies
# 166 (2020-03-05)
- Correct ftp to https in vendored file
- Warn for Django 1.11 approaching EOL, provide link to roadmap
# 165 (2020-02-27)
- Python 3.8.2 now available.
# 164 (2020-02-20)
- Update requirements.txt builds to use Pip 20.0.2
- Download get-pip.py to tmpdir instead of root dir
# 163 (2019-12-23)
- New pythons released:
Python 3.8.1, 3.7.6, 3.6.10 (CPython)
Beta Release: Pypy 2.7 and 3.6, version 7.2.0
# 162 (2019-12-06)
- Bug fix: fragile sqlite3 install
# 161 (2019-12-2)
- Bug fix: Sqlite3 version bump
# 160 (2019-10-23)
- Bugfix: Pipenv no longer installs twice in CI
# 159 (2019-10-22)
- Python 2.7.17 now available on Heroku 18 and 16.
# 158 (2019-10-21)
- Python 3.7.5 and 3.8.0 now available on Heroku 18 and 16.
- Add support for Python 3.8 branch
- Sqlite3 Update:
- Test Improvements
- Add support for staging binary testing
# 157 (2019-09-18)
- Typo fixes
# 156 (2019-09-12)
- Python 3.6.9 and 3.7.4 now available.
- Move get-pip utility to S3
- Build utility and documentation updates
- Bump Hatchet tests to point at new default python version.
# 155 (2019-08-22)
add docs and make target for heroku-18 bob builds
# 154 (2019-07-17)
Fix python 3.5.7 formula actually building 3.7.2
# 153 (2019-06-21)
Hotfix for broken heroku-16 deploys
# 152 (2019-04-04)
Python 3.7.3 now available.
# 151 (2019-03-21)
Python 3.5.7 and 3.4.10 now available on all Heroku stacks.
# 150 (2019-03-13)
Python 2.7.16 now available on all Heroku stacks.
# 149 (2019-03-04)
Hotfix for broken Cedar 14 deploys
# 148 (2019-02-21)
No user facing changes, improving internal metrics
# 147 (2019-02-07)
Python 3.7.2 and 3.6.8 now available on all Heroku stacks.
# 146 (2018-11-11)
Python 3.7.1, 3.6.7, 3.5.6 and 3.4.9 now available on all Heroku stacks.
# 145 (2018-11-08)
Testing and tooling expanded to better support new runtimes
# 144 (2018-10-10)
Switch to cautious upgrade for Pipenv install to ensure the pinned pip version
is used with Pipenv
# 143 (2018-10-09)
Add support for detecting SLUGIFY_USES_TEXT_UNIDECODE, which is required to
install Apache Airflow version 1.10 or higher.
# 142 (2018-10-08)
Improvements to Python install messaging
# 139, 140, 141
No user-facing changes, documenting for version clarity
# 138 (2018-08-01)
Use stack image SQLite3 instead of vendoring
# 137 (2018-07-17)
Prevent 3.7.0 from appearing as unsupported in buildpack messaging.
# 136 (2018-06-28)
Upgrade to 3.6.6 and support 3.7.0 on all runtimes.
# 135 (2018-05-29)
Upgrade Pipenv to v2018.5.18.
# 134 (2018-05-02)
Default to 3.6.5, bugfixes.
# 133
Fixes for Pip 10 release.
# 132
Improve pip installation, with the release of v9.0.2.
# 131
Fix bug with pip.
# 130
Better upgrade strategy for pip.
# 129
Don't upgrade pip (from v128).
# 128
Upgrade pip, pin to Pipenv v11.8.2.
# 127
Pin to Pipenv v11.7.1.
# 126
Bugfixes.
# 125
Bugfixes.
# 124
Update buildpack to automatically install `[dev-packages]` during Heroku CI Pipenv builds.
- Skip installs if Pipfile.lock hasn't changed, and uninstall stale dependencies with Pipenv.
- Set `PYTHONPATH` during collectstatic runs.
- No longer warn if there is no `Procfile`.
- Update Pipenv's "3.6" runtime specifier to point to "3.6.4".
# 123
Update gunicorn `init.d` script to allow overrides.
# 122
Update default Python to v3.6.4.
# 121
Update default Python to v3.6.3.
# 120
Use `$ pipenv --deploy`.
# 119
Improvements to Pipenv support, warning on unsupported Python versions.
- We now warn when a user is not using latest 2.x or 3.x Python.
- Heroku now supports `[requires]` `python_full_version` in addition to `python_version`.
# 118
Improvements to Pipenv support.
# 117
Bug fix.
# 116
Vendoring improvements.
- Geos libraries should work on Heroku-16 now.
- The libffi/libmemcached vendoring step is now skipped on Heroku-16 (since they are installed in the base image).
# 115
Revert a pull request.
- No longer using `sub_env` for `pip install` step.
# 114
- Bugfixes.
Blacklisting `PYTHONHOME` and `PYTHONPATH` for older apps. Upgrades to nltk support.
# 113
Updates to Pipenv support.
# 112
Bugfix.
- Fixed grep output bug.
# 111
Linting, bugfixes.
Linting , bugfixes.
# 110
Update default Python to 3.6.2.
# 109
Update Default Python to 3.6.1, bugfixes.
- Fixed automatic pip uninstall of dependencies removed from requirements.txt.
# 108
# 109
Fix output for collectstatic step.
# 108
Updated setuptools.
# 107
Bugfix for C dependency installation.
+2 -4
View File
@@ -3,11 +3,9 @@ FROM heroku/heroku:16-build
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \
S3_PREFIX="heroku-16/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-16"
S3_PREFIX="heroku-16/"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev && rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y python-pip && rm -rf /var/lib/apt/lists/*
COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
-6
View File
@@ -1,6 +0,0 @@
source "https://rubygems.org"
gem "rspec"
gem "heroku_hatchet"
gem "rspec-retry"
gem "rake"
-71
View File
@@ -1,71 +0,0 @@
GEM
remote: https://rubygems.org/
specs:
activesupport (5.2.1)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
tzinfo (~> 1.1)
concurrent-ruby (1.1.3)
diff-lcs (1.3)
erubis (2.7.0)
excon (0.62.0)
heroics (0.0.25)
erubis (~> 2.0)
excon
moneta
multi_json (>= 1.9.2)
heroku_hatchet (4.0.6)
excon (~> 0)
minitest-retry (~> 0.1.9)
platform-api (~> 2)
repl_runner (~> 0.0.3)
rrrretry (~> 1)
thor (~> 0)
threaded (~> 0)
i18n (1.1.1)
concurrent-ruby (~> 1.0)
minitest (5.11.3)
minitest-retry (0.1.9)
minitest (>= 5.0)
moneta (1.0.0)
multi_json (1.13.1)
platform-api (2.2.0)
heroics (~> 0.0.25)
moneta (~> 1.0.0)
rake (12.3.1)
repl_runner (0.0.3)
activesupport
rrrretry (1.0.0)
rspec (3.8.0)
rspec-core (~> 3.8.0)
rspec-expectations (~> 3.8.0)
rspec-mocks (~> 3.8.0)
rspec-core (3.8.0)
rspec-support (~> 3.8.0)
rspec-expectations (3.8.1)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0)
rspec-mocks (3.8.0)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.8.0)
rspec-retry (0.6.1)
rspec-core (> 3.3)
rspec-support (3.8.0)
thor (0.20.3)
thread_safe (0.3.6)
threaded (0.0.4)
tzinfo (1.2.5)
thread_safe (~> 0.1)
PLATFORMS
ruby
DEPENDENCIES
heroku_hatchet
rake
rspec
rspec-retry
BUNDLED WITH
1.16.3
+6 -24
View File
@@ -1,32 +1,27 @@
# These targets are not files
.PHONY: tests
test: test-heroku-18 test-heroku-16 test-cedar-14
test: test-heroku-16
check:
@shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings bin/default_pythons
@shellcheck -x bin/steps/collectstatic bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/gdal bin/steps/geo-libs bin/steps/mercurial bin/steps/nltk bin/steps/pip-install bin/steps/pip-uninstall bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/pylibmc bin/steps/python
@shellcheck -x bin/compile bin/detect bin/release bin/test-compile bin/utils bin/warnings
@shellcheck -x bin/steps/collectstatic bin/steps/cryptography bin/steps/eggpath-fix bin/steps/eggpath-fix2 bin/steps/gdal bin/steps/geo-libs bin/steps/mercurial bin/steps/nltk bin/steps/pip-install bin/steps/pip-uninstall bin/steps/pipenv bin/steps/pipenv-python-version bin/steps/pylibmc bin/steps/python
@shellcheck -x bin/steps/hooks/*
test-cedar-14:
@echo "Running tests in docker (cedar-14)..."
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=cedar-14" heroku/cedar:14 bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=cedar-14" heroku/cedar:14 bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run;'
@echo ""
test-heroku-16:
@echo "Running tests in docker (heroku-16)..."
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=heroku-16" heroku/heroku:16-build bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@echo ""
test-heroku-18:
@echo "Running tests in docker (heroku-18)..."
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=heroku-18" heroku/heroku:18-build bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run-deps; test/run-features; test/run-versions;'
@docker run -v $(shell pwd):/buildpack:ro --rm -it -e "STACK=heroku-16" heroku/heroku:16-build bash -c 'cp -r /buildpack /buildpack_test; cd /buildpack_test/; test/run;'
@echo ""
buildenv-heroku-16:
@echo "Creating build environment (heroku-16)..."
@echo
@docker build --pull -f $(shell pwd)/builds/heroku-16.Dockerfile -t python-buildenv-heroku-16 .
@docker build --pull -t python-buildenv-heroku-16 .
@echo
@echo "Usage..."
@echo
@@ -36,19 +31,6 @@ buildenv-heroku-16:
@echo
@docker run -it --rm python-buildenv-heroku-16
buildenv-heroku-18:
@echo "Creating build environment (heroku-18)..."
@echo
@docker build --pull -f $(shell pwd)/builds/heroku-18.Dockerfile -t python-buildenv-heroku-18 .
@echo
@echo "Usage..."
@echo
@echo " $$ export AWS_ACCESS_KEY_ID=foo AWS_SECRET_ACCESS_KEY=bar # Optional unless deploying"
@echo " $$ bob build runtimes/python-2.7.13"
@echo " $$ bob deploy runtimes/python-2.7.13"
@echo
@docker run -it --rm python-buildenv-heroku-18
tools:
git clone https://github.com/kennethreitz/pip-pop.git
mv pip-pop/bin/* vendor/pip-pop/
-9
View File
@@ -1,9 +0,0 @@
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[dev-packages]
[packages]
"bob-builder" = "==0.0.13"
Generated
-52
View File
@@ -1,52 +0,0 @@
{
"_meta": {
"hash": {
"sha256": "36d17c46a8b1b844b3cae475f6f42d6c0a9e59b2a9685cbcdc0985656a7a129f"
},
"host-environment-markers": {
"implementation_name": "cpython",
"implementation_version": "3.6.3",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_python_implementation": "CPython",
"platform_release": "16.7.0",
"platform_system": "Darwin",
"platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
"python_full_version": "3.6.3",
"python_version": "3.6",
"sys_platform": "darwin"
},
"pipfile-spec": 6,
"requires": {},
"sources": [
{
"name": "pypi",
"url": "https://pypi.python.org/simple",
"verify_ssl": true
}
]
},
"default": {
"bob-builder": {
"hashes": [
"sha256:b4de49a8e436fcaf82236ea43f78413b4a4c92100726e382ab57b6bdfb38fe64",
"sha256:288e3e765c4890fe9a63ae52ac6b4a963c13fe508482c70ff701a5ae21b9a673"
],
"version": "==0.0.13"
},
"boto": {
"hashes": [
"sha256:13be844158d1bd80a94c972c806ec8381b9ea72035aa06123c5db6bc6a6f3ead",
"sha256:deb8925b734b109679e3de65856018996338758f4b916ff4fe7bb62b6d7000d1"
],
"version": "==2.48.0"
},
"docopt": {
"hashes": [
"sha256:49b3a825280bd66b3aa83585ef59c4a8c82f2c8a522dbe754a8bc8d08c85c491"
],
"version": "==0.6.2"
}
},
"develop": {}
}
+28 -61
View File
@@ -4,54 +4,38 @@
[![Build Status](https://travis-ci.org/heroku/heroku-buildpack-python.svg?branch=master)](https://travis-ci.org/heroku/heroku-buildpack-python)
This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps.
This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps, powered by [pip](https://pip.pypa.io/) and other excellent software.
Recommended web frameworks include **Django** and **Flask**, among others. The recommended webserver is **Gunicorn**. There are no restrictions around what software can be used (as long as it's pip-installable). Web processes must bind to `$PORT`, and only the HTTP protocol is permitted for incoming connections.
Recommended web frameworks include **Django** and **Flask**. The recommended webserver is **Gunicorn**. There are no restrictions around what software can be used (as long as it's pip-installable). Web processes must bind to `$PORT`, and only the HTTP protocol is permitted for incoming connections.
Python packages with C dependencies that are not [available on the stack image](https://devcenter.heroku.com/articles/stack-packages) are generally not supported, unless `manylinux` wheels are provided by the package maintainers (common). For recommended solutions, check out [this article](https://devcenter.heroku.com/articles/python-c-deps) for more information.
Some Python packages with obscure C dependencies are [not compatible](https://devcenter.heroku.com/articles/python-c-deps).
See it in Action
----------------
```
$ ls
my-application requirements.txt runtime.txt
$ git push heroku master
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (4/4), 276 bytes | 276.00 KiB/s, done.
Total 4 (delta 0), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Python app detected
remote: -----> Installing python-3.7.4
remote: -----> Installing pip
remote: -----> Installing SQLite3
remote: -----> Installing requirements with pip
remote: Collecting flask (from -r /tmp/build_c2c067ef79ff14c9bf1aed6796f9ed1f/requirements.txt (line 1))
remote: Downloading ...
remote: Installing collected packages: Werkzeug, click, MarkupSafe, Jinja2, itsdangerous, flask
remote: Successfully installed Jinja2-2.10 MarkupSafe-1.1.0 Werkzeug-0.14.1 click-7.0 flask-1.0.2 itsdangerous-1.1.0
remote:
remote: -----> Discovering process types
remote: Procfile declares types -> (none)
remote:
```
Deploying a Python application couldn't be easier:
A `requirements.txt` must be present at the root of your application's repository to deploy.
$ ls
Procfile requirements.txt web.py
To specify your python version, you also need a `runtime.txt` file - unless you are using the default Python runtime version.
$ heroku create --buildpack heroku/python
Current default Python Runtime: Python 3.6.9
$ git push heroku master
...
-----> Python app detected
-----> Installing python-3.6.2
$ pip install -r requirements.txt
Collecting requests (from -r requirements.txt (line 1))
Downloading requests-2.12.4-py2.py3-none-any.whl (576KB)
Installing collected packages: requests
Successfully installed requests-2.12.4
Alternatively, you can provide a `setup.py` file, or a `Pipfile`. Using `Pipenv` will generate `runtime.txt` based on `python-version` at build time.
-----> Discovering process types
Procfile declares types -> (none)
Specify a Buildpack Version
---------------------------
A `requirements.txt` file must be present at the root of your application's repository.
You can specify the latest production release of this buildpack for upcoming builds of an existing application:
You can also specify the latest production release of this buildpack for upcoming builds of an existing application:
$ heroku buildpacks:set heroku/python
@@ -59,31 +43,14 @@ You can specify the latest production release of this buildpack for upcoming bui
Specify a Python Runtime
------------------------
Supported runtime options include:
Specific versions of the Python runtime can be specified with a `runtime.txt` file:
- `python-3.8.2`
- `python-3.7.6`
- `python-3.6.10`
- `python-2.7.17`
$ cat runtime.txt
python-2.7.13
## Tests
Runtime options include:
The buildpack tests use [Docker](https://www.docker.com/) to simulate
Heroku's [stack images.](https://devcenter.heroku.com/articles/stack)
To run the test suite:
```
make test
```
Or to test in a particular stack:
```
make test-heroku-18
make test-heroku-16
```
The tests are run via the vendored
[shunit2](https://github.com/kward/shunit2)
test framework.
- `python-3.6.2`
- `python-2.7.13`
- `pypy-5.7.1` (unsupported, experimental)
- `pypy3-5.5.1` (unsupported, experimental)
-1
View File
@@ -1 +0,0 @@
require 'hatchet/tasks'
+61 -196
View File
@@ -15,123 +15,61 @@
# Fail fast and fail hard.
set -eo pipefail
# Boostrap the Buildpack Standard Library.
# Standard Library.
export BPLOG_PREFIX="buildpack.python"
export BUILDPACK_LOG_FILE=${BUILDPACK_LOG_FILE:-/dev/null}
[ "$BUILDPACK_XTRACE" ] && set -o xtrace
# Prepend proper path for old-school virtualenv hackery.
# This may not be neccessary.
# Prepend proper path for virtualenv hackery. This will be deprecated soon.
export PATH=:/usr/local/bin:$PATH
# Setup Path variables, for later use in the Buildpack.
# Paths.
BIN_DIR=$(cd "$(dirname "$0")"; pwd) # absolute path
ROOT_DIR=$(dirname "$BIN_DIR")
BUILD_DIR=$1
CACHE_DIR=$2
ENV_DIR=$3
# Export Path variables, for use in sub-scripts.
export BUILD_DIR CACHE_DIR ENV_DIR
# Set the Buildpack's internet target for downloading Python distributions.
# The user can provide BUILDPACK_VENDOR_URL to specify a custom target.
# Note: this is designed for non-Heroku use, as it does not use the user-provided
# environment variable mechanism (the ENV_DIR).
VENDOR_URL="https://lang-python.s3.amazonaws.com/$STACK"
if [[ -n ${BUILDPACK_VENDOR_URL:-} ]]; then
VENDOR_URL="$BUILDPACK_VENDOR_URL"
elif [[ -n ${USE_STAGING_BINARIES} ]]; then
VENDOR_URL="$USE_STAGING_BINARIES/$STACK"
fi
export VENDOR_URL
# Default Python Versions
# shellcheck source=bin/default_pythons
source "$BIN_DIR/default_pythons"
# Supported Python Branches
PY38="python-3.8"
PY37="python-3.7"
PY36="python-3.6"
PY35="python-3.5"
PY34="python-3.4"
PY27="python-2.7"
PYPY27="pypy2.7"
PYPY36="pypy3.6"
# Which stack is used (for binary downloading), if none is provided (e.g. outside of Heroku)?
# Python defaults
DEFAULT_PYTHON_VERSION="python-3.6.2"
DEFAULT_PYTHON_STACK="cedar-14"
# If pip doesn't match this version (the version we install), run the installer.
PIP_UPDATE="20.0.2"
PIP_UPDATE="9.0.1"
for file in "$BUILD_DIR/runtime.txt" "$CACHE_DIR/.heroku/python-version" ; do
[ -f "$file" ] || continue
export DEFAULT_PYTHON_VERSION DEFAULT_PYTHON_STACK PIP_UPDATE
version=$(tr -d '[:space:]' < "$file")
case "$version" in "$PY34"*)
# Python 3.4 support was dropped in pip >= 19.2.
PIP_UPDATE="19.1.1"
break
;;
esac
done
if [[ -f "$BUILD_DIR/Pipfile" ]]; then
# Do not force pipenv users to re-install pipenv locally.
PIP_UPDATE="9.0.2"
fi
export DEFAULT_PYTHON_STACK PIP_UPDATE
export PY37 PY36 PY35 PY27 PY34
# Common Problem Warnings:
# This section creates a temporary file in which to stick the output of `pip install`.
# The `warnings` subscript then greps through this for common problems and guides
# the user towards resolution of known issues.
# Common Problem Warnings
WARNINGS_LOG=$(mktemp)
export WARNINGS_LOG
export RECOMMENDED_PYTHON_VERSION=$DEFAULT_PYTHON_VERSION
# The buildpack ships with a few executable tools (e.g. pip-grep, etc).
# This installs them into the path, so we can execute them directly.
# Setup vendored tools and pip-pop (pip-diff)
export PATH=$PATH:$ROOT_DIR/vendor/:$ROOT_DIR/vendor/pip-pop
# Set environment variables if they weren't set by the platform.
# Note: this is legacy, for a deprecated build system known as Anvil.
# This can likely be removed, with caution.
# Support Anvil Build_IDs
[ ! "$SLUG_ID" ] && SLUG_ID="defaultslug"
[ ! "$REQUEST_ID" ] && REQUEST_ID=$SLUG_ID
[ ! "$STACK" ] && STACK=$DEFAULT_PYTHON_STACK
# Sanitize externally-provided environment variables:
# The following environment variables are either problematic or simply unneccessary
# for the buildpack to have knowledge of, so we unset them, to keep the environment
# as clean and pristine as possible.
# Sanitizing environment variables.
unset GIT_DIR PYTHONHOME PYTHONPATH
unset RECEIVE_DATA RUN_KEY BUILD_INFO DEPLOY LOG_TOKEN
unset CYTOKINE_LOG_FILE GEM_PATH
# Import the utils script, which contains helper functions used throughout the buildpack.
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# Import the warnings script, which contains the `pip install` user warning mechanisms
# (mentioned and explained above)
# Import collection of warnings.
# shellcheck source=bin/warnings
source "$BIN_DIR/warnings"
# Make the directory in which we will create symlinks from the temporary build directory
# to `/app`.
# Symlinks are required, since Python is not a portable installation.
# More on this topic later.
# we need to put a bunch of symlinks in there later
mkdir -p /app/.heroku
# This buildpack programatically generates (or simply copies) a number of files for
# buildpack machinery: an export script, and a number of `.profile.d` scripts. This
# section declares the locations of those files and targets.
# Set up outputs under new context
PROFILE_PATH="$BUILD_DIR/.profile.d/python.sh"
EXPORT_PATH="$BIN_DIR/../export"
GUNICORN_PROFILE_PATH="$BUILD_DIR/.profile.d/python.gunicorn.sh"
@@ -140,77 +78,49 @@ WEB_CONCURRENCY_PROFILE_PATH="$BUILD_DIR/.profile.d/WEB_CONCURRENCY.sh"
# We'll need to send these statics to other scripts we `source`.
export BUILD_DIR CACHE_DIR BIN_DIR PROFILE_PATH EXPORT_PATH
# Python Environment Variables
# Set Python-specific environment variables, for running Python within the buildpack.
# Notes on each variable included.
# PATH is relatively obvious, we need to be able to execute 'python'.
# Prepend proper environment variables for Python use.
export PATH=/app/.heroku/python/bin:/app/.heroku/vendor/bin:$PATH
# Tell Python to not buffer it's stdin/stdout.
export PYTHONUNBUFFERED=1
# Set the locale to a well-known and expected standard.
export LANG=en_US.UTF-8
# `~/.heroku/vendor` is an place where the buildpack may stick pre-build binaries for known
# C dependencies (e.g. libmemcached on cedar-14). This section configures Python (GCC, more specifically)
# and pip to automatically include these paths when building binaries.
export C_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/app/.heroku/vendor/include:/app/.heroku/python/include:$CPLUS_INCLUDE_PATH
export LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LIBRARY_PATH
export LD_LIBRARY_PATH=/app/.heroku/vendor/lib:/app/.heroku/python/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=/app/.heroku/vendor/lib/pkg-config:/app/.heroku/python/lib/pkg-config:$PKG_CONFIG_PATH
# The Application Code
# --------------------
# Switch to the repo's context.
cd "$BUILD_DIR"
# The Cache
# ---------
# Warn for lack of Procfile.
if [[ ! -f Procfile ]]; then
puts-warn 'Warning: Your application is missing a Procfile. This file tells Heroku how to run your application.'
puts-warn 'Learn more: https://devcenter.heroku.com/articles/procfile'
fi
# The workflow for the Python Buildpack's cache is as follows:
#
# - `~/.heroku/{known-paths}` are copied from the cache into the slug.
# - The build is executed, modifying `~/.heroku/{known-paths}`.
# - Once the build is complete, `~/.heroku/{known-paths}` is copied back into the cache.
# Create the cache directory, if it doesn't exist.
mkdir -p "$CACHE_DIR/.heroku"
# Prepare the cache.
mkdir -p "$CACHE_DIR"
# Restore old artifacts from the cache.
mkdir -p .heroku
# The Python installation.
cp -R "$CACHE_DIR/.heroku/python" .heroku/ &> /dev/null || true
# A plain text file which contains the current stack being used (used for cache busting).
cp -R "$CACHE_DIR/.heroku/python-stack" .heroku/ &> /dev/null || true
# A plain text file which contains the current python version being used (used for cache busting).
cp -R "$CACHE_DIR/.heroku/python-version" .heroku/ &> /dev/null || true
# A plain text file which contains the current sqlite3 version being used (used for cache busting).
cp -R "$CACHE_DIR/.heroku/python-sqlite3-version" .heroku/ &> /dev/null || true
# Any pre-compiled binaries, provided by the buildpack.
cp -R "$CACHE_DIR/.heroku/vendor" .heroku/ &> /dev/null || true
# "editable" installations of code repositories, via pip or pipenv.
if [[ -d "$CACHE_DIR/.heroku/src" ]]; then
cp -R "$CACHE_DIR/.heroku/src" .heroku/ &> /dev/null || true
fi
# The pre_compile hook. Customers rely on this. Don't remove it.
# This part of the code is used to allow users to customize their build experience
# without forking the buildpack by providing a `bin/pre_compile` script, which gets
# run inline with the buildpack automatically.
# Experimental pre_compile hook.
# shellcheck source=bin/steps/hooks/pre_compile
source "$BIN_DIR/steps/hooks/pre_compile"
# Sticky runtimes. If there was a previous build, and it used a given version of Python,
# continue to use that version of Python in perpituity (warnings will be raised if
# they are outofdate).
# Sticky runtimes.
if [ -f "$CACHE_DIR/.heroku/python-version" ]; then
DEFAULT_PYTHON_VERSION=$(cat "$CACHE_DIR/.heroku/python-version")
fi
# We didn't always record the stack version. This code is in place because of that.
# Stack fallback for non-declared caches.
if [ -f "$CACHE_DIR/.heroku/python-stack" ]; then
CACHED_PYTHON_STACK=$(cat "$CACHE_DIR/.heroku/python-stack")
else
@@ -220,26 +130,17 @@ fi
export CACHED_PYTHON_STACK
# Pipenv Python version support.
# Detect the version of Python requested from a Pipfile (e.g. python_version or python_full_version).
# Convert it to a runtime.txt file.
# shellcheck source=bin/steps/pipenv-python-version
source "$BIN_DIR/steps/pipenv-python-version"
# If no runtime was provided by the user, assume the default Python runtime version.
# If no runtime given, assume default version.
if [ ! -f runtime.txt ]; then
echo "$DEFAULT_PYTHON_VERSION" > runtime.txt
fi
# Create the directory for .profile.d, if it doesn't exist.
mkdir -p "$(dirname "$PROFILE_PATH")"
# Create the directory for editable source code installation, if it doesn't exist.
mkdir -p /app/.heroku/src
# On Heroku CI, builds happen in `/app`. Otherwise, on the Heroku platform,
# they occur in a temp directory. Beacuse Python is not portable, we must create
# symlinks to emulate that we are operating in `/app` during the build process.
# This is (hopefully obviously) because apps end up running from `/app` in production.
if [[ $BUILD_DIR != '/app' ]]; then
# python expects to reside in /app, so set up symlinks
# we will not remove these later so subsequent buildpacks can still invoke it
@@ -248,80 +149,63 @@ if [[ $BUILD_DIR != '/app' ]]; then
# Note: .heroku/src is copied in later.
fi
# Download / Install Python, from pre-build binaries available on Amazon S3.
# This step also bootstraps pip / setuptools.
(( start=$(nowms) ))
# Install Python.
let start=$(nowms)
# shellcheck source=bin/steps/python
source "$BIN_DIR/steps/python"
mtime "python.install.time" "${start}"
# Install Pipenv dependencies, if a Pipfile was provided.
# Pipenv support.
# shellcheck source=bin/steps/pipenv
source "$BIN_DIR/steps/pipenv"
# If no requirements.txt file given, assume `setup.py develop` is intended.
# This allows for people to ship a setup.py application to Heroku
if [ ! -f requirements.txt ] && [ ! -f Pipfile ]; then
echo "-e ." > requirements.txt
fi
# Fix egg-links.
# Because we're installing things into a different path than we're running them (temp dir vs app dir),
# We must re-write all of Python's eggpath links to target the proper directory.
# shellcheck source=bin/steps/eggpath-fix
source "$BIN_DIR/steps/eggpath-fix"
# Mercurial support.
# If a customer appears to be using mercurial for dependency resolution, we install it first.
# Note: this only applies to pip, not pipenv. This can likely be removed, over time. Measure it first.
# shellcheck source=bin/steps/mercurial
source "$BIN_DIR/steps/mercurial"
# Pylibmc support.
# On cedar-14, libmemcached was not available. The buildpack provides its own version, instead.
# shellcheck source=bin/steps/pylibmc
source "$BIN_DIR/steps/pylibmc"
# Support for Geo libraries. This is deprecated functionality, only functional on cedar-14.
# It is undocumented.
# Libffi support.
# shellcheck source=bin/steps/cryptography
source "$BIN_DIR/steps/cryptography"
# Support for Geo libraries.
# shellcheck source=bin/steps/geo-libs
sub_env "$BIN_DIR/steps/geo-libs"
sub-env "$BIN_DIR/steps/geo-libs"
# GDAL support.
# This is part of the Geo support.
# shellcheck source=bin/steps/gdal
source "$BIN_DIR/steps/gdal"
# SQLite3 support.
# This sets up and installs sqlite3 dev headers and the sqlite3 binary but not the
# libsqlite3-0 library since that exists on the stack image.
# Note: This only applies to Python 2.7.15+ and Python 3.6.6+
(( start=$(nowms) ))
# shellcheck source=bin/steps/sqlite3
source "$BIN_DIR/steps/sqlite3"
buildpack_sqlite3_install
mtime "sqlite3.install.time" "${start}"
# Uninstall removed dependencies with Pip.
let start=$(nowms)
# shellcheck source=bin/steps/pip-uninstall
source "$BIN_DIR/steps/pip-uninstall"
mtime "pip.uninstall.time" "${start}"
# pip install
# -----------
# Install dependencies with pip (where the magic happens).
(( start=$(nowms) ))
# Install dependencies with Pip (where the magic happens).
let start=$(nowms)
# shellcheck source=bin/steps/pip-install
source "$BIN_DIR/steps/pip-install"
mtime "pip.install.time" "${start}"
# Support for NLTK corpora.
# Note: this may only work on Python 2.7. I don't think many customers use this functionality,
# and it should probably be undocumented.
# (there's an import error on 3.6 that should hopefully be fixed upstream at some point)
(( start=$(nowms) ))
sub_env "$BIN_DIR/steps/nltk"
let start=$(nowms)
sub-env "$BIN_DIR/steps/nltk"
mtime "nltk.download.time" "${start}"
# Support for editable installations. Here, we are copying pipcreated src directory,
# and copying it into the proper place (the logical place to do this was early, but it must be done here).
# Support for pip install -e.
# In CI, $BUILD_DIR is /app.
if [[ ! "$BUILD_DIR" == "/app" ]]; then
rm -fr "$BUILD_DIR/.heroku/src"
@@ -330,48 +214,28 @@ fi
# Django collectstatic support.
# The buildpack automatically runs collectstatic for Django applications.
# This is the cause for the majority of build failures on the Python platform.
# These failures are intentional — if collectstatic (which can be tricky, at times) fails,
# your build fails.
(( start=$(nowms) ))
sub_env "$BIN_DIR/steps/collectstatic"
let start=$(nowms)
sub-env "$BIN_DIR/steps/collectstatic"
mtime "collectstatic.time" "${start}"
# Create .profile script for application runtime environment variables.
set-env PATH "\$HOME/.heroku/python/bin:\$PATH"
set-env PYTHONUNBUFFERED true
set-env PYTHONHOME /app/.heroku/python
# Progamatically create .profile.d script for application runtime environment variables.
set-env LIBRARY_PATH "/app/.heroku/vendor/lib:/app/.heroku/python/lib:\$LIBRARY_PATH"
set-env LD_LIBRARY_PATH "/app/.heroku/vendor/lib:/app/.heroku/python/lib:\$LD_LIBRARY_PATH"
# Set the PATH to include Python / pip / pipenv / etc.
set_env PATH "\$HOME/.heroku/python/bin:\$PATH"
# Tell Python to run in unbuffered mode.
set_env PYTHONUNBUFFERED true
# Tell Python where it lives.
set_env PYTHONHOME "\$HOME/.heroku/python"
# Set variables for C libraries.
set_env LIBRARY_PATH "\$HOME/.heroku/vendor/lib:\$HOME/.heroku/python/lib:\$LIBRARY_PATH"
set_env LD_LIBRARY_PATH "\$HOME/.heroku/vendor/lib:\$HOME/.heroku/python/lib:\$LD_LIBRARY_PATH"
# Locale.
set_default_env LANG en_US.UTF-8
# The Python hash seed is set to random.
set_default_env PYTHONHASHSEED random
# Tell Python to look for Python modules in the /app dir. Don't change this.
set_default_env PYTHONPATH "\$HOME"
# Python expects to be in /app, if at runtime, it is not, set
# up symlinks… this can occur when the subdir buildpack is used.
cat <<EOT >> "$PROFILE_PATH"
if [[ \$HOME != "/app" ]]; then
mkdir -p /app/.heroku
ln -nsf "\$HOME/.heroku/python" /app/.heroku/python
ln -nsf "\$HOME/.heroku/vendor" /app/.heroku/vendor
fi
EOT
set-default-env LANG en_US.UTF-8
set-default-env PYTHONHASHSEED random
set-default-env PYTHONPATH /app/
# Install sane-default script for $WEB_CONCURRENCY and $FORWARDED_ALLOW_IPS.
cp "$ROOT_DIR/vendor/WEB_CONCURRENCY.sh" "$WEB_CONCURRENCY_PROFILE_PATH"
cp "$ROOT_DIR/vendor/python.gunicorn.sh" "$GUNICORN_PROFILE_PATH"
# Experimental post_compile hook. Don't remove this.
# Experimental post_compile hook.
# shellcheck source=bin/steps/hooks/post_compile
source "$BIN_DIR/steps/hooks/post_compile"
@@ -379,7 +243,8 @@ source "$BIN_DIR/steps/hooks/post_compile"
# shellcheck source=bin/steps/eggpath-fix2
source "$BIN_DIR/steps/eggpath-fix2"
# Store new artifacts in the cache.
# Store new artifacts in cache.
rm -rf "$CACHE_DIR/.heroku/python"
rm -rf "$CACHE_DIR/.heroku/python-version"
rm -rf "$CACHE_DIR/.heroku/python-stack"
-21
View File
@@ -1,21 +0,0 @@
#!/usr/bin/env bash
DEFAULT_PYTHON_VERSION="python-3.6.10"
LATEST_38="python-3.8.2"
LATEST_37="python-3.7.6"
LATEST_36="python-3.6.10"
LATEST_35="python-3.5.7"
LATEST_34="python-3.4.10"
LATEST_27="python-2.7.17"
PYPY_36="pypy3.6-7.2.0"
PYPY_27="pypy2.7-7.2.0"
export DEFAULT_PYTHON_VERSION \
LATEST_38 \
LATEST_37 \
LATEST_36 \
LATEST_35 \
LATEST_34 \
LATEST_27 \
PYPY_36 \
PYPY_27
-28
View File
@@ -1,28 +0,0 @@
# Python Buildpack Install Steps
TODO: Add context on Python install steps, such as why symlinking vs copying
## Installing the Pip tool
The Python Buildpack uses a tool called `get-pip` to install the pip tool. This
is done in the `python` script.
This is in part because Python historically did not come with pip by default.
## Installing Python packages using Pip
### Convention: Use `python` process to invoke Pip
We don't use this convention (yet) but this is an upcoming change being considered.
This is a bigger concern on Windows than it is in Linux environments, but an
emerging convention in the Python community is to invoke pip using:
```
python3 -m pip [options]
```
Invoking pip this way ensures correct location - python knows where these
packages are stored because it put them there (defaults to Python's pathing info).
All normal command line options are available using this method.
+2 -24
View File
@@ -11,7 +11,7 @@
# - $DEBUG_COLLECTSTATIC: upon failure, print out environment variables.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
source $BIN_DIR/utils
# Location of 'manage.py', if it exists.
MANAGE_FILE=$(find . -maxdepth 3 -type f -name 'manage.py' -printf '%d\t%P\n' | sort -nk1 | cut -f2 | head -1)
@@ -30,13 +30,7 @@ if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALL
puts-step "$ python $MANAGE_FILE collectstatic --noinput"
# Run collectstatic, cleanup some of the noisy output.
PYTHONPATH=${PYTHONPATH:-.}
export PYTHONPATH
# Create a temporary file for collecting the collectstaic logs.
COLLECTSTATIC_LOG=$(mktemp)
python "$MANAGE_FILE" collectstatic --noinput --traceback 2>&1 | tee "$COLLECTSTATIC_LOG" | sed '/^Post-processed/d;/^Copying/d;/^$/d' | indent
python "$MANAGE_FILE" collectstatic --noinput --traceback 2>&1 | sed '/^Post-processed/d;/^Copying/d;/^$/d' | indent
COLLECTSTATIC_STATUS="${PIPESTATUS[0]}"
set -e
@@ -44,22 +38,6 @@ if [ ! "$DISABLE_COLLECTSTATIC" ] && [ -f "$MANAGE_FILE" ] && [ "$DJANGO_INSTALL
# Display a warning if collectstatic failed.
[ "$COLLECTSTATIC_STATUS" -ne 0 ] && {
if grep -q 'SyntaxError' "$COLLECTSTATIC_LOG"; then
mcount "failure.collectstatic.syntax-error"
elif grep -q 'ImproperlyConfigured' "$COLLECTSTATIC_LOG"; then
mcount "failure.collectstatic.improper-configuration"
elif grep -q 'The CSS file' "$COLLECTSTATIC_LOG"; then
mcount "failure.collectstatic.fancy-references"
elif grep -q 'OSError' "$COLLECTSTATIC_LOG"; then
mcount "failure.collectstatic.missing-file"
else
mcount "failure.collectstatic.other"
fi
echo
echo " ! Error while running '$ python $MANAGE_FILE collectstatic --noinput'."
echo " See traceback above for details."
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
# This script serves as the Cryptography build step of the
# [**Python Buildpack**](https://github.com/heroku/heroku-buildpack-python)
# compiler.
#
# A [buildpack](https://devcenter.heroku.com/articles/buildpacks) is an
# adapter between a Python application and Heroku's runtime.
#
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled libffi binary.
VENDORED_LIBFFI="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/libffi.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If a package using cffi exists within requirements, use vendored libffi.
if (pip-grep -s requirements.txt argon2-cffi bcrypt cffi cryptography django[argon2] Django[argon2] django[bcrypt] Django[bcrypt] PyNaCl pyOpenSSL PyOpenSSL requests[security] misaka &> /dev/null) then
if [ ! -d ".heroku/vendor/lib/libffi-3.1" ]; then
echo "-----> Noticed cffi. Bootstrapping libffi."
mkdir -p .heroku/vendor
# Download and extract libffi into target vendor directory.
curl "$VENDORED_LIBFFI" -s | tar zxv -C .heroku/vendor &> /dev/null
fi
LIBFFI=$(pwd)/vendor
export LIBFFI
fi
+2 -6
View File
@@ -10,7 +10,7 @@
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
VENDORED_GDAL="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/gdal.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
@@ -22,17 +22,13 @@ source "$BIN_DIR/utils"
if (pip-grep -s requirements.txt GDAL gdal pygdal &> /dev/null) then
if [ ! -f ".heroku/vendor/bin/gdalserver" ]; then
puts-warn "The vendored GDAL package in the Heroku Python Buildpack now deprecated."
puts-warn "To enable GDAL use an alternative buildpack is available here - https://github.com/heroku/heroku-geo-buildpack"
echo "-----> Noticed GDAL. Bootstrapping gdal."
mkdir -p .heroku/vendor
# Download and extract cryptography into target vendor directory.
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.gdal"
fi
GDAL=$(pwd)/vendor
export GDAL
fi
+4 -21
View File
@@ -10,9 +10,9 @@
# This script is invoked by [`bin/compile`](/).
# The location of the pre-compiled cryptography binary.
VENDORED_GDAL="${VENDOR_URL}/libraries/vendor/gdal.tar.gz"
VENDORED_GEOS="${VENDOR_URL}/libraries/vendor/geos.tar.gz"
VENDORED_PROJ="${VENDOR_URL}/libraries/vendor/proj.tar.gz"
VENDORED_GDAL="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/gdal.tar.gz"
VENDORED_GEOS="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/geos.tar.gz"
VENDORED_PROJ="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/proj.tar.gz"
PKG_CONFIG_PATH="/app/.heroku/vendor/lib/pkgconfig:$PKG_CONFIG_PATH"
@@ -22,10 +22,6 @@ source "$BIN_DIR/utils"
# If GDAL exists within requirements, use vendored gdal.
if [[ "$BUILD_WITH_GEO_LIBRARIES" ]]; then
mcount "buildvar.BUILD_WITH_GEO_LIBRARIES"
puts-warn "The GDAL, GEOS and PROJ binaries and BUILD_WITH_GEO_LIBRARIES functonality are now deprecated."
puts-warn "An alternative buildpack to enable GDAL, GEOS and PROJ use is available here - https://github.com/heroku/heroku-geo-buildpack"
if [ ! -f ".heroku/vendor/bin/proj" ]; then
echo "-----> Bootstrapping gdal, geos, proj."
@@ -34,22 +30,9 @@ if [[ "$BUILD_WITH_GEO_LIBRARIES" ]]; then
curl "$VENDORED_GDAL" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_GEOS" -s | tar zxv -C .heroku/vendor &> /dev/null
curl "$VENDORED_PROJ" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.geo_libs"
# Copy libjasper from build image to slug.
if [[ "$STACK" == "heroku-16" ]]; then
cp /usr/lib/x86_64-linux-gnu/libjasper.so* ".heroku/vendor/lib/."
mcount "steps.vendor.libjasper"
fi
fi
GDAL=$(pwd)/vendor
export GDAL
# set path for post_compile hooks
export GDAL_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgdal.so"
export GEOS_LIBRARY_PATH="$BUILD_DIR/.heroku/vendor/lib/libgeos_c.so"
# set path for runtime environmeht
set_env GDAL_LIBRARY_PATH "/app/.heroku/vendor/lib/libgdal.so"
set_env GEOS_LIBRARY_PATH "/app/.heroku/vendor/lib/libgeos_c.so"
fi
+1 -1
View File
@@ -3,5 +3,5 @@
if [ -f bin/post_compile ]; then
echo "-----> Running post-compile hook"
chmod +x bin/post_compile
sub_env bin/post_compile
sub-env bin/post_compile
fi
+1 -1
View File
@@ -3,5 +3,5 @@
if [ -f bin/pre_compile ]; then
echo "-----> Running pre-compile hook"
chmod +x bin/pre_compile
sub_env bin/pre_compile
sub-env bin/pre_compile
fi
+2 -5
View File
@@ -1,9 +1,6 @@
#!/usr/bin/env bash
# Install Mercurial if it appears to be required.
if [[ -f "requirements.txt" ]]; then
if (grep -Fiq "hg+" requirements.txt) then
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
mcount "steps.mercurial"
fi
if (grep -Fiq "hg+" requirements.txt) then
/app/.heroku/python/bin/pip install mercurial | cleanup | indent
fi
+6 -7
View File
@@ -15,22 +15,21 @@ source "$BIN_DIR/utils"
# Check that nltk was installed by pip, otherwise obviously not needed
if sp-grep -s nltk; then
puts-step "Downloading NLTK corpora"
puts-step "Downloading NLTK corpora..."
nltk_packages_definition="$BUILD_DIR/nltk.txt"
if [ -f "$nltk_packages_definition" ]; then
readarray -t nltk_packages < "$nltk_packages_definition"
puts-step "Downloading NLTK packages: ${nltk_packages[*]}"
nltk_packages=$(tr "\n" " " < "$nltk_packages_definition")
puts-step "Downloading NLTK packages: $nltk_packages"
python -m nltk.downloader -d "$BUILD_DIR/.heroku/python/nltk_data" "${nltk_packages[@]}" | indent
set_env NLTK_DATA "/app/.heroku/python/nltk_data"
python -m nltk.downloader -d "$BUILD_DIR/.heroku/python/nltk_data" "$nltk_packages" | indent
set-env NLTK_DATA "/app/.heroku/python/nltk_data"
mcount "buildvar.NLTK_PACKAGES_DEFINITION"
mcount "steps.nltk"
else
puts-warn "'nltk.txt' not found, not downloading any corpora"
puts-warn "Learn more: https://devcenter.heroku.com/articles/python-nltk"
fi
fi
+4 -51
View File
@@ -1,72 +1,25 @@
#!/usr/bin/env bash
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
if [ ! "$SKIP_PIP_INSTALL" ]; then
# Install dependencies with Pip.
puts-step "Installing requirements with pip"
# Set Pip env vars
# This reads certain environment variables set on the Heroku app config
# and makes them accessible to the pip install process.
#
# PIP_EXTRA_INDEX_URL allows for an alternate pypi URL to be used.
if [[ -r "$ENV_DIR/PIP_EXTRA_INDEX_URL" ]]; then
PIP_EXTRA_INDEX_URL="$(cat "$ENV_DIR/PIP_EXTRA_INDEX_URL")"
export PIP_EXTRA_INDEX_URL
mcount "buildvar.PIP_EXTRA_INDEX_URL"
fi
set +e
# Set SLUGIFY_USES_TEXT_UNIDECODE, required for Airflow versions >=1.10
if [[ -r "$ENV_DIR/SLUGIFY_USES_TEXT_UNIDECODE" ]]; then
SLUGIFY_USES_TEXT_UNIDECODE="$(cat "$ENV_DIR/SLUGIFY_USES_TEXT_UNIDECODE")"
export SLUGIFY_USES_TEXT_UNIDECODE
mcount "buildvar.SLUGIFY_USES_TEXT_UNIDECODE"
fi
set +e
# Measure that we're using pip.
mcount "tool.pip"
# Count expected build failures.
if grep -q '==0.0.0' requirements.txt; then
mcount "failure.none-version"
fi
if grep -qi '^django==1.*' requirements.txt; then
puts-warn "Your Django version is nearing the end of its community support."
puts-warn "Upgrade to continue to receive security updates and for the best experience with Django."
puts-warn "For more information, check out https://www.djangoproject.com/download/#supported-versions"
fi
if [ ! -f "$BUILD_DIR/.heroku/python/bin/pip" ]; then
exit 1
fi
/app/.heroku/python/bin/pip install -r "$BUILD_DIR/requirements.txt" --exists-action=w --src=/app/.heroku/src --disable-pip-version-check --no-cache-dir 2>&1 | tee "$WARNINGS_LOG" | cleanup | indent
sub-env /app/.heroku/python/bin/pip install -r "$BUILD_DIR/requirements.txt" --exists-action=w --src=/app/.heroku/src --disable-pip-version-check --no-cache-dir 2>&1 | tee "$WARNINGS_LOG" | cleanup | indent
PIP_STATUS="${PIPESTATUS[0]}"
set -e
show-warnings
if [[ ! $PIP_STATUS -eq 0 ]]; then
mcount "failure.pip-install"
exit 1
fi
# Smart Requirements handling
cp requirements.txt .heroku/python/requirements-declared.txt
/app/.heroku/python/bin/pip freeze --disable-pip-version-check > .heroku/python/requirements-installed.txt
# Install test dependencies, for CI.
if [ "$INSTALL_TEST" ]; then
if [[ -f "$1/requirements-test.txt" ]]; then
puts-step "Installing test dependencies…"
/app/.heroku/python/bin/pip install -r "$1/requirements-test.txt" --exists-action=w --src=./.heroku/src --disable-pip-version-check --no-cache-dir 2>&1 | cleanup | indent
fi
fi
echo
fi
+8 -16
View File
@@ -2,27 +2,19 @@
set +e
# Install dependencies with Pip.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
if [ ! "$SKIP_PIP_INSTALL" ]; then
if [[ -f .heroku/python/requirements-declared.txt ]]; then
if [[ -f .heroku/python/requirements-declared.txt ]]; then
cp .heroku/python/requirements-declared.txt requirements-declared.txt
cp .heroku/python/requirements-declared.txt requirements-declared.txt
pip-diff --stale requirements-declared.txt requirements.txt --exclude setuptools pip wheel > .heroku/python/requirements-stale.txt
rm -fr requirements-declared.txt
if ! pip-diff --stale requirements-declared.txt requirements.txt --exclude setuptools pip wheel > .heroku/python/requirements-stale.txt; then
mcount "failure.bad-requirements"
fi
rm -fr requirements-declared.txt
if [[ -s .heroku/python/requirements-stale.txt ]]; then
puts-step "Uninstalling stale dependencies"
/app/.heroku/python/bin/pip uninstall -r .heroku/python/requirements-stale.txt -y --exists-action=w --disable-pip-version-check | cleanup | indent
fi
if [[ -s .heroku/python/requirements-stale.txt ]]; then
puts-step "Uninstalling stale dependencies"
/app/.heroku/python/bin/pip uninstall -r .heroku/python/requirements-stale.txt -y --exists-action=w | cleanup | indent
fi
fi
set -e
Executable → Regular
+10 -75
View File
@@ -1,86 +1,21 @@
#!/usr/bin/env bash
# export CLINT_FORCE_COLOR=1
# export PIPENV_FORCE_COLOR=1
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
set -e
# Pipenv support (Generate requriements.txt with pipenv).
if [[ -f Pipfile ]]; then
if [[ ! -f requirements.txt ]]; then
puts-step "Installing requirements with latest pipenv..."
if [[ -f Pipfile.lock ]]; then
if [[ -f .heroku/python/Pipfile.lock.sha256 ]]; then
if [[ $(openssl dgst -sha256 Pipfile.lock) == $(cat .heroku/python/Pipfile.lock.sha256) ]]; then
# Measure that we're using Pipenv.
mcount "tool.pipenv"
# Install pipenv.
/app/.heroku/python/bin/pip install pipenv --upgrade &> /dev/null
# Don't skip installation of there are git deps.
if ! grep -q 'git' Pipfile.lock; then
echo "Skipping installation, as Pipfile.lock hasn't changed since last deploy." | indent
mcount "tool.pipenv"
export SKIP_PIPENV_INSTALL=1
export SKIP_PIP_INSTALL=1
fi
fi
fi
fi
if [ ! "$SKIP_PIPENV_INSTALL" ]; then
# Pipenv support (Generate requirements.txt with pipenv).
if [[ -f Pipfile ]]; then
# Measure that we're using Pipenv.
mcount "tool.pipenv"
# Install the dependencies.
/app/.heroku/python/bin/pipenv install --system 2>&1 | indent
# Skip pip install, later.
export SKIP_PIP_INSTALL=1
# Set Pip env vars
# This reads certain environment variables set on the Heroku app config
# and makes them accessible to the pip install process.
#
# PIP_EXTRA_INDEX_URL allows for an alternate pypi URL to be used.
if [[ -r "$ENV_DIR/PIP_EXTRA_INDEX_URL" ]]; then
PIP_EXTRA_INDEX_URL="$(cat "$ENV_DIR/PIP_EXTRA_INDEX_URL")"
export PIP_EXTRA_INDEX_URL
mcount "buildvar.PIP_EXTRA_INDEX_URL"
fi
# Pip freeze, for compatibility.
/app/.heroku/python/bin/pip freeze > requirements.txt
# Set SLUGIFY_USES_TEXT_UNIDECODE, required for Airflow versions >=1.10
if [[ -r "$ENV_DIR/SLUGIFY_USES_TEXT_UNIDECODE" ]]; then
SLUGIFY_USES_TEXT_UNIDECODE="$(cat "$ENV_DIR/SLUGIFY_USES_TEXT_UNIDECODE")"
export SLUGIFY_USES_TEXT_UNIDECODE
mcount "buildvar.SLUGIFY_USES_TEXT_UNIDECODE"
fi
export PIPENV_VERSION="2018.5.18"
# Install pipenv.
# Due to weird old pip behavior and pipenv behavior, pipenv upgrades pip
# to latest if only --upgrade is specified. Specify upgrade strategy to
# avoid this eager behavior.
/app/.heroku/python/bin/pip install pipenv==$PIPENV_VERSION --upgrade --upgrade-strategy only-if-needed &> /dev/null
# Install the test dependencies, for CI.
if [ "$INSTALL_TEST" ]; then
puts-step "Installing test dependencies…"
/app/.heroku/python/bin/pipenv install --dev --system --deploy 2>&1 | cleanup | indent
# Install the dependencies.
elif [[ ! -f Pipfile.lock ]]; then
puts-step "Installing dependencies with Pipenv $PIPENV_VERSION…"
/app/.heroku/python/bin/pipenv install --system --skip-lock 2>&1 | indent
else
pipenv-to-pip Pipfile.lock > requirements.txt
"$BIN_DIR/steps/pip-uninstall"
cp requirements.txt .heroku/python/requirements-declared.txt
openssl dgst -sha256 Pipfile.lock > .heroku/python/Pipfile.lock.sha256
puts-step "Installing dependencies with Pipenv $PIPENV_VERSION…"
/app/.heroku/python/bin/pipenv install --system --deploy 2>&1 | indent
fi
fi
else
export SKIP_PIP_INSTALL=1
pipenv-to-pip Pipfile.lock > requirements.txt
fi
+9 -19
View File
@@ -2,34 +2,24 @@
# Detect Python-version with Pipenv.
if [[ -f $BUILD_DIR/Pipfile ]]; then
if [[ -f $BUILD_DIR/Pipfile.lock ]]; then
if [[ ! -f $BUILD_DIR/runtime.txt ]]; then
if [[ ! -f $BUILD_DIR/Pipfile.lock ]]; then
puts-warn "No 'Pipfile.lock' found! We recommend you commit this into your repository."
puts-warn "No 'pipfile.lock' found! We recommend you commit this into your repository."
fi
if [[ -f $BUILD_DIR/Pipfile.lock ]]; then
set +e
PYTHON=$(jq -r '._meta.requires.python_full_version' "$BUILD_DIR/Pipfile.lock")
if [[ "$PYTHON" != "null" ]]; then
echo "python-$PYTHON" > "$BUILD_DIR/runtime.txt"
fi
PYTHON=$(jq -r '._meta.requires.python_version' "$BUILD_DIR/Pipfile.lock")
set -e
if [[ "$PYTHON" == "null" ]]; then
PYTHON=$(jq -r '._meta.requires.python_version' "$BUILD_DIR/Pipfile.lock")
if [ "$PYTHON" = 2.7 ]; then
echo "$LATEST_27" > "$BUILD_DIR/runtime.txt"
fi
if [ "$PYTHON" = 3.6 ]; then
echo "$LATEST_36" > "$BUILD_DIR/runtime.txt"
fi
if [ "$PYTHON" = 3.7 ]; then
echo "$LATEST_37" > "$BUILD_DIR/runtime.txt"
fi
if [ "$PYTHON" = 2.7 ]; then
echo "python-2.7.13" > "$BUILD_DIR/runtime.txt"
fi
if [ "$PYTHON" = 3.6 ]; then
echo "python-3.6.2" > "$BUILD_DIR/runtime.txt"
fi
fi
fi
fi
+2 -7
View File
@@ -9,18 +9,14 @@
#
# This script is invoked by [`bin/compile`](/).
if [[ "$STACK" != "cedar-14" ]]; then
# libmemcached is pre-installed in the stack image so there is no need to vendor it.
return 0
fi
# The location of the pre-compiled libmemcached binary.
VENDORED_MEMCACHED="${VENDOR_URL}/libraries/vendor/libmemcache.tar.gz"
VENDORED_MEMCACHED="https://lang-python.s3.amazonaws.com/$STACK/libraries/vendor/libmemcache.tar.gz"
# Syntax sugar.
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# If pylibmc exists within requirements, use vendored libmemcached.
if (pip-grep -s requirements.txt pylibmc &> /dev/null) then
@@ -29,7 +25,6 @@ if (pip-grep -s requirements.txt pylibmc &> /dev/null) then
mkdir -p .heroku/vendor
# Download and extract libmemcached into target vendor directory.
curl "$VENDORED_MEMCACHED" -s | tar zxv -C .heroku/vendor &> /dev/null
mcount "steps.vendor.pylibmc"
fi
LIBMEMCACHED=$(pwd)/vendor
+7 -114
View File
@@ -4,86 +4,9 @@ set +e
runtime-fixer runtime.txt
PYTHON_VERSION=$(cat runtime.txt)
# The location of the pre-compiled python binary.
VENDORED_PYTHON="${VENDOR_URL}/runtimes/$PYTHON_VERSION.tar.gz"
SECURITY_UPDATE="Python has released a security update! Please consider upgrading to"
ONLY_SUPPORTED_2_VERSION="Only the latest version of Python 2 is supported on the platform. Please consider upgrading to"
PYTHON_2_EOL_UPDATE="Python 2 has reached it's community EOL. Upgrade your Python runtime to maintain a secure application as soon as possible."
# check if runtime exists
if curl --output /dev/null --silent --head --fail "$VENDORED_PYTHON"; then
if [[ "$PYTHON_VERSION" == $PY38* ]]; then
# do things to alert the user of security release available
if [ "$PYTHON_VERSION" != "$LATEST_38" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_38"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY37* ]]; then
# do things to alert the user of security release available
if [ "$PYTHON_VERSION" != "$LATEST_37" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_37"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY36* ]]; then
# security update note
if [ "$PYTHON_VERSION" != "$LATEST_36" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_36"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY35* ]]; then
# security update note
if [ "$PYTHON_VERSION" != "$LATEST_35" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_35"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY34* ]]; then
# security update note
if [ "$PYTHON_VERSION" != "$LATEST_34" ]; then
puts-warn "$SECURITY_UPDATE" "$LATEST_34"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PY27* ]]; then
# security update note
if [[ "$(date "+%Y")" -gt "2019" ]]; then
puts-warn "$PYTHON_2_EOL_UPDATE"
echo " Learn More: https://devcenter.heroku.com/articles/python-2-7-eol-faq"
fi
if [ "$PYTHON_VERSION" != "$LATEST_27" ]; then
puts-warn "$ONLY_SUPPORTED_2_VERSION" "$LATEST_27"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PYPY27* ]]; then
# security update note
if [ "$PYTHON_VERSION" != "$PYPY_27" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_27}?"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
if [[ "$PYTHON_VERSION" == $PYPY36* ]]; then
# security update note
if [ "$PYTHON_VERSION" != "$PYPY_36" ]; then
puts-warn "Could not find that Pypy version. Did you mean" "${PYPY_36}?"
echo " Learn More: https://devcenter.heroku.com/articles/python-runtimes"
fi
fi
else
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
exit 1
fi
if [[ "$STACK" != "$CACHED_PYTHON_STACK" ]]; then
puts-step "Stack has changed from $CACHED_PYTHON_STACK to $STACK, clearing cache"
rm -fr .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor .heroku/python .heroku/python-sqlite3-version
rm -fr .heroku/python-stack .heroku/python-version .heroku/python .heroku/vendor
fi
if [ -f .heroku/python-version ]; then
@@ -95,25 +18,6 @@ if [ -f .heroku/python-version ]; then
fi
fi
# Check if we should reinstall python dependencies
if [[ ! -f "$CACHE_DIR/.heroku/requirements.txt" ]]; then
# IF there's no cached dependencies, update cached version of requirements.txt
# This should only run for new apps and first deploys after this update
cp -R "$BUILD_DIR/requirements.txt" "$CACHE_DIR/.heroku/requirements.txt"
else
# IF there IS a cached directory, check for differences with the new one
if ! diff "$BUILD_DIR/requirements.txt" "$CACHE_DIR/.heroku/requirements.txt" &> /dev/null; then
puts-step "Requirements file has been changed, clearing cached dependencies"
# if there are any differences, clear the Python cache
# Installing Python over again does not take noticably more time
cp -R "$BUILD_DIR/requirements.txt" "$CACHE_DIR/.heroku/requirements.txt"
rm -rf .heroku/python
unset SKIP_INSTALL
else
puts-step "No change in requirements detected, installing from cache"
fi
fi
if [ ! "$SKIP_INSTALL" ]; then
puts-step "Installing $PYTHON_VERSION"
@@ -122,7 +26,7 @@ if [ ! "$SKIP_INSTALL" ]; then
mcount "version.python.$PYTHON_VERSION"
if ! curl "${VENDORED_PYTHON}" -s | tar zxv -C .heroku/python &> /dev/null; then
if ! curl "https://lang-python.s3.amazonaws.com/$STACK/runtimes/$PYTHON_VERSION.tar.gz" -s | tar zxv -C .heroku/python &> /dev/null; then
puts-warn "Requested runtime ($PYTHON_VERSION) is not available for this stack ($STACK)."
puts-warn "Aborting. More info: https://devcenter.heroku.com/articles/python-support"
exit 1
@@ -136,28 +40,17 @@ if [ ! "$SKIP_INSTALL" ]; then
hash -r
fi
# Heroku uses the get-pip utility maintained by the Python community to vendor Pip.
# https://github.com/pypa/get-pip
GETPIP_URL="https://lang-python.s3.amazonaws.com/etc/get-pip.py"
GETPIP_PY="${TMPDIR:-/tmp}/get-pip.py"
if ! curl -s "${GETPIP_URL}" -o "$GETPIP_PY" &> /dev/null; then
mcount "failure.python.get-pip"
echo "Failed to pull down get-pip"
exit 1
fi
# If a new Python has been installed or Pip isn't up to date:
# If Pip isn't up to date:
if [ "$FRESH_PYTHON" ] || [[ ! $(pip --version) == *$PIP_UPDATE* ]]; then
puts-step "Installing pip"
# Remove old installations.
rm -fr /app/.heroku/python/lib/python*/site-packages/pip-*
rm -fr /app/.heroku/python/lib/python*/site-packages/setuptools-*
rm -fr /app/.heroku/python/lib/python2.7/site-packages/pip-*
rm -fr /app/.heroku/python/lib/python2.7/site-packages/setuptools-*
/app/.heroku/python/bin/python "$ROOT_DIR/vendor/get-pip.py" &> /dev/null
/app/.heroku/python/bin/python "$GETPIP_PY" pip=="$PIP_UPDATE" &> /dev/null
/app/.heroku/python/bin/pip install "$ROOT_DIR/vendor/setuptools-39.0.1-py2.py3-none-any.whl" &> /dev/null
fi
set -e
-77
View File
@@ -1,77 +0,0 @@
#!/usr/bin/env bash
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
sqlite3_install() {
HEROKU_PYTHON_DIR="$1"
HEADERS_ONLY="$3"
mkdir -p "$HEROKU_PYTHON_DIR"
APT_CACHE_DIR="$HEROKU_PYTHON_DIR/apt/cache"
APT_STATE_DIR="$HEROKU_PYTHON_DIR/apt/state"
mkdir -p "$APT_CACHE_DIR/archives/partial"
mkdir -p "$APT_STATE_DIR/lists/partial"
APT_OPTIONS="-o debug::nolocking=true"
APT_OPTIONS="$APT_OPTIONS -o dir::cache=$APT_CACHE_DIR"
APT_OPTIONS="$APT_OPTIONS -o dir::state=$APT_STATE_DIR"
APT_OPTIONS="$APT_OPTIONS -o dir::etc::sourcelist=/etc/apt/sources.list"
apt-get $APT_OPTIONS update > /dev/null 2>&1
if [ -z "$HEADERS_ONLY" ]; then
apt-get $APT_OPTIONS -y -d --reinstall install libsqlite3-dev sqlite3 > /dev/null 2>&1
else
apt-get $APT_OPTIONS -y -d --reinstall install libsqlite3-dev
fi
find "$APT_CACHE_DIR/archives/" -name "*.deb" -exec dpkg -x {} "$HEROKU_PYTHON_DIR/sqlite3/" \;
mkdir -p "$HEROKU_PYTHON_DIR/include"
mkdir -p "$HEROKU_PYTHON_DIR/lib"
# remove old sqlite3 libraries/binaries
find "$HEROKU_PYTHON_DIR/include/" -name "sqlite3*.h" -exec rm -f {} \;
find "$HEROKU_PYTHON_DIR/lib/" -name "libsqlite3.*" -exec rm -f {} \;
rm -f "$HEROKU_PYTHON_DIR/lib/pkgconfig/sqlite3.pc"
rm -f "$HEROKU_PYTHON_DIR/bin/sqlite3"
# copy over sqlite3 headers & bins and setup linking against the stack image library
mv "$HEROKU_PYTHON_DIR/sqlite3/usr/include/"* "$HEROKU_PYTHON_DIR/include/"
mv "$HEROKU_PYTHON_DIR/sqlite3/usr/lib/x86_64-linux-gnu"/libsqlite3.*a "$HEROKU_PYTHON_DIR/lib/"
mkdir -p "$HEROKU_PYTHON_DIR/lib/pkgconfig"
# set the right prefix/lib directories
sed -e 's/prefix=\/usr/prefix=\/app\/.heroku\/python/' -e 's/\/x86_64-linux-gnu//' "$HEROKU_PYTHON_DIR/sqlite3/usr/lib/x86_64-linux-gnu/pkgconfig/sqlite3.pc" > "$HEROKU_PYTHON_DIR/lib/pkgconfig/sqlite3.pc"
# need to point the libsqlite3.so to the stack image library for /usr/bin/ld -lsqlite3
SQLITE3_LIBFILE="/usr/lib/x86_64-linux-gnu/$(readlink -n "$HEROKU_PYTHON_DIR/sqlite3/usr/lib/x86_64-linux-gnu/libsqlite3.so")"
ln -s "$SQLITE3_LIBFILE" "$HEROKU_PYTHON_DIR/lib/libsqlite3.so"
if [ -z "$HEADERS_ONLY" ]; then
mv "$HEROKU_PYTHON_DIR/sqlite3/usr/bin"/* "$HEROKU_PYTHON_DIR/bin/"
fi
# cleanup
rm -rf "$HEROKU_PYTHON_DIR/sqlite3/"
rm -rf "$HEROKU_PYTHON_DIR/apt/"
}
buildpack_sqlite3_install() {
HEROKU_PYTHON_DIR="$BUILD_DIR/.heroku/python"
SQLITE3_VERSION_FILE="$BUILD_DIR/.heroku/python-sqlite3-version"
if [ -f "$SQLITE3_VERSION_FILE" ]; then
INSTALLED_SQLITE3_VERSION=$(cat "$SQLITE3_VERSION_FILE")
fi
puts-step "Installing SQLite3"
if sqlite3_install "$BUILD_DIR/.heroku/python" ; then
mcount "success.python.sqlite3"
else
echo "Sqlite3 failed to install."
mcount "failure.python.sqlite3"
fi
mkdir -p "$CACHE_DIR/.heroku/"
}
+4 -4
View File
@@ -6,8 +6,8 @@ BIN_DIR=$(cd "$(dirname "$0")" || return; pwd) # absolute path
# shellcheck source=bin/utils
source "$BIN_DIR/utils"
# Locale support for Pipenv.
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
DISABLE_COLLECTSTATIC=1 "$(dirname "${0:-}")/compile" "$1" "$2" "$3"
DISABLE_COLLECTSTATIC=1 INSTALL_TEST=1 "$(dirname "${0:-}")/compile" "$1" "$2" "$3"
if [[ -f "$1/requirements-test.txt" ]]; then
/app/.heroku/python/bin/pip install -r "$1/requirements-test.txt" --exists-action=w --src=./.heroku/src --disable-pip-version-check --no-cache-dir 2>&1 | cleanup | indent
fi
+25 -38
View File
@@ -4,7 +4,7 @@ shopt -s nullglob
# The standard library.
if [[ ! -f /tmp/stdlib.sh ]]; then
curl --retry 3 -s https://lang-common.s3.amazonaws.com/buildpack-stdlib/v8/stdlib.sh > /tmp/stdlib.sh
curl --retry 3 -s https://lang-common.s3.amazonaws.com/buildpack-stdlib/v2/stdlib.sh > /tmp/stdlib.sh
fi
# shellcheck source=/dev/null
source /tmp/stdlib.sh
@@ -53,45 +53,32 @@ deep-cp() {
}
sub-env() {
WHITELIST=${2:-''}
BLACKLIST=${3:-'^(GIT_DIR|STACK|PYTHONHOME|LD_LIBRARY_PATH|LIBRARY_PATH|PATH)$'}
# Python-specific variables.
export PYHONHOME=$BUILD_DIR/.heroku/python
export PYTHONPATH=$BUILD_DIR/
(
if [ -d "$ENV_DIR" ]; then
# shellcheck disable=SC2045
for e in $(ls "$ENV_DIR"); do
echo "$e" | grep -E "$WHITELIST" | grep -qvE "$BLACKLIST" &&
export "$e=$(cat "$ENV_DIR/$e")"
:
done
fi
"$@"
)
}
# Measure the size of the Python installation.
measure-size() {
echo "$(du -s .heroku/python 2>/dev/null || echo 0) | awk '{print $1}')"
}
# Python version operator >
version_gt() {
test "$(printf '%s\n' "$@" | sort -V | head -n 1)" != "$1";
}
# Python verison operator >=
version_gte() {
if [ "$1" == "$2" ]; then
return 0
fi
version_gt "$1" "$2"
}
# Check if Python 2
python2_check() {
VERSION="$1"
version_gte "$VERSION" "python-2.7.0" && version_gt "python-3.0.0" "$VERSION"
}
# Check if Python 3
python3_check() {
VERSION="$1"
version_gte "$VERSION" "python-3.0.0" && version_gt "python-4.0.0" "$VERSION"
}
# Check if Python version needs to install SQLite3
python_sqlite3_check() {
VERSION="$1"
MIN_PYTHON_3="python-3.5.6"
MIN_PYTHON_2="python-2.7.15"
( python2_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_2" ) \
|| ( python3_check "$VERSION" && version_gte "$VERSION" "$MIN_PYTHON_3" )
}
-13
View File
@@ -1,13 +0,0 @@
[buildpack]
name = "Python"
[publish.Ignore]
files = [
"test/",
".gitignore",
".dockerignore",
".github/",
"Dockerfile",
"Pipfile",
"Pipfile.lock"
]
+26 -42
View File
@@ -1,57 +1,41 @@
# Python Buildpack Binaries
## Building the Docker Images
For Cedar-14 stack
------------------
**After every change to your formulae, perform the following** from the root of the Git repository (not from `builds/`) to rebuild the images for each stack:
To get started with it, create an app on Heroku inside a clone of this repository, and set your S3 config vars:
$ docker build --pull --tag heroku-python-build-cedar-14 --file $(pwd)/builds/cedar-14.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-16 --file $(pwd)/builds/heroku-16.Dockerfile .
$ docker build --pull --tag heroku-python-build-heroku-18 --file $(pwd)/builds/heroku-18.Dockerfile .
$ heroku create --buildpack https://github.com/heroku/heroku-buildpack-python#not-heroku
$ heroku config:set WORKSPACE_DIR=builds
$ heroku config:set AWS_ACCESS_KEY_ID=<your_aws_key>
$ heroku config:set AWS_SECRET_ACCESS_KEY=<your_aws_secret>
$ heroku config:set S3_BUCKET=<your_s3_bucket_name>
## Using the Image
You can e.g. `bash` into each of the images you built using their tag:
Then, shell into an instance and run a build by giving the name of the formula inside `builds`:
docker run --rm -ti heroku-python-build-cedar-14 bash
docker run --rm -ti heroku-python-build-heroku-16 bash
docker run --rm -ti heroku-python-build-heroku-18 bash
$ heroku run bash
Running `bash` attached to terminal... up, run.6880
~ $ bob build runtimes/python-2.7.6
You then have a shell where you can run `bob build`, `bob deploy`, and so forth. You can of course also invoke these programs directly with `docker run`:
Fetching dependencies... found 2:
- libraries/sqlite
docker run --rm -ti heroku-python-build-heroku-18 bob build runtimes/python-2.7.15
Building formula runtimes/python-2.7.6:
=== Building Python 2.7.6
Fetching Python v2.7.6 source...
Compiling...
In order to `bob deploy`, AWS credentials must be set up, as well as name and prefix of your custom S3 bucket (unless you're deploying to the Heroku production buckets that are pre-defined in each `Dockerfile`); see next section for details.
If this works, run `bob deploy` instead of `bob build` to have the result uploaded to S3 for you.
## Configuration
To speed things up drastically, it'll usually be a good idea to `heroku run bash --size PX` instead.
File `dockerenv.default` contains a list of required env vars; most of these have default values defined in `Dockerfile`. You can copy this file to a location outside the buildpack and modify it with the values you desire and pass its location with `--env-file`, or pass the env vars to `docker run` using `--env`.
For Heroku-16 stack
-------------------
Out of the box, each `Dockerfile` has the correct values predefined for `S3_BUCKET`, `S3_PREFIX`, and `S3_REGION`. If you're building your own packages, you'll likely want to change `S3_BUCKET` and `S3_PREFIX` to match your info. Instead of setting `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` into that file, you may also pass them to `docker run` through the environment, or explicitly using `--env`, in order to prevent accidental commits of credentials.
1. Ensure GNU Make and Docker are installed.
2. From the root of the buildpack repository, run: `make buildenv-heroku-16`
3. Follow the instructions displayed!
### Passing AWS credentials to the container
If you want to deploy packages and thus need to pass `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`, you can either pass them explicitly, through your environment, or through an env file.
#### Passing credentials explicitly
docker run --rm -ti -e AWS_ACCESS_KEY_ID=... -e AWS_SECRET_ACCESS_KEY=... heroku-python-build-heroku-18 bash
#### Passing credentials through the environment
The two environment variables `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY` are defined in `builds/dockerenv.default`, without values. This will cause Docker to "forward" values for these variables from the current environment, so you can pass them in:
AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash
or
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
docker run --rm -ti --env-file=builds/dockerenv.default heroku-python-build-heroku-18 bash
#### Passing credentials through a separate env file
This method is the easiest for users who want to build packages in their own S3 bucket, as they will have to adjust the `S3_BUCKET` and `S3_PREFIX` environment variable values anyway from their default values.
For this method, it is important to keep the credentials file in a location outside the buildpack, so that your credentials aren't accidentally committed. Copy `builds/dockerenv.default` **to a safe location outside the buildpack directory**, and insert your values for `AWS_ACCESS_KEY_ID`and `AWS_SECRET_ACCESS_KEY`.
docker run --rm -ti --env-file=../SOMEPATHOUTSIDE/s3.env heroku-python-build-heroku-18 bash
Enjoy :)
-15
View File
@@ -1,15 +0,0 @@
FROM heroku/cedar:14
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \
S3_PREFIX="cedar-14/" \
DEBIAN_FRONTEND=noninteractive \
STACK="cedar-14"
RUN apt-get update && apt-get install -y python-pip libsqlite3-dev realpath && rm -rf /var/lib/apt/lists/*
COPY requirements.txt /app/
RUN pip install -r /app/requirements.txt
COPY . /app
-5
View File
@@ -1,5 +0,0 @@
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
S3_BUCKET
S3_PREFIX
S3_REGION
-15
View File
@@ -1,15 +0,0 @@
FROM heroku/heroku:18-build.v16
WORKDIR /app
ENV WORKSPACE_DIR="/app/builds" \
S3_BUCKET="lang-python" \
S3_PREFIX="heroku-18/" \
DEBIAN_FRONTEND=noninteractive \
STACK="heroku-18"
RUN apt-get update && apt-get install --no-install-recommends -y python-pip-whl=9.0.1-2 python-pip=9.0.1-2 python-setuptools python-wheel libsqlite3-dev && rm -rf /var/lib/apt/lists/*
COPY requirements.txt /app/
RUN pip install --disable-pip-version-check --no-cache-dir -r /app/requirements.txt
COPY . /app
+1 -1
View File
@@ -3,7 +3,7 @@
OUT_PREFIX=$1
echo "Building SQLite"
echo "Building SQLite..."
SOURCE_TARBALL='https://www.sqlite.org/sqlite-autoconf-3070900.tar.gz'
+2 -2
View File
@@ -8,9 +8,9 @@ export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal"
echo "Building gdal..."
VERSION="2.2.1"
VERSION="1.11.5"
SOURCE_TARBALL="http://download.osgeo.org/gdal/${VERSION}/gdal-${VERSION}.tar.gz"
curl -L $SOURCE_TARBALL | tar zx
+4 -6
View File
@@ -8,18 +8,16 @@ export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building geos"
echo "Building geos..."
VERSION=3.6.2
SOURCE_TARBALL="http://download.osgeo.org/geos/geos-${VERSION}.tar.bz2"
SOURCE_TARBALL='http://download.osgeo.org/geos/geos-3.4.3.tar.bz2'
curl -L $SOURCE_TARBALL | tar xj
pushd "geos-${VERSION}"
cd geos-3.4.3
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
cd ..
+3 -3
View File
@@ -13,13 +13,13 @@ export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building libffi"
echo "Building libffi..."
SOURCE_TARBALL='https://github.com/libffi/libffi/archive/v3.1.tar.gz'
SOURCE_TARBALL='ftp://sourceware.org/pub/libffi/libffi-3.1.tar.gz'
curl -L $SOURCE_TARBALL | tar x
cd v3.1
cd libffi-3.1
./configure --prefix=$OUT_PREFIX --disable-static &&
make
make install
+2 -2
View File
@@ -20,7 +20,7 @@ dep_archive_name=${dep_dirname}.tar.gz
dep_url=https://launchpad.net/libmemcached/1.0/${dep_version}/+download/${dep_archive_name}
# SASL Support.
echo "-----> Building cyrus-sasl 2.1.26"
echo "-----> Building cyrus-sasl 2.1.26..."
curl -LO ftp://ftp.cyrusimap.org/cyrus-sasl/cyrus-sasl-2.1.26.tar.gz
# FTP doesn't play well with piping into tar xz
@@ -33,7 +33,7 @@ make -s -j 9
make install -s
popd
echo "-----> Building libmemcached ${dep_version}"
echo "-----> Building libmemcached ${dep_version}..."
curl -L ${dep_url} | tar xz
pushd ${dep_dirname}
+5 -6
View File
@@ -8,17 +8,16 @@ export PATH="/app/.heroku/python/bin/:$PATH"
hash -r
echo "Building gdal"
echo "Building gdal..."
VERSION=4.9.3
SOURCE_TARBALL="http://download.osgeo.org/proj/proj-${VERSION}.tar.gz"
SOURCE_TARBALL='http://download.osgeo.org/proj/proj-4.8.0.tar.gz'
curl -L $SOURCE_TARBALL | tar zx
pushd "proj-${VERSION}"
./configure --prefix=$OUT_PREFIX --enable-static=no
cd proj-4.8.0
./configure --prefix=$OUT_PREFIX --enable-static=no &&
make
make install
# Cleanup
popd
cd ..
-22
View File
@@ -1,22 +0,0 @@
#!/usr/bin/env bash
# fail hard
set -o pipefail
# fail harder
set -eu
OUT_PREFIX=$1
dep_formula=${0#$WORKSPACE_DIR/} # this is the original script, e.g. pypy-5.3.1
dep_name=$(basename $BASH_SOURCE) # this is us
dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full version name
dep_package=${dep_name}2-v${dep_version} # it's always "pypy2-…"
dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name}
echo "Building PyPy…"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python"
+10 -1
View File
@@ -2,4 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy
# NOTICE: This formula only works for the cedar-14 stack, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.3.1-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy2-v5.3.1-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy $OUT_PREFIX/bin/python
+10 -1
View File
@@ -2,4 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy
# NOTICE: This formula only works for the cedar-14 stack, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.6.0-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy2-v5.6.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy $OUT_PREFIX/bin/python
+10 -1
View File
@@ -2,4 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy
# NOTICE: This formula only works for the cedar-14 and heroku-16 stacks, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.7.0-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy2-v5.7.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy $OUT_PREFIX/bin/python
+10 -1
View File
@@ -2,4 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy
# NOTICE: This formula only works for the cedar-14 stack, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.7.1-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy2-v5.7.1-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy $OUT_PREFIX/bin/python
+8 -1
View File
@@ -2,4 +2,11 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.8.0-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy2-v5.8.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy $OUT_PREFIX/bin/python
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env bash
# fail hard
set -o pipefail
# fail harder
set -eu
OUT_PREFIX=$1
dep_formula=${0#$WORKSPACE_DIR/} # this is the original script, e.g. pypy-5.3.1
dep_name=$(basename $BASH_SOURCE) # this is us
dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full version name
dep_package=${dep_name}-v${dep_version} # it's always "pypy2-…"
dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name}
echo "Building PyPy…"
echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy" "$OUT_PREFIX/bin/python"
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy2.7
-22
View File
@@ -1,22 +0,0 @@
#!/usr/bin/env bash
# fail hard
set -o pipefail
# fail harder
set -eu
OUT_PREFIX=$1
dep_formula=${0#$WORKSPACE_DIR/} # this is the original script, e.g. pypy-5.3.1
dep_name=$(basename $BASH_SOURCE) # this is us
dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full version name
dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-}
dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name}
echo "Building PyPy3…"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python"
+10 -1
View File
@@ -2,4 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy3
# NOTICE: This formula only works for the cedar-14 stack, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy3-2.4.0-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy3-2.4.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy3 $OUT_PREFIX/bin/python
+10 -4
View File
@@ -2,7 +2,13 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
# version pypy3-5.5.0 is really called pypy3.3-v5.5.0-alpha
dep_version_prefix=".3"
dep_version_suffix="-alpha"
source $(dirname $0)/pypy3
# NOTICE: This formula only works for the cedar-14 stack, not cedar.
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy3.3-v5.5.0-alpha-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy3-v5.5.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy3 $OUT_PREFIX/bin/python
+8 -1
View File
@@ -2,4 +2,11 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy3
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy3-v5.7.1-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy3-v5.7.1-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy3 $OUT_PREFIX/bin/python
+8 -1
View File
@@ -2,4 +2,11 @@
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/pypy3
OUT_PREFIX=$1
echo "Building PyPy..."
SOURCE_TARBALL='https://bitbucket.org/pypy/pypy/downloads/pypy3-v5.8.0-linux64.tar.bz2'
curl -L $SOURCE_TARBALL | tar jx
cp -R pypy3-v5.8.0-linux64/* $OUT_PREFIX
ln $OUT_PREFIX/bin/pypy3 $OUT_PREFIX/bin/python
-23
View File
@@ -1,23 +0,0 @@
#!/usr/bin/env bash
# fail hard
set -o pipefail
# fail harder
set -eu
OUT_PREFIX=$1
dep_formula=${0#$WORKSPACE_DIR/} # this is the original script, e.g. pypy-5.3.1
dep_name=$(basename $BASH_SOURCE) # this is us
dep_version=${dep_formula##*"/${dep_name}-"} # "subtract" our name from full version name
dep_package=${dep_name}${dep_version_prefix:-}-v${dep_version}${dep_version_suffix:-}
dep_dirname=${dep_package}-linux64
dep_archive_name=${dep_dirname}.tar.bz2
dep_url=https://bitbucket.org/pypy/pypy/downloads/${dep_archive_name}
echo "Building PyPy3…"
echo "${dep_url}"
curl -L "${dep_url}" | tar jx -C "${OUT_PREFIX}" --strip-components 1 # extract to $OUT_PREFIX, drop the first directory level, which is the archive name
ln "$OUT_PREFIX/bin/pypy3" "$OUT_PREFIX/bin/python"
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/pypy3.6
+12 -1
View File
@@ -1,4 +1,15 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python2
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/2.7.10/Python-2.7.10.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.10 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
+12 -1
View File
@@ -1,4 +1,15 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python2
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/2.7.11/Python-2.7.11.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.11 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
+12 -1
View File
@@ -1,4 +1,15 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python2
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/2.7.12/Python-2.7.12.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.12 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
+16 -1
View File
@@ -1,4 +1,19 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python2
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/2.7.13/Python-2.7.13.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.13 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python2
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python2
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python2
-27
View File
@@ -1,27 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
OUT_PREFIX=$1
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/2.7.17/Python-2.7.17.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.17 src
cd src
./configure --prefix=$OUT_PREFIX --enable-unicode=ucs4 --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
+12 -1
View File
@@ -1,4 +1,15 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python2
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/2.7.9/Python-2.7.9.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-2.7.9 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
-32
View File
@@ -1,32 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
OUT_PREFIX=$1
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.7.2/Python-3.7.2.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.7.2 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
-32
View File
@@ -1,32 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
OUT_PREFIX=$1
BIN_DIR="$(cd "$(dirname "$0")"/../.. || exit; pwd)/bin"
export BIN_DIR
# shellcheck source=bin/utils
source "$BIN_DIR/steps/sqlite3"
sqlite3_version
echo "Setting up SQLite3 Headers for $SQLITE3_VERSION"
sqlite3_install "$OUT_PREFIX" "$SQLITE3_VERSION" 1
echo "Building Python…"
SOURCE_TARBALL='https://python.org/ftp/python/3.4.9/Python-3.4.9.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.4.9 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
# Remove spare /
LOCATION=${OUT_PREFIX%?}
ln $LOCATION/bin/python3 $LOCATION/bin/python
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+15 -1
View File
@@ -1,4 +1,18 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/3.5.2/Python-3.5.2.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.5.2 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# ln $OUT_PREFIX/bin/python3 $OUT_PREFIX/bin/python
source $(dirname $0)/python3
+15 -1
View File
@@ -1,4 +1,18 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/3.5.3/Python-3.5.3.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.5.3 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# ln $OUT_PREFIX/bin/python3 $OUT_PREFIX/bin/python
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+15 -1
View File
@@ -1,4 +1,18 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/3.6.0/Python-3.6.0.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.6.0 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
ln $OUT_PREFIX/bin/python3 $OUT_PREFIX/bin/python
source $(dirname $0)/python3
+15 -1
View File
@@ -1,4 +1,18 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/3.6.1/Python-3.6.1.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.6.1 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
ln $OUT_PREFIX/bin/python3 $OUT_PREFIX/bin/python
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
+18 -1
View File
@@ -1,4 +1,21 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
# Build Deps: libraries/sqlite
source $(dirname $0)/python3
OUT_PREFIX=$1
echo "Building Python..."
SOURCE_TARBALL='https://python.org/ftp/python/3.6.2/Python-3.6.2.tgz'
curl -L $SOURCE_TARBALL | tar xz
mv Python-3.6.2 src
cd src
./configure --prefix=$OUT_PREFIX --with-ensurepip=no
make
make install
# Remove unneeded test directories, similar to the official Docker Python images:
# https://github.com/docker-library/python
find "${OUT_PREFIX}" \( -type d -a \( -name test -o -name tests \) \) -exec rm -rf '{}' +
ln $OUT_PREFIX/bin/python3 $OUT_PREFIX/bin/python
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3
-4
View File
@@ -1,4 +0,0 @@
#!/usr/bin/env bash
# Build Path: /app/.heroku/python/
source $(dirname $0)/python3

Some files were not shown because too many files have changed in this diff Show More