Ed Morley
b250300b74
Since: * We want the S3 bucket to be owned by a different AWS account and it's not possible to transfer ownership of an existing bucket. * In the future we want to rebuild some of the Python runtime archives (for example to improve the sqlite3 handling, or to tweak the compile flags used), and it will be easier to reason about the change if we can guarantee only recent buildpack versions are using the assets rather than several year old unmaintained forks. The assets were synced from the old bucket using (minus the `--dryrun`): ``` aws s3 sync s3://lang-python s3://heroku-buildpack-python \ --dryrun \ --metadata-directive REPLACE \ --exclude "*" \ --include 'common/*' \ --include 'heroku-*/runtimes/*' \ --include 'heroku-*/libraries/vendor/gdal.tar.gz' \ --include 'heroku-*/libraries/vendor/geos.tar.gz' \ --include 'heroku-*/libraries/vendor/proj.tar.gz' \ --exclude 'common/pip-20.0.2-py2.py3-none-any.whl' \ --exclude '*/runtimes/*-opt.tar.gz' \ --exclude '*/runtimes/sqlite-free/*' ``` The files that were `--exclude`d are those that are no longer used, or test assets that were not officially released. The Cedar-14 assets were not migrated since it's EOL next month. The old S3 bucket will be left untouched for the foreseeable future (ie: we won't be deleting it), since builds using older versions of this buildpack (either due to pinning to a tag or via a fork) will still be using assets from it. Closes @W-8060097@.
Python Buildpack Binaries
The binaries for this buildpack are built in Docker containers based on the Heroku stack image.
Configuration
In order to publish binaries AWS credentials must be passed to the build container.
If you are testing only the build (ie: bob build), these are optional.
In addition, unless you are building the official binaries for Heroku (which use the defaults
specified in each Dockerfile), you will need to override S3_BUCKET and S3_PREFIX to
match your own S3 bucket/use case.
If you only need to set AWS credentials, you can do so by setting the environment variables
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY before calling the make commands.
For example:
set +o history # Disable bash history
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
set -o history # Re-enable bash history
make ...
If you need to override the default S3 bucket, or would prefer not to use credentials via environment variables, then you need to instead use a Docker env file like so:
-
Copy the
builds/dockerenv.defaultenv file to a location outside the buildpack repository. -
Edit the new file, adding at a minimum the values for the variables
AWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEY(see Docker env-file documentation). -
Pass the path of the file to the make commands using
ENV_FILE. For example:make ... ENV_FILE=~/.dockerenv.python-buildpack
Launching an interactive build environment
To start an interactive version of the build environment (ideal for development) use the
buildenv make target, passing in the desired STACK name. For example:
make buildenv STACK=heroku-18
This will create the builder docker image based on the latest image for that stack, and
then start a bash shell where you can run bob build, bob deploy, and so forth.
The builds/ directory is bind-mounted into the running container, so local build formula
changes will appear there immediately without the need to rebuild the image.
Bulk deploying runtimes
When a new Python version is released, binaries have to be generated for multiple stacks.
To automate this, use the deploy-runtimes make target, which will ensure the builder
image is up to date, and then run bob deploy for each runtime-stack combination.
The build formula name(s) are passed using RUNTIMES, like so:
make deploy-runtimes RUNTIMES='python-X.Y.Z'
By default this will deploy to all supported stacks (see STACKS in Makefile),
but this can be overridden using STACKS:
make deploy-runtimes RUNTIMES='python-X.Y.Z' STACKS='heroku-16 heroku-18'
Multiple runtimes can also be specified (useful for when adding a new stack), like so:
make deploy-runtimes RUNTIMES='python-A.B.C python-X.Y.Z' STACKS='heroku-20'
Note: Both RUNTIMES and STACKS are space delimited.