Due to how the version checks work - via sorting, not actual comparison - this previously resulted in _always_ installing sqlite3, even though it was already bundled for lower versions of python. The second version check also encompasses 3.7.0+, so there is no need to respecify the check.
It's occasionally useful to see at a glance when releases were pushed to understand what release might have caused a customer's issues. This back-fills dates for the last few months of releases and sets a precedent that can be followed in the future.
With inspiration from @KevinBrolly, this patch uses the stack image
SQLite3 package but also still providing the dev headers and binary that
users may still be using today. The benefit is that we won't need to
rebuild all the python binaries for this to take affect. We can just
stop shipping SQLite3 from future binaries. In addition, we don't need
to worry about what version and when to update SQLite3 and maintaining
the packages ourselves.
This also includes updates to Python 2.7.15 and Python 3.6.6 so they can
rebuilt with the stack image dev headers instead of building our own
vendored SQLite3.
- Add stage to Travis CI config and update tests.sh script to recognize
it
- Update tests to assert there is no Python 2 on Heroku-18
- Update nltk fixture to use Python 3.6 so we can test it on all stacks
Closes gh-730
Python 3.7.0 is supported but not preferred given how new it is. As a
result, we don't want it to be the default, but we also don't want users
to be confused when upgrading to it.
Closes gh-728
* Since pipenv is being pinned now, output pinned version in steps
Signed-off-by: Yury V. Zaytsev <yury.zaytsev@moneymeets.com>
* Update README.md example output to match new reality
Signed-off-by: Yury V. Zaytsev <yury.zaytsev@moneymeets.com>
* Update pipenv
Set GDAL and GEOS library locaiton explicitly in environment
variables:
* GDAL_LIBRARY_PATH "/app/.heroku/vendor/lib/libgdal.so"
* GEOS_LIBRARY_PATH "/app/.heroku/vendor/lib/libgeos_c.so"
Django has to settings with the same name. The setup now works as
described here:
https://devcenter.heroku.com/articles/postgis#geodjango-setup
* only do this on heroku-16
* history
Signed-off-by: Kenneth Reitz <me@kennethreitz.org>
* code comment
Signed-off-by: Kenneth Reitz <me@kennethreitz.org>
Found by diffing the published buildpack archives and looking at the
git log to see what changed. It looks like the v108 entry was
accidentally added in #405, when the release being published there
was actually v107. The setuptools update mentioned there never
happened, since the `SETUPTOOLS_VERSION` variable was found to be
unused and instead removed.
Previously the contents of the file being searched was not output,
making it hard to know how to fix a test so that it would pass.
Before:
```
$ make test-heroku-16
...
testCffi
ASSERT:Expected </tmp/shunit.ayVGAO/tmp/output.6gMs/stdout> to contain <ZZZZZZZ>
testPylibmc
```
After:
```
$ make test-heroku-16
...
testCffi
ASSERT:Expected </tmp/shunit.FXWeuI/tmp/output.7Vfx/stdout> to contain <ZZZZZZZ>
! Warning: Your application is missing a Procfile. This file tells Heroku how to run your application.
! Learn more: https://devcenter.heroku.com/articles/procfile
-----> Installing python-3.6.2
-----> Installing pip
-----> Installing requirements with pip
Collecting cffi (from -r /tmp/testBoLG0/requirements.txt (line 1))
Downloading cffi-1.10.0-cp36-cp36m-manylinux1_x86_64.whl (406kB)
Collecting pycparser (from cffi->-r /tmp/testBoLG0/requirements.txt (line 1))
Downloading pycparser-2.18.tar.gz (245kB)
Installing collected packages: pycparser, cffi
Running setup.py install for pycparser: started
Running setup.py install for pycparser: finished with status 'done'
Successfully installed cffi-1.10.0 pycparser-2.18
testPylibmc
```
* NLTK support: Update test to use multiple corpora
So that the incorrect handling of multiple IDs seen in #444 would
have been caught.
Also switches to some of the smaller corpora, to reduce time spent
downloading during tests (see sizes on http://www.nltk.org/nltk_data/).
* NLTK support: Fix passing of multiple corpora identifiers
As part of fixing the shellcheck warnigns in #438, double quotes had
been placed around `$nltk_packages` passed to the `nltk.downloader`,
which causes multiple identifiers to be treated as though it were just
one identifier that contains spaces.
The docs for the shellcheck warning in question recommend using arrays
if the intended behaviour really is to split on spaces:
https://github.com/koalaman/shellcheck/wiki/SC2086#exceptions
As such, `readarray` has been used, which is present in bash >=4.
The `[*]` array form is used in the log message, to prevent shellcheck
warning SC2145, whereas `[@]` is used when passed to `nltk.downloader`
to ensure the array elements are unpacked as required.
Note: Both before and after this fix, using anything but unix line
endings in `nltk.txt` will also cause breakage.
Reduces the impact on Travis runtime caused by #438, by:
* Making the shellcheck job run on the container infra (`sudo: false`)
which has faster boot times.
* Running the heroku-16 and cedar-14 stack tests in parallel, since
previously they were separate stages (which are run in series by
design).
Reduces Travis end-to-end time from ~10 minutes to ~6 minutes.
* Test that the cache is invalidated when the stack changes
* Improve UX when clearing cache due to the stack changing
Now outputs a message informing that the cache was cleared, and
clears the cache first to avoid a redundant message about removing
an old Python version.
This copies the official Docker Python images approach of stripping
the Python test suite after building from source.
For Python 3.6.2 this reduces the output from 184MB to 121MB and
removes the following directories:
```
.heroku/python/lib/python3.6/tkinter/test
.heroku/python/lib/python3.6/ctypes/test
.heroku/python/lib/python3.6/unittest/test
.heroku/python/lib/python3.6/distutils/tests
.heroku/python/lib/python3.6/sqlite3/test
.heroku/python/lib/python3.6/lib2to3/tests
.heroku/python/lib/python3.6/test
```
For Python 2.7.13 this reduces the output from 127MB to 91MB and
removes the following directories:
```
.heroku/python/lib/python2.7/ctypes/test
.heroku/python/lib/python2.7/unittest/test
.heroku/python/lib/python2.7/distutils/tests
.heroku/python/lib/python2.7/bsddb/test
.heroku/python/lib/python2.7/lib-tk/test
.heroku/python/lib/python2.7/sqlite3/test
.heroku/python/lib/python2.7/lib2to3/tests
.heroku/python/lib/python2.7/json/tests
.heroku/python/lib/python2.7/email/test
.heroku/python/lib/python2.7/test
```
Fixes#424.
The pip-uninstall step stopped working when it was moved to after
the pip-install step in f27a84e.
This regression was temporarily fixed by part of #397, however that
PR was reverted in #404.
Adds a test to hopefully catch any future regressions :-)
Fixes#393.
* Revert "Fix pyyaml (#402)"
This reverts commit ff94908505.
* Revert "Don't install packages that could mess packaging up (#397)"
This reverts commit 5496c02f9f.
* Revert "use pkg_resources to check for distributions (#395)"
This reverts commit 9b185f99d5.
* use sp-grep for django detection
* sp-grep
* fix setuptools script
* Updates bob-builder to a version that clears out the previous
build directory, which prevents the stale files seen in #379.
* Adds a `buildenv-heroku-16` Make command to simplify the
building and use of the Heroku-16 binary build environment.
* Sets `S3_BUCKET` and `S3_PREFIX` in the Docker image, to save
having to do so manually each time.
* Removes the duplication of the bob-builder dependency between
`Dockerfile` and `requirements.txt`.
* Disables the pip version check during Docker build, to avoid the
noisy stderr warning about Ubuntu 16.04 pip being older (v8.1.1).
* Adds a `.dockerignore` to speed up the Docker build, by reducing
the build context transferred to the daemon from 60MB to 2MB.
* Applies some Dockerfile best practices like disabling/removing
cached files (since layer invalidation makes caching pointless).
In cases where there is no stderr there was previously no output,
which made it hard to find the cause. The assertion messages have
also been adjusted to remove the repetition (shunit2 already outputs
expected vs actual itself).
Before:
```
testPipenv
ASSERT:Expected captured exit code to be 0; was <2> expected:<0> but was:<2>
testPipenvVersion
```
After:
```
testPipenv
ASSERT:Captured exit code - expected:<0> but was:<2>
! Warning: Your application is missing a Procfile. This file tells Heroku how to run your application.
! Learn more: https://devcenter.heroku.com/articles/procfile
-----> Installing python-2.7.13
-----> Installing pip
-----> Generating 'requirements.txt' with pipenv
testPipenvVersion
```
Fixes#389.
So that any changes made to `.heroku/` within pre_compile (such as
installing additional libraries required for the later pip install) are
not clobbered by the cache being copied over afterwards.
Fixes#320.
* update python.gunicorn.sh to WEB_CONCURRENCY.sh
* don't remove WEB_CONCURRENCY for prefixed 0s
* split gunicorn functionality into its own file
* cleanup comments
* fix var ordering
* add retry to curl
* use proper url for buildpack stdlib
The Node buildpack now exports a leading zero in its numbers. This lets us detect that it (or another buildpack) set the value, and overwrite it accordingly.
Fixes the issue where adding the node buildpack to a Python app would cause only one gunicorn worker to be spawned for a 1X dyno, and not two.
We also need to again produce leading zeroes in the value, so that e.g. the PHP buildpack can do the same on boot.
* drop duplicate mkdir for .heroku
* simplify linking and moving of python and source
* steps/python needs its own symlink setup
* fix symlink destinations for vendor and venv
* always symlink python
Since if someone previously had `GDAL` in their requirements file, they
would already have the gdalserver binary present from `bin/steps/gdal`
but be missing the proj and geos vendor files. By checking for `proj`
instead, we ensure that the vendoring isn't incorrectly skipped in this
case.
For both the cedar and cedar-14 stacks, the libffi directory is named
`libffi-3.1` not `libffi-3.1.1`, so the existing library wouldn't be
detected, causing it to be re-downloaded during every compile.
This means pip-pop will use the buildpack-installed pip 8, which doesn't
fail when encountering requirements files that use the new hash syntax.
Fixes#302.
The extraneous `format` call is harmless in itself, but it's possible for `requirement.req` to be `None` here, which results in `AttributeError: 'NoneType' object has no attribute 'project_name'`.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, “this License” refers to version 3 of the GNU Lesser General Public License, and the “GNU GPL” refers to version 3 of the GNU General Public License.
“The Library” refers to a covered work governed by this License, other than an Application or a Combined Work as defined below.
An “Application” is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library.
A “Combined Work” is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the “Linked Version”.
The “Minimal Corresponding Source” for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version.
The “Corresponding Application Code” for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version:
a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following:
a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license document.
c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.
1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version.
e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.
get-pip.py license
------------------
Copyright (c) 2008-2016 The pip developers (see AUTHORS.txt file)
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
pip-pop license
---------------
The MIT License (MIT)
Copyright (c) 2014 Kenneth Reitz.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps.
Recommended web frameworks include **Django** and **Flask**, among others. The recommended webserver is **Gunicorn**. There are no restrictions around what software can be used (as long as it's pip-installable). Web processes must bind to `$PORT`, and only the HTTP protocol is permitted for incoming connections.
Python packages with C dependencies that are not [available on the stack image](https://devcenter.heroku.com/articles/stack-packages) are generally not supported, unless `manylinux` wheels are provided by the package maintainers (common). For recommended solutions, check out [this article](https://devcenter.heroku.com/articles/python-c-deps) for more information.
This is the official [Heroku buildpack](https://devcenter.heroku.com/articles/buildpacks) for Python apps, powered by [pip](https://pip.pypa.io/) and other excellent software.
Recommended web frameworks include **Django** and **Flask**. The recommended webserver is **Gunicorn**. There are no restrictions around what software can be used (as long as it's pip-installable). Web processes must bind to `$PORT`, and only the HTTP protocol is permitted for incoming connections.
Some Python packages with obscure C dependencies (e.g. scipy) are [not compatible](https://devcenter.heroku.com/articles/python-c-deps).
See it in Action
----------------
Deploying a Python application couldn't be easier:
A `requirements.txt` file must be present at the root of your application's repository.
You can also specify the latest production release of this buildpack for upcoming builds of an existing application:
$ heroku buildpacks:set heroku/python
Specify a Python Runtime
------------------------
Specific versions of the Python runtime can be specified with a `runtime.txt` file:
$ cat runtime.txt
python-3.5.1
Runtime options include:
-`python-2.7.11`
-`python-3.5.1`
-`pypy-5.0.1` (unsupported, experimental)
Other [unsupported runtimes](https://github.com/heroku/heroku-buildpack-python/tree/master/builds/runtimes) are available as well. Use at your own risk.
sed -e 's/\.\.\.\+/.../g' | sed -e '/already satisfied/Id' | sed -e '/Overwriting/Id' | sed -e '/python executable/Id' | sed -e '/no previously-included files/Id'
}
# Buildpack Indented line.
puts-line() {
echo " $@"
sed -e 's/\.\.\.\+/.../g' | sed -e '/already satisfied/Id' | sed -e '/No files were found to uninstall/Id' | sed -e '/Overwriting/Id' | sed -e '/python executable/Id' | sed -e '/no previously-included files/Id'
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.