Commit Graph

66 Commits

Author SHA1 Message Date
David Zuelke 00a5fc7f0b call out subprocess termination from signals
when hitting Ctrl+C in a 'docker run bob …', the main script itself actually can't be terminated by Ctrl+C, because it's PID 1

in that case, rather than just shutting down silently (by terminating itself using a SIGINT), we will see the subprocess exit (because Ctrl+C is sent to the whole process group)

When this happens, the return code of the process will be negative, to indicate that it didn't exit with that code, but instead got terminated by a signal of that (absolute) number
2020-01-15 16:53:35 +01:00
David Zuelke f8b002de47 Correctly handle Ctrl+C
When receiving a signal, a process must kill itself using the same signal
sys.exit()ing 0, 1, 130, whatever will not signal to the calling program that we terminated in response to the signal

Best example: `for f in a b c; do bob deploy $f; done`, hitting Ctrl+C should interrupt Bob and stop the bash loop, but does not with `sys.exit()`:

    # for x in php-7.3.13 php-7.3.13 php-7.3.13; do bob build $x; done
    Fetching dependencies... found 1:
      - libraries/libc-client-2007f

    Building formula php-7.3.13 in /tmp/bob-35s7cr5z:

    -----> Building php-7.3.13...
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 18.7M  100 18.7M    0     0  7435k      0  0:00:02  0:00:02 --:--:-- 7434k
    ^Cool.
    Fetching dependencies... found 1:
      - libraries/libc-client-2007f

    Building formula php-7.3.13 in /tmp/bob-vmuko2ra:

    -----> Building php-7.3.13...
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
     20 18.7M   20 3887k    0     0  7479k      0  0:00:02 --:--:--  0:00:02 7476k^Cool.
    Fetching dependencies... found 1:
      - libraries/libc-client-2007f

    Building formula php-7.3.13 in /tmp/bob-p2t8as81:

    -----> Building php-7.3.13...
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
      4 18.7M    4  943k    0     0  3599k      0  0:00:05 --:--:--  0:00:05 3588k^Cool.

We want instead to have the loop end upon the first Ctrl+C.

That's only possible if Bash knows that we exited in response to Ctrl+C (=SIGINT), then it'll also terminate the loop
Bash will report the exit status as 128+$signal, so 130 for SIGINT, but sys.exit(130) does not to the same thing - the value of 130 is simply bash's representation
Killing ourselves with the signal number that we are aborting in response to does all this correctly, and bash will see the right WIFSIGNALED() status of our program, not WIFEXITED()
2020-01-15 02:01:38 +01:00
David Zuelke bda46256d9 print all informational output to stderr
all output from the formula build is on stdout already

this allows easy separating of the two into e.g. raw compile logs without bob's messages
2020-01-15 02:01:38 +01:00
David Zuelke 840c089864 Remove subprocess piping
This brings Python 3 compatibility

With this change, all output from the formula ends up on stdout, without any buffering, meaning e.g. progress bars (from curl for instance) also finally show up in real time, without line buffering like before
2020-01-14 23:15:11 +01:00
Casey Faist 9b2890fb1c make CLI override name optional 2019-04-03 11:06:13 -07:00
Casey Faist da2bff5894 update so override_path checks for None 2019-04-03 10:49:57 -07:00
Saranraj c6f983cccb Fix override_path variable. 2019-04-02 16:01:09 +05:30
Casey Faist da95c3b081 add ability to override the s3 archive file names
if given (via --name) the override name is passed to the formula as
the second argument
2019-03-25 18:05:31 -07:00
David Zuelke e4b0ce3b63 execute build formulae through shell
this prevents the need for chmod +x on formulae

fixes #35
2019-03-25 17:45:40 -07:00
David Zülke 00863664ac allow wildcards in dependencies (#39)
Example:

That would fetch the latest S3_PREFIX/php-7.3.*.tar.gz as a dependency
2019-03-25 16:12:01 -07:00
Ed Morley e838ff0027 flake8 import and whitespace fixes 2017-05-30 22:48:21 +01:00
kennethreitz 54211376a8 Merge pull request #32 from edmorley/clearer-tmpfile-names
Use clearer file and directory names for build output
2017-05-25 07:28:47 -07:00
Ed Morley 17e68dfd69 Print build directory & upload URL during build/deploy
Previously only the temporary build directory was output, not the
final build output location.

The upload URL is generated using:
http://boto.cloudhackers.com/en/latest/ref/s3.html#boto.s3.key.Key.generate_url

Fixes #10.
2017-05-25 14:03:34 +01:00
Ed Morley 480ba2ecab Fail gracefully if S3_BUCKET is not set
Previously this raised with the unclear:
`TypeError: cannot concatenate 'str' and 'NoneType' objects`
2017-05-25 01:51:16 +01:00
Ed Morley bce1e3509d Make errors output to stderr
Using `print_function` is the cleanest option out of those on:
https://stackoverflow.com/questions/5574702/how-to-print-to-stderr-in-python
2017-05-25 01:51:07 +01:00
Ed Morley 9df32fadee Use clearer file and directory names for build output
This makes it easier to differentiate between the archives for
dependencies and that of the final build output by bob when sifting
through `/tmp/`. The archive type is now also apparent when reading
the console output.

Before:
```
Building formula foo in /tmp/bobrmyuSG:
    ...
Archiving.
/tmp/tmp4JUwdj
Deploying.
...
```

After (when combined with #28):
```
Building formula foo in /tmp/bob-rmyuSG:
    ...
Archiving.
Created: /tmp/bob-build-tmp4JUwdj.tar.gz
Deploying.
...
```
2017-05-24 19:52:00 +01:00
Ed Morley 3f5018c69c Defer S3 connections until Formula instantiation
This prevents unnecessary connections to S3 or resultant error
messages when running `bob --help`.

Fixes #27.
2017-05-24 18:22:17 +01:00
Ed Morley 3cd9532ad2 Support no AWS credentials being found in the environment
This allows the build command to be used with public buckets even
if the user doesn't have their own AWS credentials.

Fixes #26.
2017-05-24 18:22:12 +01:00
Ed Morley 13f3689087 Move S3 connection handling to its own class
This means that the main bucket also benefits from the access denied
`anon=True` fall-back previously only used for the upstream bucket.
This helps with the case where AWS credentials are found, but are
for an account unrelated to either bucket.

Since the main bucket can now fall back to anonymous mode, a check
has been added to the deploy command to improve the UX.

Fixes:
https://github.com/kennethreitz/bob-builder/issues/26#issuecomment-301058112
2017-05-24 18:19:54 +01:00
Ed Morley 7053c3e708 Remove unused dependencies and imports 2017-05-12 14:52:32 +01:00
David Zülke 5ce25e7b9b Allow fallback to anon UPSTREAM_S3_BUCKET access (#24)
An UPSTREAM_S3_BUCKET should usually allow "Everyone" for listing bucket contents.
If the S3 credentials for the normal bucket do not explicitly have access, a 403 will occur.
In that case, we re-try fetching the upstream bucket with anonymous authentication.
2016-12-22 17:13:58 +01:00
noexng 8b08462b0b fix setting bucket for upstream 2016-04-28 16:54:58 +06:00
David Zülke 3ef883e21e fix undefined upstream 2016-03-29 17:25:05 +02:00
David Zuelke fc2e325372 UPSTREAM_S3_BUCKET 2016-03-29 12:07:36 +02:00
kennethreitz a1fe32c184 Merge pull request #20 from kennethreitz/dzuelke-mkdir_p-file
Raise in mkdir_p if path is existing file
2016-03-28 16:39:06 -04:00
David Zülke de2a44e310 Clear build_dir before build
Intentionally uses os.path.exists without checking if it's a dir so that if it's a file or link an error is raised and the user sees that they're doing something really wrong somehow, somewhere
2016-03-28 03:09:41 +02:00
David Zülke 00af02bd7c Raise in mkdir_p if path is existing file 2016-03-28 02:58:56 +02:00
David Zuelke 21570bb939 programs interrupted by SIGINT must exit with code 130 (128+INT) 2016-02-17 06:07:38 +01:00
David Zuelke 6b4891d719 use sys.exit instead of site.exit 2016-02-17 06:07:38 +01:00
David Zuelke c669c195db redirect subprocess stderr to stdout 2016-02-16 22:24:37 +01:00
David Zuelke c27f8e6beb do not buffer when relaying process stream data 2016-02-16 21:58:46 +01:00
kennethreitz 9788a39583 Update models.py 2016-02-16 13:53:36 -05:00
kennethreitz c4c1a71641 Update models.py 2016-02-16 13:51:41 -05:00
kennethreitz 89ecb9061f S3_PREFIX-fix for @dzuelke
/
2016-02-16 13:08:58 -05:00
kennethreitz 105c03a402 exit(1), /s/warnings/errors 2016-02-05 14:42:27 -05:00
David Zuelke e1841bb47e fix empty leading slash entry, for real
this also preserves empty directories in tar files
2014-04-07 20:58:46 +02:00
David Zuelke b1439fd014 Revert "fixed tar bug :)"
This reverts commit 88e18ff8cc.
2014-04-07 20:00:08 +02:00
David Zuelke 182b3b67fb Revert "cleaned up archive_tree function"
This reverts commit 1d03df5c85.
2014-04-07 19:59:55 +02:00
Kenneth Reitz 08e3138770 doh 2014-04-02 12:56:18 -04:00
Kenneth Reitz ebdf9b57c8 builds occur in temp directories 2014-04-02 12:51:50 -04:00
Kenneth Reitz e818b632b1 Merge remote-tracking branch 'origin/master' 2014-04-02 11:30:10 -04:00
Kenneth Reitz 1d03df5c85 cleaned up archive_tree function 2014-04-02 11:21:09 -04:00
Kenneth Reitz 88e18ff8cc fixed tar bug :) 2014-04-02 11:19:35 -04:00
kennethreitz e9392d6baf Update cli.py 2014-03-25 18:59:14 -04:00
Kenneth Reitz cda128e492 S3_BUCKET, S3_PREFIX, WORKSACE_DIR
Fixes #9
Fixes #8
2014-03-17 16:01:31 -04:00
Kenneth Reitz fa28c99dcb pep8 2014-03-13 16:59:01 -04:00
Kenneth Reitz 492425d1cf remove ARCHIVE_PATH for now 2014-03-13 16:36:49 -04:00
Kenneth Reitz 4a0596c683 dependency fetching 2014-03-13 16:33:31 -04:00
Kenneth Reitz fe3e9bf072 extract_tree 2014-03-13 15:18:50 -04:00
Kenneth Reitz 3310ab8d93 refactor deps workflow 2014-03-13 14:56:02 -04:00