diff --git a/content/admin-processes.md b/content/admin-processes.md index 870a560..33d4286 100644 --- a/content/admin-processes.md +++ b/content/admin-processes.md @@ -1,14 +1,14 @@ ## XII. Admin processes ### Run admin/management tasks as one-off processes -The [process formation](./concurrency) is the array of processes that are used to do the app's regular business (such as handling web requests) as it runs. Separately, developers will often wish to do one-off administrative or maintenance tasks for the app, such as: +The [process formation](./concurrency.md) is the array of processes that are used to do the app's regular business (such as handling web requests) as it runs. Separately, developers will often wish to do one-off administrative or maintenance tasks for the app, such as: * Running database migrations (e.g. `manage.py migrate` in Django, `rake db:migrate` in Rails). * Running a console (also known as a [REPL](http://en.wikipedia.org/wiki/Read-eval-print_loop) shell) to run arbitrary code or inspect the app's models against the live database. Most languages provide a REPL by running the interpreter without any arguments (e.g. `python` or `perl`) or in some cases have a separate command (e.g. `irb` for Ruby, `rails console` for Rails). * Running one-time scripts committed into the app's repo (e.g. `php scripts/fix_bad_records.php`). -One-off admin processes should be run in an identical environment as the regular [long-running processes](./processes) of the app. They run against a [release](./build-release-run), using the same [codebase](./codebase) and [config](./config) as any process run against that release. Admin code must ship with application code to avoid synchronization issues. +One-off admin processes should be run in an identical environment as the regular [long-running processes](./processes.md) of the app. They run against a [release](./build-release-run.md), using the same [codebase](./codebase.md) and [config](./config.md) as any process run against that release. Admin code must ship with application code to avoid synchronization issues. -The same [dependency isolation](./dependencies) techniques should be used on all process types. For example, if the Ruby web process uses the command `bundle exec thin start`, then a database migration should use `bundle exec rake db:migrate`. Likewise, a Python program using Virtualenv should use the vendored `bin/python` for running both the Tornado webserver and any `manage.py` admin processes. +The same [dependency isolation](./dependencies.md) techniques should be used on all process types. For example, if the Ruby web process uses the command `bundle exec thin start`, then a database migration should use `bundle exec rake db:migrate`. Likewise, a Python program using Virtualenv should use the vendored `bin/python` for running both the Tornado webserver and any `manage.py` admin processes. Twelve-factor strongly favors languages which provide a REPL shell out of the box, and which make it easy to run one-off scripts. In a local deploy, developers invoke one-off admin processes by a direct shell command inside the app's checkout directory. In a production deploy, developers can use ssh or other remote command execution mechanism provided by that deploy's execution environment to run such a process. diff --git a/content/backing-services.md b/content/backing-services.md index c8f5994..f21300b 100644 --- a/content/backing-services.md +++ b/content/backing-services.md @@ -5,10 +5,10 @@ A *backing service* is any service the app consumes over the network as part of Backing services like the database are traditionally managed by the same systems administrators who deploy the app's runtime. In addition to these locally-managed services, the app may also have services provided and managed by third parties. Examples include SMTP services (such as [Postmark](http://postmarkapp.com/)), metrics-gathering services (such as [New Relic](http://newrelic.com/) or [Loggly](http://www.loggly.com/)), binary asset services (such as [Amazon S3](http://aws.amazon.com/s3/)), and even API-accessible consumer services (such as [Twitter](http://dev.twitter.com/), [Google Maps](https://developers.google.com/maps/), or [Last.fm](http://www.last.fm/api)). -**The code for a twelve-factor app makes no distinction between local and third party services.** To the app, both are attached resources, accessed via a URL or other locator/credentials stored in the [config](./config). A [deploy](./codebase) of the twelve-factor app should be able to swap out a local MySQL database with one managed by a third party (such as [Amazon RDS](http://aws.amazon.com/rds/)) without any changes to the app's code. Likewise, a local SMTP server could be swapped with a third-party SMTP service (such as Postmark) without code changes. In both cases, only the resource handle in the config needs to change. +**The code for a twelve-factor app makes no distinction between local and third party services.** To the app, both are attached resources, accessed via a URL or other locator/credentials stored in the [config](./config.md). A [deploy](./codebase.md) of the twelve-factor app should be able to swap out a local MySQL database with one managed by a third party (such as [Amazon RDS](http://aws.amazon.com/rds/)) without any changes to the app's code. Likewise, a local SMTP server could be swapped with a third-party SMTP service (such as Postmark) without code changes. In both cases, only the resource handle in the config needs to change. Each distinct backing service is a *resource*. For example, a MySQL database is a resource; two MySQL databases (used for sharding at the application layer) qualify as two distinct resources. The twelve-factor app treats these databases as *attached resources*, which indicates their loose coupling to the deploy they are attached to. -A production deploy attached to four backing services. +![A production deploy attached to four backing services.](/images/attached-resources.png) Resources can be attached to and detached from deploys at will. For example, if the app's database is misbehaving due to a hardware issue, the app's administrator might spin up a new database server restored from a recent backup. The current production database could be detached, and the new database attached -- all without any code changes. diff --git a/content/build-release-run.md b/content/build-release-run.md index 83525c1..52b5168 100644 --- a/content/build-release-run.md +++ b/content/build-release-run.md @@ -1,11 +1,11 @@ ## V. Build, release, run ### Strictly separate build and run stages -A [codebase](./codebase) is transformed into a (non-development) deploy through three stages: +A [codebase](./codebase.md) is transformed into a (non-development) deploy through three stages: -* The *build stage* is a transform which converts a code repo into an executable bundle known as a *build*. Using a version of the code at a commit specified by the deployment process, the build stage fetches vendors [dependencies](./dependencies) and compiles binaries and assets. -* The *release stage* takes the build produced by the build stage and combines it with the deploy's current [config](./config). The resulting *release* contains both the build and the config and is ready for immediate execution in the execution environment. -* The *run stage* (also known as "runtime") runs the app in the execution environment, by launching some set of the app's [processes](./processes) against a selected release. +* The *build stage* is a transform which converts a code repo into an executable bundle known as a *build*. Using a version of the code at a commit specified by the deployment process, the build stage fetches vendors [dependencies](./dependencies.md) and compiles binaries and assets. +* The *release stage* takes the build produced by the build stage and combines it with the deploy's current [config](./config.md). The resulting *release* contains both the build and the config and is ready for immediate execution in the execution environment. +* The *run stage* (also known as "runtime") runs the app in the execution environment, by launching some set of the app's [processes](./processes.md) against a selected release. ![Code becomes a build, which is combined with config to create a release.](/images/release.png) diff --git a/content/codebase.md b/content/codebase.md index 234ad67..44af7f5 100644 --- a/content/codebase.md +++ b/content/codebase.md @@ -10,7 +10,7 @@ A *codebase* is any single repo (in a centralized revision control system like S There is always a one-to-one correlation between the codebase and the app: * If there are multiple codebases, it's not an app -- it's a distributed system. Each component in a distributed system is an app, and each can individually comply with twelve-factor. -* Multiple apps sharing the same code is a violation of twelve-factor. The solution here is to factor shared code into libraries which can be included through the [dependency manager](./dependencies). +* Multiple apps sharing the same code is a violation of twelve-factor. The solution here is to factor shared code into libraries which can be included through the [dependency manager](./dependencies.md). There is only one codebase per app, but there will be many deploys of the app. A *deploy* is a running instance of the app. This is typically a production site, and one or more staging sites. Additionally, every developer has a copy of the app running in their local development environment, each of which also qualifies as a deploy. diff --git a/content/config.md b/content/config.md index 0bc603b..8031789 100644 --- a/content/config.md +++ b/content/config.md @@ -1,9 +1,9 @@ ## III. Config ### Store config in the environment -An app's *config* is everything that is likely to vary between [deploys](./codebase) (staging, production, developer environments, etc). This includes: +An app's *config* is everything that is likely to vary between [deploys](./codebase.md) (staging, production, developer environments, etc). This includes: -* Resource handles to the database, Memcached, and other [backing services](./backing-services) +* Resource handles to the database, Memcached, and other [backing services](./backing-services.md) * Credentials to external services such as Amazon S3 or Twitter * Per-deploy values such as the canonical hostname for the deploy diff --git a/content/dev-prod-parity.md b/content/dev-prod-parity.md index e771d16..2437d59 100644 --- a/content/dev-prod-parity.md +++ b/content/dev-prod-parity.md @@ -1,7 +1,7 @@ ## X. Dev/prod parity ### Keep development, staging, and production as similar as possible -Historically, there have been substantial gaps between development (a developer making live edits to a local [deploy](./codebase) of the app) and production (a running deploy of the app accessed by end users). These gaps manifest in three areas: +Historically, there have been substantial gaps between development (a developer making live edits to a local [deploy](./codebase.md) of the app) and production (a running deploy of the app accessed by end users). These gaps manifest in three areas: * **The time gap**: A developer may work on code that takes days, weeks, or even months to go into production. * **The personnel gap**: Developers write code, ops engineers deploy it. @@ -38,7 +38,7 @@ Summarizing the above into a table: -[Backing services](./backing-services), such as the app's database, queueing system, or cache, is one area where dev/prod parity is important. Many languages offer libraries which simplify access to the backing service, including *adapters* to different types of services. Some examples are in the table below. +[Backing services](./backing-services.md), such as the app's database, queueing system, or cache, is one area where dev/prod parity is important. Many languages offer libraries which simplify access to the backing service, including *adapters* to different types of services. Some examples are in the table below. diff --git a/content/disposability.md b/content/disposability.md index 4725ef5..f1cbdb2 100644 --- a/content/disposability.md +++ b/content/disposability.md @@ -1,7 +1,7 @@ ## IX. Disposability ### Maximize robustness with fast startup and graceful shutdown -**The twelve-factor app's [processes](./processes) are *disposable*, meaning they can be started or stopped at a moment's notice.** This facilitates fast elastic scaling, rapid deployment of [code](./codebase) or [config](./config) changes, and robustness of production deploys. +**The twelve-factor app's [processes](./processes.md) are *disposable*, meaning they can be started or stopped at a moment's notice.** This facilitates fast elastic scaling, rapid deployment of [code](./codebase.md) or [config](./config.md) changes, and robustness of production deploys. Processes should strive to **minimize startup time**. Ideally, a process takes a few seconds from the time the launch command is executed until the process is up and ready to receive requests or jobs. Short startup time provides more agility for the [release](./build-release-run) process and scaling up; and it aids robustness, because the process manager can more easily move processes to new physical machines when warranted. diff --git a/content/port-binding.md b/content/port-binding.md index b12de8b..5b6b6b7 100644 --- a/content/port-binding.md +++ b/content/port-binding.md @@ -7,8 +7,8 @@ Web apps are sometimes executed inside a webserver container. For example, PHP In a local development environment, the developer visits a service URL like `http://localhost:5000/` to access the service exported by their app. In deployment, a routing layer handles routing requests from a public-facing hostname to the port-bound web processes. -This is typically implemented by using [dependency declaration](./dependencies) to add a webserver library to the app, such as [Tornado](http://www.tornadoweb.org/) for Python, [Thin](http://code.macournoyer.com/thin/) for Ruby, or [Jetty](http://www.eclipse.org/jetty/) for Java and other JVM-based languages. This happens entirely in *user space*, that is, within the app's code. The contract with the execution environment is binding to a port to serve requests. +This is typically implemented by using [dependency declaration](./dependencies.md) to add a webserver library to the app, such as [Tornado](http://www.tornadoweb.org/) for Python, [Thin](http://code.macournoyer.com/thin/) for Ruby, or [Jetty](http://www.eclipse.org/jetty/) for Java and other JVM-based languages. This happens entirely in *user space*, that is, within the app's code. The contract with the execution environment is binding to a port to serve requests. HTTP is not the only service that can be exported by port binding. Nearly any kind of server software can be run via a process binding to a port and awaiting incoming requests. Examples include [ejabberd](http://www.ejabberd.im/) (speaking [XMPP](http://xmpp.org/)), and [Redis](http://redis.io/) (speaking the [Redis protocol](http://redis.io/topics/protocol)). -Note also that the port-binding approach means that one app can become the [backing service](./backing-services) for another app, by providing the URL to the backing app as a resource handle in the [config](./config) for the consuming app. +Note also that the port-binding approach means that one app can become the [backing service](./backing-services.md) for another app, by providing the URL to the backing app as a resource handle in the [config](./config.md) for the consuming app. diff --git a/content/processes.md b/content/processes.md index 97d222a..05a767c 100644 --- a/content/processes.md +++ b/content/processes.md @@ -3,12 +3,12 @@ The app is executed in the execution environment as one or more *processes*. -In the simplest case, the code is a stand-alone script, the execution environment is a developer's local laptop with an installed language runtime, and the process is launched via the command line (for example, `python my_script.py`). On the other end of the spectrum, a production deploy of a sophisticated app may use many [process types, instantiated into zero or more running processes](./concurrency). +In the simplest case, the code is a stand-alone script, the execution environment is a developer's local laptop with an installed language runtime, and the process is launched via the command line (for example, `python my_script.py`). On the other end of the spectrum, a production deploy of a sophisticated app may use many [process types, instantiated into zero or more running processes](./concurrency.md). -**Twelve-factor processes are stateless and [share-nothing](http://en.wikipedia.org/wiki/Shared_nothing_architecture).** Any data that needs to persist must be stored in a stateful [backing service](./backing-services), typically a database. +**Twelve-factor processes are stateless and [share-nothing](http://en.wikipedia.org/wiki/Shared_nothing_architecture).** Any data that needs to persist must be stored in a stateful [backing service](./backing-services.md), typically a database. The memory space or filesystem of the process can be used as a brief, single-transaction cache. For example, downloading a large file, operating on it, and storing the results of the operation in the database. The twelve-factor app never assumes that anything cached in memory or on disk will be available on a future request or job -- with many processes of each type running, chances are high that a future request will be served by a different process. Even when running only one process, a restart (triggered by code deploy, config change, or the execution environment relocating the process to a different physical location) will usually wipe out all local (e.g., memory and filesystem) state. -Asset packagers like [django-assetpackager](http://code.google.com/p/django-assetpackager/) use the filesystem as a cache for compiled assets. A twelve-factor app prefers to do this compiling during the [build stage](/build-release-run). Asset packagers such as [Jammit](http://documentcloud.github.io/jammit/) and the [Rails asset pipeline](http://ryanbigg.com/guides/asset_pipeline.html) can be configured to package assets during the build stage. +Asset packagers like [django-assetpackager](http://code.google.com/p/django-assetpackager/) use the filesystem as a cache for compiled assets. A twelve-factor app prefers to do this compiling during the [build stage](/build-release-run). Asset packagers such as [Jammit](http://documentcloud.github.io/jammit/) and the [Rails asset pipeline](http://ryanbigg.com/guides/asset_pipeline.html) can be configured to package assets during the build stage. Some web systems rely on ["sticky sessions"](http://en.wikipedia.org/wiki/Load_balancing_%28computing%29#Persistence) -- that is, caching user session data in memory of the app's process and expecting future requests from the same visitor to be routed to the same process. Sticky sessions are a violation of twelve-factor and should never be used or relied upon. Session state data is a good candidate for a datastore that offers time-expiration, such as [Memcached](http://memcached.org/) or [Redis](http://redis.io/). diff --git a/images/attached-resources.png b/images/attached-resources.png new file mode 100644 index 0000000..31af2be Binary files /dev/null and b/images/attached-resources.png differ diff --git a/images/codebase-deploys.png b/images/codebase-deploys.png new file mode 100644 index 0000000..c7e1a46 Binary files /dev/null and b/images/codebase-deploys.png differ diff --git a/images/process-types.png b/images/process-types.png new file mode 100644 index 0000000..a9d46e0 Binary files /dev/null and b/images/process-types.png differ diff --git a/images/release.png b/images/release.png new file mode 100644 index 0000000..dc12ee6 Binary files /dev/null and b/images/release.png differ