tweak remaining sections

This commit is contained in:
Adam Wiggins
2011-07-29 00:21:11 -07:00
parent 82463836ba
commit 06c3455d2f
6 changed files with 26 additions and 12 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ Any computer program, once run, is represented by one or more processes. Web ap
**In the twelve-factor app, processes are a first class citizen.** Processes in the twelve-factor app take strong cues from [the unix process model for running service daemons](http://adam.heroku.com/past/2011/5/9/applying_the_unix_process_model_to_web_apps/). Using this model, the developer can architect their app to handle diverse workloads by assigning each type of work to a *process type*. For example, HTTP requests may be handled by a web process, and long-running background tasks handled by a worker process.
This does not exclude individual processes from handling their own internal multiplexing, via threads inside the runtime VM, or the async/evented model found in tools such as EventMachine, Twited, and Node.js. But an individual VM can only grow so large, so the application must also be able to span multiple processes running on multiple physical machines.
This does not exclude individual processes from handling their own internal multiplexing, via threads inside the runtime VM, or the async/evented model found in tools such as [EventMachine](http://rubyeventmachine.com/), [Twisted](http://twistedmatrix.com/trac/), or [Node.js](http://nodejs.org/). But an individual VM can only grow so large (vertical scale), so the application must also be able to span multiple processes running on multiple physical machines.
The process model truly shines when it comes time to scale out. The [share-nothing, horizontally partitionable nature of twelve-factor app processes](/processes) means that adding more concurrency is a simple and reliable operation. The array of process types and number of processes of each type is known as the *process formation*:
+14 -4
View File
@@ -1,9 +1,19 @@
## X. Dev/prod parity
### Keep development and production and similar as possible
Historically, there is a substantial gap between development (a developer making live edits to a local [deploy](/codebase) of the app) and production (a running deploy of the app accessed by end users). A developer may work on code that doesn't go to production for days, weeks, or even months (a time gap). Developers write code, ops engineers deploy it (a personnel gap). Developers may be using a stack like Nginx, SQLite, and OS X, while the production deploy uses Apache, MySQL, and Linux (a tools gap).
Historically, there is a substantial gap between development (a developer making live edits to a local [deploy](/codebase) of the app) and production (a running deploy of the app accessed by end users). Gaps manifest in three areas:
**The twelve-factor app is designed for [continuous deployment](http://www.avc.com/a_vc/2011/02/continuous-deployment.html).** The gap in time between writing code and deploying it is small: a developer may write code and have it deployed hours or even just minutes later. The gap in personnel is small: developers who wrote code are closely involved in deploying it and watching its behavior in production. So it follows that the gap between development and production environments should also be made small.
* **The time gap:** A developer may work on code that takes days, weeks, or even months to go into production.
* **The personnel gap**: Developers write code, ops engineers deploy it.
* **The tools gap**: Developers may be using a stack like Nginx, SQLite, and OS X, while the production deploy uses Apache, MySQL, and Linux.
**The twelve-factor app is designed for [continuous deployment](http://www.avc.com/a_vc/2011/02/continuous-deployment.html)** by minimizing all three:
* The time gap is small: a developer may write code and have it deployed hours or even just minutes later.
* The personnel gap is small: developers who wrote code are closely involved in deploying it and watching its behavior in production.
* Thus, the tools gap between development and production environments should also be made small.
Summarizing the above into a table:
<table>
<tr>
@@ -59,8 +69,8 @@ Historically, there is a substantial gap between development (a developer making
Developers sometimes find great appeal in using a lightweight backing service in their local environments, while a more serious and robust backing service will be used in production. For example, using SQLite locally and PostgreSQL in production; or local process memory for caching in development and Memcached in production.
**The twelve-factor developer resists the urge to use different backing services between development and production**, even when adapters theoretically abstract away any differences in backing services. Differences between backing services mean that tiny incompatibilities crop up, causing code that worked and passed tests in development or staging to fail in production. These types of errors, though infrequent, are maddening to debug, and create friction that disincentives continuous deployment. The cost of this frustration and the subsequent dampening of continuous deployment is extremely high, when considered in aggregate over the lifetime of an application.
**The twelve-factor developer resists the urge to use different backing services between development and production**, even when adapters theoretically abstract away any differences in backing services. Differences between backing services mean that tiny incompatibilities crop up, causing code that worked and passed tests in development or staging to fail in production. These types of errors create friction that disincentives continuous deployment. The cost of this friction and the subsequent dampening of continuous deployment is extremely high when considered in aggregate over the lifetime of an application.
The value of lightweight local services is also much lower than it used to be. Modern backing services such as Memcached, PostgreSQL, and RabbitMQ are not difficult to install and run thanks to modern packaging systems, such as [Homebrew](http://mxcl.github.com/homebrew/) and [apt-get](https://help.ubuntu.com/community/AptGet/Howto). The cost of installing and using one of these is low compared to the high benefit of dev/prod parity and continuous deployment.
Lightweight local services are less compelling than they once were. Modern backing services such as Memcached, PostgreSQL, and RabbitMQ are not difficult to install and run thanks to modern packaging systems, such as [Homebrew](http://mxcl.github.com/homebrew/) and [apt-get](https://help.ubuntu.com/community/AptGet/Howto). The cost of installing and using these is low compared to the benefit of dev/prod parity and continuous deployment.
Adapters to different backing services are still useful, because they make porting to new backing services relatively painless. But all deploys of the app (developer environments, staging, production) should be using the same type and version of each of the backing services.
+1 -1
View File
@@ -9,6 +9,6 @@ Processes shut down gracefully when they receive a [SIGTERM](http://en.wikipedia
For a worker process, graceful shutdown is achieved by returning the current job to the work queue. For example, on [RabbitMQ](http://www.rabbitmq.com/) the worker can send a [`NACK`](http://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.nack); on [Beanstalkd](http://kr.github.com/beanstalkd/), the job is returned to the queue automatically whenever a worker disconnects. Lock-based systems such as [Delayed Job](https://github.com/collectiveidea/delayed_job#readme) need to be sure to release their lock on the job record. Implicit in this model is that all jobs are [reentrant](http://en.wikipedia.org/wiki/Reentrant_%28subroutine%29), which typically is achieved by wrapping the results in a transaction, or making the operation [idempotent](http://en.wikipedia.org/wiki/Idempotence).
Processes should also be robust against sudden death, in the case of a failure in the underlying hardware. While this is a much less common occurrence than a graceful shutdown with `SIGTERM`, it can still happen. Use of a robust queueing backend (such as Beanstalkd) that returns jobs to the queue when clients disconnect or time out, can make all the difference here. Either way, a twelve-factor app is architected to handle unexpected, non-graceful terminations. [Crash-only design](http://lwn.net/Articles/191059/) takes this concept to its [logical extreme](http://couchdb.apache.org/docs/overview.html).
Processes should also be robust against sudden death, in the case of a failure in the underlying hardware. While this is a much less common occurrence than a graceful shutdown with `SIGTERM`, it can still happen. A recommended approach is use of a robust queueing backend, such as Beanstalkd, that returns jobs to the queue when clients disconnect or time out. Either way, a twelve-factor app is architected to handle unexpected, non-graceful terminations. [Crash-only design](http://lwn.net/Articles/191059/) takes this concept to its [logical extreme](http://couchdb.apache.org/docs/overview.html).
+5 -1
View File
@@ -9,4 +9,8 @@ A twelve-factor app never concerns itself with routing or storage of its output
In staging or production deploys, each process' stream will be captured by the execution environment, collated together with all other streams from the app, and routed to one or more final destinations for viewing and long-term archival. These archival destinations are not visible to or configurable by the app, and instead are completely managed by the execution environment.
The event stream for an app can be routed to a file, or watched via realtime tail in a terminal. Most significantly, the stream can be sent to into an log indexing and analysis system such as [Splunk](http://www.splunk.com/), or a general-purpose data warehousing system such as [Hadoop/Hive](http://hive.apache.org/). These systems allow for great power and flexibility with introspecting the app's behavior over time: from finding specific events in the past, to large-scale graphing of trends (such as requests per minute), to active alerting according to user-defined heuristics (such as an alert when the quantity of errors per minute exceeds a certain threshold).
The event stream for an app can be routed to a file, or watched via realtime tail in a terminal. Most significantly, the stream can be sent to into an log indexing and analysis system such as [Splunk](http://www.splunk.com/), or a general-purpose data warehousing system such as [Hadoop/Hive](http://hive.apache.org/). These systems allow for great power and flexibility for introspecting an app's behavior over time, including:
* Finding specific events in the past.
* Large-scale graphing of trends (such as requests per minute).
* Active alerting according to user-defined heuristics (such as an alert when the quantity of errors per minute exceeds a certain threshold).
+3 -3
View File
@@ -1,14 +1,14 @@
## VII. Port binding
### Services exported via port binding
Traditionally, web apps are executed inside some kind of runtime container. For example, PHP apps might run as a module inside [Apache HTTPD](http://httpd.apache.org/), or Java apps might run inside [Tomcat](http://tomcat.apache.org/).
Web apps are sometimes executed inside a webserver container. For example, PHP apps might run as a module inside [Apache HTTPD](http://httpd.apache.org/), or Java apps might run inside [Tomcat](http://tomcat.apache.org/).
**The twelve-factor app is completely self-contained** and does not rely on runtime injection of a webserver into the execution environment to create a web-facing service. The web app **exports HTTP as a service by binding to a port**, and listening to requests coming in on that port.
In a local development environment, the developer visits a service URL like `http://localhost:5000/` to access the service exported by their app. In deployment, a routing layer handles routing requests from a public-facing hostname to the port-bound web processes.
This is typically implemented by using [dependency declaration](/dependencies) to add a webserver library to the app, such as [Tornado](http://www.tornadoweb.org/) for Python, [Thin](http://code.macournoyer.com/thin/) for Ruby, or [Jetty](http://jetty.codehaus.org/jetty/) for Java and other JVM-based languages. This happens entirely in [*user space*](http://en.wikipedia.org/wiki/User_space) (within the app's code), and if the developers chose to, they could write app code to accept raw TCP requests and parse the HTTP without any supporting libraries. The contract with the execution environment is binding to a port to serve requests, leaving the implementation details of what webserver is being used up to the web framework and/or the app's developers.
This is typically implemented by using [dependency declaration](/dependencies) to add a webserver library to the app, such as [Tornado](http://www.tornadoweb.org/) for Python, [Thin](http://code.macournoyer.com/thin/) for Ruby, or [Jetty](http://jetty.codehaus.org/jetty/) for Java and other JVM-based languages. This happens entirely in *user space*, that is, within the app's code. The contract with the execution environment is binding to a port to serve requests.
HTTP is not the only service that can be exported by port binding. Nearly any kind of server software can be run via process-binds-to-a-port-to-receive-requests model. To examples include [ejabberd](http://www.ejabberd.im/) (speaking [XMPP](http://xmpp.org/)), and [Redis](http://redis.io/) (speaking the [Redis protocol](http://redis.io/topics/protocol)).
HTTP is not the only service that can be exported by port binding. Nearly any kind of server software can be run via a process binding to a port and awaiting incoming requests. Examples include [ejabberd](http://www.ejabberd.im/) (speaking [XMPP](http://xmpp.org/)), and [Redis](http://redis.io/) (speaking the [Redis protocol](http://redis.io/topics/protocol)).
Note also that the port-binding approach means that one app can become the [backing service](/backing-services) for another app, by providing the URL to the backing app as a resource handle in the [config](/config) for the consuming app.
+2 -2
View File
@@ -7,9 +7,9 @@ In the simplest case, the code is a stand-alone script, the execution environmen
**Twelve-factor processes are stateless and [share-nothing](http://en.wikipedia.org/wiki/Shared_nothing_architecture).** The filesystem sitting beneath the process is either read-only (attempting a write will produce an error), or completely ephemeral (when the process terminates, all state written to disk will be discarded). Any data that needs to persist must be stored in a stateful [backing service](/backing-services), typically a database.
Process memory space and ephemeral filesystem can potentially be used as a brief, single-transaction cache. For example, downloading a large file, operating on it, and storing the results of the operation in the database. The twelve-factor app never assumes that anything cached in memory or on disk will be available on a future request or job -- with many processes of each type running, chances are high that that a future request will be served by a different process. Even when running only one process, a restart (triggered by code deploy, config change, or the execution environment relocating the process to a different physical location) will wipe all filesystem and in-memory state.
Process memory space or an ephemeral filesystem can potentially be used as a brief, single-transaction cache. For example, downloading a large file, operating on it, and storing the results of the operation in the database. The twelve-factor app never assumes that anything cached in memory or on disk will be available on a future request or job -- with many processes of each type running, chances are high that that a future request will be served by a different process. Even when running only one process, a restart (triggered by code deploy, config change, or the execution environment relocating the process to a different physical location) will wipe all filesystem and in-memory state.
Asset packagers (such as [Jammit](http://documentcloud.github.com/jammit/) or [django-assetpackager](http://code.google.com/p/django-assetpackager/)) use the filesystem as a cache for compiled assets. A twelve-factor app is best served by doing this compiling during the [build stage](/build-release-run) rather than at runtime.
Asset packagers (such as [Jammit](http://documentcloud.github.com/jammit/) or [django-assetpackager](http://code.google.com/p/django-assetpackager/)) use the filesystem as a cache for compiled assets. A twelve-factor app prefers to do this compiling during the [build stage](/build-release-run), such as the [Rails asset pipeline](http://ryanbigg.com/guides/asset_pipeline.html), rather than at runtime.
Some web systems rely on "sticky sessions" -- that is, caching user session data in memory of the app's process and expecting future requests from the same visitor to be routed to the same process. Sticky sessions are a violation of twelve-factor and should never be used or relied upon. Session state is a good candidate for a datastore that offers time-expiration, such as [Memcached](http://memcached.org/) or [Redis](http://redis.io/).