From 45506d1f39be8cc055814227952e53d94d3cb10d Mon Sep 17 00:00:00 2001 From: Cory Benfield Date: Thu, 17 May 2012 10:16:42 +0100 Subject: [PATCH 1/3] Rewrite quickstart docs. This should help with issue #503. --- docs/user/quickstart.rst | 225 ++++++++++++++++++--------------------- 1 file changed, 102 insertions(+), 123 deletions(-) diff --git a/docs/user/quickstart.rst b/docs/user/quickstart.rst index bdb6635c..36275362 100644 --- a/docs/user/quickstart.rst +++ b/docs/user/quickstart.rst @@ -15,53 +15,71 @@ First, make sure that: * Requests is :ref:`up-to-date ` -Lets gets started with some simple use cases and examples. +Let's get started with some simple examples. -Make a GET Request +Make a Request ------------------ -Making a standard request with Requests is very simple. +Making a request with Requests is very simple. -Let's get GitHub's public timeline :: +Begin by importing the Requests module:: + + >>> import requests - r = requests.get('https://github.com/timeline.json') +Now, let's try to get a webpage. For this example, let's get GitHub's public +timeline :: + + >>> r = requests.get('https://github.com/timeline.json') Now, we have a :class:`Response` object called ``r``. We can get all the -information we need from this. +information we need from this object. -Typically, you want to send some sort of data in the urls query string. -To do this, simply pass a dictionary to the `params` argument. Your -dictionary of data will automatically be encoded when the request is made:: +Requests' simple API means that all forms of HTTP request are as obvious. For +example, this is how you make an HTTP POST request:: + + >>> r = requests.post("http://httpbin.org/post") + +Nice, right? What about the other HTTP request types: PUT, DELETE, HEAD and +OPTIONS? These are all just as simple:: + + >>> r = requests.put("http://httpbin.org/put") + >>> r = requests.delete("http://httpbin.org/delete") + >>> r = requests.head("http://httpbin.org/get") + >>> r = requests.options("http://httpbin.org/get") + +That's all well and good, but it's also only the start of what Requests can +do. + + +Passing Parameters In URLs +-------------------------- + +You often want to send some sort of data in the URL's query string. If +you were constructing the URL by hand, this data would be given as key/value +pairs in the URL after a question mark, e.g. ``httpbin.org/get?key=val``. +Requests allows you to provide these arguments as a dictionary, using the +``params`` keyword argument. As an example, if you wanted to pass +``key1=value1`` and ``key2=value2`` to ``httpbin.org/get``, you would use the +following code:: >>> payload = {'key1': 'value1', 'key2': 'value2'} >>> r = requests.get("http://httpbin.org/get", params=payload) - >>> print r.text - { - "origin": "179.13.100.4", - "args": { - "key2": "value2", - "key1": "value1" - }, - "url": "http://httpbin.org/get", - "headers": { - "Connections": "keep-alive", - "Content-Length": "", - "Accept-Encoding": "identity, deflate, compress, gzip", - "Accept": "*/*", - "User-Agent": "python-requests/0.11.0", - "Host": httpbin.org", - "Content-Type": "" - }, - } +You can see that the URL has been correctly encoded by printing the URL:: + >>> print r.url + u'http://httpbin.org/get?key2=value2&key1=value1' + Response Content ---------------- -We can read the content of the server's response:: +We can read the content of the server's response. Consider the GitHub timeline +again:: + >>> import requests + >>> r = requests.get('https://github.com/timeline.json') >>> r.text '[{"repository":{"open_issues":0,"url":"https://github.com/... @@ -85,7 +103,7 @@ You can also access the response body as bytes, for non-text requests:: The ``gzip`` and ``deflate`` transfer-encodings are automatically decoded for you. -For example to create an image from binary data returned by a request, you can +For example, to create an image from binary data returned by a request, you can use the following code: >>> from PIL import Image @@ -106,14 +124,24 @@ you can access ``r.raw``:: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03' +Custom Headers +-------------- -Make a POST Request -------------------- +If you'd like to add HTTP headers to a request, simply pass in a ``dict`` to the +``headers`` parameter. -POST requests are equally simple:: +For example, we didn't specify our content-type in the previous example:: - r = requests.post("http://httpbin.org/post") + >>> import json + >>> url = 'https://api.github.com/some/endpoint' + >>> payload = {'some': 'data'} + >>> headers = {'content-type': 'application/json'} + >>> r = requests.post(url, data=json.dumps(payload), headers=headers) + + +More complicated POST requests +------------------------------ Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the `data` argument. Your @@ -123,48 +151,23 @@ dictionary of data will automatically be form-encoded when the request is made:: >>> r = requests.post("http://httpbin.org/post", data=payload) >>> print r.text { - "origin": "179.13.100.4", - "files": {}, + // ...snip... // "form": { "key2": "value2", "key1": "value1" }, - "url": "http://httpbin.org/post", - "args": {}, - "headers": { - "Content-Length": "23", - "Accept-Encoding": "identity, deflate, compress, gzip", - "Accept": "*/*", - "User-Agent": "python-requests/0.8.0", - "Host": "127.0.0.1:7077", - "Content-Type": "application/x-www-form-urlencoded" - }, - "data": "" + // ...snip... // } There are many times that you want to send data that is not form-encoded. If you pass in a ``string`` instead of a ``dict``, that data will be posted directly. For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data:: - url = 'https://api.github.com/some/endpoint' - payload = {'some': 'data'} + >>> import json + >>> url = 'https://api.github.com/some/endpoint' + >>> payload = {'some': 'data'} - r = requests.post(url, data=json.dumps(payload)) - - -Custom Headers --------------- - -If you'd like to add HTTP headers to a request, simply pass in a ``dict`` to the -``headers`` parameter. - -For example, we didn't specify our content-type in the previous example:: - - url = 'https://api.github.com/some/endpoint' - payload = {'some': 'data'} - headers = {'content-type': 'application/json'} - - r = requests.post(url, data=json.dumps(payload), headers=headers) + >>> r = requests.post(url, data=json.dumps(payload)) POST a Multipart-Encoded File @@ -178,25 +181,14 @@ Requests makes it simple to upload Multipart-encoded files:: >>> r = requests.post(url, files=files) >>> r.text { - "origin": "179.13.100.4", + // ...snip... // "files": { "report.xls": "" }, - "form": {}, - "url": "http://httpbin.org/post", - "args": {}, - "headers": { - "Content-Length": "3196", - "Accept-Encoding": "identity, deflate, compress, gzip", - "Accept": "*/*", - "User-Agent": "python-requests/0.8.0", - "Host": "httpbin.org:80", - "Content-Type": "multipart/form-data; boundary=127.0.0.1.502.21746.1321131593.786.1" - }, - "data": "" + // ...snip... // } -Setting filename explicitly:: +You can set the filename explicitly:: >>> url = 'http://httpbin.org/post' >>> files = {'file': ('report.xls', open('report.xls', 'rb'))} @@ -204,25 +196,14 @@ Setting filename explicitly:: >>> r = requests.post(url, files=files) >>> r.text { - "origin": "179.13.100.4", + // ...snip... // "files": { "file": "" }, - "form": {}, - "url": "http://httpbin.org/post", - "args": {}, - "headers": { - "Content-Length": "3196", - "Accept-Encoding": "identity, deflate, compress, gzip", - "Accept": "*/*", - "User-Agent": "python-requests/0.8.0", - "Host": "httpbin.org:80", - "Content-Type": "multipart/form-data; boundary=127.0.0.1.502.21746.1321131593.786.1" - }, - "data": "" + // ...snip... // } -Sending strings to be received as files:: +If you want, you can send strings to be received as files:: >>> url = 'http://httpbin.org/post' >>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')} @@ -230,24 +211,11 @@ Sending strings to be received as files:: >>> r = requests.post(url, files=files) >>> r.text { - "origin": "179.13.100.4", + // ...snip... // "files": { "file": "some,data,to,send\\nanother,row,to,send\\n" }, - "form": {}, - "url": "http://httpbin.org/post", - "args": {}, - "headers": { - "Content-Length": "216", - "Accept-Encoding": "identity, deflate, compress, gzip", - "Connection": "keep-alive", - "Accept": "*/*", - "User-Agent": "python-requests/0.11.1", - "Host": "httpbin.org", - "Content-Type": "multipart/form-data; boundary=127.0.0.1.502.41433.1335385481.788.1" - }, - "json": null, - "data": "" + // ...snip... // } @@ -256,6 +224,7 @@ Response Status Codes We can check the response status code:: + >>> r = requests.get("http://httpbin.org/get') >>> r.status_code 200 @@ -278,7 +247,8 @@ If we made a bad request (non-200 response), we can raise it with raise self.error urllib2.HTTPError: HTTP Error 404: NOT FOUND -But, since our ``status_code`` was ``200``, when we call it:: +But, since our ``status_code`` for ``r`` was ``200``, when we call +``raise_for_status()`` we get:: >>> r.raise_for_status() None @@ -289,8 +259,7 @@ All is well. Response Headers ---------------- -We can view the server's response headers with a simple Python dictionary -interface:: +We can view the server's response headers using a Python dictionary:: >>> r.headers { @@ -347,7 +316,7 @@ parameter:: Basic Authentication -------------------- -Most web services require authentication. There many different types of +Many web services require authentication. There many different types of authentication, but the most common is HTTP Basic Auth. Making requests with Basic Auth is extremely simple:: @@ -380,16 +349,20 @@ Another popular form of web service protection is Digest Authentication:: OAuth Authentication -------------------- -Miguel Araujo's `requests-oauth `_ project provides a simple interface for -establishing OAuth connections. Documentation and examples can be found on the requests-oauth `git repository `_. +Miguel Araujo's `requests-oauth `_ +project provides a simple interface for establishing OAuth connections. +Documentation and examples can be found on the requests-oauth +`git repository `_. Redirection and History ----------------------- -Requests will automatically perform location redirection while using idempotent methods. +Requests will automatically perform location redirection while using +idempotent methods, such as GET. -GitHub redirects all HTTP requests to HTTPS. Let's see what happens:: +GitHub redirects all HTTP requests to HTTPS. We can use the ``history`` method +of the Response object to track redirection. Let's see what Github does:: >>> r = requests.get('http://github.com') >>> r.url @@ -411,7 +384,8 @@ handling with the ``allow_redirects`` parameter:: >>> r.history [] -If you're using POST, PUT, PATCH, *&c*, you can also explicitly enable redirection as well:: +If you're using POST, PUT, PATCH, *&c*, you can also explicitly enable +redirection as well:: >>> r = requests.post('http://github.com', allow_redirects=True) >>> r.url @@ -423,16 +397,18 @@ If you're using POST, PUT, PATCH, *&c*, you can also explicitly enable redirecti Timeouts -------- -You can tell requests to stop waiting for a response after a given number of seconds with the ``timeout`` parameter:: +You can tell requests to stop waiting for a response after a given number of +seconds with the ``timeout`` parameter:: >>> requests.get('http://github.com', timeout=0.001) Traceback (most recent call last): File "", line 1, in requests.exceptions.Timeout: Request timed out. -.. admonition:: Note +.. admonition:: Note: - ``timeout`` only effects the connection process itself, not the downloading of the response body. + ``timeout`` only effects the connection process itself, not the + downloading of the response body. Errors and Exceptions @@ -446,14 +422,17 @@ an :class:`HTTPError` exception. If a request times out, a :class:`Timeout` exception is raised. -If a request exceeds the configured number of maximum redirections, a :class:`TooManyRedirects` exception is raised. +If a request exceeds the configured number of maximum redirections, a +:class:`TooManyRedirects` exception is raised. All exceptions that Requests explicitly raises inherit from :class:`requests.exceptions.RequestException`. -You can refer to :ref:`Configuration API Docs ` for immediate raising of :class:`HTTPError` exceptions -via the ``danger_mode`` option or have Requests catch the majority of :class:`requests.exceptions.RequestException` exceptions -with the ``safe_mode`` option. +You can refer to :ref:`Configuration API Docs ` for immediate +raising of :class:`HTTPError` exceptions via the ``danger_mode`` option or +have Requests catch the majority of +:class:`requests.exceptions.RequestException` exceptions with the ``safe_mode`` +option. ----------------------- From 36fd8da1c07090c25a386dbc0c8ba22213e50f93 Mon Sep 17 00:00:00 2001 From: Cory Benfield Date: Thu, 17 May 2012 19:25:35 +0100 Subject: [PATCH 2/3] Correctly reflect redirection behaviour. The docs incorrectly listed HEAD as a method that follows redirects by default: it does not. This commit resolves this issue, and thus resolves issue #504. --- docs/user/quickstart.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user/quickstart.rst b/docs/user/quickstart.rst index 36275362..775970e7 100644 --- a/docs/user/quickstart.rst +++ b/docs/user/quickstart.rst @@ -358,8 +358,8 @@ Documentation and examples can be found on the requests-oauth Redirection and History ----------------------- -Requests will automatically perform location redirection while using -idempotent methods, such as GET. +Requests will automatically perform location redirection while using the GET +and OPTIONS verbs. GitHub redirects all HTTP requests to HTTPS. We can use the ``history`` method of the Response object to track redirection. Let's see what Github does:: @@ -375,8 +375,8 @@ of the Response object to track redirection. Let's see what Github does:: The :class:`Response.history` list contains a list of the :class:`Request` objects that were created in order to complete the request. -If you're using GET, HEAD, or OPTIONS, you can disable redirection -handling with the ``allow_redirects`` parameter:: +If you're using GET or OPTIONS, you can disable redirection handling with the +``allow_redirects`` parameter:: >>> r = requests.get('http://github.com', allow_redirects=False) >>> r.status_code @@ -384,7 +384,7 @@ handling with the ``allow_redirects`` parameter:: >>> r.history [] -If you're using POST, PUT, PATCH, *&c*, you can also explicitly enable +If you're using POST, PUT, PATCH, DELETE or HEAD, you can enable redirection as well:: >>> r = requests.post('http://github.com', allow_redirects=True) From 2b8e01bff7f8331b09c4ec599e24e57c10b1ad39 Mon Sep 17 00:00:00 2001 From: Cory Benfield Date: Sat, 19 May 2012 21:34:36 +0100 Subject: [PATCH 3/3] First draft of full verbs. --- docs/user/advanced.rst | 156 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 156 insertions(+) diff --git a/docs/user/advanced.rst b/docs/user/advanced.rst index cee2d4b9..677ef05a 100644 --- a/docs/user/advanced.rst +++ b/docs/user/advanced.rst @@ -319,3 +319,159 @@ You can also configure proxies by environment variables ``HTTP_PROXY`` and ``HTT $ python >>> import requests >>> requests.get("http://example.org") + +HTTP Verbs +---------- + +Requests provides access to almost the full range of HTTP verbs: GET, OPTIONS, +HEAD, POST, PUT, PATCH and DELETE. The following provides detailed examples of +using these various verbs in Requests, using the GitHub API. + +We will begin with the verb most commonly used: GET. HTTP GET is an idempotent +method that returns a resource from a given URL. As a result, it is the verb +you ought to use when attempting to retrieve data from a web location. An +example usage would be attempting to get information about a specific commit +from GitHub. Suppose we wanted commit ``a050faf`` on Requests. We would get it +like so:: + + >>> import requests + >>> r = requests.get('https://api.github.com/repos/kennethreitz/requests/git/commits/a050faf084662f3a352dd1a941f2c7c9f886d4ad') + +We should confirm that GitHub responded correctly. If it has, we want to work +out what type of content it is. Do this like so:: + + >>> if (r.status_code == requests.codes.ok): + ... print r.headers['content-type'] + ... + application/json; charset=utf-8 + +So, GitHub returns JSON. That's great, we can use the JSON module to turn it +into Python objects. Because GitHub returned UTF-8, we should use the +``r.text`` method, not the ``r.content`` method. ``r.content`` returns a +bytestring, while ``r.text`` returns a Unicode-encoded string. I have no plans +to perform byte-manipulation on this response, so I want any Unicode code +points encoded.:: + + >>> import json + >>> commit_data = json.loads(r.text) + >>> print commit_data.keys() + [u'committer', u'author', u'url', u'tree', u'sha', u'parents', u'message'] + >>> print commit_data[u'committer'] + {u'date': u'2012-05-10T11:10:50-07:00', u'email': u'me@kennethreitz.com', u'name': u'Kenneth Reitz'} + >>> print commit_data[u'message'] + makin' history + +So far, so simple. Well, let's investigate the GitHub API a little bit. Now, +we could look at the documentation, but we might have a little more fun if we +use Requests instead. We can take advantage of the Requests OPTIONS verb to +see what kinds of HTTP methods are supported on the url we just used.:: + + >>> verbs = requests.options(r.url) + >>> verbs.status_code + 500 + +Uh, what? That's unhelpful! Turns out GitHub, like many API providers, don't +actually implement the OPTIONS method. This is an annoying oversight, but it's +OK, we can just use the boring documentation. If GitHub had correctly +implemented OPTIONS, however, they should return the allowed methods in the +headers, e.g.:: + + >>> verbs = requests.options('http://a-good-website.com/api/cats') + >>> print verbs.headers['allow'] + GET,HEAD,POST,OPTIONS + +Turning to the documentation, we see that the only other method allowed for +commits is POST, which creates a new commit. As we're using the Requests repo, +we should probably avoid making ham-handed POSTS to it. Instead, let's play +with the Issues feature of GitHub. + +This documentation was added in response to Issue #482. Given that this issue +already exists, we will use it as an example. Let's start by getting it.:: + + >>> r = requests.get('https://api.github.com/repos/kennethreitz/requests/issues/482') + >>> r.status_code + 200 + >>> issue = json.loads(r.text) + >>> print issue[u'title'] + Feature any http verb in docs + >>> print issue[u'comments'] + 3 + +Cool, we have three comments. Let's take a look at the last of them.:: + + >>> r = requests.get(r.url + u'/comments') + >>> r.status_code + 200 + >>> comments = json.loads(r.text) + >>> print comments[0].keys() + [u'body', u'url', u'created_at', u'updated_at', u'user', u'id'] + >>> print comments[2][u'body'] + Probably in the "advanced" section + +Well, that seems like a silly place. Let's post a comment telling the poster +that he's silly. Who is the poster, anyway?:: + + >>> print comments[2][u'user'][u'login'] + kennethreitz + +OK, so let's tell this Kenneth guy that we think this example should go in the +quickstart guide instead. According to the GitHub API doc, the way to do this +is to POST to the thread. Let's do it.:: + + >>> body = json.dumps({u"body": u"Sounds great! I'll get right on it!"}) + >>> url = u"https://api.github.com/repos/kennethreitz/requests/issues/482/comments" + >>> r = requests.post(url=url, data=body) + >>> r.status_code + 404 + +Huh, that's weird. We probably need to authenticate. That'll be a pain, right? +Wrong. Requests makes it easy to use many forms of authentication, including +the very common Basic Auth.:: + + >>> from requests.auth import HTTPBasicAuth + >>> auth = HTTPBasicAuth('fake@example.com', 'not_a_real_password') + >>> r = requests.post(url=url, data=body, auth=auth) + >>> r.status_code + 201 + >>> content = json.loads(r.text) + >>> print content[u'body'] + Sounds great! I'll get right on it. + +Brilliant. Oh, wait, no! I meant to add that it would take me a while, because +I had to go feed my cat. If only I could edit this comment! Happily, GitHub +allows us to use another HTTP verb, PATCH, to edit this comment. Let's do +that.:: + + >>> print content[u"id"] + 5804413 + >>> body = json.dumps({u"body": u"Sounds great! I'll get right on it once I feed my cat."}) + >>> url = u"https://api.github.com/repos/kennethreitz/requests/issues/comments/5804413" + >>> r = requests.patch(url=url, data=body, auth=auth) + >>> r.status_code + 200 + +Excellent. Now, just to torture this Kenneth guy, I've decided to let him +sweat and not tell him that I'm working on this. That means I want to delete +this comment. GitHub lets us delete comments using the incredibly aptly named +DELETE method. Let's get rid of it.:: + + >>> r = requests.delete(url=url, auth=auth) + >>> r.status_code + 204 + >>> r.headers['status'] + '204 No Content' + +Excellent. All gone. The last thing I want to know is how much of my ratelimit +I've used. Let's find out. GitHub sends that information in the headers, so +rather than download the whole page I'll send a HEAD request to get the +headers.:: + + >>> r = requests.head(url=url, auth=auth) + >>> print r.headers + // ...snip... // + 'x-ratelimit-remaining': '4995' + 'x-ratelimit-limit': '5000' + // ...snip... // + +Excellent. Time to write a Python program that abuses the GitHub API in all +kinds of exciting ways, 4995 more times.