mirror of
https://github.com/kennethreitz/dive-into-python3.git
synced 2026-06-05 15:00:18 +00:00
Mark Pilgrim
249 lines
13 KiB
HTML
249 lines
13 KiB
HTML
<!DOCTYPE html>
|
|
<head>
|
|
<meta charset=utf-8>
|
|
<title>Packaging Python Libraries - Dive into Python 3</title>
|
|
<!--[if IE]><script src=j/html5.js></script><![endif]-->
|
|
<link rel=stylesheet href=dip3.css>
|
|
<style>
|
|
body{counter-reset:h1 16}
|
|
</style>
|
|
<link rel=stylesheet media='only screen and (max-device-width: 480px)' href=mobile.css>
|
|
<link rel=stylesheet media=print href=print.css>
|
|
<meta name=viewport content='initial-scale=1.0'>
|
|
</head>
|
|
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
|
|
<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#packaging>Dive Into Python 3</a> <span class=u>‣</span>
|
|
<p id=level>Difficulty level: <span class=u title=advanced>♦♦♦♦♢</span>
|
|
<h1>Packaging Python Libraries</h1>
|
|
<blockquote class=q>
|
|
<p><span class=u>❝</span> FIXME <span class=u>❞</span><br>— FIXME
|
|
</blockquote>
|
|
<p id=toc>
|
|
<h2 id=divingin>Diving In</h2>
|
|
<p class=f>So you want to release a Python library. Excellent. The world needs more Python libraries.
|
|
|
|
<p>Python 3 comes with a packaging framework called Distutils. Distutils is many things: a build tool (for you), an installation tool (for your users), a package metadata format (for search engines), and more. It integrates with the <a href=http://pypi.python.org/>Python Package Index</a>, a central repository for open source Python libraries.
|
|
|
|
<p>All of these facets of Distutils center around the <i>setup script</i>, traditionally called <code>setup.py</code>. In fact, you’ve already seen a Distutils setup script: <a href=http-web-services.html#introducing-httplib2>you used one to install <code>httplib2</code></a> in the “HTTP Web Services” chapter.
|
|
|
|
<p>In this chapter, we’ll see how the setup script for <a href=http://code.google.com/p/httplib2/><code>httplib2</code></a> works and step through the process of releasing your own Python library.
|
|
|
|
<pre><code class=pp># httplib2's setup.py
|
|
from distutils.core import setup
|
|
VERSION = '0.5.0'
|
|
setup(name='httplib2',
|
|
version=VERSION,
|
|
author='Joe Gregorio',
|
|
author_email='joe@example.com',
|
|
url='http://code.google.com/p/httplib2/',
|
|
download_url='http://httplib2.googlecode.com/files/httplib2-python3-{}.tar.gz'.format(VERSION),
|
|
description='A comprehensive HTTP client library.',
|
|
license='MIT',
|
|
packages=['httplib2'],
|
|
classifiers=[
|
|
'Development Status :: 4 - Beta',
|
|
'Environment :: Web Environment',
|
|
'Intended Audience :: Developers',
|
|
'License :: OSI Approved :: MIT License',
|
|
'Operating System :: OS Independent',
|
|
'Programming Language :: Python',
|
|
'Programming Language :: Python :: 3',
|
|
'Topic :: Internet :: WWW/HTTP',
|
|
'Topic :: Software Development :: Libraries',
|
|
],
|
|
long_description="""
|
|
|
|
A comprehensive HTTP client library, ``httplib2`` supports many features left out of other HTTP libraries.
|
|
|
|
**HTTP and HTTPS**
|
|
HTTPS support is only available if the socket module was compiled with SSL support.
|
|
|
|
|
|
**Keep-Alive**
|
|
Supports HTTP 1.1 Keep-Alive, keeping the socket open and performing multiple requests over the same connection if possible.
|
|
|
|
|
|
**Authentication**
|
|
The following three types of HTTP Authentication are supported. These can be used over both HTTP and HTTPS.
|
|
|
|
* Digest
|
|
* Basic
|
|
* WSSE
|
|
|
|
**Caching**
|
|
The module can optionally operate with a private cache that understands the Cache-Control:
|
|
header and uses both the ETag and Last-Modified cache validators. Both file system
|
|
and memcached based caches are supported.
|
|
|
|
|
|
**All Methods**
|
|
The module can handle any HTTP request method, not just GET and POST.
|
|
|
|
|
|
**Redirects**
|
|
Automatically follows 3XX redirects on GETs.
|
|
|
|
|
|
**Compression**
|
|
Handles both 'deflate' and 'gzip' types of compression.
|
|
|
|
|
|
**Lost update support**
|
|
Automatically adds back ETags into PUT requests to resources we have already cached. This implements Section 3.2 of Detecting the Lost Update Problem Using Unreserved Checkout
|
|
|
|
|
|
**Unit Tested**
|
|
A large and growing set of unit tests.
|
|
"""
|
|
)</code></pre>
|
|
|
|
<blockquote class=note>
|
|
<p><span class=u>☞</span><code>httplib2</code> is open source, but there’s no requirement that you release your own Python libraries under any particular license. The process described in this chapter will work for any Python library, regardless of license.
|
|
</blockquote>
|
|
|
|
<p class=a>⁂
|
|
|
|
<h2 id=cantdo>Things Distutils Can’t Do For You</h2>
|
|
|
|
<p>Releasing your first Python library is a daunting process. (Releasing your second one is easier.) Distutils tries to automate as much of it as possible, but there are some things you simply must do yourself.
|
|
|
|
<ul>
|
|
<li><b>Choose a license</b>. This is a complicated topic, fraught with politics and peril. If you wish to release your software as open source, I humbly offer five pieces of advice:
|
|
<ol>
|
|
<li>Don’t write your own license.
|
|
<li>Don’t write your own license.
|
|
<li>Don’t write your own license.
|
|
<li>It doesn’t need to be <abbr>GPL</abbr>, but <a href=http://www.dwheeler.com/essays/gpl-compatible.html>it needs to be <abbr>GPL</abbr>-compatible</a>.
|
|
<li>Don’t write your own license.
|
|
</ol>
|
|
<li><b>Classify your library</b> using the Trove classification system. I’ll explain what this means later in this chapter.
|
|
<li><b>Write a “read me” file</b>. Don’t skimp on this. At a minimum, it should give your users an overview of what your library does and how to install it.
|
|
</ul>
|
|
|
|
<h2 id=structure>Directory Structure</h2>
|
|
|
|
<p>To start packaging your Python library, you need to get your files and directories in order. The <code>httplib2</code> directory looks like this:
|
|
|
|
<pre class=screen>
|
|
<a>httplib2/ <span class=u>①</span></a>
|
|
|
|
|
<a>+--README.txt <span class=u>②</span></a>
|
|
|
|
|
<a>+--setup.py <span class=u>③</span></a>
|
|
|
|
|
<a>+--httplib2/ <span class=u>④</span></a>
|
|
|
|
|
+--__init__.py
|
|
|
|
|
+--iri2uri.py</pre>
|
|
<ol>
|
|
<li>Make a root directory to hold everything. Give it the same name as your Python module.
|
|
<li>To accomodate Windows users, your “read me” file should include a <code>.txt</code> extension, and it should use Windows-style carriage returns. Just because <em>you</em> use a fancy text editor that runs from the command line and includes its own macro language, that doesn’t mean you need to make life difficult for your users. (Your users use Notepad. Sad but true.) Even if you’re on Linux or Mac OS X, your fancy text editor undoubtedly has an option to save files with Windows-style carriage returns.
|
|
<li>Your Distutils setup script should be named <code>setup.py</code> unless you have a good reason not to. You do not have a good reason not to.
|
|
<li>If your Python library is a single <code>.py</code> file, you should put it in the root directory along with your “read me” file and your setup script. If your library is a directory with a main <code>__init__.py</code> script, like <code>httplib2</code>, you should put the entire directory here. Yes, that means you’ll have an <code>httplib2/</code> directory within an <code>httplib2/</code> directory. Trust me, that’s not a problem. In fact, any other arrangement would be a problem.
|
|
</ol>
|
|
|
|
<p>Depending on the license you chose, you may include the license text within your <code>.py</code> files themselves, or you may have a separate file containing license text, or both. <abbr>GPL</abbr>-licensed programs generally include a file called <code>COPYING</code> that includes the entire text of the <abbr>GPL</abbr>. If you have a separate license file, it should go in the root directory along with your “read me” file and your setup script.
|
|
|
|
<h2 id=setuppy>Writing Your Setup Script</h2>
|
|
|
|
<p>The Distutils setup script is a Python script. In theory, it can do anything Python can do. In practice, it should do as little as possible, in as standard a way as possible. Setup scripts should be boring. The more exotic your installation process is, the more exotic your bug reports will be.
|
|
|
|
<p>The first line of every Distutils setup script is always the same:
|
|
|
|
<pre class=nd><code class=pp>from distutils.core import setup</code></pre>
|
|
|
|
<p>This imports the <code>setup()</code> function, which is the main entry point into Distutils. 95% of all Distutils setup scripts consist of a single call to <code>setup()</code> and nothing else. (I totally just made up that statistic, but if your Distutils setup script is doing more than calling the Distutils <code>setup()</code> function, you should have a good reason.)
|
|
|
|
<p>...FIXME...required setup() parameters, optional but recommended setup() parameters, always use named parameters, etc.
|
|
|
|
<h3 id=trove>Classifying Your Package</h3>
|
|
|
|
<h3 id=check>Checking Your Setup Script for Errors</h3>
|
|
|
|
<h2 id=setupcfg>Writing Your Setup Configuration File</h2>
|
|
|
|
<h2 id=sdist>Creating Source Distributions</h2>
|
|
|
|
<!-- archive formats -->
|
|
<!-- manifest -->
|
|
|
|
<h2 id=bdist>Creating Binary Distributions</h2>
|
|
|
|
<!-- python3 setup.py bdist --help-formats -->
|
|
|
|
<h3 id=wininst>Building A Windows Installer</h3>
|
|
|
|
<!-- 32-bit vs. 64-bit -->
|
|
<!-- UAC -->
|
|
|
|
<h3 id=rpm>Building a Linux RPM Package</h3>
|
|
|
|
<h2 id=pypi>The Python Package Index</h2>
|
|
|
|
<h3 id=register-user>Registering Yourself</h3>
|
|
|
|
<h3 id=register-package>Registering Your Python Library</h3>
|
|
|
|
<h3 id=upload>Uploading New Versions</h3>
|
|
|
|
<!--
|
|
@jessenoller sez:
|
|
* distutils, how to make a setup.py (and include data files, tests, docs, etc for a project)
|
|
* how to upload it to pypi properly (label it for python 3 for the love of pete)
|
|
* how to build build bdist/RPMs/DEBs.
|
|
* If you can - and I've forgotten how much distutils supports of this, cover dependency management.
|
|
* Oh, I almost forgot - cover the Per-User Site packages stuff, PEP 370
|
|
-->
|
|
|
|
<h2 id=future>The Many Possible Futures of Python Packaging</h2>
|
|
|
|
<p>Distutils is not the be-all and end-all of Python packaging, but as of this writing (August 2009), it’s the only one that works in Python 3. There are a number of other frameworks for Python 2; some focus on installation, others on testing and deployment. Some or all of these may end up being ported to Python 3 in the future.
|
|
|
|
<p>These frameworks focus on installation:
|
|
|
|
<ul>
|
|
<li><a href=http://pypi.python.org/pypi/setuptools>Setuptools</a>
|
|
<li><a href=http://pypi.python.org/pypi/pip>Pip</a>
|
|
<li><a href=http://bitbucket.org/tarek/distribute/src/tip/README.txt>Distribute</a>
|
|
</ul>
|
|
|
|
<p>These focus on testing and deployment:
|
|
|
|
<ul>
|
|
<li><a href=http://pypi.python.org/pypi/virtualenv><code>virtualenv</code></a>
|
|
<li><a href=http://pypi.python.org/pypi/zc.buildout><code>zc.buildout</code></a>
|
|
<li><a href=http://www.blueskyonmars.com/projects/paver/>Paver</a>
|
|
<li><a href=http://fabfile.org/>Fabric</a>
|
|
<li><a href=http://www.py2exe.org/><code>py2exe</code></a>
|
|
</ul>
|
|
|
|
<h2 id=furtherreading>Further Reading</h2>
|
|
|
|
<p>On Distutils:
|
|
|
|
<ul>
|
|
<li><a href=http://docs.python.org/3.1/distutils/>Distributing Python Modules with Distutils</a>
|
|
<li><a href=http://docs.python.org/3.1/distutils/apiref.html#module-distutils.core>Core Distutils functionality</a> lists all the possible arguments to the <code>setup()</code> function
|
|
<li><a href=http://wiki.python.org/moin/Distutils/Cookbook>Distutils Cookbook</a>
|
|
<li><a href=http://www.python.org/dev/peps/pep-0370/><abbr>PEP</abbr> 370: Per user <code>site-packages</code> directory</a>
|
|
<li><a href=http://jessenoller.com/2009/07/19/pep-370-per-user-site-packages-and-environment-stew/><abbr>PEP</abbr> 370 and “environment stew”</a>
|
|
</ul>
|
|
|
|
<p>On other packaging frameworks:
|
|
|
|
<ul>
|
|
<li><a href=http://groups.google.com/group/django-developers/msg/5407cdb400157259>The Python packaging ecosystem</a>
|
|
<li><a href=http://www.b-list.org/weblog/2008/dec/14/packaging/>On packaging</a>
|
|
<li><a href=http://blog.ianbicking.org/2008/12/14/a-few-corrections-to-on-packaging/>A few corrections to “On packaging”</a>
|
|
<li><a href=http://www.b-list.org/weblog/2008/dec/15/pip/>Why I like Pip</a>
|
|
<li><a href=http://cournape.wordpress.com/2009/04/01/python-packaging-a-few-observations-cabal-for-a-solution/>Python packaging: a few observations</a>
|
|
<li><a href=http://jacobian.org/writing/nobody-expects-python-packaging/>Nobody expects Python packaging!</a>
|
|
</ul>
|
|
|
|
<p class=v><a rel=prev href=multiprocessing.html title='back to “Threading & Multiprocessing”'><span class=u>☜</span></a> <a rel=next href=case-study-porting-chardet-to-python-3.html title='onward to “Case Study: Porting chardet to Python 3”'><span class=u>☞</span></a>
|
|
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
|
|
<script src=j/jquery.js></script>
|
|
<script src=j/prettify.js></script>
|
|
<script src=j/dip3.js></script>
|