Packaging Python Libraries

❝ FIXME ❞
— FIXME

Diving In

So you want to release a Python library. Excellent. The world needs more Python libraries.

Python 3 comes with a packaging framework called Distutils. Distutils is many things: a build tool (for you), an installation tool (for your users), a package metadata format (for search engines), and more. It integrates with the Python Package Index, a central repository for open source Python libraries.

All of these facets of Distutils center around the setup script, traditionally called setup.py. In fact, you’ve already seen a Distutils setup script: you used one to install httplib2 in the “HTTP Web Services” chapter.

In this chapter, we’ll see how the setup script for httplib2 works and step through the process of releasing your own Python library.

# httplib2's setup.py
from distutils.core import setup
VERSION = '0.5.0'
setup(name='httplib2',
      version=VERSION, 
      author='Joe Gregorio',
      author_email='joe@example.com',
      url='http://code.google.com/p/httplib2/',
      download_url='http://httplib2.googlecode.com/files/httplib2-python3-{}.tar.gz'.format(VERSION),
      description='A comprehensive HTTP client library.',
      license='MIT',
      packages=['httplib2'],
      classifiers=[
          'Development Status :: 4 - Beta',
          'Environment :: Web Environment',
          'Intended Audience :: Developers',
          'License :: OSI Approved :: MIT License',
          'Operating System :: OS Independent',
          'Programming Language :: Python',
          'Programming Language :: Python :: 3',
          'Topic :: Internet :: WWW/HTTP',
          'Topic :: Software Development :: Libraries',
      ],
        long_description="""

A comprehensive HTTP client library, ``httplib2`` supports many features left out of other HTTP libraries.

**HTTP and HTTPS**
  HTTPS support is only available if the socket module was compiled with SSL support. 
 

**Keep-Alive**
  Supports HTTP 1.1 Keep-Alive, keeping the socket open and performing multiple requests over the same connection if possible. 


**Authentication**
  The following three types of HTTP Authentication are supported. These can be used over both HTTP and HTTPS.

  * Digest
  * Basic
  * WSSE

**Caching**
  The module can optionally operate with a private cache that understands the Cache-Control: 
  header and uses both the ETag and Last-Modified cache validators. Both file system
  and memcached based caches are supported.


**All Methods**
  The module can handle any HTTP request method, not just GET and POST.


**Redirects**
  Automatically follows 3XX redirects on GETs.


**Compression**
  Handles both 'deflate' and 'gzip' types of compression.


**Lost update support**
  Automatically adds back ETags into PUT requests to resources we have already cached. This implements Section 3.2 of Detecting the Lost Update Problem Using Unreserved Checkout


**Unit Tested**
  A large and growing set of unit tests.
        """
)

☞httplib2 is open source, but there’s no requirement that you release your own Python libraries under any particular license. The process described in this chapter will work for any Python library, regardless of license.

⁂

Things Distutils Can’t Do For You

Releasing your first Python library is a daunting process. (Releasing your second one is easier.) Distutils tries to automate as much of it as possible, but there are some things you simply must do yourself.

Choose a license. This is a complicated topic, fraught with politics and peril. If you wish to release your software as open source, I humbly offer five pieces of advice:
1. Don’t write your own license.
2. Don’t write your own license.
3. Don’t write your own license.
4. It doesn’t need to be GPL, but it needs to be GPL-compatible.
5. Don’t write your own license.
Classify your library using the Trove classification system. I’ll explain what this means later in this chapter.
Write a “read me” file. Don’t skimp on this. At a minimum, it should give your users an overview of what your library does and how to install it.

Directory Structure

To start packaging your Python library, you need to get your files and directories in order. The httplib2 directory looks like this:

httplib2/                 ①
|
+--README.txt             ②
|
+--setup.py               ③
|
+--httplib2/              ④
   |
   +--__init__.py
   |
   +--iri2uri.py

Make a root directory to hold everything. Give it the same name as your Python module.
To accomodate Windows users, your “read me” file should include a .txt extension, and it should use Windows-style carriage returns. Just because you use a fancy text editor that runs from the command line and includes its own macro language, that doesn’t mean you need to make life difficult for your users. (Your users use Notepad. Sad but true.) Even if you’re on Linux or Mac OS X, your fancy text editor undoubtedly has an option to save files with Windows-style carriage returns.
Your Distutils setup script should be named setup.py unless you have a good reason not to. You do not have a good reason not to.
If your Python library is a single .py file, you should put it in the root directory along with your “read me” file and your setup script. If your library is a directory with a main __init__.py script, like httplib2, you should put the entire directory here. Yes, that means you’ll have an httplib2/ directory within an httplib2/ directory. Trust me, that’s not a problem. In fact, any other arrangement would be a problem.

Depending on the license you chose, you may include the license text within your .py files themselves, or you may have a separate file containing license text, or both. GPL-licensed programs generally include a file called COPYING that includes the entire text of the GPL. If you have a separate license file, it should go in the root directory along with your “read me” file and your setup script.

Writing Your Setup Script

The Distutils setup script is a Python script. In theory, it can do anything Python can do. In practice, it should do as little as possible, in as standard a way as possible. Setup scripts should be boring. The more exotic your installation process is, the more exotic your bug reports will be.

The first line of every Distutils setup script is always the same:

from distutils.core import setup

This imports the setup() function, which is the main entry point into Distutils. 95% of all Distutils setup scripts consist of a single call to setup() and nothing else. (I totally just made up that statistic, but if your Distutils setup script is doing more than calling the Distutils setup() function, you should have a good reason.)

...FIXME...required setup() parameters, optional but recommended setup() parameters, always use named parameters, etc.

Packaging Python Libraries

Diving In

Things Distutils Can’t Do For You

Directory Structure

Writing Your Setup Script

Classifying Your Package

Checking Your Setup Script for Errors

Writing Your Setup Configuration File

Creating Source Distributions

Creating Binary Distributions

Building A Windows Installer

Building a Linux RPM Package

The Python Package Index

Registering Yourself

Registering Your Python Library

Uploading New Versions

Further Reading