You are here: Home ‣ Dive Into Python 3 ‣

Difficulty level: ♦♦♦♦♦

Special Method Names

❝ My specialty is being right when other people are wrong. ❞
— George Bernard Shaw

Diving in

FIXME

Basics

If you’ve read the introduction to classes, you’ve already seen the most common special method: the __init__() method. The majority of classes I write end up needing some initialization.

1. The __init__() method is called after the instance is created. If you want to control the actual creation process, use the __new__() method.
2. By convention, the __repr__() method should return a string that is a valid Python expression.
3. The __str__() method is also called when you print(x).
4. New in Python 3, since the bytes type was introduced.
5. By convention, format_spec should conform to the Format Specification Mini-Language.

Classes That Act Like Iterators

In the Iterators chapter, you saw how to build an iterator from the ground up using the __iter__() and __next__() methods.

1. The __iter__() method is called whenever you create a new iterator. It’s a good place to initialize the iterator with initial values.
2. The __next__() method is called whenever you retrieve the next value from an iterator.
3. The __reversed__() method is uncommon. It takes an existing sequence and returns an iterator that yields the items in the sequence in reverse order, from last to first.

Computed Attributes

FIXME not sure of the wording/depth required here because I don't yet know if I'm going to cover these in a previous chapter. Let's assume I'm not, and I can move the examples later if need be.

1. If your class defines a __getattribute__() method, Python will call it on every reference to any attribute or method name (except special method names, since that would cause an unpleasant infinite loop).
2. If your class defines a __getattr__() method, Python will call it only after looking for the attribute in all the normal places. If an instance x defines an attribute color, x.color will not call x.__getattr__("color"); it will simply return the already-defined value of x.color.
3. The __setattr__() method is called whenever you assign a value to an attribute.
4. The __delattr__() method is called whenever you delete an attribute.
5. The __dir__() method is useful if you define a __getattr__() or __getattribute__() method. Normally, calling dir(x) would only list the regular attributes and methods. If your __getattr()__ method handles a color attribute dynamically, dir(x) would not list color as one of the available attributes. Overriding the __dir__() method allows you to list color as an available attribute, which is helpful for other people who wish to use your class without digging into the internals of it.

The distinction between the __getattr__() and __getattribute__() methods is subtle but important. I can explain it with two examples:

>>> class Dynamo:
...   def __getattr__(self, key):
...     if key == "color":         ①
...       return "PapayaWhip"
...     else:
...       raise AttributeError     ②
... 
>>> dyn = Dynamo()
>>> dyn.color                      ③
'PapayaWhip'
>>> dyn.color = "LemonChiffon"
>>> dyn.color                      ④
'LemonChiffon'

1. The attribute name is passed into the __getattr()__ method as a string. If the name is "color", the method returns a value. (In this case, it’s just a hard-coded string, but you would normally do some sort of computation and return the result.)
2. If the attribute name is unknown, the __getattr()__ method needs to raise an AttributeError exception, otherwise your code will silently fail when accessing undefined attributes. (Technically, if the method doesn’t raise an exception or explicitly return a value, it returns None, the Python null value. This means that all attributes not explicitly defined will be None, which is almost certainly not what you want.)
3. The dyn instance does not have an attribute named color, so the __getattr__() method is called to provide a computed value.
4. After explicitly setting dyn.color, the __getattr__() method will no longer be called to provide a value for dyn.color, because dyn.color is already defined on the instance.

On the other hand, the __getattribute__() method is absolute and unconditional.

>>> class SuperDynamo:
...   def __getattribute__(self, key):
...     if key == 'color':
...       return "PapayaWhip"
...     else:
...       raise AttributeError
... 
>>> dyn = SuperDynamo()
>>> dyn.color                      ①
"PapayaWhip"
>>> dyn.color = "LemonChiffon"
>>> dyn.color                      ②
"PapayaWhip"

1. The __getattribute__() method is called to provide a value for dyn.color.
2. Even after explicitly setting dyn.color, the __getattribute__() method is still called to provide a value for dyn.color. If present, the __getattribute__() method is called unconditionally for every attribute and method lookup, even for attributes that you explicitly set after creating an instance.

☞If your class defines a __getattribute__() method, you probably also want to define a __setattr__() method and coordinate between them to keep track of attribute values. Otherwise, any attributes you set after creating an instance will disappear into a black hole.

Classes That Act Like Functions

FIXME

Classes That Act Like Sequences

FIXME sequence intro

Classes That Act Like Dictionaries

FIXME

Classes That Act Like Numbers

Using the appropriate special methods, you can define your own classes that act like numbers. That is, you can add them, subtract them, and perform other mathematical operations on them. This is how fractions are implemented — the Fraction class implements these special methods, then you can do things like this:

>>> from fractions import Fraction
>>> x = Fraction(1, 3)
>>> x / 3
Fraction(1, 9)

Here is the comprehensive list of special methods you need to implement a number-like class.

That’s all well and good if x is an instance of a class that implements those methods. But what if it doesn’t implement one of them? Or worse, what if it implements it, but it can’t handle certain kinds of arguments? For example:

>>> from fractions import Fraction
>>> x = Fraction(1, 3)
>>> 1 / x
Fraction(3, 1)

This is not a case of taking a Fraction and dividing it by an integer (as in the previous example). That case was straightforward: x / 3 calls x.__truediv__(3), and the __truediv__() method of the Fraction class handles all the math. But integers don’t “know” how to do arithmetic operations with fractions. So why does this example work?

The answer lies in a second set of arithmetic special methods with reflected operands. Given an arithmetic operation that takes two operands (e.g. x / y), there are two ways to go about it:

1. Tell x to divide itself by y, or
2. Tell y to divide itself into x

The set of special methods above take the first approach: given x / y, they provide a way for x to say “I know how to divide myself by y.” The following set of special methods tackle the second approach: they provide a way for y to say “I know how to be the denominator and divide myself into x.”

But wait! There’s more! If you’re doing “in-place” operations, like x /= 3, there are even more special methods you can define.

Note: for the most part, the in-place operation methods are not required. If you don’t define an in-place method for a particular operation, Python will try the methods. For example, to execute the expression x /= y, Python will:

1. Try calling x.__itruediv__(y). If this method is defined and returns a value other than NotImplemented, we’re done.
2. Try calling x.__truediv__(y). If this method is defined and returns a value other than NotImplemented, the old value of x is discarded and replaced with the return value, just as if you had done x = x / y instead.
3. Try calling y.__rtruediv__(x). If this method is defined and returns a value other than NotImplemented, the old value of x is discarded and replaced with the return value.

So you only need to define in-place methods like the __itruediv__() method if you want to do some special optimization for in-place operands. Otherwise Python will essentially reformulate the in-place operand to use a regular operand + a variable assignment.

There are also a few “unary” mathematical operations you can perform on number-like objects by themselves.

Classes That Can Be Compared

I broke this section out from the previous one because comparisons are not strictly the purview of numbers. Many datatypes can be compared — strings, lists, even dictionaries. If you’re creating your own class and it makes sense to compare your objects to other objects, you can use the following special methods to implement comparisons.

Classes That Can Be Pickled

see http://docs.python.org/3.0/library/pickle.html:

__copy__ (*) - covered in fractions.py
__deepcopy__ (*) - covered in fractions.py
__getnewargs__ (*)
__getinitargs__ (*)
__getstate__ (*)
__setstate__ (*)
__reduce__ (*) - covered in ordereddict.py, fractions.py
__reduce_ex__ (*)

Classes That Can Be Used in a with Block

Python 3 supports the with statement, which allows you to access an object’s properties and methods without explicitly referencing the object every time. A with block defines a runtime context; you “enter” the context when you execute the with statement, and you “exit” the context after you execute the last statement in the block.

Any class can be used in a with block; no special methods are required. The Python interpreter will automatically set up the runtime context and dispatch all the property and method lookups to your class. However, if you want your class to do something special upon entering or exiting a runtime context, you can define the following special methods.

This is how the [FIXME-xref] with file idiom works.

# excerpt from io.py:
def _checkClosed(self, msg=None):
    """Internal: raise an ValueError if file is closed
    """
    if self.closed:
        raise ValueError("I/O operation on closed file."
                         if msg is None else msg)

def __enter__(self) -> "IOBase":
    """Context management protocol.  Returns self."""
    self._checkClosed()                                ①
    return self                                        ②

def __exit__(self, *args) -> None:
    """Context management protocol.  Calls close()"""
    self.close()                                       ③

1. The file object defines both an __enter__() and an __exit__() method. The __enter__() method checks that the file is open; if it’s not, the _checkClosed() method raises an exception.
2. The __enter__() method should almost always return self — this is the object that the with block will use to dispatch properties and methods.
3. After the with block, the file object automatically closes. How? In the __exit__() method, it calls self.close().

☞The __exit__() method will always be called, even if an exception is raised inside the with block. In fact, if an exception is raises, the exception information will be passed to the __exit__() method. See With Statement Context Managers for more details.

Really Esoteric Stuff

__new__ - covered in fractions.py
__del__
__slots__
__hash__ - covered in fractions.py
__get__
__set__
__delete__
__subclasshook__ (*) see http://docs.python.org/3.0/library/abc.html
__instancecheck__ (*) see http://www.ibm.com/developerworks/linux/library/l-python3-2/
__subclasscheck__ (*)

☜ ☞

© 2001–9 Mark Pilgrim