diff --git a/about.html b/about.html index 9db0863..79ec50b 100644 --- a/about.html +++ b/about.html @@ -12,15 +12,15 @@ h1:before{content:""}

You are here: Home ‣ Dive Into Python 3 ‣

About the book

The text of Dive Into Python 3 is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License. -

The chardet library referenced in Case study: porting chardet to Python 3 is licensed under the LGPL 2.1 or later. All other example code is licensed under the MIT license. Full licensing terms are included in each source code file. -

The dynamic highlighting effects in the online edition are built on top of jQuery, which is dual-licensed under the MIT and GPL licenses. +

The chardet library referenced in Case study: porting chardet to Python 3 is licensed under the LGPL 2.1 or later. The alphametics solver referenced in Advanced Iterators is based on Raymond Hettinger's solver for Python 2, which he has graciously relicensed under the MIT license so I could port it to Python 3. Advanced Classes and Special Method Names contain snippets of code from the Python standard library which are released under the Python Software Foundation License version 2. All other example code is my original work and is licensed under the MIT license. Full licensing terms are included in each source code file. +

The dynamic highlighting effects in the online edition are built on top of jQuery, which is dual-licensed under the MIT and GPL licenses.

The online edition loads as quickly as it does because

  1. jQuery is served by Google AJAX Libraries API.
  2. Other Javascript and CSS resources are minimized by YUI Compressor.
  3. HTTP caching and other server-side options are optimized based on advice from YSlow. -
  4. The text uses Unicode characters in place of graphics wherever possible. -
  5. The entire book was lovingly hand-authored in HTML 5 to avoid markup cruft. +
  6. The text uses Unicode characters in place of graphics wherever possible. +
  7. The entire book was lovingly hand-authored in HTML 5 to avoid markup cruft.

Send corrections and feedback to mark@diveintomark.org.

© 2001–9 Mark Pilgrim diff --git a/special-method-names.html b/special-method-names.html index 7f3f231..d5da949 100644 --- a/special-method-names.html +++ b/special-method-names.html @@ -29,11 +29,11 @@ td a:link, td a:visited{border:0}

 

Diving in

-

FIXME +

We’ve already covered a few special method names elsewhere in this book — “magic” methods that Python invokes when you use certain syntax. Using special methods, your classes can act like sequences, like dictionaries, like functions, like iterators, or even like numbers! This appendix serves both as a reference for the special methods we’ve seen already and a brief introduction to some of the more esoteric ones.

Basics

-

If you’ve read the introduction to classes, you’ve already seen the most common special method: the __init__() method. The majority of classes I write end up needing some initialization. +

If you’ve read the introduction to classes, you’ve already seen the most common special method: the __init__() method. The majority of classes I write end up needing some initialization. There are also a few other basic special methods that are especially useful for debugging your custom classes.
Notes @@ -66,7 +66,7 @@ td a:link, td a:visited{border:0}
  • By convention, the __repr__() method should return a string that is a valid Python expression.
  • The __str__() method is also called when you print(x).
  • New in Python 3, since the bytes type was introduced. -
  • By convention, format_spec should conform to the Format Specification Mini-Language. +
  • By convention, format_spec should conform to the Format Specification Mini-Language. decimal.py in the Python standard library provides its own __format__() method.

    Classes That Act Like Iterators

    @@ -138,13 +138,13 @@ td a:link, td a:visited{border:0}

    The distinction between the __getattr__() and __getattribute__() methods is subtle but important. I can explain it with two examples: 

    ->>> class Dynamo:
-...   def __getattr__(self, key):
-...     if key == "color":         ①
-...       return "PapayaWhip"
-...     else:
-...       raise AttributeError     ②
-... 
+class Dynamo:
+    def __getattr__(self, key):
+        if key == "color":         ①
+            return "PapayaWhip"
+        else:
+            raise AttributeError   ②
+
 >>> dyn = Dynamo()
 >>> dyn.color                      ③
 'PapayaWhip'
@@ -161,13 +161,13 @@ td a:link, td a:visited{border:0}
 
    On the other hand, the __getattribute__() method is absolute and unconditional.
 
 
    ->>> class SuperDynamo:
-...   def __getattribute__(self, key):
-...     if key == 'color':
-...       return "PapayaWhip"
-...     else:
-...       raise AttributeError
-... 
+class SuperDynamo:
+    def __getattribute__(self, key):
+        if key == 'color':
+            return "PapayaWhip"
+        else:
+            raise AttributeError
+
 >>> dyn = SuperDynamo()
 >>> dyn.color                      ①
 "PapayaWhip"
@@ -183,9 +183,29 @@ td a:link, td a:visited{border:0}
 
    ☞If your class defines a __getattribute__() method, you probably also want to define a __setattr__() method and coordinate between them to keep track of attribute values. Otherwise, any attributes you set after creating an instance will disappear into a black hole.
 
 
+
    You need to be extra careful with the __getattribute__() method, because it is also called when Python looks up a method name on your class.
+
+
    +class Rastan:
+    def __getattribute__(self, key):
+        raise AttributeError           ①
+    def swim(self):
+        pass
+
+>>> hero = Rastan()
+>>> hero.swim()                        ②
+Traceback (most recent call last):
+  File "<stdin>", line 1, in <module>
+  File "<stdin>", line 3, in __getattribute__
+AttributeError
    
+
      
+
    1. This class defines a __getattribute__() method which always raises an AttributeError exception. No attribute or method lookups will succeed.
+
    2. When you call hero.swim(), Python looks for a swim() method in the Rastan class. This lookup goes through the __getattribute__() method, because all attribute and method lookups go through the __getattribute__() method. In this case, the __getattribute__() method raises an AttributeError exception, so the method lookup fails, so the method call fails.
+
    
+
 
    Classes That Act Like Functions
    
 
-
    FIXME
+
    You can make an instance of a class callable — exactly like a function is callable — by defining the __call__() method.
 
 
    
 
    Notes
@@ -198,9 +218,43 @@ td a:link, td a:visited{border:0}
 my_instance.__call__()
 
    
 
+
    The zipfile module uses this to define a class that can decrypt an encrypted zip file with a given password. The zip decryption algorithm requires you to store state during decryption. Defining the decryptor as a class allows you to maintain this state within a single instance of the decryptor class.  The state is initialized in the __init__() method and updated as the file is decrypted. But since the class is also “callable” like a function, you can pass the instance as the first argument of the map() function, like so:
+
+
    
+# excerpt from zipfile.py
+class _ZipDecrypter:
+.
+.
+.
+    def __init__(self, pwd):
+        self.key0 = 305419896               ①
+        self.key1 = 591751049
+        self.key2 = 878082192
+        for p in pwd:
+            self._UpdateKeys(p)
+
+    def __call__(self, c):                  ②
+        assert isinstance(c, int)
+        k = self.key2 | 2
+        c = c ^ (((k * (k^1)) >> 8) & 255)
+        self._UpdateKeys(c)
+        return c
+.
+.
+.
+zd = _ZipDecrypter(pwd)                    ③
+bytes = zef_file.read(12)
+h = list(map(zd, bytes[0:12]))             ④
    
+
      
+
    1. The _ZipDecryptor class maintains state in the form of three rotating keys, which are later updated in the _UpdateKeys() method (not shown here).
+
    2. The class defines a __call__() method, which makes class instances callable like functions. In this case, the __call__() method decrypts a single byte of the zip file, then updates the rotating keys based on the byte that was decrypted.
+
    3. zd is an instance of the _ZipDecryptor class. The pwd variable is passed to the __init__() method, where it is stored and used to update the rotating keys for the first time.
+
    4. Given the first 12 bytes of a zip file, decrypt them by mapping the bytes to zd, in effect “calling” zd 12 times, which invokes the __call__() method 12 times, which updates its internal state and returns a resulting byte 12 times.
+
    
+
 
    Classes That Act Like Sequences
    
 
-
    FIXME sequence intro
+
    If your class acts as a container for a set of values — that is, if it makes sense to ask whether your class “contains” a value — then it should probably define the following special methods that make it act like a sequence.
 
 
    
 
    Notes
@@ -217,9 +271,37 @@ td a:link, td a:visited{border:0}
 seq.__contains__(x)
 
    
 
+
    The cgi module uses these methods in its FieldStorage class, which represents all of the form fields or query parameters submitted to a dynamic web page.
+
+
    
+# A script which responds to http://example.com/search?q=cgi
+import cgi
+fs = cgi.FieldStorage()
+if "q" in fs:                                               ①
+  do_search()
+
+# An excerpt from cgi.py that explains how that works
+class FieldStorage:
+.
+.
+.
+    def __contains__(self, key):                            ②
+        if self.list is None:
+            raise TypeError("not indexable")
+        return any(item.name == key for item in self.list)  ③
+
+    def __len__(self):                                      ④
+        return len(self.keys())
    
+
      
+
    1. Once you create an instance of the cgi.FieldStorage class, you can use the “in” operator to check whether a particular parameter was included in the query string.
+
    2. The __contains__() method is the magic that makes this work.
+
    3. When you say if "q" in fs, Python looks for the __contains__() method on the fs object, which is defined in cgi.py. The value "q" is passed into the __contains__() method as the key argument.
+
    4. The same FieldStorage class also supports returning its length, so you can say len(fs) and it will call the __len__() method on the FieldStorage class to return the number of query parameters that it identified.
+
    
+
 
    Classes That Act Like Dictionaries
    
 
-
    FIXME
+
    Extending the previous section a bit, you can define classes that not only respond to the “in” operator and the len() function, but they act like full-blown dictionaries, returning values based on keys.
 
 
    
 
    Notes
@@ -244,6 +326,37 @@ td a:link, td a:visited{border:0}
 x.__missing__("nonexistent_key")
 
    
 
+
    The FieldStorage class from the cgi module also defines these special methods, which means you can do things like this:
+
+
    
+# A script which responds to http://example.com/search?q=cgi
+import cgi
+fs = cgi.FieldStorage()
+if "q" in fs:
+  do_search(fs["q"])                              ①
+
+# An excerpt from cgi.py that shows how it works
+class FieldStorage:
+.
+.
+.
+    def __getitem__(self, key):                   ②
+        if self.list is None:
+            raise TypeError("not indexable")
+        found = []
+        for item in self.list:
+            if item.name == key: found.append(item)
+        if not found:
+            raise KeyError(key)
+        if len(found) == 1:
+            return found[0]
+        else:
+            return found
    
+
      
+
    1. The fs object is an instance of cgi.FieldStorage, but you can still evaluate expressions like fs["q"].
+
    2. fs["q"] invokes the __getitem__() method with the key parameter set to "q". It then looks up in its internally maintained list of query parameters (self.list) for an item whose .name matches the given key.
+
    
+
 
    Classes That Act Like Numbers
    
 
 
    Using the appropriate special methods, you can define your own classes that act like numbers. That is, you can add them, subtract them, and perform other mathematical operations on them. This is how fractions are implemented — the Fraction class implements these special methods, then you can do things like this:
@@ -515,9 +628,9 @@ td a:link, td a:visited{border:0}
    		• truncate x to nearest integer toward 0 math.trunc(x) x.__trunc__() -
    -??? FIXME what the hell is this? -??? +
    PEP 357 +number as a list index +a_list[x] x.__index__()
    @@ -560,20 +673,45 @@ td a:link, td a:visited{border:0} x.__bool__() -

    Classes That Can Be Pickled

    +

    Classes That Can Be Serialized

    + -
    -see http://docs.python.org/3.0/library/pickle.html:
+
    Python supports serializing and unserializing arbitrary objects. (Most Python references call this process “pickling” and “unpickling.”) This can be useful for saving state to a file and restoring it later. All of the native datatypes support pickling already. If you create a custom class that you to be able to pickle, read up on the pickle protocol to see when and how the following special methods are called.
 
-__copy__ (*) - covered in fractions.py
-__deepcopy__ (*) - covered in fractions.py
-__getnewargs__ (*)
-__getinitargs__ (*)
-__getstate__ (*)
-__setstate__ (*)
-__reduce__ (*) - covered in ordereddict.py, fractions.py
-__reduce_ex__ (*)
-
    + +
    Notes +You Want… +So You Write… +And Python Calls… +
    +a custom object copy +copy.copy(x) +x.__copy__() +
    +a custom object deepcopy +copy.deepcopy(x) +x.__deepcopy__() +
    +to get an object’s state before pickling +pickle.dump(x, file) +x.__getstate__() +
    +to serialize an object +pickle.dump(x, file) +x.__reduce__() +
    +to serialize an object (new pickling protocol) +pickle.dump(x, file, protocol_version) +x.__reduce_ex__(protocol_version) +
    +control over how an object is created during unpickling +x = pickle.load(file) +x.__getnewargs__() +
    +to restore an object’s state after unpickling +x = pickle.load(file) +x.__setstate__() +

    Classes That Can Be Used in a with Block

    @@ -626,18 +764,54 @@ def __exit__(self, *args) -> None:

    Really Esoteric Stuff

    -
    -__new__ - covered in fractions.py
-__del__
-__slots__
-__hash__ - covered in fractions.py
-__get__
-__set__
-__delete__
-__subclasshook__ (*) see http://docs.python.org/3.0/library/abc.html
-__instancecheck__ (*) see http://www.ibm.com/developerworks/linux/library/l-python3-2/
-__subclasscheck__ (*)
-
    +

    If you know what you’re doing, you can gain almost complete control over how classes are compared, how attributes are defined, and what kinds of classes are considered subclasses of your class. + + +
    Notes +You Want… +So You Write… +And Python Calls… +
    +a class constructor +x = MyClass() +x.__new__() +
    +a class destructor +del x +x.__del__() +
    +only a specific set of attributes to be defined + +x.__slots__() +
    +a custom hash value +hash(x) +x.__hash__() +
    +to get an attribute’s value +x.color +type(x).__dict__['color'].__get__(x, type(x)) +
    +to set an attribute’s value +x.color = 'PapayaWhip' +type(x).__dict__['color'].__set__(x, 'PapayaWhip') +
    +to delete an attribute +del x.color +type(x).__dict__['color'].__del__(x) +
    +to control whether an object is an instance of your class +isinstance(x, MyClass) +MyClass.__instancecheck__(x) +
    +to control whether a class is a subclass of your class +issubclass(C, MyClass) +MyClass.__subclasscheck__(C) +
    +to control whether a class is a subclass of your abstract base class +issubclass(C, MyABC) +MyABC.__subclasshook__(C) +

    © 2001–9 Mark Pilgrim diff --git a/table-of-contents.html b/table-of-contents.html index db1b74c..e9d621a 100644 --- a/table-of-contents.html +++ b/table-of-contents.html @@ -135,29 +135,28 @@ ul li ol{margin:0;padding:0 0 0 2.5em}

  • Iterators
      -
    1. Iterators -
        -
      1. A Fibonacci iterator -
      2. A plural rule iterator -
      -
    2. Further reading +
    3. Diving In +
    4. Defining Classes +
    5. Instantiating Classes +
    6. Instance Variables +
    7. A Fibonacci iterator +
    8. A Plural Rule Iterator +
    9. Further Reading
  • Advanced Iterators
      -
    1. Diving In -
    2. Finding all occurrences of a pattern -
    3. Finding the unique items in a sequence -
    4. Making assertions -
    5. Generator expressions -
    6. Calculating Permutations… The Lazy Way! -
    7. Other Fun Stuff in the itertools Module -
    8. A New Kind Of String Manipulation -
    9. Evaluating Arbitrary Strings As Python Expressions -
    10. Putting It All Together -
    11. Further Reading -
    12. Objects and object-orientation -
        -
      1. ...major changes afoot... +
      2. Diving In +
      3. Finding all occurrences of a pattern +
      4. Finding the unique items in a sequence +
      5. Making assertions +
      6. Generator expressions +
      7. Calculating Permutations… The Lazy Way! +
      8. Other Fun Stuff in the itertools Module +
      9. A New Kind Of String Manipulation +
      10. Evaluating Arbitrary Strings As Python Expressions +
      11. Putting It All Together +
      12. Further Reading +
    13. Unit testing
      1. (Not) diving in @@ -170,13 +169,12 @@ ul li ol{margin:0;padding:0 0 0 2.5em}
        1. ...
        -
      2. Refactoring your code +
      3. Refactoring your code
          -
        1. Handling bugs -
        2. Handling changing requirements -
        3. The art of refactoring -
        4. Postscript -
        5. Summary +
        6. Diving in +
        7. Handling changing requirements +
        8. Refactoring +
        9. Summary
      4. Advanced Classes
          @@ -367,14 +365,14 @@ ul li ol{margin:0;padding:0 0 0 2.5em}
          1. Diving in
          2. Basics -
          3. Rich Comparisons -
          4. Custom Attributes +
          5. Classes That Act Like Iterators +
          6. Computed Attributes
          7. Classes That Act Like Functions
          8. Classes That Act Like Sequences
          9. Classes That Act Like Dictionaries -
          10. Classes That Act Like Iterators
          11. Classes That Act Like Numbers -
          12. Support For Pickling +
          13. Classes That Can Be Compared +
          14. Classes That Can Be Serialized
          15. Classes That Can Be Used in a with Block
          16. Really Esoteric Stuff