You are here: Home ‣ Dive Into Python 3 ‣ +
You are here: Home ‣ Dive Into Python 3 ‣
The text of Dive Into Python 3 is licensed under the Creative Commons Attribution-ShareAlike 3.0 Unported License.
The chardet library referenced in Case study: porting chardet to Python 3 is licensed under the LGPL 2.1 or later. The alphametics solver referenced in Advanced Iterators is based on Raymond Hettinger's solver for Python 2, which he has graciously relicensed under the MIT license so I could port it to Python 3. Advanced Classes and Special Method Names contain snippets of code from the Python standard library which are released under the Python Software Foundation License version 2. All other example code is my original work and is licensed under the MIT license. Full licensing terms are included in each source code file.
diff --git a/advanced-classes.html b/advanced-classes.html
index eff6d0a..7b5f163 100644
--- a/advanced-classes.html
+++ b/advanced-classes.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 11}
You are here: Home ‣ Dive Into Python 3 ‣ +
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♦♢
-❝ FIXME ❞
— FIXME +❝ FIXME ❞
— FIXME
[FIXME here's why ordered dicts are useful: http://www.gossamer-threads.com/lists/python/dev/656556 ]
import collections
+import collections
import itertools
class OrderedDict(dict, collections.MutableMapping):
@@ -107,7 +107,7 @@ class OrderedDict(dict, collections.MutableMapping):
>>> import ordereddict
>>> od = ordereddict.OrderedDict()
->>> klass = od.__class__ ①
+>>> klass = od.__class__ ①
>>> type(klass)
<class 'abc.ABCMeta'>
>>> klass.__name__
@@ -163,7 +163,8 @@ class OrderedDict(dict, collections.MutableMapping):
Implementing Fractions
-© 2001–9 Mark Pilgrim
+
diff --git a/advanced-iterators.html b/advanced-iterators.html
index 2a2729b..10b2529 100644
--- a/advanced-iterators.html
+++ b/advanced-iterators.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 7}
-You are here: Home ‣ Dive Into Python 3 ‣
+
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♦♢
Advanced Iterators
-❝ Great fleas have little fleas upon their backs to bite ’em,
And little fleas have lesser fleas, and so ad infinitum. ❞
— Augustus De Morgan
+
❝ Great fleas have little fleas upon their backs to bite ’em,
And little fleas have lesser fleas, and so ad infinitum. ❞
— Augustus De Morgan
Diving In
@@ -44,7 +44,7 @@ E = 4
In this chapter, we’ll dive into an incredible Python program originally written by Raymond Hettinger. This program solves alphametic puzzles in just 14 lines of code.
import re
+import re
import itertools
def solve(puzzle):
@@ -91,9 +91,9 @@ if __name__ == '__main__':
>>> import re
->>> re.findall('[0-9]+', '16 2-by-4s in rows of 8') ①
+>>> re.findall('[0-9]+', '16 2-by-4s in rows of 8') ①
['16', '2', '4', '8']
->>> re.findall('[A-Z]+', 'SEND + MORE == MONEY') ②
+>>> re.findall('[A-Z]+', 'SEND + MORE == MONEY') ②
['SEND', 'MORE', 'MONEY']
- The
re module is Python’s implementation of regular expressions. It has a nifty function called findall() which takes a regular expression pattern and a string, and finds all occurrences of the pattern within the string. In this case, the pattern matches sequences of numbers. The findall() function returns a list of all the substrings that matched the pattern.
@@ -108,15 +108,15 @@ if __name__ == '__main__':
>>> a_list = ['a', 'c', 'b', 'a', 'd', 'b']
->>> {c for c in a_list} ①
+>>> {c for c in a_list} ①
{'a', 'c', 'b', 'd'}
>>> a_string = 'EAST IS EAST'
->>> {c for c in a_string} ②
+>>> {c for c in a_string} ②
{'A', ' ', 'E', 'I', 'S', 'T'}
>>> words = ['SEND', 'MORE', 'MONEY']
->>> ''.join(words) ③
+>>> ''.join(words) ③
'SENDMOREMONEY'
->>> {c for c in ''.join(words)} ④
+>>> {c for c in ''.join(words)} ④
{'E', 'D', 'M', 'O', 'N', 'S', 'R', 'Y'}
- Given a list of several strings, a set comprehension with the identity function will return a set of unique strings from the list. This makes sense if you think of it like a
for loop. Take the first item from the list, put it in the set. Second. Third. Fourth — wait, that’s in the set already, so it only gets listed once. Fifth. Sixth — again, a duplicate, so it only gets listed once. The end result? All the unique items in the original list, without any duplicates. The original list doesn’t even need to be sorted first.
@@ -127,7 +127,7 @@ if __name__ == '__main__':
The alphametics solver uses this technique to get a list of all the unique characters in the puzzle.
-
unique_characters = {c for c in ''.join(words)}
+unique_characters = {c for c in ''.join(words)}
This list is later used to assign digits to characters as the solver iterates through the possible solutions.
@@ -138,8 +138,8 @@ if __name__ == '__main__':
Like many programming languages, Python has an assert statement. Here’s how it works.
->>> assert 1 + 1 == 2 ①
->>> assert 1 + 1 == 3 ②
+>>> assert 1 + 1 == 2 ①
+>>> assert 1 + 1 == 3 ②
Traceback (most recent call last):
File "<stdin>", line 1, in
AssertionError
@@ -150,11 +150,11 @@ AssertionError
Therefore, this line of code:
-
assert len(unique_characters) <= 10
+assert len(unique_characters) <= 10
…is equivalent to…
-
if len(unique_characters) > 10:
+if len(unique_characters) > 10:
raise AssertionError
But a bit easier to read and write.
@@ -169,14 +169,14 @@ AssertionError
>>> unique_characters = {'E', 'D', 'M', 'O', 'N', 'S', 'R', 'Y'}
->>> gen = (ord(c) for c in unique_characters) ①
->>> gen ②
+>>> gen = (ord(c) for c in unique_characters) ①
+>>> gen ②
<generator object <genexpr> at 0x00BADC10>
->>> next(gen) ③
+>>> next(gen) ③
69
>>> next(gen)
68
->>> tuple(ord(c) for c in unique_characters) ④
+>>> tuple(ord(c) for c in unique_characters) ④
(69, 68, 77, 79, 78, 83, 82, 89)
- A generator expression is like an anonymous function that yields values. The expression itself looks like a list comprehension [FIXME have we introduced this yet?], but it’s wrapped in parentheses instead of square brackets.
@@ -187,7 +187,7 @@ AssertionError
Here’s another way to accomplish the same thing, using a generator function:
-
def ord_map(a_string):
+def ord_map(a_string):
for c in a_string:
yield ord(c)
@@ -202,21 +202,21 @@ gen = ord_map(unique_characters)
The idea is that you take a list of things (could be numbers, could be letters, could be dancing bears) and find all the possible ways to split them up into smaller lists. All the smaller lists have the same size, which can be as small as 1 and as large as the total number of items. Oh, and nothing can be repeated. Mathematicians say things like “let’s find the permutations of 3 different items taken 2 at a time,” which means you have a sequence of 3 items and you want to find all the possible ordered pairs.
->>> import itertools ①
->>> perms = itertools.permutations([1, 2, 3], 2) ②
->>> next(perms) ③
+>>> import itertools ①
+>>> perms = itertools.permutations([1, 2, 3], 2) ②
+>>> next(perms) ③
(1, 2)
>>> next(perms)
(1, 3)
>>> next(perms)
-(2, 1) ④
+(2, 1) ④
>>> next(perms)
(2, 3)
>>> next(perms)
(3, 1)
>>> next(perms)
(3, 2)
->>> next(perms) ⑤
+>>> next(perms) ⑤
Traceback (most recent call last):
File "<stdin>", line 1, in
StopIteration
@@ -232,9 +232,9 @@ StopIteration
>>> import itertools
->>> perms = itertools.permutations('ABC', 3) ①
+>>> perms = itertools.permutations('ABC', 3) ①
>>> next(perms)
-('A', 'B', 'C') ②
+('A', 'B', 'C') ②
>>> next(perms)
('A', 'C', 'B')
>>> next(perms)
@@ -249,7 +249,7 @@ StopIteration
Traceback (most recent call last):
File "<stdin>", line 1, in
StopIteration
->>> list(itertools.permutations('ABC', 3)) ③
+>>> list(itertools.permutations('ABC', 3)) ③
[('A', 'B', 'C'), ('A', 'C', 'B'),
('B', 'A', 'C'), ('B', 'C', 'A'),
('C', 'A', 'B'), ('C', 'B', 'A')]itertools Module>>> import itertools ->>> list(itertools.product('ABC', '123')) ① +>>> list(itertools.product('ABC', '123')) ① [('A', '1'), ('A', '2'), ('A', '3'), ('B', '1'), ('B', '2'), ('B', '3'), ('C', '1'), ('C', '2'), ('C', '3')] ->>> list(itertools.combinations('ABC', 2)) ② +>>> list(itertools.combinations('ABC', 2)) ② [('A', 'B'), ('A', 'C'), ('B', 'C')]
itertools.product() function returns an iterator containing the Cartesian product of two sequences.
@@ -277,19 +277,19 @@ StopIteration
[download favorite-people.txt]
->>> names = list(open('examples/favorite-people.txt')) ① +>>> names = list(open('examples/favorite-people.txt')) ① >>> names ['Dora\n', 'Ethan\n', 'Wesley\n', 'John\n', 'Anne\n', 'Mike\n', 'Chris\n', 'Sarah\n', 'Alex\n', 'Lizzie\n'] ->>> names = [name.rstrip() for name in names] ② +>>> names = [name.rstrip() for name in names] ② >>> names ['Dora', 'Ethan', 'Wesley', 'John', 'Anne', 'Mike', 'Chris', 'Sarah', 'Alex', 'Lizzie'] ->>> names = sorted(names) ③ +>>> names = sorted(names) ③ >>> names ['Alex', 'Anne', 'Chris', 'Dora', 'Ethan', 'John', 'Lizzie', 'Mike', 'Sarah', 'Wesley'] ->>> names = sorted(names, key=len) ④ +>>> names = sorted(names, key=len) ④ >>> names ['Alex', 'Anne', 'Dora', 'John', 'Mike', 'Chris', 'Ethan', 'Sarah', 'Lizzie', 'Wesley']@@ -305,7 +305,7 @@ StopIteration
[0, 1, 2] >>> list(range(10, 13)) [10, 11, 12] ->>> list(itertools.chain(range(0, 3), range(10, 13))) ① +>>> list(itertools.chain(range(0, 3), range(10, 13))) ① [0, 1, 2, 10, 11, 12] ->>> list(zip(range(0, 3), range(10, 13))) ② +>>> list(zip(range(0, 3), range(10, 13))) ② [(0, 10), (1, 11), (2, 12)] ->>> list(zip(range(0, 3), range(10, 14))) ③ +>>> list(zip(range(0, 3), range(10, 14))) ③ [(0, 10), (1, 11), (2, 12)] ->>> list(itertools.zip_longest(range(0, 3), range(10, 14))) ④ +>>> list(itertools.zip_longest(range(0, 3), range(10, 14))) ④ [(0, 10), (1, 11), (2, 12), (None, 13)]…continuing from the previous interactive shell… >>> import itertools ->>> groups = itertools.groupby(names, len) ① +>>> groups = itertools.groupby(names, len) ① >>> groups <itertools.groupby object at 0x00BB20C0> >>> list(groups) @@ -313,7 +313,7 @@ StopIteration (5, <itertools._grouper object at 0x00BB4050>), (6, <itertools._grouper object at 0x00BB4030>)] >>> groups = itertools.groupby(names, len) ->>> for name_length, name_iter in groups: ② +>>> for name_length, name_iter in groups: ② ... print('Names with {0:d} letters:'.format(name_length)) ... for name in name_iter: ... print(name) @@ -342,13 +342,13 @@ Wesley
itertools.chain() function takes two iterators and returns an iterator that contains all the items from the first iterator, followed by all the items from the second iterator. (Actually, it can take any number of iterators, and it chains them all in the order they were passed to the function.)
@@ -362,10 +362,10 @@ Wesley
>>> characters = ('S', 'M', 'E', 'D', 'O', 'N', 'R', 'Y')
>>> guess = ('1', '2', '0', '3', '4', '5', '6', '7')
->>> tuple(zip(characters, guess)) ①
+>>> tuple(zip(characters, guess)) ①
(('S', '1'), ('M', '2'), ('E', '0'), ('D', '3'),
('O', '4'), ('N', '5'), ('R', '6'), ('Y', '7'))
->>> dict(zip(characters, guess)) ②
+>>> dict(zip(characters, guess)) ②
{'E': '0', 'D': '3', 'M': '2', 'O': '4',
'N': '5', 'S': '1', 'R': '6', 'Y': '7'}
The alphametics solver uses this technique to create a dictionary that maps letters in the puzzle to digits in the solution, for each possible solution. -
characters = tuple(ord(c) for c in sorted_characters)
+characters = tuple(ord(c) for c in sorted_characters)
digits = tuple(ord(c) for c in '0123456789')
...
for guess in itertools.permutations(digits, len(characters)):
@@ -391,10 +391,10 @@ for guess in itertools.permutations(digits, len(characters)):
Python strings have many methods. You learned about some of those methods in the Strings chapter: lower(), count(), and format(). Now I want to introduce you to a powerful but little-known string manipulation technique: the translate() method.
->>> translation_table = {ord('A'): ord('O')} ①
->>> translation_table ②
+>>> translation_table = {ord('A'): ord('O')} ①
+>>> translation_table ②
{65: 79}
->>> 'MARK'.translate(translation_table) ③
+>>> 'MARK'.translate(translation_table) ③
'MORK'
- String translation starts with a translation table, which is just a dictionary that maps one character to another. Actually, “character” is incorrect — the translation table really maps one byte to another.
@@ -405,16 +405,16 @@ for guess in itertools.permutations(digits, len(characters)):
What does this have to do with solving alphametic puzzles? As it turns out, everything.
->>> characters = tuple(ord(c) for c in 'SMEDONRY') ①
+>>> characters = tuple(ord(c) for c in 'SMEDONRY') ①
>>> characters
(83, 77, 69, 68, 79, 78, 82, 89)
->>> guess = tuple(ord(c) for c in '91570682') ②
+>>> guess = tuple(ord(c) for c in '91570682') ②
>>> guess
(57, 49, 53, 55, 48, 54, 56, 50)
->>> translation_table = dict(zip(characters, guess)) ③
+>>> translation_table = dict(zip(characters, guess)) ③
>>> translation_table
{68: 55, 69: 53, 77: 49, 78: 54, 79: 48, 82: 56, 83: 57, 89: 50}
->>> 'SEND + MORE == MONEY'.translate(translation_table) ④
+>>> 'SEND + MORE == MONEY'.translate(translation_table) ④
'9567 + 1085 == 10652'
- Using a generator expression, we quickly compute the byte values for each character in a string. characters is an example of the value of sorted_characters in the
alphametics.solve() function.
@@ -455,12 +455,12 @@ for guess in itertools.permutations(digits, len(characters)):
>>> x = 5
->>> eval("x * 5") ①
+>>> eval("x * 5") ①
25
->>> eval("pow(x, 2)") ②
+>>> eval("pow(x, 2)") ②
25
>>> import math
->>> eval("math.sqrt(x)") ③
+>>> eval("math.sqrt(x)") ③
2.2360679774997898
- The expression that
eval() takes can reference global variables defined outside the eval(). If called within a function, it can reference local variables too.
@@ -472,11 +472,11 @@ for guess in itertools.permutations(digits, len(characters)):
>>> import subprocess
->>> eval("subprocess.getoutput('ls ~')") ①
+>>> eval("subprocess.getoutput('ls ~')") ①
'Desktop Library Pictures \
Documents Movies Public \
Music Sites'
->>> eval("subprocess.getoutput('rm -rf /')") ②
+>>> eval("subprocess.getoutput('rm -rf /')") ②
- The
subprocess module allows you to run arbitrary shell commands and get the result as a Python string.
- Don’t do this.
@@ -485,7 +485,7 @@ for guess in itertools.permutations(digits, len(characters)):
It’s even worse than that, because there’s a global __import__() function that takes a module name as a string, imports the module, and returns a reference to it. Combined with the power of eval(), you can construct a single expression that will wipe out all your files:
->>> eval("__import__('subprocess').getoutput('rm -rf /')") ①
+>>> eval("__import__('subprocess').getoutput('rm -rf /')") ①
>>> x = 5 ->>> eval("x * 5", {}, {}) ① +>>> eval("x * 5", {}, {}) ① Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> NameError: name 'x' is not defined ->>> eval("x * 5", {"x": x}, {}) ② +>>> eval("x * 5", {"x": x}, {}) ② >>> import math ->>> eval("math.sqrt(x)", {"x": x}, {}) ② +>>> eval("math.sqrt(x)", {"x": x}, {}) ② Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1, in <module> @@ -519,9 +519,9 @@ NameError: name 'math' is not defined
Gee, that was easy. Lemme make an alphametics web service now!
->>> eval("pow(5, 2)", {}, {}) ① +>>> eval("pow(5, 2)", {}, {}) ① 25 ->>> eval("__import__('math').sqrt(5)", {}, {}) ② +>>> eval("__import__('math').sqrt(5)", {}, {}) ② 2.2360679774997898
pow(5, 2) works, because 5 and 2 are literals, and pow() is a built-in function.
@@ -531,7 +531,7 @@ NameError: name 'math' is not defined
Yeah, that means you can still do nasty things, even if you explicitly set the global and local namespaces to empty dictionaries when calling eval():
->>> eval("__import__('subprocess').getoutput('rm -rf /')", {}, {}) ①+>>> eval("__import__('subprocess').getoutput('rm -rf /')", {}, {}) ①
>>> eval("__import__('math').sqrt(5)",
-... {"__builtins__":None}, {}) ①
+... {"__builtins__":None}, {}) ①
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name '__import__' is not defined
>>> eval("__import__('subprocess').getoutput('rm -rf /')",
-... {"__builtins__":None}, {}) ②
+... {"__builtins__":None}, {}) ②
Traceback (most recent call last):
File "", line 1, in
File "", line 1, in
@@ -591,9 +591,10 @@ NameError: name '__import__' is not defined
Many, many thanks to Raymond Hettinger for agreeing to relicense his code so I could port it to Python 3 and use it as the basis for this chapter. -
© 2001–9 Mark Pilgrim + diff --git a/case-study-porting-chardet-to-python-3.html b/case-study-porting-chardet-to-python-3.html index ded5b85..9601890 100644 --- a/case-study-porting-chardet-to-python-3.html +++ b/case-study-porting-chardet-to-python-3.html @@ -15,11 +15,11 @@ del{background:#f87}
-You are here: Home ‣ Dive Into Python 3 ‣ +
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♦♦
chardet to Python 3-❝ Words, words. They’re all we have to go on. ❞
— Rosencrantz and Guildenstern are Dead +❝ Words, words. They’re all we have to go on. ❞
— Rosencrantz and Guildenstern are Dead
Hmm, a small snag. In Python 3, False is a reserved word, so you can’t use it as a variable name. Let’s look at constants.py to see where it’s defined. Here’s the original version from constants.py, before the 2to3 script changed it:
-
import __builtin__
+import __builtin__
if not hasattr(__builtin__, 'False'):
False = 0
True = 1
@@ -603,9 +603,9 @@ else:
This piece of code is designed to allow this library to run under older versions of Python 2. Prior to Python 2.3 [FIXME-LINK], Python had no built-in bool type. This code detects the absence of the built-in constants True and False, and defines them if necessary.
However, Python 3 will always have a bool type, so this entire code snippet is unnecessary. The simplest solution is to replace all instances of constants.True and constants.False with True and False, respectively, then delete this dead code from constants.py.
So this line in universaldetector.py:
-
self.done = constants.False
+self.done = constants.False
Becomes
-
self.done = False
+self.done = False
Ah, wasn’t that satisfying? The code is shorter and more readable already.
No module named constants
Time to run test.py again and see how far it gets.
@@ -617,12 +617,12 @@ else:
import constants, sys
ImportError: No module named constants
What’s that you say? No module named constants? Of course there’s a module named constants. …Oh wait, no there isn’t. Remember when the 2to3 script fixed up all those import statements? This library has a lot of relative imports — that is, modules that import other modules within the library. In Python 3, all import statements are absolute by default [FIXME-LINK PEP 0328]. To do relative imports, you need to do something like this instead:
-
from . import constants
+from . import constants
But wait. Wasn’t the 2to3 script supposed to take care of these for you? Well, it did, but this particular import statement combines two different types of imports into one line: a relative import of the constants module within the library, and an absolute import of the sys module that is pre-installed in the Python standard library. In Python 2, you could combine these into one import statement. In Python 3, you can’t, and the 2to3 script is not smart enough to split the import statement into two.
The solution is to split the import statement manually. So this two-in-one import:
-
import constants, sys
+import constants, sys
Needs to become two separate imports:
-
from . import constants
+from . import constants
import sys
There are variations of this problem scattered throughout the chardet library. In some places it’s “import constants, sys”; in other places, it’s “import constants, re”. The fix is the same: manually split the import statement into two lines, one for the relative import, the other for the absolute import.
FIXME-xref to as-yet-unwritten PEP 8 style section (which says you should put all imports on their own line)
@@ -638,7 +638,7 @@ import sys
NameError: name 'file' is not definedThis one surprised me, because I’ve been using this idiom as long as I can remember. In Python 2, the global file() function was an alias for the open() function, which was the standard way of opening files for reading. In Python 3, the entire system for reading and writing files has been refactored into the io module. [FIXME-LINK PEP 3116] I’ll cover the new I/O module in more detail in Chapter FIXME, but for now, the important bit is that the global file() function no longer exists. However, the open() function does still exist. (Technically, it’s an alias for io.open(), but never mind that right now.)
Thus, the simplest solution to the problem of the missing file() is to call the open() function instead:
-
for line in open(f, 'rb'):for line in open(f, 'rb'):And that’s all I have to say about that.
Now things are starting to get interesting. And by “interesting,” I mean “confusing as all hell.” @@ -651,20 +651,20 @@ NameError: name 'file' is not defined if self._highBitDetector.search(aBuf): TypeError: can't use a string pattern on a bytes-like object
To debug this, let’s see what self._highBitDetector is. It’s defined in the __init__ method of the UniversalDetector class: -
class UniversalDetector:
+class UniversalDetector:
def __init__(self):
self._highBitDetector = re.compile(r'[\x80-\xFF]')
This pre-compiles a regular expression designed to find non-ASCII characters in the range 128–255 (0x80–0xFF). Wait, that’s not quite right; I need to be more precise with my terminology. This pattern is designed to find non-ASCII bytes in the range 128-255.
And therein lies the problem.
In Python 2, a string was an array of bytes whose character encoding was tracked separately. If you wanted Python 2 to keep track of the character encoding, you had to use a Unicode string (u'') instead. But in Python 3, a string is always what Python 2 called a Unicode string — that is, an array of Unicode characters (of possibly varying byte lengths). Since this regular expression is defined by a string pattern, it can only be used to search a string — again, an array of characters. But what we’re searching is not a string, it’s a byte array. Looking at the traceback, this error occurred in universaldetector.py:
-
def feed(self, aBuf):
+def feed(self, aBuf):
.
.
.
if self._mInputState == ePureAscii:
if self._highBitDetector.search(aBuf):
And what is aBuf? Let’s backtrack further to a place that calls UniversalDetector.feed(). One place that calls it is the test harness, test.py.
-
u = UniversalDetector()
+u = UniversalDetector()
.
.
.
@@ -674,7 +674,7 @@ for line in open(f, 'rb'):
And here we find our answer: in the UniversalDetector.feed() method, aBuf is a line read from a file on disk. Look carefully at the parameters used to open the file: 'rb'. 'r' is for “read”; OK, big deal, we’re reading the file. Ah, but 'b' is for “binary.” Without the 'b' flag, this for loop would read the file, line by line, and convert each line into a string — an array of Unicode characters — according to the system default character encoding. (You could override the system encoding with another parameter to the open() function, but never mind that for now.) But with the 'b' flag, this for loop reads the file, line by line, and stores each line exactly as it appears in the file, as an array of bytes. That byte array gets passed to UniversalDetector.feed(), and eventually gets passed to the pre-compiled regular expression, self._highBitDetector, to search for high-bit… characters. But we don’t have characters; we have bytes. Oops.
What we need this regular expression to search is not an array of characters, but an array of bytes.
Once you realize that, the solution is not difficult. Regular expressions defined with strings can search strings. Regular expressions defined with byte arrays can search byte arrays. To define a byte array pattern, we simply change the type of the argument we use to define the regular expression to a byte array. (There is one other case of this same problem, on the very next line.)
-
class UniversalDetector:
+ class UniversalDetector:
def __init__(self):
- self._highBitDetector = re.compile(r'[\x80-\xFF]')
- self._escDetector = re.compile(r'(\033|~{)')
@@ -684,7 +684,7 @@ for line in open(f, 'rb'):
self._mCharSetProbers = []
self.reset()
Searching the entire codebase for other uses of the re module turns up two more instances, in charsetprober.py. Again, the code is defining regular expressions as strings but executing them on aBuf, which is a byte array. The solution is the same: define the regular expression patterns as byte arrays.
-
class CharSetProber:
+ class CharSetProber:
.
.
.
@@ -709,7 +709,7 @@ for line in open(f, 'rb'):
elif (self._mInputState == ePureAscii) and self._escDetector.search(self._mLastChar + aBuf):
TypeError: Can't convert 'bytes' object to str implicitly
There’s an unfortunate clash of coding style and Python interpreter here. The TypeError could be anywhere on that line, but the traceback doesn’t tell you exactly where it is. It could be in the first conditional or the second, and the traceback would look the same. To narrow it down, you should split the line in half, like this:
-
elif (self._mInputState == ePureAscii) and \
+elif (self._mInputState == ePureAscii) and \
self._escDetector.search(self._mLastChar + aBuf):
And re-run the test:
C:\home\chardet> python test.py tests\*\*
@@ -722,7 +722,7 @@ TypeError: Can't convert 'bytes' object to str implicitly
TypeError: Can't convert 'bytes' object to str implicitly
Aha! The problem was not in the first conditional (self._mInputState == ePureAscii) but in the second one. So what could cause a TypeError there? Perhaps you’re thinking that the search() method is expecting a value of a different type, but that wouldn’t generate this traceback. Python functions can take any value; if you pass the right number of arguments, the function will execute. It may crash if you pass it a value of a different type than it’s expecting, but if that happened, the traceback would point to somewhere inside the function. But this traceback says it never got as far as calling the search() method. So the problem must be in that + operation, as it’s trying to construct the value that it will eventually pass to the search() method.
We know from previous debugging that aBuf is a byte array. So what is self._mLastChar? It’s an instance variable, defined in the reset() method, which is actually called from the __init__() method.
-
class UniversalDetector:
+class UniversalDetector:
def __init__(self):
self._highBitDetector = re.compile(b'[\x80-\xFF]')
self._escDetector = re.compile(b'(\033|~{)')
@@ -739,7 +739,7 @@ TypeError: Can't convert 'bytes' object to str implicitly
self._mLastChar = ''
And now we have our answer. Do you see it? self._mLastChar is a string, but aBuf is a byte array. And you can’t concatenate a string to a byte array — not even a zero-length string.
So what is self._mLastChar anyway? The answer is in the feed() method, just a few lines down from where the trackback occurred.
-
if self._mInputState == ePureAscii:
+if self._mInputState == ePureAscii:
if self._highBitDetector.search(aBuf):
self._mInputState = eHighbyte
elif (self._mInputState == ePureAscii) and \
@@ -748,15 +748,14 @@ TypeError: Can't convert 'bytes' object to str implicitly
self._mLastChar = aBuf[-1]
The calling function calls this feed() method over and over again with a few bytes at a time. The method processes the bytes it was given (passed in as aBuf), then stores the last byte in self._mLastChar in case it’s needed during the next call. (In a multi-byte encoding, the feed() method might get called with half of a character, then called again with the other half.) But because aBuf is now a byte array instead of a string, self._mLastChar needs to be a byte array as well. Thus:
-
def reset(self):
+ def reset(self):
.
.
.
- self._mLastChar = ''
+ self._mLastChar = b''
Searching the entire codebase for “mLastChar” turns up a similar problem in mbcharsetprober.py, but instead of tracking the last character, it tracks the last two characters. The MultiByteCharSetProber class uses a list of 1-character strings to track the last two characters; in Python 3, it needs to use a list of integers.
-
- class MultiByteCharSetProber(CharSetProber):
+ class MultiByteCharSetProber(CharSetProber):
def __init__(self):
CharSetProber.__init__(self)
self._mDistributionAnalyzer = None
@@ -785,7 +784,7 @@ TypeError: unsupported operand type(s) for +: 'int' and 'bytes'
…The bad news is it doesn’t always feel like progress.
But this is progress! Really! Even though the traceback calls out the same line of code, it’s a different error than it used to be. Progress! So what’s the problem now? The last time I checked, this line of code didn’t try to concatenate an int with a byte array (bytes). In fact, you just spent a lot of time ensuring that self._mLastChar was a byte array. How did it turn into an int?
The answer lies not in the previous lines of code, but in the following lines.
-
if self._mInputState == ePureAscii:
+if self._mInputState == ePureAscii:
if self._highBitDetector.search(aBuf):
self._mInputState = eHighbyte
elif (self._mInputState == ePureAscii) and \
@@ -796,22 +795,22 @@ TypeError: unsupported operand type(s) for +: 'int' and 'bytes'
This error doesn’t occur the first time the feed() method gets called; it occurs the second time, after self._mLastChar has been set to the last byte of aBuf. Well, what’s the problem with that? Getting a single element from a byte array yields an integer, not a byte array. To see the difference, follow me to the interactive shell:
->>> aBuf = b'\xEF\xBB\xBF' ①
+>>> aBuf = b'\xEF\xBB\xBF' ①
>>> len(aBuf)
3
>>> mLastChar = aBuf[-1]
->>> mLastChar ②
+>>> mLastChar ②
191
->>> type(mLastChar) ③
+>>> type(mLastChar) ③
<class 'int'>
->>> mLastChar + aBuf ④
+>>> mLastChar + aBuf ④
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'bytes'
->>> mLastChar = aBuf[-1:] ⑤
+>>> mLastChar = aBuf[-1:] ⑤
>>> mLastChar
b'\xbf'
->>> mLastChar + aBuf ⑥
+>>> mLastChar + aBuf ⑥
b'\xbf\xef\xbb\xbf'
- Define a byte array of length 3.
@@ -822,7 +821,7 @@ TypeError: unsupported operand type(s) for +: 'int' and 'bytes'
- Concatenating a byte array of length 1 with a byte array of length 3 returns a new byte array of length 4.
So, to ensure that the feed() method in universaldetector.py continues to work no matter how often it’s called, you need to initialize self._mLastChar as a 0-length byte array, then make sure it stays a byte array.
-
self._escDetector.search(self._mLastChar + aBuf):
+ self._escDetector.search(self._mLastChar + aBuf):
self._mInputState = eEscAscii
- self._mLastChar = aBuf[-1]
@@ -845,25 +844,25 @@ tests\Big5\0804.blogspot.com.xml
byteCls = self._mModel['classTable'][ord(c)]
TypeError: ord() expected string of length 1, but int found
OK, so c is an int, but the ord() function was expecting a 1-character string. Fair enough. Where is c defined?
-
# codingstatemachine.py
+# codingstatemachine.py
def next_state(self, c):
# for each byte we get its class
# if it is first byte, we also get byte length
byteCls = self._mModel['classTable'][ord(c)]
That’s no help; it’s just passed into the function. Let’s pop the stack.
-
# utf8prober.py
+# utf8prober.py
def feed(self, aBuf):
for c in aBuf:
codingState = self._mCodingSM.next_state(c)
And now we have the answer. Do you see it? In Python 2, aBuf was a string, so c was a 1-character string. (That’s what you get when you iterate over a string — all the characters, one by one.) But now, aBuf is a byte array, so c is an int, not a 1-character string. In other words, there’s no need to call the ord() function because c is already an int!
Thus:
-
def next_state(self, c):
+ def next_state(self, c):
# for each byte we get its class
# if it is first byte, we also get byte length
- byteCls = self._mModel['classTable'][ord(c)]
+ byteCls = self._mModel['classTable'][c]
Searching the entire codebase for instances of “ord(c)” uncovers similar problems in sbcharsetprober.py…
-
# sbcharsetprober.py
+# sbcharsetprober.py
def feed(self, aBuf):
if not self._mModel['keepEnglishLetter']:
aBuf = self.filter_without_english_letters(aBuf)
@@ -873,13 +872,13 @@ def feed(self, aBuf):
for c in aBuf:
order = self._mModel['charToOrderMap'][ord(c)]
…and latin1prober.py…
-
# latin1prober.py
+# latin1prober.py
def feed(self, aBuf):
aBuf = self.filter_with_english_letters(aBuf)
for c in aBuf:
charClass = Latin1_CharToClass[ord(c)]
c is iterating over aBuf, which means it is an integer, not a 1-character string. The solution is the same: change ord(c) to just plain c.
-
# sbcharsetprober.py
+ # sbcharsetprober.py
def feed(self, aBuf):
if not self._mModel['keepEnglishLetter']:
aBuf = self.filter_without_english_letters(aBuf)
@@ -918,7 +917,7 @@ tests\Big5\0804.blogspot.com.xml
TypeError: unorderable types: int() >= str()
Did you notice? This time around, the code passed the first test case (tests\ascii\howto.diveintomark.org.xml). You’re making real progress here.
So what’s this all about? “Unorderable types”? Once again, the difference between byte arrays and strings is rearing its ugly head. Take a look at the code:
-
class SJISContextAnalysis(JapaneseContextAnalysis):
+class SJISContextAnalysis(JapaneseContextAnalysis):
def get_order(self, aStr):
if not aStr: return -1, 1
# find out current char's byte length
@@ -928,7 +927,7 @@ TypeError: unorderable types: int() >= str()
else:
charLen = 1
And where does aStr come from? Let’s pop the stack:
-
def feed(self, aBuf, aLen):
+def feed(self, aBuf, aLen):
.
.
.
@@ -938,7 +937,7 @@ TypeError: unorderable types: int() >= str()
Oh look, it’s our old friend, aBuf. As you might have guessed from every other issue we’ve encountered in this chapter, aBuf is a byte array. Here, the feed() method isn’t just passing it on wholesale; it’s slicing it. But as you saw earlier in this chapter, slicing a byte array returns a byte array, so the aStr parameter that gets passed to the get_order() method is still a byte array.
And what is this code trying to do with aStr? It’s taking the first element of the byte array and comparing it to a string of length 1. In Python 2, that worked, because aStr and aBuf were strings, and aStr[0] would be a string, and you can compare strings for inequality. But in Python 3, aStr and aBuf are byte arrays, aStr[0] is an integer, and you can’t compare integers and strings for inequality without explicitly coercing one of them.
In this case, there’s no need to make the code more complicated by adding an explicit coercion. aStr[0] yields an integer; the things you’re comparing to are all constants. Let’s change them from 1-character strings to integers.
-
class SJISContextAnalysis(JapaneseContextAnalysis):
+ class SJISContextAnalysis(JapaneseContextAnalysis):
def get_order(self, aStr):
if not aStr: return -1, 1
# find out current char's byte length
@@ -1009,7 +1008,7 @@ tests\Big5\0804.blogspot.com.xml
if (aStr[0] >= '\x81') and (aStr[0] <= '\x9F'):
TypeError: unorderable types: int() >= str()
The fix is the same:
-
class EUCTWDistributionAnalysis(CharDistributionAnalysis):
+ class EUCTWDistributionAnalysis(CharDistributionAnalysis):
def __init__(self):
CharDistributionAnalysis.__init__(self)
self._mCharToFreqOrder = EUCTWCharToFreqOrder
@@ -1127,21 +1126,21 @@ tests\Big5\0804.blogspot.com.xml
total = reduce(operator.add, self._mFreqCounter)
NameError: global name 'reduce' is not defined
According to the official What’s New In Python 3.0 guide, the reduce() function has been moved out of the global namespace and into the functools module. Quoting the guide: “Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable.” You can read more about the decision from Guido van Rossum’s weblog: The fate of reduce() in Python 3000.
-
def get_confidence(self):
+def get_confidence(self):
if self.get_state() == constants.eNotMe:
return 0.01
total = reduce(operator.add, self._mFreqCounter)
The reduce() function takes two arguments — a function and a list (strictly speaking, any iterable object will do) — and applies the function cumulatively to each item of the list. In other words, this is a fancy and roundabout way of adding up all the items in a list and returning the result.
This monstrosity was so common that Python added a global sum() function.
-
def get_confidence(self):
+ def get_confidence(self):
if self.get_state() == constants.eNotMe:
return 0.01
- total = reduce(operator.add, self._mFreqCounter)
+ total = sum(self._mFreqCounter)
Since you’re no longer using the operator module, you can remove that import from the top of the file as well.
-
from .charsetprober import CharSetProber
+ from .charsetprober import CharSetProber
from . import constants
- import operator
I CAN HAZ TESTZ?
@@ -1192,7 +1191,8 @@ tests\EUC-JP\arclamp.jp.xml EUC-JP with confide
- Test cases are essential. Don’t port anything without them. Don’t even try. The only reason I have any confidence at all that
chardet works in Python 3 is because I had a test suite that exercised every line of code in the entire library. I never would have found half of these problems with manual spot-checking.
© 2001–9 Mark Pilgrim + diff --git a/dip3.css b/dip3.css index c5326e8..0dfb0e0 100644 --- a/dip3.css +++ b/dip3.css @@ -37,6 +37,7 @@ Classname Legend .c = "centered" = centered footer text (also clears floats) .a = "asterism" = section break .v = "navigation" = prev/next navigation links (not breadcrumbs) +.u = "Unicode" = text contains Unicode characters (requires special font declaration) .nm = "no mobile" = hide this section on mobile devices .nd = "no decoration" = hide the widgets on this code block @@ -53,6 +54,7 @@ Acknowledgements & Inspirations "Use the Best Available Ampersand" ....................... http://simplebits.com/notebook/2008/08/14/ampersands.html "Unicode Support in HTML, Fonts, and Web Browsers" ....... http://alanwood.net/unicode/ "Punctuation" ............................................ http://en.wikipedia.org/wiki/Punctuation +"Google Code Prettify" ................................... http://code.google.com/p/google-code-prettify/ */ /* typography */ @@ -61,15 +63,15 @@ body, .w a { font: medium/1.75 'Gill Sans', 'Gill Sans MT', Corbel, Helvetica, 'Nimbus Sans L', sans-serif; word-spacing: 0.1em; } -pre, kbd, samp, code, var, .b { +pre, kbd, samp, code, var, .b, pre span { font: small/2.154 Consolas, 'Andale Mono', Monaco, 'Liberation Mono', 'Bitstream Vera Sans Mono', 'DejaVu Sans Mono', monospace; word-spacing: 0; } -span { - font: medium 'Arial Unicode MS', FreeSerif, OpenSymbol, 'DejaVu Sans', sans-serif; +span.u { + font: medium/1.75 'Arial Unicode MS', FreeSerif, OpenSymbol, 'DejaVu Sans', sans-serif; } -pre span, .a { - font-family: 'Arial Unicode MS', 'DejaVu Sans', FreeSerif, OpenSymbol, sans-serif; +pre span.u, pre span.u span, .a { + font: medium/1.75 'Arial Unicode MS', 'DejaVu Sans', FreeSerif, OpenSymbol, sans-serif; } .baa { font: oblique large Constantia, Baskerville, Palatino, 'Palatino Linotype', 'URW Palladio L', serif; @@ -201,7 +203,7 @@ li ol, .q { code, var, samp { line-height:inherit !important; } -pre a, .w a, pre a:hover { +pre a, td code a, .w a, pre a:hover { border: 0; } @@ -271,6 +273,7 @@ aside { #level span { color: #82b445; } + /* previous/next navigation links */ .v a { @@ -290,3 +293,17 @@ aside { margin: 0; text-shadow: gainsboro 3px 3px 3px; } + +/* syntax highlighting */ + +.str { color: #080; } +.kwd { color: #008; } +.com { color: #800; } +.typ { color: #606; } +.lit { color: #066; } +.pun { color: #660; } +.pln { color: #000; } +.tag { color: #008; } +.atn { color: #606; } +.atv { color: #080; } +.dec { color: #606; } diff --git a/files.html b/files.html index c54b07d..0041b23 100644 --- a/files.html +++ b/files.html @@ -12,11 +12,11 @@ body{counter-reset:h1 12}
You are here: Home ‣ Dive Into Python 3 ‣ +
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♢♢
-❝ FIXME ❞
— FIXME +❝ FIXME ❞
— FIXME
© 2001–9 Mark Pilgrim + diff --git a/generators.html b/generators.html index 22d02b3..73e3a9a 100644 --- a/generators.html +++ b/generators.html @@ -12,11 +12,11 @@ body{counter-reset:h1 5}
-You are here: Home ‣ Dive Into Python 3 ‣ +
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♢♢
-❝ My spelling is Wobbly. It’s good spelling but it Wobbles, and the letters get in the wrong places. ❞
— Winnie-the-Pooh +❝ My spelling is Wobbly. It’s good spelling but it Wobbles, and the letters get in the wrong places. ❞
— Winnie-the-Pooh
So you’re looking at words, which, at least in English, means you’re looking at strings of characters. You have rules that say you need to find different combinations of characters, then do different things to them. This sounds like a job for regular expressions!
import re
+import re
def plural(noun):
- if re.search('[sxz]$', noun): ①
- return re.sub('$', 'es', noun) ②
+ if re.search('[sxz]$', noun): ①
+ return re.sub('$', 'es', noun) ②
elif re.search('[^aeioudgkprt]h$', noun):
return re.sub('$', 'es', noun)
elif re.search('[^aeiou]y$', noun):
@@ -57,13 +57,13 @@ def plural(noun):
Let’s look at regular expression substitutions in more detail.
>>> import re
->>> re.search('[abc]', 'Mark') ①
+>>> re.search('[abc]', 'Mark') ①
<_sre.SRE_Match object at 0x001C1FA8>
->>> re.sub('[abc]', 'o', 'Mark') ②
+>>> re.sub('[abc]', 'o', 'Mark') ②
'Mork'
->>> re.sub('[abc]', 'o', 'rock') ③
+>>> re.sub('[abc]', 'o', 'rock') ③
'rook'
->>> re.sub('[abc]', 'o', 'caps') ④
+>>> re.sub('[abc]', 'o', 'caps') ④
'oops'
- Does the string
Mark contain a, b, or c? Yes, it contains a.
@@ -74,11 +74,11 @@ def plural(noun):
And now, back to the plural() function…
-
def plural(noun):
+def plural(noun):
if re.search('[sxz]$', noun):
- return re.sub('$', 'es', noun) ①
- elif re.search('[^aeioudgkprt]h$', noun): ②
- return re.sub('$', 'es', noun) ③
+ return re.sub('$', 'es', noun) ①
+ elif re.search('[^aeioudgkprt]h$', noun): ②
+ return re.sub('$', 'es', noun) ③
elif re.search('[^aeiou]y$', noun):
return re.sub('y$', 'ies', noun)
else:
@@ -93,13 +93,13 @@ def plural(noun):
>>> import re
->>> re.search('[^aeiou]y$', 'vacancy') ①
+>>> re.search('[^aeiou]y$', 'vacancy') ①
<_sre.SRE_Match object at 0x001C1FA8>
->>> re.search('[^aeiou]y$', 'boy') ②
+>>> re.search('[^aeiou]y$', 'boy') ②
>>>
>>> re.search('[^aeiou]y$', 'day')
>>>
->>> re.search('[^aeiou]y$', 'pita') ③
+>>> re.search('[^aeiou]y$', 'pita') ③
>>>
vacancy matches this regular expression, because it ends in cy, and c is not a, e, i, o, or u.
@@ -107,11 +107,11 @@ def plural(noun):
pita does not match, because it does not end in y.
->>> re.sub('y$', 'ies', 'vacancy') ①
+>>> re.sub('y$', 'ies', 'vacancy') ①
'vacancies'
>>> re.sub('y$', 'ies', 'agency')
'agencies'
->>> re.sub('([^aeiou])y$', r'\1ies', 'vacancy') ②
+>>> re.sub('([^aeiou])y$', r'\1ies', 'vacancy') ②
'vacancies'
- This regular expression turns
vacancy into vacancies and agency into agencies, which is what you wanted. Note that it would also turn boy into boies, but that will never happen in the function because you did that re.search first to find out whether you should do this re.sub.
@@ -126,7 +126,7 @@ def plural(noun):
Now you’re going to add a level of abstraction. You started by defining a list of rules: if this, do that, otherwise go to the next rule. Let’s temporarily complicate part of the program so you can simplify another part.
import re
+import re
def match_sxz(noun):
return re.search('[sxz]$', noun)
@@ -140,10 +140,10 @@ def match_h(noun):
def apply_h(noun):
return re.sub('$', 'es', noun)
-def match_y(noun): ①
+def match_y(noun): ①
return re.search('[^aeiou]y$', noun)
-def apply_y(noun): ②
+def apply_y(noun): ②
return re.sub('y$', 'ies', noun)
def match_default(noun):
@@ -152,14 +152,14 @@ def match_default(noun):
def apply_default(noun):
return noun + 's'
-rules = [[match_sxz, apply_sxz], ③
+rules = [[match_sxz, apply_sxz], ③
[match_h, apply_h],
[match_y, apply_y],
[match_default, apply_default]
]
def plural(noun):
- for matches_rule, apply_rule in rules: ④
+ for matches_rule, apply_rule in rules: ④
if matches_rule(noun):
return apply_rule(noun)
@@ -174,7 +174,7 @@ def plural(noun):
If this additional level of abstraction is confusing, try unrolling the function to see the equivalence. The entire for loop is equivalent to the following:
-
+
def plural(noun):
if match_sxz(noun):
return apply_sxz(noun)
@@ -206,14 +206,14 @@ def plural(noun):
Defining separate named functions for each match and apply rule isn’t really necessary. You never call them directly; you add them to the rules list and call them through there. Furthermore, each function follows one of two patterns. All the match functions call re.search(), and all the apply functions call re.sub(). Let’s factor out the patterns so that defining new rules can be easier.
import re
+import re
def build_match_and_apply_functions(pattern, search, replace):
- def matches_rule(word): ①
+ def matches_rule(word): ①
return re.search(pattern, word)
- def apply_rule(word): ②
+ def apply_rule(word): ②
return re.sub(search, replace, word)
- return [matches_rule, apply_rule] ③
+ return [matches_rule, apply_rule] ③
build_match_and_apply_functions() is a function that builds other functions dynamically. It takes pattern, search and replace, then defines a matches_rule() function which calls re.search() with the pattern that was passed to the build_match_and_apply_functions() function, and the word that was passed to the matches_rule() function you’re building. Whoa.
- Building the apply function works the same way. The apply function is a function that takes one parameter, and calls
re.sub() with the search and replace parameters that were passed to the build_match_and_apply_functions() function, and the word that was passed to the apply_rule() function you’re building. This technique of using the values of outside parameters within a dynamic function is called closures. You’re essentially defining constants within the apply function you’re building: it takes one parameter (word), but it then acts on that plus two other values (search and replace) which were set when you defined the apply function.
@@ -222,15 +222,14 @@ def build_match_and_apply_functions(pattern, search, replace):
If this is incredibly confusing (and it should be, this is weird stuff), it may become clearer when you see how to use it.
-
-patterns = \ ①
+patterns = \ ①
[
['[sxz]$', '$', 'es'],
['[^aeioudgkprt]h$', '$', 'es'],
['(qu|[^aeiou])y$', 'y$', 'ies'],
['$', '$', 's']
]
-rules = [build_match_and_apply_functions(pattern, search, replace) ②
+rules = [build_match_and_apply_functions(pattern, search, replace) ②
for (pattern, search, replace) in patterns]
- Our pluralization rules are now defined as a list of lists of strings (not functions). The first string in each group is the regular expression pattern that you would use in
re.search() to see if this rule matches. The second and third strings in each group are the search and replace expressions you would use in re.sub() to actually apply the rule to turn a noun into its plural.
@@ -239,8 +238,8 @@ def build_match_and_apply_functions(pattern, search, replace):
Rounding out this version of the script is the main entry point, the plural() function.
-
def plural(noun):
- for matches_rule, apply_rule in rules: ①
+def plural(noun):
+ for matches_rule, apply_rule in rules: ①
if matches_rule(noun):
return apply_rule(noun)
@@ -256,7 +255,7 @@ def build_match_and_apply_functions(pattern, search, replace):
First, let’s create a text file that contains the rules you want. No fancy data structures, just whitespace-delimited strings in three columns. Let’s call it plural4-rules.txt.
[download plural4-rules.txt]
-
[sxz]$ $ es
+[sxz]$ $ es
[^aeioudgkprt]h$ $ es
[^aeiou]y$ y$ ies
$ $ s
@@ -266,9 +265,9 @@ $ $ s
[FIXME: now that this chapter comes before the I/O chapter, need to at least mention what open() does]
[FIXME: try/finally -> with]
import re
+import re
-def build_match_and_apply_functions(pattern, search, replace): ①
+def build_match_and_apply_functions(pattern, search, replace): ①
def matches_rule(word):
return re.search(pattern, word)
def apply_rule(word):
@@ -276,14 +275,14 @@ $ $ s
return [matches_rule, apply_rule]
rules = []
-pattern_file = open('plural4-rules.txt') ②
+pattern_file = open('plural4-rules.txt') ②
try:
- for line in pattern_file: ③
- pattern, search, replace = line.split(None, 3) ④
- rules.append(build_match_and_apply_functions( ⑤
+ for line in pattern_file: ③
+ pattern, search, replace = line.split(None, 3) ④
+ rules.append(build_match_and_apply_functions( ⑤
pattern, search, replace))
finally:
- pattern_file.close() ⑥
+ pattern_file.close() ⑥
- The
build_match_and_apply_functions() function has not changed. You’re still using closures to build two functions dynamically that use variables defined in the outer function.
- Open the file that contains the pattern strings.
@@ -301,7 +300,7 @@ finally:
Wouldn’t it be grand to have a generic plural() function that parses the rules file? Get rules, check for a match, apply appropriate transformation, go to next rule. That’s all the plural() function has to do, and that’s all the plural() function should do.
def rules():
+def rules():
for line in open('plural5-rules.txt'):
pattern, search, replace = line.split(None, 3)
yield build_match_and_apply_functions(pattern, search, replace)
@@ -317,20 +316,20 @@ def plural(noun):
>>> def make_counter(x):
... print('entering make_counter')
... while True:
-... yield x ①
+... yield x ①
... print('incrementing x')
... x = x + 1
...
->>> counter = make_counter(2) ②
->>> counter ③
+>>> counter = make_counter(2) ②
+>>> counter ③
<generator object at 0x001C9C10>
->>> next(counter) ④
+>>> next(counter) ④
entering make_counter
2
->>> next(counter) ⑤
+>>> next(counter) ⑤
incrementing x
3
->>> next(counter) ⑥
+>>> next(counter) ⑥
incrementing x
4
@@ -347,11 +346,11 @@ def plural(noun):
A Fibonacci Generator
def fib(max):
- a, b = 0, 1 ①
+def fib(max):
+ a, b = 0, 1 ①
while a < max:
- yield a ②
- a, b = b, a + b ③
+ yield a ②
+ a, b = b, a + b ③
- The Fibonacci sequence is a sequence of numbers where each number is the sum of the two numbers before it. It starts with
0 and 1, goes up slowly at first, then more and more rapidly. To start the sequence, you need two variables: a starts at 0, and b starts at 1.
- a is the current number in the sequence, so yield it.
@@ -364,8 +363,8 @@ def plural(noun):
>>> from fibonacci import fib
->>> for n in fib(1000): ①
-... print(n, end=' ') ②
+>>> for n in fib(1000): ①
+... print(n, end=' ') ②
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
- You can use a generator like
fib() in a for loop directly. The for loop will automatically call the next() function to get values from the fib() generator and assign them to the for loop index variable (n).
@@ -376,13 +375,13 @@ def plural(noun):
Let’s go back to plural5.py and see how this version of the plural() function works.
-
def rules():
+def rules():
for line in open('plural5-rules.txt'):
- pattern, search, replace = line.split(None, 3) ②
- yield build_match_and_apply_functions(pattern, search, replace) ③
+ pattern, search, replace = line.split(None, 3) ②
+ yield build_match_and_apply_functions(pattern, search, replace) ③
def plural(noun):
- for matches_rule, apply_rule in rules(): ④
+ for matches_rule, apply_rule in rules(): ④
if matches_rule(noun):
return apply_rule(noun)
@@ -406,8 +405,9 @@ def plural(noun):
- PEP 255: Simple Generators
-
© 2001–9 Mark Pilgrim
+
diff --git a/http-web-services.html b/http-web-services.html
index 107ddb6..36b45b2 100644
--- a/http-web-services.html
+++ b/http-web-services.html
@@ -13,11 +13,11 @@ mark{display:inline}
-You are here: Home ‣ Dive Into Python 3 ‣
+
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♦♢
HTTP Web Services
-❝ A ruffled mind makes a restless pillow. ❞
— Charlotte Brontë
+
❝ A ruffled mind makes a restless pillow. ❞
— Charlotte Brontë
Diving In
@@ -137,7 +137,7 @@ The second time you request the same data, you include the ETag hash in an Again with the curl:
-you@localhost:~$ curl -I -H "If-None-Match: \"3075-ddc8d800\"" http://wearehugh.com/m.jpg ①
+you@localhost:~$ curl -I -H "If-None-Match: \"3075-ddc8d800\"" http://wearehugh.com/m.jpg ①
HTTP/1.1 304 Not Modified
Date: Sun, 31 May 2009 18:04:39 GMT
Server: Apache
@@ -188,7 +188,7 @@ Cache-Control: max-age=31536000, public
Let’s say you want to download a resource over HTTP, such as an Atom feed. Being a feed, you’re not just going to download it once; you’re going to download it over and over again. (Most feed readers will check for changes once an hour.) Let’s do it the quick-and-dirty way first, and then see how you can do better.
>>> import urllib.request
->>> data = urllib.request.urlopen('http://diveintopython3.org/examples/feed.xml').read() ①
+>>> data = urllib.request.urlopen('http://diveintopython3.org/examples/feed.xml').read() ①
>>> print(data)
<?xml version='1.0' encoding='utf-8'?>
<feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
@@ -213,13 +213,13 @@ Cache-Control: max-age=31536000, public
>>> from http.client import HTTPConnection
->>> HTTPConnection.debuglevel = 1 ①
+>>> HTTPConnection.debuglevel = 1 ①
>>> from urllib.request import urlopen
->>> response = urlopen('http://diveintopython3.org/examples/feed.xml') ②
-send: b'GET /examples/feed.xml HTTP/1.1 ③
-Host: diveintopython3.org ④
-Accept-Encoding: identity ⑤
-User-Agent: Python-urllib/3.0' ⑥
+>>> response = urlopen('http://diveintopython3.org/examples/feed.xml') ②
+send: b'GET /examples/feed.xml HTTP/1.1 ③
+Host: diveintopython3.org ④
+Accept-Encoding: identity ⑤
+User-Agent: Python-urllib/3.0' ⑥
Connection: close
reply: 'HTTP/1.1 200 OK'
…further debugging information omitted…
@@ -236,19 +236,19 @@ reply: 'HTTP/1.1 200 OK'
# continued from previous example
->>> print(response.headers.as_string()) ①
-Date: Sun, 31 May 2009 19:23:06 GMT ②
+>>> print(response.headers.as_string()) ①
+Date: Sun, 31 May 2009 19:23:06 GMT ②
Server: Apache
-Last-Modified: Sun, 31 May 2009 06:39:55 GMT ③
-ETag: "bfe-93d9c4c0" ④
+Last-Modified: Sun, 31 May 2009 06:39:55 GMT ③
+ETag: "bfe-93d9c4c0" ④
Accept-Ranges: bytes
-Content-Length: 3070 ⑤
-Cache-Control: max-age=86400 ⑥
+Content-Length: 3070 ⑤
+Cache-Control: max-age=86400 ⑥
Expires: Mon, 01 Jun 2009 19:23:06 GMT
Vary: Accept-Encoding
Connection: close
Content-Type: application/xml
->>> data = response.read() ⑦
+>>> data = response.read() ⑦
>>> len(data)
3070
@@ -282,7 +282,7 @@ reply: 'HTTP/1.1 200 OK'
# continued from the previous example
->>> print(response2.headers.as_string()) ①
+>>> print(response2.headers.as_string()) ①
Date: Mon, 01 Jun 2009 03:58:00 GMT
Server: Apache
Last-Modified: Sun, 31 May 2009 22:51:11 GMT
@@ -295,9 +295,9 @@ Vary: Accept-Encoding
Connection: close
Content-Type: application/xml
>>> data2 = response2.read()
->>> len(data2) ②
+>>> len(data2) ②
3070
->>> data2 == data ③
+>>> data2 == data ③
True
- The server is still sending the same array of “smart” headers:
Cache-Control and Expires to allow caching, Last-Modified and ETag to enable “not-modified” tracking. Even the Vary: Accept-Encoding header hints that the server would support compression, if only you would ask for it. But you didn’t.
@@ -315,11 +315,11 @@ Content-Type: application/xml
>>> import httplib2
->>> h = httplib2.Http('.cache') ①
->>> response, content = h.request('http://diveintopython3.org/examples/feed.xml') ②
->>> response.status ③
+>>> h = httplib2.Http('.cache') ①
+>>> response, content = h.request('http://diveintopython3.org/examples/feed.xml') ②
+>>> response.status ③
200
->>> content[:52] ④
+>>> content[:52] ④
b"<?xml version='1.0' encoding='utf-8'?>\r\n<feed xmlns="
>>> len(content)
3070
@@ -331,7 +331,7 @@ Content-Type: application/xml
-☞You probably only need one httplib2.Http object. There are valid reasons for creating more than one, but you should only do so if you know why you need them. “I need to request data from two different URLs” is not a valid reason. Re-use the Http object and just call the request() method twice.
+
☞You probably only need one httplib2.Http object. There are valid reasons for creating more than one, but you should only do so if you know why you need them. “I need to request data from two different URLs” is not a valid reason. Re-use the Http object and just call the request() method twice.
How httplib2 Handles Caching
@@ -340,10 +340,10 @@ Content-Type: application/xml
# continued from the previous example
->>> response2, content2 = h.request('http://diveintopython3.org/examples/feed.xml') ①
->>> response2.status ②
+>>> response2, content2 = h.request('http://diveintopython3.org/examples/feed.xml') ①
+>>> response2.status ②
200
->>> content2[:52] ③
+>>> content2[:52] ③
b"<?xml version='1.0' encoding='utf-8'?>\r\n<feed xmlns="
>>> len(content2)
3070
@@ -360,14 +360,14 @@ Content-Type: application/xml
# Please exit out of the interactive shell
# and launch a new one.
>>> import httplib2
->>> httplib2.debuglevel = 1 ①
->>> h = httplib2.Http('.cache') ②
->>> response, content = h.request('http://diveintopython3.org/examples/feed.xml') ③
->>> len(content) ④
+>>> httplib2.debuglevel = 1 ①
+>>> h = httplib2.Http('.cache') ②
+>>> response, content = h.request('http://diveintopython3.org/examples/feed.xml') ③
+>>> len(content) ④
3070
->>> response.status ⑤
+>>> response.status ⑤
200
->>> response.fromcache ⑥
+>>> response.fromcache ⑥
True
- Let’s turn on debugging and see what’s on the wire. This is the
httplib2 equivalent of turning on debugging in http.client. httplib2 will print all the data being sent to the server and some key information being sent back.
@@ -389,8 +389,8 @@ Content-Type: application/xml
# continued from the previous example
>>> response2, content2 = h.request('http://diveintopython3.org/examples/feed.xml',
-... headers={'cache-control':'no-cache'}) ①
-connect: (diveintopython3.org, 80) ②
+... headers={'cache-control':'no-cache'}) ①
+connect: (diveintopython3.org, 80) ②
send: b'GET /examples/feed.xml HTTP/1.1
Host: diveintopython3.org
user-agent: Python-httplib2/$Rev: 259 $
@@ -400,9 +400,9 @@ reply: 'HTTP/1.1 200 OK'
…further debugging information omitted…
>>> response2.status
200
->>> response2.fromcache ③
+>>> response2.fromcache ③
False
->>> print(dict(response2.items())) ④
+>>> print(dict(response2.items())) ④
{'status': '200',
'content-length': '3070',
'content-location': 'http://diveintopython3.org/examples/feed.xml',
@@ -434,14 +434,14 @@ reply: 'HTTP/1.1 200 OK'
>>> import httplib2
>>> httplib2.debuglevel = 1
>>> h = httplib2.Http('.cache')
->>> response, content = h.request('http://diveintopython3.org/') ①
+>>> response, content = h.request('http://diveintopython3.org/') ①
connect: (diveintopython3.org, 80)
send: b'GET / HTTP/1.1
Host: diveintopython3.org
accept-encoding: deflate, gzip
user-agent: Python-httplib2/$Rev: 259 $'
reply: 'HTTP/1.1 200 OK'
->>> print(dict(response.items())) ②
+>>> print(dict(response.items())) ②
{'-content-encoding': 'gzip',
'accept-ranges': 'bytes',
'connection': 'close',
@@ -454,7 +454,7 @@ reply: 'HTTP/1.1 200 OK'
'server': 'Apache',
'status': '304',
'vary': 'Accept-Encoding,User-Agent'}
->>> len(content) ③
+>>> len(content) ③
6657
- Instead of the feed, this time we’re going to download the site’s home page, which is HTML. Since this is the first time you’lve ever requested this page,
httplib2 has little to work with, and it sends out a minimum of headers with the request.
@@ -464,22 +464,22 @@ reply: 'HTTP/1.1 200 OK'
# continued from the previous example
->>> response, content = h.request('http://diveintopython3.org/') ①
+>>> response, content = h.request('http://diveintopython3.org/') ①
connect: (diveintopython3.org, 80)
send: b'GET / HTTP/1.1
Host: diveintopython3.org
-if-none-match: "7f806d-1a01-9fb97900" ②
-if-modified-since: Tue, 02 Jun 2009 02:51:48 GMT ③
+if-none-match: "7f806d-1a01-9fb97900" ②
+if-modified-since: Tue, 02 Jun 2009 02:51:48 GMT ③
accept-encoding: deflate, gzip
user-agent: Python-httplib2/$Rev: 259 $'
-reply: 'HTTP/1.1 304 Not Modified' ④
->>> response.fromcache ⑤
+reply: 'HTTP/1.1 304 Not Modified' ④
+>>> response.fromcache ⑤
True
->>> response.status ⑥
+>>> response.status ⑥
200
->>> response.dict['status'] ⑦
+>>> response.dict['status'] ⑦
'304'
->>> len(content) ⑧
+>>> len(content) ⑧
6657
- You request the same page again, with the same
Http object (and the same local cache).
@@ -501,11 +501,11 @@ user-agent: Python-httplib2/$Rev: 259 $'
connect: (diveintopython3.org, 80)
send: b'GET / HTTP/1.1
Host: diveintopython3.org
-accept-encoding: deflate, gzip ①
+accept-encoding: deflate, gzip ①
user-agent: Python-httplib2/$Rev: 259 $'
reply: 'HTTP/1.1 200 OK'
>>> print(dict(response.items()))
-{'-content-encoding': 'gzip', ②
+{'-content-encoding': 'gzip', ②
'accept-ranges': 'bytes',
'connection': 'close',
'content-length': '6657',
@@ -681,7 +681,8 @@ reply: 'HTTP/1.1 301 Moved Permanently'
- How to control caching with HTTP headers on Google Doctype
-
© 2001–9 Mark Pilgrim
+
diff --git a/iterators-and-generators.html b/iterators-and-generators.html
index e79d2b2..da6adde 100644
--- a/iterators-and-generators.html
+++ b/iterators-and-generators.html
@@ -13,10 +13,10 @@ h1:before{counter-increment:h1;content:''}
-You are here: Home ‣ Dive Into Python 3 ‣
+
You are here: Home ‣ Dive Into Python 3 ‣
Secret Leftover Page
-❝ You step in the stream / but the water has moved on. / This page is not here. ❞
— 404 Not Found haiku
+
❝ You step in the stream / but the water has moved on. / This page is not here. ❞
— 404 Not Found haiku
Huh?
diff --git a/iterators.html b/iterators.html
index 018af2a..f2554c9 100644
--- a/iterators.html
+++ b/iterators.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 6}
-You are here: Home ‣ Dive Into Python 3 ‣
+
You are here: Home ‣ Dive Into Python 3 ‣
Difficulty level: ♦♦♦♢♢
Iterators
-❝ East is East, and West is West, and never the twain shall meet. ❞
— Rudyard Kipling
+
❝ East is East, and West is West, and never the twain shall meet. ❞
— Rudyard Kipling
Diving In
@@ -25,7 +25,7 @@ body{counter-reset:h1 6}
Remember the Fibonacci generator? Here it is as a built-from-scratch iterator:
class Fib:
+class Fib:
'''iterator that yields numbers in the Fibonacci sequence'''
def __init__(self, max):
@@ -45,7 +45,7 @@ body{counter-reset:h1 6}
Let’s take that one line at a time.
-
class Fib:
+class Fib:
class? What’s a class?
@@ -57,9 +57,8 @@ body{counter-reset:h1 6}
Defining a class in Python is simple. As with functions, there is no separate interface definition. Just define the class and start coding. A Python class starts with the reserved word class, followed by the class name. Technically, that’s all that’s required, since a class doesn’t need to inherit from any other class.
-
-class PapayaWhip: ①
- pass ②
+class PapayaWhip: ①
+ pass ②
- The name of this class is
PapayaWhip, and it doesn’t inherit from any other class. Class names are usually capitalized, EachWordLikeThis, but this is only a convention, not a requirement.
- You probably guessed this, but everything in a class is indented, just like the code within a function,
if statement, for loop, or any other block of code. The first line not indented is outside the class.
@@ -68,7 +67,7 @@ class PapayaWhip: ①
This PapayaWhip class doesn’t define any methods or attributes, but syntactically, there needs to be something in the definition, thus the pass statement. This is a Python reserved word that just means “move along, nothing to see here”. It’s a statement that does nothing, and it’s a good placeholder when you’re stubbing out functions or classes.
-☞The pass statement in Python is like a empty set of curly braces ({}) in Java or C.
+
☞The pass statement in Python is like a empty set of curly braces ({}) in Java or C.
Many classes are inherited from other classes, but this one is not. Many classes define methods, but this one does not. There is nothing that a Python class absolutely must have, other than a name. In particular, C++ programmers may find it odd that Python classes don’t have explicit constructors and destructors. Although it’s not required, Python classes can have something similar to a constructor: the __init__() method.
@@ -77,11 +76,10 @@ class PapayaWhip: ①
This example shows the initialization of the Fib class using the __init__ method.
-
-class Fib:
- '''iterator that yields numbers in the Fibonacci sequence''' ①
+class Fib:
+ '''iterator that yields numbers in the Fibonacci sequence''' ①
- def __init__(self, max): ②
+ def __init__(self, max): ②
- Classes can (and should) have
docstrings too, just like modules and functions.
- The
__init__() method is called immediately after an instance of the class is created. It would be tempting but incorrect to call this the constructor of the class. It’s tempting, because it looks like a constructor (by convention, the __init__() method is the first method defined for the class), acts like one (it’s the first piece of code executed in a newly created instance of the class), and even sounds like one. Incorrect, because the object has already been constructed by the time the __init__() method is called, and you already have a valid reference to the new instance of the class.
@@ -98,12 +96,12 @@ class Fib:
Instantiating classes in Python is straightforward. To instantiate a class, simply call the class as if it were a function, passing the arguments that the __init__() method requires. The return value will be the newly created object.
>>> import fibonacci2
->>> fib = fibonacci2.Fib(100) ①
->>> fib ②
+>>> fib = fibonacci2.Fib(100) ①
+>>> fib ②
<fibonacci2.Fib object at 0x00DB8810>
->>> fib.__class__ ③
+>>> fib.__class__ ③
<class 'fibonacci2.Fib'>
->>> fib.__doc__ ④
+>>> fib.__doc__ ④
'iterator that yields numbers in the Fibonacci sequence'
- You are creating an instance of the
Fib class (defined in the fibonacci2 module) and assigning the newly created instance to the variable fib. You are passing one parameter, 100, which will end up as the max argument in Fib’s __init__() method.
@@ -113,7 +111,7 @@ class Fib:
-☞In Python, simply call a class as if it were a function to create a new instance of the class. There is no explicit new operator like C++ or Java.
+
☞In Python, simply call a class as if it were a function to create a new instance of the class. There is no explicit new operator like C++ or Java.
⁂
@@ -122,22 +120,22 @@ class Fib:
On to the next line:
-
class Fib:
+class Fib:
def __init__(self, max):
- self.max = max ①
+ self.max = max ①
- What is self.max? It’s an instance variable. It is completely separate from max, which was passed into the
__init__() method as an argument. self.max is “global” to the instance. That means that you can access it from other methods.
-class Fib:
+class Fib:
def __init__(self, max):
- self.max = max ①
+ self.max = max ①
.
.
.
def __next__(self):
fib = self.a
- if fib > self.max: ②
+ if fib > self.max: ②
- self.max is defined in the
__init__() method…
- …and referenced in the
__next__() method.
@@ -161,20 +159,20 @@ class Fib:
Now you’re ready to learn how to build an iterator. An iterator is just a class that defines an __iter__() method.
class Fib: ①
- def __init__(self, max): ②
+class Fib: ①
+ def __init__(self, max): ②
self.max = max
- def __iter__(self): ③
+ def __iter__(self): ③
self.a, self.b = 0, 1
return self
- def __next__(self): ④
+ def __next__(self): ④
fib = self.a
if fib > self.max:
- raise StopIteration ⑤
+ raise StopIteration ⑤
self.a, self.b = self.b, self.a + self.b
- return fib ⑥
+ return fib ⑥
- To build an iterator from scratch,
fib needs to be a class, not a function.
- “Calling”
Fib(max) is really creating an instance of this class and calling its __init__() method with max. The __init__() method saves the maximum value as an instance variable so other methods can refer to it later.
@@ -211,7 +209,7 @@ class Fib:
Now it’s time for the finale. Let’s rewrite the plural rules generator as an iterator.
class LazyRules:
+class LazyRules:
rules_filename = 'plural6-rules.txt'
def __init__(self):
@@ -247,12 +245,12 @@ rules = LazyRules()
Let’s take the class one bite at a time.
-
class LazyRules:
+class LazyRules:
rules_filename = 'plural6-rules.txt'
- def __init__(self): ①
- self.pattern_file = open(self.rules_filename) ③
- self.cache = [] ②
+ def __init__(self): ①
+ self.pattern_file = open(self.rules_filename) ③
+ self.cache = [] ②
- The
__init__() method is only going to be called once, when you instantiate the class and assign it to rules.
- Since this is only going to get called once, it’s the perfect place to open the pattern file. You’ll read it later; no point doing more than you absolutely have to until absolutely necessary!
@@ -265,16 +263,16 @@ rules = LazyRules()
>>> import plural6
>>> r1 = plural6.LazyRules()
>>> r2 = plural6.LazyRules()
->>> r1.rules_filename ①
+>>> r1.rules_filename ①
'plural6-rules.txt'
>>> r2.rules_filename
'plural6-rules.txt'
->>> r1.__class__.rules_filename ②
+>>> r1.__class__.rules_filename ②
'plural6-rules.txt'
->>> r1.__class__.rules_filename = 'papayawhip.txt' ③
+>>> r1.__class__.rules_filename = 'papayawhip.txt' ③
>>> r1.rules_filename
'papayawhip.txt'
->>> r2.rules_filename ④
+>>> r2.rules_filename ④
'papayawhip.txt'
- FIXME
@@ -285,9 +283,9 @@ rules = LazyRules()
And now back to our show.
-
def __iter__(self): ①
- self.cache_index = 0 ②
- return self ③
+ def __iter__(self): ①
+ self.cache_index = 0 ②
+ return self ③
- The
__iter__() method will be called every time someone — say, a for loop — calls iter(rules).
@@ -295,14 +293,14 @@ rules = LazyRules()
- Finally, the
__iter__() method returns self, which signals that this class will take care of returning its own values throughout an iteration.
- def __next__(self): ①
+ def __next__(self): ①
.
.
.
pattern, search, replace = line.split(None, 3)
- funcs = build_match_and_apply_functions( ②
+ funcs = build_match_and_apply_functions( ②
pattern, search, replace)
- self.cache.append(funcs) ③
+ self.cache.append(funcs) ③
return funcs
- The
__next__() method gets called whenever someone — say, a for loop — calls next(rules). This method will only make sense if we start at the end and work backwards. So let’s do that.
@@ -312,32 +310,32 @@ rules = LazyRules()
Moving backwards…
-
def __next__(self):
+ def __next__(self):
.
.
.
- line = self.pattern_file.readline() ①
- if not line: ②
+ line = self.pattern_file.readline() ①
+ if not line: ②
self.pattern_file.close()
- raise StopIteration ③
+ raise StopIteration ③
.
.
.
- A bit of advanced file trickery here. The
readline() method (note: singular, not the plural readlines()) reads exactly one line from an open file. Specifically, the next line. (File objects are iterators too! It’s iterators all the way down…)
- If there was a line for
readline() to read, line will not be an empty string. Even if the file contained a blank line, line would end up as the one-character string '\n' (a carriage return). If line is really an empty string, that means there are no more lines to read from the file.
- - When we reach the end of the file, we should close the file and raise the magic
StopIteration exception. Remember, we got to this point because we needed a match and apply function for the next rule. The next rule comes from the next line of the file… but there is no next line! Therefore, we have no value to return. The iteration is over. (♫ The party’s over… ♫)
+ - When we reach the end of the file, we should close the file and raise the magic
StopIteration exception. Remember, we got to this point because we needed a match and apply function for the next rule. The next rule comes from the next line of the file… but there is no next line! Therefore, we have no value to return. The iteration is over. (♫ The party’s over… ♫)
Moving backwards all the way to the start of the __next__() method…
-
def __next__(self):
+ def __next__(self):
self.cache_index += 1
if len(self.cache) >= self.cache_index:
- return self.cache[self.cache_index - 1] ①
+ return self.cache[self.cache_index - 1] ①
if self.pattern_file.closed:
- raise StopIteration ②
+ raise StopIteration ②
.
.
.
@@ -374,8 +372,9 @@ rules = LazyRules()
- PEP 255: Simple Generators
-
© 2001–9 Mark Pilgrim
+
diff --git a/j/dip3.js b/j/dip3.js
index cb23697..789a221 100644
--- a/j/dip3.js
+++ b/j/dip3.js
@@ -29,7 +29,8 @@ POSSIBILITY OF SUCH DAMAGE.
var HS = {'visible': 'hide', 'hidden': 'show'};
$(document).ready(function() {
hideTOC();
-
+ prettyPrint();
+
/* "hide", "open in new window", and (optionally) "download" widgets on code & screen blocks */
$("pre > code").each(function(i) {
var pre = $(this.parentNode);
@@ -90,6 +91,7 @@ $(document).ready(function() {
}
});
});
+
}); /* document.ready */
function toggleCodeBlock(id) {
@@ -100,7 +102,7 @@ function toggleCodeBlock(id) {
function plainTextOnClick(id) {
var clone = $("#" + id).clone();
- clone.find("div.w, span").remove();
+ clone.find("div.w, span.u").remove();
var win = window.open("about:blank", "plaintext", "toolbar=0,scrollbars=1,location=0,statusbar=0,menubar=0,resizable=1,width=600,height=400,left=35,top=75");
win.document.open();
win.document.write('
' + clone.html());
diff --git a/j/prettify.js b/j/prettify.js
new file mode 100644
index 0000000..80a5554
--- /dev/null
+++ b/j/prettify.js
@@ -0,0 +1,1427 @@
+// Copyright (C) 2006 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+// Changes from upstream:
+// - use class=pp instead of class=prettyprint to declare blocks-to-colorize
+
+
+/**
+ * @fileoverview
+ * some functions for browser-side pretty printing of code contained in html.
+ *
+ * The lexer should work on a number of languages including C and friends,
+ * Java, Python, Bash, SQL, HTML, XML, CSS, Javascript, and Makefiles.
+ * It works passably on Ruby, PHP and Awk and a decent subset of Perl, but,
+ * because of commenting conventions, doesn't work on Smalltalk, Lisp-like, or
+ * CAML-like languages.
+ *
+ * If there's a language not mentioned here, then I don't know it, and don't
+ * know whether it works. If it has a C-like, Bash-like, or XML-like syntax
+ * then it should work passably.
+ *
+ * Usage:
+ * 1) include this source file in an html page via
+ *
+ * 2) define style rules. See the example page for examples.
+ * 3) mark the and tags in your source with class=pp.
+ * You can also use the (html deprecated) tag, but the pretty printer
+ * needs to do more substantial DOM manipulations to support that, so some
+ * css styles may not be preserved.
+ * That's it. I wanted to keep the API as simple as possible, so there's no
+ * need to specify which language the code is in.
+ *
+ * Change log:
+ * cbeust, 2006/08/22
+ * Java annotations (start with "@") are now captured as literals ("lit")
+ */
+
+// JSLint declarations
+/*global console, document, navigator, setTimeout, window */
+
+/**
+ * Split {@code prettyPrint} into multiple timeouts so as not to interfere with
+ * UI events.
+ * If set to {@code false}, {@code prettyPrint()} is synchronous.
+ */
+window['PR_SHOULD_USE_CONTINUATION'] = true;
+
+/** the number of characters between tab columns */
+window['PR_TAB_WIDTH'] = 8;
+
+/** Walks the DOM returning a properly escaped version of innerHTML.
+ * @param {Node} node
+ * @param {Array.<string>} out output buffer that receives chunks of HTML.
+ */
+window['PR_normalizedHtml']
+
+/** Contains functions for creating and registering new language handlers.
+ * @type {Object}
+ */
+ = window['PR']
+
+/** Pretty print a chunk of code.
+ *
+ * @param {string} sourceCodeHtml code as html
+ * @return {string} code as html, but prettier
+ */
+ = window['prettyPrintOne']
+/** Find all the {@code <pre>} and {@code <code>} tags in the DOM with
+ * {@code class=pp} and prettify them.
+ * @param {Function?} opt_whenDone if specified, called when the last entry
+ * has been finished.
+ */
+ = window['prettyPrint'] = void 0;
+
+/** browser detection. @extern */
+window['_pr_isIE6'] = function () {
+ var isIE6 = navigator && navigator.userAgent &&
+ /\bMSIE 6\./.test(navigator.userAgent);
+ window['_pr_isIE6'] = function () { return isIE6; };
+ return isIE6;
+};
+
+
+(function () {
+ // Keyword lists for various languages.
+ var FLOW_CONTROL_KEYWORDS =
+ "break continue do else for if return while ";
+ var C_KEYWORDS = FLOW_CONTROL_KEYWORDS + "auto case char const default " +
+ "double enum extern float goto int long register short signed sizeof " +
+ "static struct switch typedef union unsigned void volatile ";
+ var COMMON_KEYWORDS = C_KEYWORDS + "catch class delete false import " +
+ "new operator private protected public this throw true try ";
+ var CPP_KEYWORDS = COMMON_KEYWORDS + "alignof align_union asm axiom bool " +
+ "concept concept_map const_cast constexpr decltype " +
+ "dynamic_cast explicit export friend inline late_check " +
+ "mutable namespace nullptr reinterpret_cast static_assert static_cast " +
+ "template typeid typename typeof using virtual wchar_t where ";
+ var JAVA_KEYWORDS = COMMON_KEYWORDS +
+ "boolean byte extends final finally implements import instanceof null " +
+ "native package strictfp super synchronized throws transient ";
+ var CSHARP_KEYWORDS = JAVA_KEYWORDS +
+ "as base by checked decimal delegate descending event " +
+ "fixed foreach from group implicit in interface internal into is lock " +
+ "object out override orderby params partial readonly ref sbyte sealed " +
+ "stackalloc string select uint ulong unchecked unsafe ushort var ";
+ var JSCRIPT_KEYWORDS = COMMON_KEYWORDS +
+ "debugger eval export function get null set undefined var with " +
+ "Infinity NaN ";
+ var PERL_KEYWORDS = "caller delete die do dump elsif eval exit foreach for " +
+ "goto if import last local my next no our print package redo require " +
+ "sub undef unless until use wantarray while BEGIN END ";
+ var PYTHON_KEYWORDS = FLOW_CONTROL_KEYWORDS + "and as assert class def del " +
+ "elif except exec finally from global import in is lambda " +
+ "nonlocal not or pass print raise try with yield " +
+ "False True None ";
+ var RUBY_KEYWORDS = FLOW_CONTROL_KEYWORDS + "alias and begin case class def" +
+ " defined elsif end ensure false in module next nil not or redo rescue " +
+ "retry self super then true undef unless until when yield BEGIN END ";
+ var SH_KEYWORDS = FLOW_CONTROL_KEYWORDS + "case done elif esac eval fi " +
+ "function in local set then until ";
+ var ALL_KEYWORDS = (
+ CPP_KEYWORDS + CSHARP_KEYWORDS + JSCRIPT_KEYWORDS + PERL_KEYWORDS +
+ PYTHON_KEYWORDS + RUBY_KEYWORDS + SH_KEYWORDS);
+
+ // token style names. correspond to css classes
+ /** token style for a string literal */
+ var PR_STRING = 'str';
+ /** token style for a keyword */
+ var PR_KEYWORD = 'kwd';
+ /** token style for a comment */
+ var PR_COMMENT = 'com';
+ /** token style for a type */
+ var PR_TYPE = 'typ';
+ /** token style for a literal value. e.g. 1, null, true. */
+ var PR_LITERAL = 'lit';
+ /** token style for a punctuation string. */
+ var PR_PUNCTUATION = 'pun';
+ /** token style for a punctuation string. */
+ var PR_PLAIN = 'pln';
+
+ /** token style for an sgml tag. */
+ var PR_TAG = 'tag';
+ /** token style for a markup declaration such as a DOCTYPE. */
+ var PR_DECLARATION = 'dec';
+ /** token style for embedded source. */
+ var PR_SOURCE = 'src';
+ /** token style for an sgml attribute name. */
+ var PR_ATTRIB_NAME = 'atn';
+ /** token style for an sgml attribute value. */
+ var PR_ATTRIB_VALUE = 'atv';
+
+ /**
+ * A class that indicates a section of markup that is not code, e.g. to allow
+ * embedding of line numbers within code listings.
+ */
+ var PR_NOCODE = 'nocode';
+
+ /** A set of tokens that can precede a regular expression literal in
+ * javascript.
+ * http://www.mozilla.org/js/language/js20/rationale/syntax.html has the full
+ * list, but I've removed ones that might be problematic when seen in
+ * languages that don't support regular expression literals.
+ *
+ * <p>Specifically, I've removed any keywords that can't precede a regexp
+ * literal in a syntactically legal javascript program, and I've removed the
+ * "in" keyword since it's not a keyword in many languages, and might be used
+ * as a count of inches.
+ *
+ * <p>The link a above does not accurately describe EcmaScript rules since
+ * it fails to distinguish between (a=++/b/i) and (a++/b/i) but it works
+ * very well in practice.
+ *
+ * @private
+ */
+ var REGEXP_PRECEDER_PATTERN = function () {
+ var preceders = [
+ "!", "!=", "!==", "#", "%", "%=", "&", "&&", "&&=",
+ "&=", "(", "*", "*=", /* "+", */ "+=", ",", /* "-", */ "-=",
+ "->", /*".", "..", "...", handled below */ "/", "/=", ":", "::", ";",
+ "<", "<<", "<<=", "<=", "=", "==", "===", ">",
+ ">=", ">>", ">>=", ">>>", ">>>=", "?", "@", "[",
+ "^", "^=", "^^", "^^=", "{", "|", "|=", "||",
+ "||=", "~" /* handles =~ and !~ */,
+ "break", "case", "continue", "delete",
+ "do", "else", "finally", "instanceof",
+ "return", "throw", "try", "typeof"
+ ];
+ var pattern = '(?:^^|[+-]';
+ for (var i = 0; i < preceders.length; ++i) {
+ pattern += '|' + preceders[i].replace(/([^=<>:&a-z])/g, '\\$1');
+ }
+ pattern += ')\\s*'; // matches at end, and matches empty string
+ return pattern;
+ // CAVEAT: this does not properly handle the case where a regular
+ // expression immediately follows another since a regular expression may
+ // have flags for case-sensitivity and the like. Having regexp tokens
+ // adjacent is not valid in any language I'm aware of, so I'm punting.
+ // TODO: maybe style special characters inside a regexp as punctuation.
+ }();
+
+ // Define regexps here so that the interpreter doesn't have to create an
+ // object each time the function containing them is called.
+ // The language spec requires a new object created even if you don't access
+ // the $1 members.
+ var pr_amp = /&/g;
+ var pr_lt = /</g;
+ var pr_gt = />/g;
+ var pr_quot = /\"/g;
+ /** like textToHtml but escapes double quotes to be attribute safe. */
+ function attribToHtml(str) {
+ return str.replace(pr_amp, '&')
+ .replace(pr_lt, '<')
+ .replace(pr_gt, '>')
+ .replace(pr_quot, '"');
+ }
+
+ /** escapest html special characters to html. */
+ function textToHtml(str) {
+ return str.replace(pr_amp, '&')
+ .replace(pr_lt, '<')
+ .replace(pr_gt, '>');
+ }
+
+
+ var pr_ltEnt = /</g;
+ var pr_gtEnt = />/g;
+ var pr_aposEnt = /'/g;
+ var pr_quotEnt = /"/g;
+ var pr_ampEnt = /&/g;
+ var pr_nbspEnt = / /g;
+ /** unescapes html to plain text. */
+ function htmlToText(html) {
+ var pos = html.indexOf('&');
+ if (pos < 0) { return html; }
+ // Handle numeric entities specially. We can't use functional substitution
+ // since that doesn't work in older versions of Safari.
+ // These should be rare since most browsers convert them to normal chars.
+ for (--pos; (pos = html.indexOf('&#', pos + 1)) >= 0;) {
+ var end = html.indexOf(';', pos);
+ if (end >= 0) {
+ var num = html.substring(pos + 3, end);
+ var radix = 10;
+ if (num && num.charAt(0) === 'x') {
+ num = num.substring(1);
+ radix = 16;
+ }
+ var codePoint = parseInt(num, radix);
+ if (!isNaN(codePoint)) {
+ html = (html.substring(0, pos) + String.fromCharCode(codePoint) +
+ html.substring(end + 1));
+ }
+ }
+ }
+
+ return html.replace(pr_ltEnt, '<')
+ .replace(pr_gtEnt, '>')
+ .replace(pr_aposEnt, "'")
+ .replace(pr_quotEnt, '"')
+ .replace(pr_ampEnt, '&')
+ .replace(pr_nbspEnt, ' ');
+ }
+
+ /** is the given node's innerHTML normally unescaped? */
+ function isRawContent(node) {
+ return 'XMP' === node.tagName;
+ }
+
+ function normalizedHtml(node, out) {
+ switch (node.nodeType) {
+ case 1: // an element
+ var name = node.tagName.toLowerCase();
+ out.push('<', name);
+ for (var i = 0; i < node.attributes.length; ++i) {
+ var attr = node.attributes[i];
+ if (!attr.specified) { continue; }
+ out.push(' ');
+ normalizedHtml(attr, out);
+ }
+ out.push('>');
+ for (var child = node.firstChild; child; child = child.nextSibling) {
+ normalizedHtml(child, out);
+ }
+ if (node.firstChild || !/^(?:br|link|img)$/.test(name)) {
+ out.push('<\/', name, '>');
+ }
+ break;
+ case 2: // an attribute
+ out.push(node.name.toLowerCase(), '="', attribToHtml(node.value), '"');
+ break;
+ case 3: case 4: // text
+ out.push(textToHtml(node.nodeValue));
+ break;
+ }
+ }
+
+ /**
+ * Given a group of {@link RegExp}s, returns a {@code RegExp} that globally
+ * matches the union o the sets o strings matched d by the input RegExp.
+ * Since it matches globally, if the input strings have a start-of-input
+ * anchor (/^.../), it is ignored for the purposes of unioning.
+ * @param {Array.<RegExpr>} regexs non multiline, non-global regexs.
+ * @return {RegExp} a global regex.
+ */
+ function combinePrefixPatterns(regexs) {
+ var capturedGroupIndex = 0;
+
+ var needToFoldCase = false;
+ var ignoreCase = false;
+ for (var i = 0, n = regexs.length; i < n; ++i) {
+ var regex = regexs[i];
+ if (regex.ignoreCase) {
+ ignoreCase = true;
+ } else if (/[a-z]/i.test(regex.source.replace(
+ /\\u[0-9a-f]{4}|\\x[0-9a-f]{2}|\\[^ux]/gi, ''))) {
+ needToFoldCase = true;
+ ignoreCase = false;
+ break;
+ }
+ }
+
+ function decodeEscape(charsetPart) {
+ if (charsetPart.charAt(0) !== '\\') { return charsetPart.charCodeAt(0); }
+ switch (charsetPart.charAt(1)) {
+ case 'b': return 8;
+ case 't': return 9;
+ case 'n': return 0xa;
+ case 'v': return 0xb;
+ case 'f': return 0xc;
+ case 'r': return 0xd;
+ case 'u': case 'x':
+ return parseInt(charsetPart.substring(2), 16)
+ || charsetPart.charCodeAt(1);
+ case '0': case '1': case '2': case '3': case '4':
+ case '5': case '6': case '7':
+ return parseInt(charsetPart.substring(1), 8);
+ default: return charsetPart.charCodeAt(1);
+ }
+ }
+
+ function encodeEscape(charCode) {
+ if (charCode < 0x20) {
+ return (charCode < 0x10 ? '\\x0' : '\\x') + charCode.toString(16);
+ }
+ var ch = String.fromCharCode(charCode);
+ if (ch === '\\' || ch === '-' || ch === '[' || ch === ']') {
+ ch = '\\' + ch;
+ }
+ return ch;
+ }
+
+ function caseFoldCharset(charSet) {
+ var charsetParts = charSet.substring(1, charSet.length - 1).match(
+ new RegExp(
+ '\\\\u[0-9A-Fa-f]{4}'
+ + '|\\\\x[0-9A-Fa-f]{2}'
+ + '|\\\\[0-3][0-7]{0,2}'
+ + '|\\\\[0-7]{1,2}'
+ + '|\\\\[\\s\\S]'
+ + '|-'
+ + '|[^-\\\\]',
+ 'g'));
+ var groups = [];
+ var ranges = [];
+ var inverse = charsetParts[0] === '^';
+ for (var i = inverse ? 1 : 0, n = charsetParts.length; i < n; ++i) {
+ var p = charsetParts[i];
+ switch (p) {
+ case '\\B': case '\\b':
+ case '\\D': case '\\d':
+ case '\\S': case '\\s':
+ case '\\W': case '\\w':
+ groups.push(p);
+ continue;
+ }
+ var start = decodeEscape(p);
+ var end;
+ if (i + 2 < n && '-' === charsetParts[i + 1]) {
+ end = decodeEscape(charsetParts[i + 2]);
+ i += 2;
+ } else {
+ end = start;
+ }
+ ranges.push([start, end]);
+ // If the range might intersect letters, then expand it.
+ if (!(end < 65 || start > 122)) {
+ if (!(end < 65 || start > 90)) {
+ ranges.push([Math.max(65, start) | 32, Math.min(end, 90) | 32]);
+ }
+ if (!(end < 97 || start > 122)) {
+ ranges.push([Math.max(97, start) & ~32, Math.min(end, 122) & ~32]);
+ }
+ }
+ }
+
+ // [[1, 10], [3, 4], [8, 12], [14, 14], [16, 16], [17, 17]]
+ // -> [[1, 12], [14, 14], [16, 17]]
+ ranges.sort(function (a, b) { return (a[0] - b[0]) || (b[1] - a[1]); });
+ var consolidatedRanges = [];
+ var lastRange = [NaN, NaN];
+ for (var i = 0; i < ranges.length; ++i) {
+ var range = ranges[i];
+ if (range[0] <= lastRange[1] + 1) {
+ lastRange[1] = Math.max(lastRange[1], range[1]);
+ } else {
+ consolidatedRanges.push(lastRange = range);
+ }
+ }
+
+ var out = ['['];
+ if (inverse) { out.push('^'); }
+ out.push.apply(out, groups);
+ for (var i = 0; i < consolidatedRanges.length; ++i) {
+ var range = consolidatedRanges[i];
+ out.push(encodeEscape(range[0]));
+ if (range[1] > range[0]) {
+ if (range[1] + 1 > range[0]) { out.push('-'); }
+ out.push(encodeEscape(range[1]));
+ }
+ }
+ out.push(']');
+ return out.join('');
+ }
+
+ function allowAnywhereFoldCaseAndRenumberGroups(regex) {
+ // Split into character sets, escape sequences, punctuation strings
+ // like ('(', '(?:', ')', '^'), and runs of characters that do not
+ // include any of the above.
+ var parts = regex.source.match(
+ new RegExp(
+ '(?:'
+ + '\\[(?:[^\\x5C\\x5D]|\\\\[\\s\\S])*\\]' // a character set
+ + '|\\\\u[A-Fa-f0-9]{4}' // a unicode escape
+ + '|\\\\x[A-Fa-f0-9]{2}' // a hex escape
+ + '|\\\\[0-9]+' // a back-reference or octal escape
+ + '|\\\\[^ux0-9]' // other escape sequence
+ + '|\\(\\?[:!=]' // start of a non-capturing group
+ + '|[\\(\\)\\^]' // start/emd of a group, or line start
+ + '|[^\\x5B\\x5C\\(\\)\\^]+' // run of other characters
+ + ')',
+ 'g'));
+ var n = parts.length;
+
+ // Maps captured group numbers to the number they will occupy in
+ // the output or to -1 if that has not been determined, or to
+ // undefined if they need not be capturing in the output.
+ var capturedGroups = [];
+
+ // Walk over and identify back references to build the capturedGroups
+ // mapping.
+ var groupIndex;
+ for (var i = 0, groupIndex = 0; i < n; ++i) {
+ var p = parts[i];
+ if (p === '(') {
+ // groups are 1-indexed, so max group index is count of '('
+ ++groupIndex;
+ } else if ('\\' === p.charAt(0)) {
+ var decimalValue = +p.substring(1);
+ if (decimalValue && decimalValue <= groupIndex) {
+ capturedGroups[decimalValue] = -1;
+ }
+ }
+ }
+
+ // Renumber groups and reduce capturing groups to non-capturing groups
+ // where possible.
+ for (var i = 1; i < capturedGroups.length; ++i) {
+ if (-1 === capturedGroups[i]) {
+ capturedGroups[i] = ++capturedGroupIndex;
+ }
+ }
+ for (var i = 0, groupIndex = 0; i < n; ++i) {
+ var p = parts[i];
+ if (p === '(') {
+ ++groupIndex;
+ if (capturedGroups[groupIndex] === undefined) {
+ parts[i] = '(?:';
+ }
+ } else if ('\\' === p.charAt(0)) {
+ var decimalValue = +p.substring(1);
+ if (decimalValue && decimalValue <= groupIndex) {
+ parts[i] = '\\' + capturedGroups[groupIndex];
+ }
+ }
+ }
+
+ // Remove any prefix anchors so that the output will match anywhere.
+ // ^^ really does mean an anchored match though.
+ for (var i = 0, groupIndex = 0; i < n; ++i) {
+ if ('^' === parts[i] && '^' !== parts[i + 1]) { parts[i] = ''; }
+ }
+
+ // Expand letters to groupts to handle mixing of case-sensitive and
+ // case-insensitive patterns if necessary.
+ if (regex.ignoreCase && needToFoldCase) {
+ for (var i = 0; i < n; ++i) {
+ var p = parts[i];
+ var ch0 = p.charAt(0);
+ if (p.length >= 2 && ch0 === '[') {
+ parts[i] = caseFoldCharset(p);
+ } else if (ch0 !== '\\') {
+ // TODO: handle letters in numeric escapes.
+ parts[i] = p.replace(
+ /[a-zA-Z]/g,
+ function (ch) {
+ var cc = ch.charCodeAt(0);
+ return '[' + String.fromCharCode(cc & ~32, cc | 32) + ']';
+ });
+ }
+ }
+ }
+
+ return parts.join('');
+ }
+
+ var rewritten = [];
+ for (var i = 0, n = regexs.length; i < n; ++i) {
+ var regex = regexs[i];
+ if (regex.global || regex.multiline) { throw new Error('' + regex); }
+ rewritten.push(
+ '(?:' + allowAnywhereFoldCaseAndRenumberGroups(regex) + ')');
+ }
+
+ return new RegExp(rewritten.join('|'), ignoreCase ? 'gi' : 'g');
+ }
+
+ var PR_innerHtmlWorks = null;
+ function getInnerHtml(node) {
+ // inner html is hopelessly broken in Safari 2.0.4 when the content is
+ // an html description of well formed XML and the containing tag is a PRE
+ // tag, so we detect that case and emulate innerHTML.
+ if (null === PR_innerHtmlWorks) {
+ var testNode = document.createElement('PRE');
+ testNode.appendChild(
+ document.createTextNode('<!DOCTYPE foo PUBLIC "foo bar">\n<foo />'));
+ PR_innerHtmlWorks = !/</.test(testNode.innerHTML);
+ }
+
+ if (PR_innerHtmlWorks) {
+ var content = node.innerHTML;
+ // XMP tags contain unescaped entities so require special handling.
+ if (isRawContent(node)) {
+ content = textToHtml(content);
+ }
+ return content;
+ }
+
+ var out = [];
+ for (var child = node.firstChild; child; child = child.nextSibling) {
+ normalizedHtml(child, out);
+ }
+ return out.join('');
+ }
+
+ /** returns a function that expand tabs to spaces. This function can be fed
+ * successive chunks of text, and will maintain its own internal state to
+ * keep track of how tabs are expanded.
+ * @return {function (string) : string} a function that takes
+ * plain text and return the text with tabs expanded.
+ * @private
+ */
+ function makeTabExpander(tabWidth) {
+ var SPACES = ' ';
+ var charInLine = 0;
+
+ return function (plainText) {
+ // walk over each character looking for tabs and newlines.
+ // On tabs, expand them. On newlines, reset charInLine.
+ // Otherwise increment charInLine
+ var out = null;
+ var pos = 0;
+ for (var i = 0, n = plainText.length; i < n; ++i) {
+ var ch = plainText.charAt(i);
+
+ switch (ch) {
+ case '\t':
+ if (!out) { out = []; }
+ out.push(plainText.substring(pos, i));
+ // calculate how much space we need in front of this part
+ // nSpaces is the amount of padding -- the number of spaces needed
+ // to move us to the next column, where columns occur at factors of
+ // tabWidth.
+ var nSpaces = tabWidth - (charInLine % tabWidth);
+ charInLine += nSpaces;
+ for (; nSpaces >= 0; nSpaces -= SPACES.length) {
+ out.push(SPACES.substring(0, nSpaces));
+ }
+ pos = i + 1;
+ break;
+ case '\n':
+ charInLine = 0;
+ break;
+ default:
+ ++charInLine;
+ }
+ }
+ if (!out) { return plainText; }
+ out.push(plainText.substring(pos));
+ return out.join('');
+ };
+ }
+
+ var pr_chunkPattern = new RegExp(
+ '[^<]+' // A run of characters other than '<'
+ + '|<\!--[\\s\\S]*?--\>' // an HTML comment
+ + '|<!\\[CDATA\\[[\\s\\S]*?\\]\\]>' // a CDATA section
+ + '|</?[a-zA-Z][^>]*>' // a probable tag that should not be highlighted
+ + '|<', // A '<' that does not begin a larger chunk
+ 'g');
+ var pr_commentPrefix = /^<\!--/;
+ var pr_cdataPrefix = /^<\[CDATA\[/;
+ var pr_brPrefix = /^<br\b/i;
+ var pr_tagNameRe = /^<(\/?)([a-zA-Z]+)/;
+
+ /** split markup into chunks of html tags (style null) and
+ * plain text (style {@link #PR_PLAIN}), converting tags which are
+ * significant for tokenization (<br>) into their textual equivalent.
+ *
+ * @param {string} s html where whitespace is considered significant.
+ * @return {Object} source code and extracted tags.
+ * @private
+ */
+ function extractTags(s) {
+ // since the pattern has the 'g' modifier and defines no capturing groups,
+ // this will return a list of all chunks which we then classify and wrap as
+ // PR_Tokens
+ var matches = s.match(pr_chunkPattern);
+ var sourceBuf = [];
+ var sourceBufLen = 0;
+ var extractedTags = [];
+ if (matches) {
+ for (var i = 0, n = matches.length; i < n; ++i) {
+ var match = matches[i];
+ if (match.length > 1 && match.charAt(0) === '<') {
+ if (pr_commentPrefix.test(match)) { continue; }
+ if (pr_cdataPrefix.test(match)) {
+ // strip CDATA prefix and suffix. Don't unescape since it's CDATA
+ sourceBuf.push(match.substring(9, match.length - 3));
+ sourceBufLen += match.length - 12;
+ } else if (pr_brPrefix.test(match)) {
+ // <br> tags are lexically significant so convert them to text.
+ // This is undone later.
+ sourceBuf.push('\n');
+ ++sourceBufLen;
+ } else {
+ if (match.indexOf(PR_NOCODE) >= 0 && isNoCodeTag(match)) {
+ // A <span class="nocode"> will start a section that should be
+ // ignored. Continue walking the list until we see a matching end
+ // tag.
+ var name = match.match(pr_tagNameRe)[2];
+ var depth = 1;
+ var j;
+ end_tag_loop:
+ for (j = i + 1; j < n; ++j) {
+ var name2 = matches[j].match(pr_tagNameRe);
+ if (name2 && name2[2] === name) {
+ if (name2[1] === '/') {
+ if (--depth === 0) { break end_tag_loop; }
+ } else {
+ ++depth;
+ }
+ }
+ }
+ if (j < n) {
+ extractedTags.push(
+ sourceBufLen, matches.slice(i, j + 1).join(''));
+ i = j;
+ } else { // Ignore unclosed sections.
+ extractedTags.push(sourceBufLen, match);
+ }
+ } else {
+ extractedTags.push(sourceBufLen, match);
+ }
+ }
+ } else {
+ var literalText = htmlToText(match);
+ sourceBuf.push(literalText);
+ sourceBufLen += literalText.length;
+ }
+ }
+ }
+ return { source: sourceBuf.join(''), tags: extractedTags };
+ }
+
+ /** True if the given tag contains a class attribute with the nocode class. */
+ function isNoCodeTag(tag) {
+ return !!tag
+ // First canonicalize the representation of attributes
+ .replace(/\s(\w+)\s*=\s*(?:\"([^\"]*)\"|'([^\']*)'|(\S+))/g,
+ ' $1="$2$3$4"')
+ // Then look for the attribute we want.
+ .match(/[cC][lL][aA][sS][sS]=\"[^\"]*\bnocode\b/);
+ }
+
+ /**
+ * Apply the given language handler to sourceCode and add the resulting
+ * decorations to out.
+ * @param {number} basePos the index of sourceCode within the chunk of source
+ * whose decorations are already present on out.
+ */
+ function appendDecorations(basePos, sourceCode, langHandler, out) {
+ if (!sourceCode) { return; }
+ var job = {
+ source: sourceCode,
+ basePos: basePos
+ };
+ langHandler(job);
+ out.push.apply(out, job.decorations);
+ }
+
+ /** Given triples of [style, pattern, context] returns a lexing function,
+ * The lexing function interprets the patterns to find token boundaries and
+ * returns a decoration list of the form
+ * [index_0, style_0, index_1, style_1, ..., index_n, style_n]
+ * where index_n is an index into the sourceCode, and style_n is a style
+ * constant like PR_PLAIN. index_n-1 <= index_n, and style_n-1 applies to
+ * all characters in sourceCode[index_n-1:index_n].
+ *
+ * The stylePatterns is a list whose elements have the form
+ * [style : string, pattern : RegExp, DEPRECATED, shortcut : string].
+ *
+ * Style is a style constant like PR_PLAIN, or can be a string of the
+ * form 'lang-FOO', where FOO is a language extension describing the
+ * language of the portion of the token in $1 after pattern executes.
+ * E.g., if style is 'lang-lisp', and group 1 contains the text
+ * '(hello (world))', then that portion of the token will be passed to the
+ * registered lisp handler for formatting.
+ * The text before and after group 1 will be restyled using this decorator
+ * so decorators should take care that this doesn't result in infinite
+ * recursion. For example, the HTML lexer rule for SCRIPT elements looks
+ * something like ['lang-js', /<[s]cript>(.+?)<\/script>/]. This may match
+ * '<script>foo()<\/script>', which would cause the current decorator to
+ * be called with '<script>' which would not match the same rule since
+ * group 1 must not be empty, so it would be instead styled as PR_TAG by
+ * the generic tag rule. The handler registered for the 'js' extension would
+ * then be called with 'foo()', and finally, the current decorator would
+ * be called with '<\/script>' which would not match the original rule and
+ * so the generic tag rule would identify it as a tag.
+ *
+ * Pattern must only match prefixes, and if it matches a prefix, then that
+ * match is considered a token with the same style.
+ *
+ * Context is applied to the last non-whitespace, non-comment token
+ * recognized.
+ *
+ * Shortcut is an optional string of characters, any of which, if the first
+ * character, gurantee that this pattern and only this pattern matches.
+ *
+ * @param {Array} shortcutStylePatterns patterns that always start with
+ * a known character. Must have a shortcut string.
+ * @param {Array} fallthroughStylePatterns patterns that will be tried in
+ * order if the shortcut ones fail. May have shortcuts.
+ *
+ * @return {function (Object)} a
+ * function that takes source code and returns a list of decorations.
+ */
+ function createSimpleLexer(shortcutStylePatterns, fallthroughStylePatterns) {
+ var shortcuts = {};
+ var tokenizer;
+ (function () {
+ var allPatterns = shortcutStylePatterns.concat(fallthroughStylePatterns);
+ var allRegexs = [];
+ var regexKeys = {};
+ for (var i = 0, n = allPatterns.length; i < n; ++i) {
+ var patternParts = allPatterns[i];
+ var shortcutChars = patternParts[3];
+ if (shortcutChars) {
+ for (var c = shortcutChars.length; --c >= 0;) {
+ shortcuts[shortcutChars.charAt(c)] = patternParts;
+ }
+ }
+ var regex = patternParts[1];
+ var k = '' + regex;
+ if (!regexKeys.hasOwnProperty(k)) {
+ allRegexs.push(regex);
+ regexKeys[k] = null;
+ }
+ }
+ allRegexs.push(/[\0-\uffff]/);
+ tokenizer = combinePrefixPatterns(allRegexs);
+ })();
+
+ var nPatterns = fallthroughStylePatterns.length;
+ var notWs = /\S/;
+
+ /**
+ * Lexes job.source and produces an output array job.decorations of style
+ * classes preceded by the position at which they start in job.source in
+ * order.
+ *
+ * @param {Object} job an object like {@code
+ * source: {string} sourceText plain text,
+ * basePos: {int} position of job.source in the larger chunk of
+ * sourceCode.
+ * }
+ */
+ var decorate = function (job) {
+ var sourceCode = job.source, basePos = job.basePos;
+ /** Even entries are positions in source in ascending order. Odd enties
+ * are style markers (e.g., PR_COMMENT) that run from that position until
+ * the end.
+ * @type {Array.<number|string>}
+ */
+ var decorations = [basePos, PR_PLAIN];
+ var pos = 0; // index into sourceCode
+ var tokens = sourceCode.match(tokenizer) || [];
+ var styleCache = {};
+
+ for (var ti = 0, nTokens = tokens.length; ti < nTokens; ++ti) {
+ var token = tokens[ti];
+ var style = styleCache[token];
+ var match;
+
+ var isEmbedded;
+ if (typeof style === 'string') {
+ isEmbedded = false;
+ } else {
+ var patternParts = shortcuts[token.charAt(0)];
+ if (patternParts) {
+ match = token.match(patternParts[1]);
+ style = patternParts[0];
+ } else {
+ for (var i = 0; i < nPatterns; ++i) {
+ patternParts = fallthroughStylePatterns[i];
+ match = token.match(patternParts[1]);
+ if (match) {
+ style = patternParts[0];
+ break;
+ }
+ }
+
+ if (!match) { // make sure that we make progress
+ style = PR_PLAIN;
+ }
+ }
+
+ isEmbedded = style.length >= 5 && 'lang-' === style.substring(0, 5);
+ if (isEmbedded && !(match && match[1])) {
+ isEmbedded = false;
+ style = PR_SOURCE;
+ }
+
+ if (!isEmbedded) { styleCache[token] = style; }
+ }
+
+ var tokenStart = pos;
+ pos += token.length;
+
+ if (!isEmbedded) {
+ decorations.push(basePos + tokenStart, style);
+ } else { // Treat group 1 as an embedded block of source code.
+ var embeddedSource = match[1];
+ var embeddedSourceStart = token.indexOf(embeddedSource);
+ var embeddedSourceEnd = embeddedSourceStart + embeddedSource.length;
+ var lang = style.substring(5);
+ var size = decorations.length - 10;
+ // Decorate the left of the embedded source
+ appendDecorations(
+ basePos + tokenStart,
+ token.substring(0, embeddedSourceStart),
+ decorate, decorations);
+ // Decorate the embedded source
+ appendDecorations(
+ basePos + tokenStart + embeddedSourceStart,
+ embeddedSource,
+ langHandlerForExtension(lang, embeddedSource),
+ decorations);
+ // Decorate the right of the embedded section
+ appendDecorations(
+ basePos + tokenStart + embeddedSourceEnd,
+ token.substring(embeddedSourceEnd),
+ decorate, decorations);
+ }
+ }
+ job.decorations = decorations;
+ };
+ return decorate;
+ }
+
+ /** returns a function that produces a list of decorations from source text.
+ *
+ * This code treats ", ', and ` as string delimiters, and \ as a string
+ * escape. It does not recognize perl's qq() style strings.
+ * It has no special handling for double delimiter escapes as in basic, or
+ * the tripled delimiters used in python, but should work on those regardless
+ * although in those cases a single string literal may be broken up into
+ * multiple adjacent string literals.
+ *
+ * It recognizes C, C++, and shell style comments.
+ *
+ * @param {Object} options a set of optional parameters.
+ * @return {function (Object)} a function that examines the source code
+ * in the input job and builds the decoration list.
+ */
+ function sourceDecorator(options) {
+ var shortcutStylePatterns = [], fallthroughStylePatterns = [];
+ if (options['tripleQuotedStrings']) {
+ // '''multi-line-string''', 'single-line-string', and double-quoted
+ shortcutStylePatterns.push(
+ [PR_STRING, /^(?:\'\'\'(?:[^\'\\]|\\[\s\S]|\'{1,2}(?=[^\']))*(?:\'\'\'|$)|\"\"\"(?:[^\"\\]|\\[\s\S]|\"{1,2}(?=[^\"]))*(?:\"\"\"|$)|\'(?:[^\\\']|\\[\s\S])*(?:\'|$)|\"(?:[^\\\"]|\\[\s\S])*(?:\"|$))/,
+ null, '\'"']);
+ } else if (options['multiLineStrings']) {
+ // 'multi-line-string', "multi-line-string"
+ shortcutStylePatterns.push(
+ [PR_STRING, /^(?:\'(?:[^\\\']|\\[\s\S])*(?:\'|$)|\"(?:[^\\\"]|\\[\s\S])*(?:\"|$)|\`(?:[^\\\`]|\\[\s\S])*(?:\`|$))/,
+ null, '\'"`']);
+ } else {
+ // 'single-line-string', "single-line-string"
+ shortcutStylePatterns.push(
+ [PR_STRING,
+ /^(?:\'(?:[^\\\'\r\n]|\\.)*(?:\'|$)|\"(?:[^\\\"\r\n]|\\.)*(?:\"|$))/,
+ null, '"\'']);
+ }
+ if (options['hashComments']) {
+ if (options['cStyleComments']) {
+ // Stop C preprocessor declarations at an unclosed open comment
+ shortcutStylePatterns.push(
+ [PR_COMMENT, /^#(?:[^\r\n\/]|\/(?!\*)|\/\*[^\r\n]*?\*\/)*/,
+ null, '#']);
+ } else {
+ shortcutStylePatterns.push([PR_COMMENT, /^#[^\r\n]*/, null, '#']);
+ }
+ }
+ if (options['cStyleComments']) {
+ fallthroughStylePatterns.push([PR_COMMENT, /^\/\/[^\r\n]*/, null]);
+ fallthroughStylePatterns.push(
+ [PR_COMMENT, /^\/\*[\s\S]*?(?:\*\/|$)/, null]);
+ }
+ if (options['regexLiterals']) {
+ var REGEX_LITERAL = (
+ // A regular expression literal starts with a slash that is
+ // not followed by * or / so that it is not confused with
+ // comments.
+ '/(?=[^/*])'
+ // and then contains any number of raw characters,
+ + '(?:[^/\\x5B\\x5C]'
+ // escape sequences (\x5C),
+ + '|\\x5C[\\s\\S]'
+ // or non-nesting character sets (\x5B\x5D);
+ + '|\\x5B(?:[^\\x5C\\x5D]|\\x5C[\\s\\S])*(?:\\x5D|$))+'
+ // finally closed by a /.
+ + '/');
+ fallthroughStylePatterns.push(
+ ['lang-regex',
+ new RegExp('^' + REGEXP_PRECEDER_PATTERN + '(' + REGEX_LITERAL + ')')
+ ]);
+ }
+
+ var keywords = options['keywords'].replace(/^\s+|\s+$/g, '');
+ if (keywords.length) {
+ fallthroughStylePatterns.push(
+ [PR_KEYWORD,
+ new RegExp('^(?:' + keywords.replace(/\s+/g, '|') + ')\\b'), null]);
+ }
+
+ shortcutStylePatterns.push([PR_PLAIN, /^\s+/, null, ' \r\n\t\xA0']);
+ fallthroughStylePatterns.push(
+ // TODO(mikesamuel): recognize non-latin letters and numerals in idents
+ [PR_LITERAL, /^@[a-z_$][a-z_$@0-9]*/i, null, '@'],
+ [PR_TYPE, /^@?[A-Z]+[a-z][A-Za-z_$@0-9]*/, null],
+ [PR_PLAIN, /^[a-z_$][a-z_$@0-9]*/i, null],
+ [PR_LITERAL,
+ new RegExp(
+ '^(?:'
+ // A hex number
+ + '0x[a-f0-9]+'
+ // or an octal or decimal number,
+ + '|(?:\\d(?:_\\d+)*\\d*(?:\\.\\d*)?|\\.\\d\\+)'
+ // possibly in scientific notation
+ + '(?:e[+\\-]?\\d+)?'
+ + ')'
+ // with an optional modifier like UL for unsigned long
+ + '[a-z]*', 'i'),
+ null, '0123456789'],
+ [PR_PUNCTUATION, /^.[^\s\w\.$@\'\"\`\/\#]*/, null]);
+
+ return createSimpleLexer(shortcutStylePatterns, fallthroughStylePatterns);
+ }
+
+ var decorateSource = sourceDecorator({
+ 'keywords': ALL_KEYWORDS,
+ 'hashComments': true,
+ 'cStyleComments': true,
+ 'multiLineStrings': true,
+ 'regexLiterals': true
+ });
+
+ /** Breaks {@code job.source} around style boundaries in
+ * {@code job.decorations} while re-interleaving {@code job.extractedTags},
+ * and leaves the result in {@code job.prettyPrintedHtml}.
+ * @param {Object} job like {
+ * source: {string} source as plain text,
+ * extractedTags: {Array.<number|string>} extractedTags chunks of raw
+ * html preceded by their position in {@code job.source}
+ * in order
+ * decorations: {Array.<number|string} an array of style classes preceded
+ * by the position at which they start in job.source in order
+ * }
+ * @private
+ */
+ function recombineTagsAndDecorations(job) {
+ var sourceText = job.source;
+ var extractedTags = job.extractedTags;
+ var decorations = job.decorations;
+
+ var html = [];
+ // index past the last char in sourceText written to html
+ var outputIdx = 0;
+
+ var openDecoration = null;
+ var currentDecoration = null;
+ var tagPos = 0; // index into extractedTags
+ var decPos = 0; // index into decorations
+ var tabExpander = makeTabExpander(window['PR_TAB_WIDTH']);
+
+ var adjacentSpaceRe = /([\r\n ]) /g;
+ var startOrSpaceRe = /(^| ) /gm;
+ var newlineRe = /\r\n?|\n/g;
+ var trailingSpaceRe = /[ \r\n]$/;
+ var lastWasSpace = true; // the last text chunk emitted ended with a space.
+
+ // A helper function that is responsible for opening sections of decoration
+ // and outputing properly escaped chunks of source
+ function emitTextUpTo(sourceIdx) {
+ if (sourceIdx > outputIdx) {
+ if (openDecoration && openDecoration !== currentDecoration) {
+ // Close the current decoration
+ html.push('</span>');
+ openDecoration = null;
+ }
+ if (!openDecoration && currentDecoration) {
+ openDecoration = currentDecoration;
+ html.push('<span class="', openDecoration, '">');
+ }
+ // This interacts badly with some wikis which introduces paragraph tags
+ // into pre blocks for some strange reason.
+ // It's necessary for IE though which seems to lose the preformattedness
+ // of <pre> tags when their innerHTML is assigned.
+ // http://stud3.tuwien.ac.at/~e0226430/innerHtmlQuirk.html
+ // and it serves to undo the conversion of <br>s to newlines done in
+ // chunkify.
+ var htmlChunk = textToHtml(
+ tabExpander(sourceText.substring(outputIdx, sourceIdx)))
+ .replace(lastWasSpace
+ ? startOrSpaceRe
+ : adjacentSpaceRe, '$1 ');
+ // Keep track of whether we need to escape space at the beginning of the
+ // next chunk.
+ lastWasSpace = trailingSpaceRe.test(htmlChunk);
+ // IE collapses multiple adjacient <br>s into 1 line break.
+ // Prefix every <br> with ' ' can prevent such IE's behavior.
+ var lineBreakHtml = window['_pr_isIE6']() ? ' <br />' : '<br />';
+ html.push(htmlChunk.replace(newlineRe, lineBreakHtml));
+ outputIdx = sourceIdx;
+ }
+ }
+
+ while (true) {
+ // Determine if we're going to consume a tag this time around. Otherwise
+ // we consume a decoration or exit.
+ var outputTag;
+ if (tagPos < extractedTags.length) {
+ if (decPos < decorations.length) {
+ // Pick one giving preference to extractedTags since we shouldn't open
+ // a new style that we're going to have to immediately close in order
+ // to output a tag.
+ outputTag = extractedTags[tagPos] <= decorations[decPos];
+ } else {
+ outputTag = true;
+ }
+ } else {
+ outputTag = false;
+ }
+ // Consume either a decoration or a tag or exit.
+ if (outputTag) {
+ emitTextUpTo(extractedTags[tagPos]);
+ if (openDecoration) {
+ // Close the current decoration
+ html.push('</span>');
+ openDecoration = null;
+ }
+ html.push(extractedTags[tagPos + 1]);
+ tagPos += 2;
+ } else if (decPos < decorations.length) {
+ emitTextUpTo(decorations[decPos]);
+ currentDecoration = decorations[decPos + 1];
+ decPos += 2;
+ } else {
+ break;
+ }
+ }
+ emitTextUpTo(sourceText.length);
+ if (openDecoration) {
+ html.push('</span>');
+ }
+ job.prettyPrintedHtml = html.join('');
+ }
+
+ /** Maps language-specific file extensions to handlers. */
+ var langHandlerRegistry = {};
+ /** Register a language handler for the given file extensions.
+ * @param {function (Object)} handler a function from source code to a list
+ * of decorations. Takes a single argument job which describes the
+ * state of the computation. The single parameter has the form
+ * {@code {
+ * source: {string} as plain text.
+ * decorations: {Array.<number|string>} an array of style classes
+ * preceded by the position at which they start in
+ * job.source in order.
+ * The language handler should assigned this field.
+ * basePos: {int} the position of source in the larger source chunk.
+ * All positions in the output decorations array are relative
+ * to the larger source chunk.
+ * } }
+ * @param {Array.<string>} fileExtensions
+ */
+ function registerLangHandler(handler, fileExtensions) {
+ for (var i = fileExtensions.length; --i >= 0;) {
+ var ext = fileExtensions[i];
+ if (!langHandlerRegistry.hasOwnProperty(ext)) {
+ langHandlerRegistry[ext] = handler;
+ } else if ('console' in window) {
+ console.warn('cannot override language handler %s', ext);
+ }
+ }
+ }
+ function langHandlerForExtension(extension, source) {
+ if (!(extension && langHandlerRegistry.hasOwnProperty(extension))) {
+ // Treat it as markup if the first non whitespace character is a < and
+ // the last non-whitespace character is a >.
+ extension = /^\s*</.test(source)
+ ? 'default-markup'
+ : 'default-code';
+ }
+ return langHandlerRegistry[extension];
+ }
+ registerLangHandler(decorateSource, ['default-code']);
+ registerLangHandler(
+ createSimpleLexer(
+ [],
+ [
+ [PR_PLAIN, /^[^<?]+/],
+ [PR_DECLARATION, /^<!\w[^>]*(?:>|$)/],
+ [PR_COMMENT, /^<\!--[\s\S]*?(?:-\->|$)/],
+ // Unescaped content in an unknown language
+ ['lang-', /^<\?([\s\S]+?)(?:\?>|$)/],
+ ['lang-', /^<%([\s\S]+?)(?:%>|$)/],
+ [PR_PUNCTUATION, /^(?:<[%?]|[%?]>)/],
+ ['lang-', /^<xmp\b[^>]*>([\s\S]+?)<\/xmp\b[^>]*>/i],
+ // Unescaped content in javascript. (Or possibly vbscript).
+ ['lang-js', /^<script\b[^>]*>([\s\S]+?)<\/script\b[^>]*>/i],
+ // Contains unescaped stylesheet content
+ ['lang-css', /^<style\b[^>]*>([\s\S]+?)<\/style\b[^>]*>/i],
+ ['lang-in.tag', /^(<\/?[a-z][^<>]*>)/i]
+ ]),
+ ['default-markup', 'htm', 'html', 'mxml', 'xhtml', 'xml', 'xsl']);
+ registerLangHandler(
+ createSimpleLexer(
+ [
+ [PR_PLAIN, /^[\s]+/, null, ' \t\r\n'],
+ [PR_ATTRIB_VALUE, /^(?:\"[^\"]*\"?|\'[^\']*\'?)/, null, '\"\'']
+ ],
+ [
+ [PR_TAG, /^^<\/?[a-z](?:[\w:-]*\w)?|\/?>$/],
+ [PR_ATTRIB_NAME, /^(?!style\b|on)[a-z](?:[\w:-]*\w)?/],
+ ['lang-uq.val', /^=\s*([^>\'\"\s]*(?:[^>\'\"\s\/]|\/(?=\s)))/],
+ [PR_PUNCTUATION, /^[=<>\/]+/],
+ ['lang-js', /^on\w+\s*=\s*\"([^\"]+)\"/i],
+ ['lang-js', /^on\w+\s*=\s*\'([^\']+)\'/i],
+ ['lang-js', /^on\w+\s*=\s*([^\"\'>\s]+)/i],
+ ['lang-css', /^sty\w+\s*=\s*\"([^\"]+)\"/i],
+ ['lang-css', /^sty\w+\s*=\s*\'([^\']+)\'/i],
+ ['lang-css', /^sty\w+\s*=\s*([^\"\'>\s]+)/i]
+ ]),
+ ['in.tag']);
+ registerLangHandler(
+ createSimpleLexer([], [[PR_ATTRIB_VALUE, /^[\s\S]+/]]), ['uq.val']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': CPP_KEYWORDS,
+ 'hashComments': true,
+ 'cStyleComments': true
+ }), ['c', 'cc', 'cpp', 'cxx', 'cyc', 'm']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': 'null true false'
+ }), ['json']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': CSHARP_KEYWORDS,
+ 'hashComments': true,
+ 'cStyleComments': true
+ }), ['cs']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': JAVA_KEYWORDS,
+ 'cStyleComments': true
+ }), ['java']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': SH_KEYWORDS,
+ 'hashComments': true,
+ 'multiLineStrings': true
+ }), ['bsh', 'csh', 'sh']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': PYTHON_KEYWORDS,
+ 'hashComments': true,
+ 'multiLineStrings': true,
+ 'tripleQuotedStrings': true
+ }), ['cv', 'py']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': PERL_KEYWORDS,
+ 'hashComments': true,
+ 'multiLineStrings': true,
+ 'regexLiterals': true
+ }), ['perl', 'pl', 'pm']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': RUBY_KEYWORDS,
+ 'hashComments': true,
+ 'multiLineStrings': true,
+ 'regexLiterals': true
+ }), ['rb']);
+ registerLangHandler(sourceDecorator({
+ 'keywords': JSCRIPT_KEYWORDS,
+ 'cStyleComments': true,
+ 'regexLiterals': true
+ }), ['js']);
+ registerLangHandler(
+ createSimpleLexer([], [[PR_STRING, /^[\s\S]+/]]), ['regex']);
+
+ function applyDecorator(job) {
+ var sourceCodeHtml = job.sourceCodeHtml;
+ var opt_langExtension = job.langExtension;
+
+ // Prepopulate output in case processing fails with an exception.
+ job.prettyPrintedHtml = sourceCodeHtml;
+
+ try {
+ // Extract tags, and convert the source code to plain text.
+ var sourceAndExtractedTags = extractTags(sourceCodeHtml);
+ /** Plain text. @type {string} */
+ var source = sourceAndExtractedTags.source;
+ job.source = source;
+ job.basePos = 0;
+
+ /** Even entries are positions in source in ascending order. Odd entries
+ * are tags that were extracted at that position.
+ * @type {Array.<number|string>}
+ */
+ job.extractedTags = sourceAndExtractedTags.tags;
+
+ // Apply the appropriate language handler
+ langHandlerForExtension(opt_langExtension, source)(job);
+ // Integrate the decorations and tags back into the source code to produce
+ // a decorated html string which is left in job.prettyPrintedHtml.
+ recombineTagsAndDecorations(job);
+ } catch (e) {
+ if ('console' in window) {
+ console.log(e);
+ console.trace();
+ }
+ }
+ }
+
+ function prettyPrintOne(sourceCodeHtml, opt_langExtension) {
+ var job = {
+ sourceCodeHtml: sourceCodeHtml,
+ langExtension: opt_langExtension
+ };
+ applyDecorator(job);
+ return job.prettyPrintedHtml;
+ }
+
+ function prettyPrint(opt_whenDone) {
+ var isIE6 = window['_pr_isIE6']();
+
+ // fetch a list of nodes to rewrite
+ var codeSegments = [
+ document.getElementsByTagName('pre'),
+ document.getElementsByTagName('code'),
+ document.getElementsByTagName('xmp') ];
+ var elements = [];
+ for (var i = 0; i < codeSegments.length; ++i) {
+ for (var j = 0, n = codeSegments[i].length; j < n; ++j) {
+ elements.push(codeSegments[i][j]);
+ }
+ }
+ codeSegments = null;
+
+ var clock = Date;
+ if (!clock['now']) {
+ clock = { 'now': function () { return (new Date).getTime(); } };
+ }
+
+ // The loop is broken into a series of continuations to make sure that we
+ // don't make the browser unresponsive when rewriting a large page.
+ var k = 0;
+ var prettyPrintingJob;
+
+ function doWork() {
+ var endTime = (window['PR_SHOULD_USE_CONTINUATION'] ?
+ clock.now() + 250 /* ms */ :
+ Infinity);
+ for (; k < elements.length && clock.now() < endTime; k++) {
+ var cs = elements[k];
+ if (cs.className && cs.className.indexOf('pp') >= 0) {
+ // If the classes includes a language extensions, use it.
+ // Language extensions can be specified like
+ // <pre class="pp lang-cpp">
+ // the language extension "cpp" is used to find a language handler as
+ // passed to PR_registerLangHandler.
+ var langExtension = cs.className.match(/\blang-(\w+)\b/);
+ if (langExtension) { langExtension = langExtension[1]; }
+
+ // make sure this is not nested in an already prettified element
+ var nested = false;
+ for (var p = cs.parentNode; p; p = p.parentNode) {
+ if ((p.tagName === 'pre' || p.tagName === 'code' ||
+ p.tagName === 'xmp') &&
+ p.className && p.className.indexOf('pp') >= 0) {
+ nested = true;
+ break;
+ }
+ }
+ if (!nested) {
+ // fetch the content as a snippet of properly escaped HTML.
+ // Firefox adds newlines at the end.
+ var content = getInnerHtml(cs);
+ content = content.replace(/(?:\r\n?|\n)$/, '');
+
+ // do the pretty printing
+ prettyPrintingJob = {
+ sourceCodeHtml: content,
+ langExtension: langExtension,
+ sourceNode: cs
+ };
+ applyDecorator(prettyPrintingJob);
+ replaceWithPrettyPrintedHtml();
+ }
+ }
+ }
+ if (k < elements.length) {
+ // finish up in a continuation
+ setTimeout(doWork, 250);
+ } else if (opt_whenDone) {
+ opt_whenDone();
+ }
+ }
+
+ function replaceWithPrettyPrintedHtml() {
+ var newContent = prettyPrintingJob.prettyPrintedHtml;
+ if (!newContent) { return; }
+ var cs = prettyPrintingJob.sourceNode;
+
+ // push the prettified html back into the tag.
+ if (!isRawContent(cs)) {
+ // just replace the old html with the new
+ cs.innerHTML = newContent;
+ } else {
+ // we need to change the tag to a <pre> since <xmp>s do not allow
+ // embedded tags such as the span tags used to attach styles to
+ // sections of source code.
+ var pre = document.createElement('PRE');
+ for (var i = 0; i < cs.attributes.length; ++i) {
+ var a = cs.attributes[i];
+ if (a.specified) {
+ var aname = a.name.toLowerCase();
+ if (aname === 'class') {
+ pre.className = a.value; // For IE 6
+ } else {
+ pre.setAttribute(a.name, a.value);
+ }
+ }
+ }
+ pre.innerHTML = newContent;
+
+ // remove the old
+ cs.parentNode.replaceChild(pre, cs);
+ cs = pre;
+ }
+
+ // Replace <br>s with line-feeds so that copying and pasting works
+ // on IE 6.
+ // Doing this on other browsers breaks lots of stuff since \r\n is
+ // treated as two newlines on Firefox, and doing this also slows
+ // down rendering.
+ if (isIE6 && cs.tagName === 'PRE') {
+ var lineBreaks = cs.getElementsByTagName('br');
+ for (var j = lineBreaks.length; --j >= 0;) {
+ var lineBreak = lineBreaks[j];
+ lineBreak.parentNode.replaceChild(
+ document.createTextNode('\r'), lineBreak);
+ }
+ }
+ }
+
+ doWork();
+ }
+
+ window['PR_normalizedHtml'] = normalizedHtml;
+ window['prettyPrintOne'] = prettyPrintOne;
+ window['prettyPrint'] = prettyPrint;
+ window['PR'] = {
+ 'combinePrefixPatterns': combinePrefixPatterns,
+ 'createSimpleLexer': createSimpleLexer,
+ 'registerLangHandler': registerLangHandler,
+ 'sourceDecorator': sourceDecorator,
+ 'PR_ATTRIB_NAME': PR_ATTRIB_NAME,
+ 'PR_ATTRIB_VALUE': PR_ATTRIB_VALUE,
+ 'PR_COMMENT': PR_COMMENT,
+ 'PR_DECLARATION': PR_DECLARATION,
+ 'PR_KEYWORD': PR_KEYWORD,
+ 'PR_LITERAL': PR_LITERAL,
+ 'PR_NOCODE': PR_NOCODE,
+ 'PR_PLAIN': PR_PLAIN,
+ 'PR_PUNCTUATION': PR_PUNCTUATION,
+ 'PR_SOURCE': PR_SOURCE,
+ 'PR_STRING': PR_STRING,
+ 'PR_TAG': PR_TAG,
+ 'PR_TYPE': PR_TYPE
+ };
+})();
diff --git a/native-datatypes.html b/native-datatypes.html
index 11f17fa..7e9ac98 100644
--- a/native-datatypes.html
+++ b/native-datatypes.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 2}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#native-datatypes>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#native-datatypes>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=beginner>♦♦♢♢♢</span>
<h1>Native Datatypes</h1>
<blockquote class=q>
-<p><span>❝</span> Wonder is the foundation of all philosophy, inquiry its progress, ignorance its end. <span>❞</span><br>— Michel de Montaigne
+<p><span class=u>❝</span> Wonder is the foundation of all philosophy, inquiry its progress, ignorance its end. <span class=u>❞</span><br>— Michel de Montaigne
</blockquote>
<p id=toc>
<h2 id=divingin>Diving In</h2>
@@ -39,7 +39,7 @@ body{counter-reset:h1 2}
<aside>You can use virtually any expression in a boolean context.</aside>
<p>Booleans are either true or false. Python has two constants, <code>True</code> and <code>False</code>, which can be used to assign boolean values directly. Expressions can also evaluate to a boolean value. In certain places (like <code>if</code> statements), Python expects an expression to evaluate to a boolean value. These places are called <i>boolean contexts</i>. You can use virtually any expression in a boolean context, and Python will try to determine its truth value. Different datatypes have different rules about which values are true or false in a boolean context. (This will make more sense once you see some concrete examples later in this chapter.)
<p>For example, take this snippet from <a href=your-first-python-program.html#divingin><code>humansize.py</code></a>:
-<pre><code>if size < 0:
+<pre><code class=pp>if size < 0:
raise ValueError('number must be non-negative')</code></pre>
<p><var>size</var> is an integer, <code>0</code> is an integer, and <code><</code> is a numerical operator. The result of the expression <code>size < 0</code> is always a boolean. You can test this yourself in the Python interactive shell:
<pre class=screen>
@@ -57,11 +57,11 @@ body{counter-reset:h1 2}
<h2 id=numbers>Numbers</h2>
<p>Numbers are awesome. There are so many to choose from. Python supports both integers and floating point numbers. There’s no type declaration to distinguish them; Python tells them apart by the presence or absence of a decimal point.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>type(1)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>type(1)</kbd> <span class=u>①</span></a>
<samp><class 'int'></samp>
-<a><samp class=p>>>> </samp><kbd>1 + 1</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>1 + 1</kbd> <span class=u>②</span></a>
<samp>2</samp>
-<a><samp class=p>>>> </samp><kbd>1 + 1.0</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>1 + 1.0</kbd> <span class=u>③</span></a>
<samp>2.0</samp>
<samp class=p>>>> </samp><kbd>type(2.0)</kbd>
<samp><class 'float'></samp></pre>
@@ -73,17 +73,17 @@ body{counter-reset:h1 2}
<h3 id=number-coercion>Coercing Integers To Floats And Vice-Versa</h3>
<p>As you just saw, some operators (like addition) will coerce integers to floating point numbers as needed. You can also coerce them by yourself.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>float(2)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>float(2)</kbd> <span class=u>①</span></a>
<samp>2.0</samp>
-<a><samp class=p>>>> </samp><kbd>int(2.0)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>int(2.0)</kbd> <span class=u>②</span></a>
<samp>2</samp>
-<a><samp class=p>>>> </samp><kbd>int(2.5)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>int(2.5)</kbd> <span class=u>③</span></a>
<samp>2</samp>
-<a><samp class=p>>>> </samp><kbd>int(-2.5)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>int(-2.5)</kbd> <span class=u>④</span></a>
<samp>-2</samp>
-<a><samp class=p>>>> </samp><kbd>1.12345678901234567890</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>1.12345678901234567890</kbd> <span class=u>⑤</span></a>
<samp>1.1234567890123457</samp>
-<a><samp class=p>>>> </samp><kbd>type(1000000000000000)</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>type(1000000000000000)</kbd> <span class=u>⑥</span></a>
<samp><class 'int'></samp></pre>
<ol>
<li>You can explicitly coerce an <code>int</code> to a <code>float</code> by calling the <code>float()</code> function.
@@ -94,22 +94,22 @@ body{counter-reset:h1 2}
<li>Integers can be arbitrarily large.
</ol>
<blockquote class='note compare python2'>
-<p><span>☞</span>Python 2 had separate types for <code>int</code> and <code>long</code>. The <code>int</code> datatype was limited by <code>sys.maxint</code>, which varied by platform but was usually <code>2<sup>32</sup>-1</code>. Python 3 has just one integer type, which behaves mostly like the old <code>long</code> type from Python 2. See <a href=http://www.python.org/dev/peps/pep-0237><abbr>PEP</abbr> 237</a> for details.
+<p><span class=u>☞</span>Python 2 had separate types for <code>int</code> and <code>long</code>. The <code>int</code> datatype was limited by <code>sys.maxint</code>, which varied by platform but was usually <code>2<sup>32</sup>-1</code>. Python 3 has just one integer type, which behaves mostly like the old <code>long</code> type from Python 2. See <a href=http://www.python.org/dev/peps/pep-0237><abbr>PEP</abbr> 237</a> for details.
</blockquote>
<h3 id=common-numerical-operations>Common Numerical Operations</h3>
<p>You can do all kinds of things with numbers.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>11 / 2</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>11 / 2</kbd> <span class=u>①</span></a>
<samp>5.5</samp>
-<a><samp class=p>>>> </samp><kbd>11 // 2</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>11 // 2</kbd> <span class=u>②</span></a>
<samp>5</samp>
-<a><samp class=p>>>> </samp><kbd>−11 // 2</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>−11 // 2</kbd> <span class=u>③</span></a>
<samp>−6</samp>
-<a><samp class=p>>>> </samp><kbd>11.0 // 2</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>11.0 // 2</kbd> <span class=u>④</span></a>
<samp>5.0</samp>
-<a><samp class=p>>>> </samp><kbd>11 ** 2</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>11 ** 2</kbd> <span class=u>⑤</span></a>
<samp>121</samp>
-<a><samp class=p>>>> </samp><kbd>11 % 2</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>11 % 2</kbd> <span class=u>⑥</span></a>
<samp>1</samp>
</pre>
<ol>
@@ -121,18 +121,18 @@ body{counter-reset:h1 2}
<li>The <code>%</code> operator gives the remainder after performing integer division. <code>11</code> divided by <code>2</code> is <code>5</code> with a remainder of <code>1</code>, so the result here is <code>1</code>.
</ol>
<blockquote class='note compare python2'>
-<p><span>☞</span>In Python 2, the <code>/</code> operator usually meant integer division, but you could make it behave like floating point division by including a special directive in your code. In Python 3, the <code>/</code> operator always means floating point division. See <a href=http://www.python.org/dev/peps/pep-0238/><abbr>PEP</abbr> 238</a> for details.
+<p><span class=u>☞</span>In Python 2, the <code>/</code> operator usually meant integer division, but you could make it behave like floating point division by including a special directive in your code. In Python 3, the <code>/</code> operator always means floating point division. See <a href=http://www.python.org/dev/peps/pep-0238/><abbr>PEP</abbr> 238</a> for details.
</blockquote>
<h3 id=fractions>Fractions</h3>
<p>Python isn’t limited to integers and floating point numbers. It can also do all the fancy math you learned in high school and promptly forgot about.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>import fractions</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>x = fractions.Fraction(1, 3)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>import fractions</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>x = fractions.Fraction(1, 3)</kbd> <span class=u>②</span></a>
<samp class=p>>>> </samp><kbd>x</kbd>
<samp>Fraction(1, 3)</samp>
-<a><samp class=p>>>> </samp><kbd>x * 2</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>x * 2</kbd> <span class=u>③</span></a>
<samp>Fraction(2, 3)</samp>
-<a><samp class=p>>>> </samp><kbd>fractions.Fraction(6, 4)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>fractions.Fraction(6, 4)</kbd> <span class=u>④</span></a>
<samp>Fraction(3, 2)</samp></pre>
<ol>
<li>To start using fractions, import the <code>fractions</code> module.
@@ -144,11 +144,11 @@ body{counter-reset:h1 2}
<p>You can also do basic trigonometry in Python.
<pre class=screen>
<samp class=p>>>> </samp><kbd>import math</kbd>
-<a><samp class=p>>>> </samp><kbd>math.pi</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>math.pi</kbd> <span class=u>①</span></a>
<samp>3.1415926535897931</samp>
-<a><samp class=p>>>> </samp><kbd>math.sin(math.pi / 2)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>math.sin(math.pi / 2)</kbd> <span class=u>②</span></a>
<samp>1.0</samp>
-<a><samp class=p>>>> </samp><kbd>math.tan(math.pi / 4)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>math.tan(math.pi / 4)</kbd> <span class=u>③</span></a>
<samp>0.99999999999999989</samp></pre>
<ol>
<li>The <code>math</code> module has a constant for π, the ratio of a circle’s circumference to its diameter.
@@ -159,24 +159,24 @@ body{counter-reset:h1 2}
<aside>Zero values are false, and non-zero values are true.</aside>
<p>You can use numbers <a href=#booleans>in a boolean context</a>, such as an <code>if</code> statement. Zero values are false, and non-zero values are true.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>def is_it_true(anything):</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>def is_it_true(anything):</kbd> <span class=u>①</span></a>
<samp class=p>... </samp><kbd> if anything:</kbd>
<samp class=p>... </samp><kbd> print('yes, it's true')</kbd>
<samp class=p>... </samp><kbd> else:</kbd>
<samp class=p>... </samp><kbd> print('no, it's false')</kbd>
<samp class=p>...</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true(1)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true(1)</kbd> <span class=u>②</span></a>
<samp>yes, it's true</samp>
<samp class=p>>>> </samp><kbd>is_it_true(-1)</kbd>
<samp>yes, it's true</samp>
<samp class=p>>>> </samp><kbd>is_it_true(0)</kbd>
<samp>no, it's false</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true(0.1)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true(0.1)</kbd> <span class=u>③</span></a>
<samp>yes, it's true</samp>
<samp class=p>>>> </samp><kbd>is_it_true(0.0)</kbd>
<samp>no, it's false</samp>
<samp class=p>>>> </samp><kbd>import fractions</kbd>
-<a><samp class=p>>>> </samp><kbd>is_it_true(fractions.Fraction(1, 2))</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true(fractions.Fraction(1, 2))</kbd> <span class=u>④</span></a>
<samp>yes, it's true</samp>
<samp class=p>>>> </samp><kbd>is_it_true(fractions.Fraction(0, 1))</kbd>
<samp>no, it's false</samp></pre>
@@ -191,24 +191,24 @@ body{counter-reset:h1 2}
<h2 id=lists>Lists</h2>
<p>Lists are Python’s workhorse datatype. When I say “list,” you might be thinking “array whose size I have to declare in advance, that can only contain items of the same type, <i class=baa>&</i>c.” Don’t think that. Lists are much cooler than that.
<blockquote class='note compare perl5'>
-<p><span>☞</span>A list in Python is like an array in Perl 5. In Perl 5, variables that store arrays always start with the <code>@</code> character; in Python, variables can be named anything, and Python keeps track of the datatype internally.
+<p><span class=u>☞</span>A list in Python is like an array in Perl 5. In Perl 5, variables that store arrays always start with the <code>@</code> character; in Python, variables can be named anything, and Python keeps track of the datatype internally.
</blockquote>
<blockquote class='note compare java'>
-<p><span>☞</span>A list in Python is much more than an array in Java (although it can be used as one if that’s really all you want out of life). A better analogy would be to the <code>ArrayList</code> class, which can hold arbitrary objects and can expand dynamically as new items are added.
+<p><span class=u>☞</span>A list in Python is much more than an array in Java (although it can be used as one if that’s really all you want out of life). A better analogy would be to the <code>ArrayList</code> class, which can hold arbitrary objects and can expand dynamically as new items are added.
</blockquote>
<h3 id=creatinglists>Creating A List</h3>
<p>Creating a list is easy: use square brackets to wrap a comma-separated list of values.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>a_list = ['a', 'b', 'mpilgrim', 'z', 'example']</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list = ['a', 'b', 'mpilgrim', 'z', 'example']</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
['a', 'b', 'mpilgrim', 'z', 'example']
-<a><samp class=p>>>> </samp><kbd>a_list[0]</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[0]</kbd> <span class=u>②</span></a>
<samp>'a'</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[4]</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[4]</kbd> <span class=u>③</span></a>
<samp>'example'</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[-1]</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[-1]</kbd> <span class=u>④</span></a>
<samp>'example'</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[-3]</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[-3]</kbd> <span class=u>⑤</span></a>
<samp>'mpilgrim'</samp></pre>
<ol>
<li>First, you define a list of five items. Note that they retain their original order. This is not an accident. A list is an ordered set of items.
@@ -223,17 +223,17 @@ body{counter-reset:h1 2}
<pre class=screen>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 'b', 'mpilgrim', 'z', 'example']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[1:3]</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[1:3]</kbd> <span class=u>①</span></a>
<samp>['b', 'mpilgrim']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[1:-1]</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[1:-1]</kbd> <span class=u>②</span></a>
<samp>['b', 'mpilgrim', 'z']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[0:3]</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[0:3]</kbd> <span class=u>③</span></a>
<samp>['a', 'b', 'mpilgrim']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[:3]</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[:3]</kbd> <span class=u>④</span></a>
<samp>['a', 'b', 'mpilgrim']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[3:]</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[3:]</kbd> <span class=u>⑤</span></a>
<samp>['z', 'example']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list[:]</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list[:]</kbd> <span class=u>⑥</span></a>
['a', 'b', 'mpilgrim', 'z', 'example']</pre>
<ol>
<li>You can get a part of a list, called a “slice”, by specifying two indices. The return value is a new list containing all the items of the list, in order, starting with the first slice index (in this case <code>a_list[1]</code>), up to but not including the second slice index (in this case <code>a_list[3]</code>).
@@ -247,16 +247,16 @@ body{counter-reset:h1 2}
<p>There are four ways to add items to a list.
<pre class=screen>
<samp class=p>>>> </samp><kbd>a_list = ['a']</kbd>
-<a><samp class=p>>>> </samp><kbd>a_list = a_list + [2.0, 3]</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list = a_list + [2.0, 3]</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 2.0, 3]</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.append(True)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.append(True)</kbd> <span class=u>②</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 2.0, 3, True]</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.extend(['four', 'e'])</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.extend(['four', 'e'])</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 2.0, 3, True, 'four', 'e']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.insert(1, 'a')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.insert(1, 'a')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 'a', 2.0, 3, True, 'four', 'e']</samp></pre>
<ol>
@@ -268,17 +268,17 @@ body{counter-reset:h1 2}
<p>Let’s look closer at the difference between <code>append()</code> and <code>extend()</code>.
<pre class=screen>
<samp class=p>>>> </samp><kbd>a_list = ['a', 'b', 'c']</kbd>
-<a><samp class=p>>>> </samp><kbd>a_list.extend(['d', 'e', 'f'])</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.extend(['d', 'e', 'f'])</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 'b', 'c', 'd', 'e', 'f']</samp>
-<a><samp class=p>>>> </samp><kbd>len(a_list)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>len(a_list)</kbd> <span class=u>②</span></a>
<samp>6</samp>
<samp class=p>>>> </samp><kbd>a_list[-1]</kbd>
<samp>'f'</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.append(['g', 'h', 'i'])</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.append(['g', 'h', 'i'])</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['a', 'b', 'c', 'd', 'e', 'f', ['g', 'h', 'i']]</samp>
-<a><samp class=p>>>> </samp><kbd>len(a_list)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>len(a_list)</kbd> <span class=u>④</span></a>
<samp>7</samp>
<samp class=p>>>> </samp><kbd>a_list[-1]</kbd>
<samp>['g', 'h', 'i']</samp></pre>
@@ -291,15 +291,15 @@ body{counter-reset:h1 2}
<h3 id=searchinglists>Searching For Values In A List</h3>
<pre class=screen>
<samp class=p>>>> </samp><kbd>a_list = ['a', 'b', 'new', 'mpilgrim', 'new']</kbd>
-<a><samp class=p>>>> </samp><kbd>'mpilgrim' in a_list</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>'mpilgrim' in a_list</kbd> <span class=u>①</span></a>
<samp>True</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.index('mpilgrim')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.index('mpilgrim')</kbd> <span class=u>②</span></a>
<samp>3</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.index('new')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.index('new')</kbd> <span class=u>③</span></a>
<samp>2</samp>
-<a><samp class=p>>>> </samp><kbd>'c' in a_list</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>'c' in a_list</kbd> <span class=u>④</span></a>
<samp>False</samp>
-<a><samp class=p>>>> </samp><kbd>a_list.index('c')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list.index('c')</kbd> <span class=u>⑤</span></a>
<samp class=traceback>Traceback (innermost last):
File "<interactive input>", line 1, in ?
ValueError: list.index(x): x not in list</samp></pre>
@@ -320,11 +320,11 @@ ValueError: list.index(x): x not in list</samp></pre>
<samp class=p>... </samp><kbd> else:</kbd>
<samp class=p>... </samp><kbd> print('no, it's false')</kbd>
<samp class=p>...</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true([])</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true([])</kbd> <span class=u>②</span></a>
<samp>no, it's false</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true(['a'])</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true(['a'])</kbd> <span class=u>③</span></a>
<samp>yes, it's true</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true([False])</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true([False])</kbd> <span class=u>④</span></a>
<samp>yes, it's true</samp></pre>
<ol>
<li>In a boolean context, an empty list is false.
@@ -342,19 +342,19 @@ ValueError: list.index(x): x not in list</samp></pre>
<h2 id=dictionaries>Dictionaries</h2>
<p>One of Python’s most important datatypes is the dictionary, which defines one-to-one relationships between keys and values.
<blockquote class='note compare perl5'>
-<p><span>☞</span>A dictionary in Python is like a hash in Perl 5. In Perl 5, variables that store hashes always start with a <code>%</code> character. In Python, variables can be named anything, and Python keeps track of the datatype internally.
+<p><span class=u>☞</span>A dictionary in Python is like a hash in Perl 5. In Perl 5, variables that store hashes always start with a <code>%</code> character. In Python, variables can be named anything, and Python keeps track of the datatype internally.
</blockquote>
<h3 id=creating-dictionaries>Creating A Dictionary</h3>
<p>Creating a dictionary is easy. The syntax is similar to <a href=#sets>sets</a>, but instead of values, you have key-value pairs. Once you have a dictionary, you can look up values by their key.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>a_dict = {'server':'db.diveintopython3.org', 'database':'mysql'}</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict = {'server':'db.diveintopython3.org', 'database':'mysql'}</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'server': 'db.diveintopython3.org', 'database': 'mysql'}</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict['server']</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['server']</kbd> <span class=u>②</span></a>
'db.diveintopython3.org'
-<a><samp class=p>>>> </samp><kbd>a_dict['database']</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['database']</kbd> <span class=u>③</span></a>
'mysql'
-<a><samp class=p>>>> </samp><kbd>a_dict['db.diveintopython3.org']</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['db.diveintopython3.org']</kbd> <span class=u>④</span></a>
<samp class=traceback>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'db.diveintopython3.org'</samp></pre>
@@ -369,16 +369,16 @@ KeyError: 'db.diveintopython3.org'</samp></pre>
<pre class=screen>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'server': 'db.diveintopython3.org', 'database': 'mysql'}</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict['database'] = 'blog'</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['database'] = 'blog'</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'server': 'db.diveintopython3.org', 'database': 'blog'}</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict['user'] = 'mark'</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>a_dict</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['user'] = 'mark'</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict</kbd> <span class=u>③</span></a>
<samp>{'server': 'db.diveintopython3.org', 'user': 'mark', 'database': 'blog'}</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict['user'] = 'dora'</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['user'] = 'dora'</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'server': 'db.diveintopython3.org', 'user': 'dora', 'database': 'blog'}</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict['User'] = 'mark'</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict['User'] = 'mark'</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'User': 'mark', 'server': 'db.diveintopython3.org', 'user': 'dora', 'database': 'blog'}</samp></pre>
<ol>
@@ -391,19 +391,19 @@ KeyError: 'db.diveintopython3.org'</samp></pre>
<h3 id=mixed-value-dictionaries>Mixed-Value Dictionaries</h3>
<p>Dictionaries aren’t just for strings. Dictionary values can be any datatype, including integers, booleans, arbitrary objects, or even other dictionaries. And within a single dictionary, the values don’t all need to be the same type; you can mix and match as needed. Dictionary keys are more restricted, but they can be strings, integers, and a few other types. You can also mix and match key datatypes within a dictionary.
<p>In fact, you’ve already seen a dictionary with non-string keys and values, in <a href=your-first-python-program.html#divingin>your first Python program</a>.
-<pre><code>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],
+<pre><code class=pp>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],
1024: ['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']}</code></pre>
<p>Let's tear that apart in the interactive shell.
<pre class=screen>
<samp class=p>>>> </samp><kbd>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],</kbd>
<samp class=p>... </samp><kbd> 1024: ['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']}</kbd>
-<a><samp class=p>>>> </samp><kbd>len(SUFFIXES)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>len(SUFFIXES)</kbd> <span class=u>①</span></a>
<samp>2</samp>
-<a><samp class=p>>>> </samp><kbd>SUFFIXES[1000]</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>SUFFIXES[1000]</kbd> <span class=u>②</span></a>
<samp>['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']</samp>
-<a><samp class=p>>>> </samp><kbd>SUFFIXES[1024]</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>SUFFIXES[1024]</kbd> <span class=u>③</span></a>
<samp>['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']</samp>
-<a><samp class=p>>>> </samp><kbd>SUFFIXES[1000][3]</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>SUFFIXES[1000][3]</kbd> <span class=u>④</span></a>
<samp>'TB'</samp></pre>
<ol>
<li>As with <a href=#lists>lists</a><!-- and <a href=#sets>sets</a>-->, the <code>len()</code> function gives you the number of items in a dictionary.
@@ -421,9 +421,9 @@ KeyError: 'db.diveintopython3.org'</samp></pre>
<samp class=p>... </samp><kbd> else:</kbd>
<samp class=p>... </samp><kbd> print('no, it's false')</kbd>
<samp class=p>...</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true({})</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true({})</kbd> <span class=u>①</span></a>
<samp>no, it's false</samp>
-<a><samp class=p>>>> </samp><kbd>is_it_true({'a': 1})</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>is_it_true({'a': 1})</kbd> <span class=u>②</span></a>
<samp>yes, it's true</samp></pre>
<ol>
<li>In a boolean context, an empty dictionary is false.
@@ -474,7 +474,8 @@ KeyError: 'db.diveintopython3.org'</samp></pre>
<li><a href=http://www.python.org/dev/peps/pep-0237/><abbr>PEP</abbr> 237: Unifying Long Integers and Integers</a>
<li><a href=http://www.python.org/dev/peps/pep-0238/><abbr>PEP</abbr> 238: Changing the Division Operator</a>
</ul>
-<p class=v><a href=your-first-python-program.html rel=prev title='back to “Your First Python Program”'><span>☜</span></a> <a href=strings.html rel=next title='onward to “Strings”'><span>☞</span></a>
+<p class=v><a href=your-first-python-program.html rel=prev title='back to “Your First Python Program”'><span class=u>☜</span></a> <a href=strings.html rel=next title='onward to “Strings”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/porting-code-to-python-3-with-2to3.html b/porting-code-to-python-3-with-2to3.html
index 40a8356..259354d 100644
--- a/porting-code-to-python-3-with-2to3.html
+++ b/porting-code-to-python-3-with-2to3.html
@@ -22,11 +22,11 @@ td pre{padding:0;border:0}
<meta content='initial-scale=1.0' name=viewport>
</head>
<form action=http://www.google.com/cse><div><input name=cx type=hidden value=014021643941856155761:l5eihuescdw><input name=ie type=hidden value=UTF-8> <input name=q size=25> <input name=sa type=submit value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#porting-code-to-python-3-with-2to3>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#porting-code-to-python-3-with-2to3>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=pro>♦♦♦♦♦</span>
<h1>Porting Code to Python 3 with <code>2to3</code></h1>
<blockquote class=q>
-<p><span>❝</span> Life is pleasant. Death is peaceful. It’s the transition that’s troublesome. <span>❞</span><br>— Isaac Asimov (attributed)
+<p><span class=u>❝</span> Life is pleasant. Death is peaceful. It’s the transition that’s troublesome. <span class=u>❞</span><br>— Isaac Asimov (attributed)
</blockquote>
<p id=toc>
<h2 id=divingin>Diving in</h2>
@@ -38,20 +38,20 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>print</code>
-<td><code>print()</code>
+<td><code class=pp>print</code>
+<td><code class=pp>print()</code>
<tr><th>②
-<td><code>print 1</code>
-<td><code>print(1)</code>
+<td><code class=pp>print 1</code>
+<td><code class=pp>print(1)</code>
<tr><th>③
-<td><code>print 1, 2</code>
-<td><code>print(1, 2)</code>
+<td><code class=pp>print 1, 2</code>
+<td><code class=pp>print(1, 2)</code>
<tr><th>④
-<td><code>print 1, 2,</code>
-<td><code>print(1, 2, end=' ')</code>
+<td><code class=pp>print 1, 2,</code>
+<td><code class=pp>print(1, 2, end=' ')</code>
<tr><th>⑤
-<td><code>print >>sys.stderr, 1, 2, 3</code>
-<td><code>print(1, 2, 3, file=sys.stderr)</code>
+<td><code class=pp>print >>sys.stderr, 1, 2, 3</code>
+<td><code class=pp>print(1, 2, 3, file=sys.stderr)</code>
</table>
<ol>
<li>To print a blank line, call <code>print()</code> without any arguments.
@@ -67,11 +67,11 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>u'PapayaWhip'</code>
-<td><code>'PapayaWhip'</code>
+<td><code class=pp>u'PapayaWhip'</code>
+<td><code class=pp>'PapayaWhip'</code>
<tr><th>②
-<td><code>ur'PapayaWhip\foo'</code>
-<td><code>r'PapayaWhip\foo'</code>
+<td><code class=pp>ur'PapayaWhip\foo'</code>
+<td><code class=pp>r'PapayaWhip\foo'</code>
</table>
<ol>
<li>Unicode string literals are simply converted into string literals, which, in Python 3, are always Unicode.
@@ -84,8 +84,8 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>unicode(anything)</code>
-<td><code>str(anything)</code>
+<td><code class=pp>unicode(anything)</code>
+<td><code class=pp>str(anything)</code>
</table>
<h2 id=long><code>long</code> data type</h2>
<p>Python 2 had separate <code>int</code> and <code>long</code> types for non-floating-point numbers. An <code>int</code> could not be any larger than <a href=#renames><code>sys.maxint</code></a>, which varied by platform. Longs were defined by appending an <code>L</code> to the end of the number, and they could be, well, longer than ints. In Python 3, there is only one integer type, called <code>int</code>, which mostly behaves like the <code>long</code> type in Python 2. Since there are no longer two types, there is no need for special syntax to distinguish them.
@@ -95,20 +95,20 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>x = 1000000000000L</code>
-<td><code>x = 1000000000000</code>
+<td><code class=pp>x = 1000000000000L</code>
+<td><code class=pp>x = 1000000000000</code>
<tr><th>②
-<td><code>x = 0xFFFFFFFFFFFFL</code>
-<td><code>x = 0xFFFFFFFFFFFF</code>
+<td><code class=pp>x = 0xFFFFFFFFFFFFL</code>
+<td><code class=pp>x = 0xFFFFFFFFFFFF</code>
<tr><th>③
-<td><code>long(x)</code>
-<td><code>int(x)</code>
+<td><code class=pp>long(x)</code>
+<td><code class=pp>int(x)</code>
<tr><th>④
-<td><code>type(x) is long</code>
-<td><code>type(x) is int</code>
+<td><code class=pp>type(x) is long</code>
+<td><code class=pp>type(x) is int</code>
<tr><th>⑤
-<td><code>isinstance(x, long)</code>
-<td><code>isinstance(x, int)</code>
+<td><code class=pp>isinstance(x, long)</code>
+<td><code class=pp>isinstance(x, int)</code>
</table>
<ol>
<li>Base 10 long integer literals become base 10 integer literals.
@@ -124,11 +124,11 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>if x <> y:</code>
-<td><code>if x != y:</code>
+<td><code class=pp>if x <> y:</code>
+<td><code class=pp>if x != y:</code>
<tr><th>②
-<td><code>if x <> y <> z:</code>
-<td><code>if x != y != z:</code>
+<td><code class=pp>if x <> y <> z:</code>
+<td><code class=pp>if x != y != z:</code>
</table>
<ol>
<li>A simple comparison.
@@ -141,20 +141,20 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>a_dictionary.has_key('PapayaWhip')</code>
-<td><code>'PapayaWhip' in a_dictionary</code>
+<td><code class=pp>a_dictionary.has_key('PapayaWhip')</code>
+<td><code class=pp>'PapayaWhip' in a_dictionary</code>
<tr><th>②</th >
-<td><code>a_dictionary.has_key(x) or a_dictionary.has_key(y)</code>
-<td><code>x in a_dictionary or y in a_dictionary</code>
+<td><code class=pp>a_dictionary.has_key(x) or a_dictionary.has_key(y)</code>
+<td><code class=pp>x in a_dictionary or y in a_dictionary</code>
<tr><th>③
-<td><code>a_dictionary.has_key(x or y)</code>
-<td><code>(x or y) in a_dictionary</code>
+<td><code class=pp>a_dictionary.has_key(x or y)</code>
+<td><code class=pp>(x or y) in a_dictionary</code>
<tr><th>④
-<td><code>a_dictionary.has_key(x + y)</code>
-<td><code>(x + y) in a_dictionary</code>
+<td><code class=pp>a_dictionary.has_key(x + y)</code>
+<td><code class=pp>(x + y) in a_dictionary</code>
<tr><th>⑤
-<td><code>x + a_dictionary.has_key(y)</code>
-<td><code>x + (y in a_dictionary)</code>
+<td><code class=pp>x + a_dictionary.has_key(y)</code>
+<td><code class=pp>x + (y in a_dictionary)</code>
</table>
<ol>
<li>The simplest form.
@@ -170,19 +170,19 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>a_dictionary.keys()</code>
-<td><code>list(a_dictionary.keys())</code>
+<td><code class=pp>a_dictionary.keys()</code>
+<td><code class=pp>list(a_dictionary.keys())</code>
<tr><th>②
-<td><code>a_dictionary.items()</code>
-<td><code>list(a_dictionary.items())</code>
+<td><code class=pp>a_dictionary.items()</code>
+<td><code class=pp>list(a_dictionary.items())</code>
<tr><th>③
-<td><code>a_dictionary.iterkeys()</code>
-<td><code>iter(a_dictionary.keys())</code>
+<td><code class=pp>a_dictionary.iterkeys()</code>
+<td><code class=pp>iter(a_dictionary.keys())</code>
<tr><th>④
-<td><code>[i for i in a_dictionary.iterkeys()]</code>
-<td><code>[i for i in a_dictionary.keys()]</code>
+<td><code class=pp>[i for i in a_dictionary.iterkeys()]</code>
+<td><code class=pp>[i for i in a_dictionary.keys()]</code>
<tr><th>⑤
-<td><code>min(a_dictionary.keys())</code>
+<td><code class=pp>min(a_dictionary.keys())</code>
<td><i>no change</i>
</table>
<ol>
@@ -201,19 +201,19 @@ td pre{padding:0;border:0}
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>import httplib</code>
-<td><code>import http.client</code>
+<td><code class=pp>import httplib</code>
+<td><code class=pp>import http.client</code>
<tr><th>②
-<td><code>import Cookie</code>
-<td><code>import http.cookies</code>
+<td><code class=pp>import Cookie</code>
+<td><code class=pp>import http.cookies</code>
<tr><th>③
-<td><code>import cookielib</code>
-<td><code>import http.cookiejar</code>
+<td><code class=pp>import cookielib</code>
+<td><code class=pp>import http.cookiejar</code>
<tr><th>④
-<td><pre><code>import BaseHTTPServer
+<td><pre><code class=pp>import BaseHTTPServer
import SimpleHTTPServer
import CGIHttpServer</code></pre>
-<td><code>import http.server</code>
+<td><code class=pp>import http.server</code>
</table>
<ol>
<li>The <code>http.client</code> module implements a low-level library that can request <abbr>HTTP</abbr> resources and interpret <abbr>HTTP</abbr> responses.
@@ -228,26 +228,26 @@ import CGIHttpServer</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>import urllib</code>
-<td><code>import urllib.request, urllib.parse, urllib.error</code>
+<td><code class=pp>import urllib</code>
+<td><code class=pp>import urllib.request, urllib.parse, urllib.error</code>
<tr><th>②
-<td><code>import urllib2</code>
-<td><code>import urllib.request, urllib.error</code>
+<td><code class=pp>import urllib2</code>
+<td><code class=pp>import urllib.request, urllib.error</code>
<tr><th>③
-<td><code>import urlparse</code>
-<td><code>import urllib.parse</code>
+<td><code class=pp>import urlparse</code>
+<td><code class=pp>import urllib.parse</code>
<tr><th>④
-<td><code>import robotparser</code>
-<td><code>import urllib.robotparser</code>
+<td><code class=pp>import robotparser</code>
+<td><code class=pp>import urllib.robotparser</code>
<tr><th>⑤
-<td><pre><code>from urllib import FancyURLopener
+<td><pre><code class=pp>from urllib import FancyURLopener
from urllib import urlencode</code></pre>
-<td><pre><code>from urllib.request import FancyURLopener
+<td><pre><code class=pp>from urllib.request import FancyURLopener
from urllib.parse import urlencode</code></pre>
<tr><th>⑥
-<td><pre><code>from urllib2 import Request
+<td><pre><code class=pp>from urllib2 import Request
from urllib2 import HTTPError</code></pre>
-<td><pre><code>from urllib.request import Request
+<td><pre><code class=pp>from urllib.request import Request
from urllib.error import HTTPError</code></pre>
</table>
<ol>
@@ -265,21 +265,21 @@ from urllib.error import HTTPError</code></pre>
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>import dbm</code>
-<td><code>import dbm.ndbm</code>
+<td><code class=pp>import dbm</code>
+<td><code class=pp>import dbm.ndbm</code>
<tr><th>
-<td><code>import gdbm</code>
-<td><code>import dbm.gnu</code>
+<td><code class=pp>import gdbm</code>
+<td><code class=pp>import dbm.gnu</code>
<tr><th>
-<td><code>import dbhash</code>
-<td><code>import dbm.bsd</code>
+<td><code class=pp>import dbhash</code>
+<td><code class=pp>import dbm.bsd</code>
<tr><th>
-<td><code>import dumbdbm</code>
-<td><code>import dbm.dumb</code>
+<td><code class=pp>import dumbdbm</code>
+<td><code class=pp>import dbm.dumb</code>
<tr><th>
-<td><pre><code>import anydbm
+<td><pre><code class=pp>import anydbm
import whichdb</code></pre>
-<td><code>import dbm</code>
+<td><code class=pp>import dbm</code>
</table>
<h3 id=xmlrpc><code>xmlrpc</code></h3>
<p><abbr>XML-RPC</abbr> is a lightweight method of performing remote <abbr>RPC</abbr> calls over <abbr>HTTP</abbr>. The <abbr>XML-RPC</abbr> client library and several <abbr>XML-RPC</abbr> server implementations are now combined in a single package, <code>xmlrpc</code>.
@@ -288,12 +288,12 @@ import whichdb</code></pre>
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>import xmlrpclib</code>
-<td><code>import xmlrpc.client</code>
+<td><code class=pp>import xmlrpclib</code>
+<td><code class=pp>import xmlrpc.client</code>
<tr><th>
-<td><pre><code>import DocXMLRPCServer
+<td><pre><code class=pp>import DocXMLRPCServer
import SimpleXMLRPCServer</code></pre>
-<td><code>import xmlrpc.server</code>
+<td><code class=pp>import xmlrpc.server</code>
</table>
<h3 id=othermodules>Other modules</h3>
<table>
@@ -301,38 +301,38 @@ import SimpleXMLRPCServer</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import cStringIO as StringIO
except ImportError:
import StringIO</code></pre>
-<td><code>import io</code>
+<td><code class=pp>import io</code>
<tr><th>②
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import cPickle as pickle
except ImportError:
import pickle</code></pre>
-<td><code>import pickle</code>
+<td><code class=pp>import pickle</code>
<tr><th>③
-<td><code>import __builtin__</code>
-<td><code>import builtins</code>
+<td><code class=pp>import __builtin__</code>
+<td><code class=pp>import builtins</code>
<tr><th>④
-<td><code>import copy_reg</code>
-<td><code>import copyreg</code>
+<td><code class=pp>import copy_reg</code>
+<td><code class=pp>import copyreg</code>
<tr><th>⑤
-<td><code>import Queue</code>
-<td><code>import queue</code>
+<td><code class=pp>import Queue</code>
+<td><code class=pp>import queue</code>
<tr><th>⑥
-<td><code>import SocketServer</code>
-<td><code>import socketserver</code>
+<td><code class=pp>import SocketServer</code>
+<td><code class=pp>import socketserver</code>
<tr><th>⑦
-<td><code>import ConfigParser</code>
-<td><code>import configparser</code>
+<td><code class=pp>import ConfigParser</code>
+<td><code class=pp>import configparser</code>
<tr><th>⑧
-<td><code>import repr</code>
-<td><code>import reprlib</code>
+<td><code class=pp>import repr</code>
+<td><code class=pp>import reprlib</code>
<tr><th>⑨
-<td><code>import commands</code>
-<td><code>import subprocess</code>
+<td><code class=pp>import commands</code>
+<td><code class=pp>import subprocess</code>
</table>
<ol>
<li>A common idiom in Python 2 was to try to import <code>cStringIO as StringIO</code>, and if that failed, to import <code>StringIO</code> instead. Do not do this in Python 3; the <code>io</code> module does it for you. It will find the fastest implementation available and use it automatically.
@@ -363,11 +363,11 @@ except ImportError:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>import constants</code>
-<td><code>from . import constants</code>
+<td><code class=pp>import constants</code>
+<td><code class=pp>from . import constants</code>
<tr><th>②
-<td><code>from mbcharsetprober import MultiByteCharSetProber</code>
-<td><code>from .mbcharsetprober import MultiByteCharsetProber</code>
+<td><code class=pp>from mbcharsetprober import MultiByteCharSetProber</code>
+<td><code class=pp>from .mbcharsetprober import MultiByteCharsetProber</code>
</table>
<ol>
<li>When you need to import an entire module from elsewhere in your package, use the new <code>from . import</code> syntax. The period is actually a relative path from this file (<code>universaldetector.py</code>) to the file you want to import (<code>constants.py</code>). In this case, they are in the same directory, thus the single period. You can also import from the parent directory (<code>from .. import anothermodule</code>) or a subdirectory.
@@ -380,28 +380,28 @@ except ImportError:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>anIterator.next()</code>
-<td><code>next(anIterator)</code>
+<td><code class=pp>anIterator.next()</code>
+<td><code class=pp>next(anIterator)</code>
<tr><th>②
-<td><code>a_function_that_returns_an_iterator().next()</code>
-<td><code>next(a_function_that_returns_an_iterator())</code>
+<td><code class=pp>a_function_that_returns_an_iterator().next()</code>
+<td><code class=pp>next(a_function_that_returns_an_iterator())</code>
<tr><th>③
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def next(self):
pass</code></pre>
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def __next__(self):
pass</code></pre>
<tr><th>④
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def next(self, x, y):
pass</code></pre>
<td><i>no change</i>
<tr><th>⑤
-<td><pre><code>next = 42
+<td><pre><code class=pp>next = 42
for an_iterator in a_sequence_of_iterators:
an_iterator.next()</code></pre>
-<td><pre><code>next = 42
+<td><pre><code class=pp>next = 42
for an_iterator in a_sequence_of_iterators:
an_iterator.__next__()</code></pre>
</table>
@@ -419,19 +419,19 @@ for an_iterator in a_sequence_of_iterators:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>filter(a_function, a_sequence)</code>
-<td><code>list(filter(a_function, a_sequence))</code>
+<td><code class=pp>filter(a_function, a_sequence)</code>
+<td><code class=pp>list(filter(a_function, a_sequence))</code>
<tr><th>②
-<td><code>list(filter(a_function, a_sequence))</code>
+<td><code class=pp>list(filter(a_function, a_sequence))</code>
<td><i>no change</i>
<tr><th>③
-<td><code>filter(None, a_sequence)</code>
-<td><code>[i for i in a_sequence if i]</code>
+<td><code class=pp>filter(None, a_sequence)</code>
+<td><code class=pp>[i for i in a_sequence if i]</code>
<tr><th>④
-<td><code>for i in filter(None, a_sequence):</code>
+<td><code class=pp>for i in filter(None, a_sequence):</code>
<td><i>no change</i>
<tr><th>⑤
-<td><code>[i for i in filter(a_function, a_sequence)]</code>
+<td><code class=pp>[i for i in filter(a_function, a_sequence)]</code>
<td><i>no change</i>
</table>
<ol>
@@ -448,19 +448,19 @@ for an_iterator in a_sequence_of_iterators:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>map(a_function, 'PapayaWhip')</code>
-<td><code>list(map(a_function, 'PapayaWhip'))</code>
+<td><code class=pp>map(a_function, 'PapayaWhip')</code>
+<td><code class=pp>list(map(a_function, 'PapayaWhip'))</code>
<tr><th>②
-<td><code>map(None, 'PapayaWhip')</code>
-<td><code>list('PapayaWhip')</code>
+<td><code class=pp>map(None, 'PapayaWhip')</code>
+<td><code class=pp>list('PapayaWhip')</code>
<tr><th>③
-<td><code>map(lambda x: x+1, range(42))</code>
-<td><code>[x+1 for x in range(42)]</code>
+<td><code class=pp>map(lambda x: x+1, range(42))</code>
+<td><code class=pp>[x+1 for x in range(42)]</code>
<tr><th>④
-<td><code>for i in map(a_function, a_sequence):</code>
+<td><code class=pp>for i in map(a_function, a_sequence):</code>
<td><i>no change</i>
<tr><th>⑤
-<td><code>[i for i in map(a_function, a_sequence)]</code>
+<td><code class=pp>[i for i in map(a_function, a_sequence)]</code>
<td><i>no change</i>
</table>
<ol>
@@ -477,12 +477,12 @@ for an_iterator in a_sequence_of_iterators:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>reduce(a, b, c)</code>
-<td><pre><code>from functtools import reduce
+<td><code class=pp>reduce(a, b, c)</code>
+<td><pre><code class=pp>from functtools import reduce
reduce(a, b, c)</code></pre>
</table>
<blockquote class=note>
-<p><span>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix the <code>reduce()</code> function automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
+<p><span class=u>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix the <code>reduce()</code> function automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
</blockquote>
<h2 id=apply><code>apply()</code> global function</h2>
<p>Python 2 had a global function called <code>apply()</code>, which took a function <var>f</var> and a list <code>[a, b, c]</code> and returned <code>f(a, b, c)</code>. In Python 3, the <code>apply()</code> function no longer exists. Instead, there is a new function calling syntax that allows you to pass a list and have Python apply the list as the function’s arguments.
@@ -491,17 +491,17 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>apply(a_function, a_list_of_args)</code>
-<td><code>a_function(*a_list_of_args)</code>
+<td><code class=pp>apply(a_function, a_list_of_args)</code>
+<td><code class=pp>a_function(*a_list_of_args)</code>
<tr><th>②
-<td><code>apply(a_function, a_list_of_args, a_dictionary_of_named_args)</code>
-<td><code>a_function(*a_list_of_args, **a_dictionary_of_named_args)</code>
+<td><code class=pp>apply(a_function, a_list_of_args, a_dictionary_of_named_args)</code>
+<td><code class=pp>a_function(*a_list_of_args, **a_dictionary_of_named_args)</code>
<tr><th>③
-<td><code>apply(a_function, a_list_of_args + z)</code>
-<td><code>a_function(*a_list_of_args + z)</code>
+<td><code class=pp>apply(a_function, a_list_of_args + z)</code>
+<td><code class=pp>a_function(*a_list_of_args + z)</code>
<tr><th>④
-<td><code>apply(aModule.a_function, a_list_of_args)</code>
-<td><code>aModule.a_function(*a_list_of_args)</code>
+<td><code class=pp>apply(aModule.a_function, a_list_of_args)</code>
+<td><code class=pp>aModule.a_function(*a_list_of_args)</code>
</table>
<ol>
<li>In the simplest form, you can call a function with a list of arguments (an actual list like <code>[a, b, c]</code>) by prepending the list with an asterisk (<code>*</code>). This is exactly equivalent to the old <code>apply()</code> function in Python 2.
@@ -516,8 +516,8 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>intern(aString)</code>
-<td><code>sys.intern(aString)</code>
+<td><code class=pp>intern(aString)</code>
+<td><code class=pp>sys.intern(aString)</code>
</table>
<h2 id=exec><code>exec</code> statement</h2>
<p>Just as <a href=#print>the <code>print</code> statement</a> became a function in Python 3, so too has the <code>exec</code> statement. The <code>exec()</code> function takes a string which contains arbitrary Python code and executes it as if it were just another statement or expression.
@@ -526,14 +526,14 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>exec codeString</code>
-<td><code>exec(codeString)</code>
+<td><code class=pp>exec codeString</code>
+<td><code class=pp>exec(codeString)</code>
<tr><th>②
-<td><code>exec codeString in a_global_namespace</code>
-<td><code>exec(codeString, a_global_namespace)</code>
+<td><code class=pp>exec codeString in a_global_namespace</code>
+<td><code class=pp>exec(codeString, a_global_namespace)</code>
<tr><th>③
-<td><code>exec codeString in a_global_namespace, a_local_namespace</code>
-<td><code>exec(codeString, a_global_namespace, a_local_namespace)</code>
+<td><code class=pp>exec codeString in a_global_namespace, a_local_namespace</code>
+<td><code class=pp>exec(codeString, a_global_namespace, a_local_namespace)</code>
</table>
<ol>
<li>In the simplest form, the <code>2to3</code> script simply encloses the code-as-a-string in parentheses, since <code>exec()</code> is now a function instead of a statement.
@@ -547,11 +547,11 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>execfile('a_filename')</code>
-<td><code>exec(compile(open('a_filename').read(), 'a_filename', 'exec'))</code>
+<td><code class=pp>execfile('a_filename')</code>
+<td><code class=pp>exec(compile(open('a_filename').read(), 'a_filename', 'exec'))</code>
</table>
<blockquote class=note>
-<p><span>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix the <code>execfile</code> statement automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
+<p><span class=u>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix the <code>execfile</code> statement automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
</blockquote>
<h2 id=repr><code>repr</code> literals (backticks)</h2>
<p>In Python 2, there was a special syntax of wrapping any object in backticks (like <code>`x`</code>) to get a representation of the object. In Python 3, this capability still exists, but you can no longer use backticks to get it. Instead, use the global <code>repr()</code> function.
@@ -560,11 +560,11 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>`x`</code>
-<td><code>repr(x)</code>
+<td><code class=pp>`x`</code>
+<td><code class=pp>repr(x)</code>
<tr><th>②
-<td><code>`'PapayaWhip' + `2``</code>
-<td><code>repr('PapayaWhip' + repr(2))</code>
+<td><code class=pp>`'PapayaWhip' + `2``</code>
+<td><code class=pp>repr('PapayaWhip' + repr(2))</code>
</table>
<ol>
<li>Remember, <var>x</var> can be anything — a class, a function, a module, a primitive data type, etc. The <code>repr()</code> function works on everything.
@@ -577,31 +577,31 @@ reduce(a, b, c)</code></pre>
<th>Python 2
<th>Python 3
<tr><th>①
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except ImportError, e
pass</code></pre>
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except ImportError as e:
pass</code></pre>
<tr><th>②
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except (RuntimeError, ImportError), e
pass</code></pre>
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except (RuntimeError, ImportError) as e:
pass</code></pre>
<tr><th>③
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except ImportError:
pass</code></pre>
<td><i>no change</i>
<tr><th>④
-<td><pre><code>try:
+<td><pre><code class=pp>try:
import mymodule
except:
pass</code></pre>
@@ -614,7 +614,7 @@ except:
<li>Similarly, if you use a fallback to catch <em>all</em> exceptions, the syntax is identical.
</ol>
<blockquote class=note>
-<p><span>☞</span>You should never use a fallback to catch <em>all</em> exceptions when importing modules (or most other times). Doing so will catch things like <code>KeyboardInterrupt</code> (if the user pressed <kbd>Ctrl-C</kbd> to interrupt the program) and can make it more difficult to debug errors.
+<p><span class=u>☞</span>You should never use a fallback to catch <em>all</em> exceptions when importing modules (or most other times). Doing so will catch things like <code>KeyboardInterrupt</code> (if the user pressed <kbd>Ctrl-C</kbd> to interrupt the program) and can make it more difficult to debug errors.
</blockquote>
<h2 id=raise><code>raise</code> statement</h2>
<p>The syntax for raising your own exceptions has changed slightly between Python 2 and Python 3.
@@ -623,16 +623,16 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>raise MyException</code>
+<td><code class=pp>raise MyException</code>
<td><i>unchanged</i>
<tr><th>②
-<td><code>raise MyException, 'error message'</code>
-<td><code>raise MyException('error message')</code>
+<td><code class=pp>raise MyException, 'error message'</code>
+<td><code class=pp>raise MyException('error message')</code>
<tr><th>③
-<td><code>raise MyException, 'error message', a_traceback</code>
-<td><code>raise MyException('error message').with_traceback(a_traceback)</code>
+<td><code class=pp>raise MyException, 'error message', a_traceback</code>
+<td><code class=pp>raise MyException('error message').with_traceback(a_traceback)</code>
<tr><th>④
-<td><code>raise 'error message'</code>
+<td><code class=pp>raise 'error message'</code>
<td><i>unsupported</i>
</table>
<ol>
@@ -648,13 +648,13 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>a_generator.throw(MyException)</code>
+<td><code class=pp>a_generator.throw(MyException)</code>
<td><i>no change</i>
<tr><th>②
-<td><code>a_generator.throw(MyException, 'error message')</code>
-<td><code>a_generator.throw(MyException('error message'))</code>
+<td><code class=pp>a_generator.throw(MyException, 'error message')</code>
+<td><code class=pp>a_generator.throw(MyException('error message'))</code>
<tr><th>③
-<td><code>a_generator.throw('error message')</code>
+<td><code class=pp>a_generator.throw('error message')</code>
<td><i>unsupported</i>
</table>
<ol>
@@ -669,19 +669,19 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>xrange(10)</code>
-<td><code>range(10)</code>
+<td><code class=pp>xrange(10)</code>
+<td><code class=pp>range(10)</code>
<tr><th>②
-<td><code>a_list = range(10)</code>
-<td><code>a_list = list(range(10))</code>
+<td><code class=pp>a_list = range(10)</code>
+<td><code class=pp>a_list = list(range(10))</code>
<tr><th>③
-<td><code>[i for i in xrange(10)]</code>
-<td><code>[i for i in range(10)]</code>
+<td><code class=pp>[i for i in xrange(10)]</code>
+<td><code class=pp>[i for i in range(10)]</code>
<tr><th>④
-<td><code>for i in range(10):</code>
+<td><code class=pp>for i in range(10):</code>
<td><i>no change</i>
<tr><th>⑤
-<td><code>sum(range(10))</code>
+<td><code class=pp>sum(range(10))</code>
<td><i>no change</i>
</table>
<ol>
@@ -698,14 +698,14 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>raw_input()</code>
-<td><code>input()</code>
+<td><code class=pp>raw_input()</code>
+<td><code class=pp>input()</code>
<tr><th>②
-<td><code>raw_input('prompt')</code>
-<td><code>input('prompt')</code>
+<td><code class=pp>raw_input('prompt')</code>
+<td><code class=pp>input('prompt')</code>
<tr><th>③
-<td><code>input()</code>
-<td><code>eval(input())</code>
+<td><code class=pp>input()</code>
+<td><code class=pp>eval(input())</code>
</table>
<ol>
<li>In the simplest form, <code>raw_input()</code> becomes <code>input()</code>.
@@ -719,26 +719,26 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>a_function.func_name</code>
-<td><code>a_function.__name__</code>
+<td><code class=pp>a_function.func_name</code>
+<td><code class=pp>a_function.__name__</code>
<tr><th>②
-<td><code>a_function.func_doc</code>
-<td><code>a_function.__doc__</code>
+<td><code class=pp>a_function.func_doc</code>
+<td><code class=pp>a_function.__doc__</code>
<tr><th>③
-<td><code>a_function.func_defaults</code>
-<td><code>a_function.__defaults__</code>
+<td><code class=pp>a_function.func_defaults</code>
+<td><code class=pp>a_function.__defaults__</code>
<tr><th>④
-<td><code>a_function.func_dict</code>
-<td><code>a_function.__dict__</code>
+<td><code class=pp>a_function.func_dict</code>
+<td><code class=pp>a_function.__dict__</code>
<tr><th>⑤
-<td><code>a_function.func_closure</code>
-<td><code>a_function.__closure__</code>
+<td><code class=pp>a_function.func_closure</code>
+<td><code class=pp>a_function.__closure__</code>
<tr><th>⑥
-<td><code>a_function.func_globals</code>
-<td><code>a_function.__globals__</code>
+<td><code class=pp>a_function.func_globals</code>
+<td><code class=pp>a_function.__globals__</code>
<tr><th>⑦
-<td><code>a_function.func_code</code>
-<td><code>a_function.__code__</code>
+<td><code class=pp>a_function.func_code</code>
+<td><code class=pp>a_function.__code__</code>
</table>
<ol>
<li>The <code>__name__</code> attribute (previously <code>func_name</code>) contains the function’s name.
@@ -756,10 +756,10 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>for line in a_file.xreadlines():</code>
-<td><code>for line in a_file:</code>
+<td><code class=pp>for line in a_file.xreadlines():</code>
+<td><code class=pp>for line in a_file:</code>
<tr><th>②
-<td><code>for line in a_file.xreadlines(5):</code>
+<td><code class=pp>for line in a_file.xreadlines(5):</code>
<td><i>no change</i>
</table>
<ol>
@@ -774,16 +774,16 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>lambda (x,): x + f(x)</code>
-<td><code>lambda x1: x1[0] + f(x1[0])</code>
+<td><code class=pp>lambda (x,): x + f(x)</code>
+<td><code class=pp>lambda x1: x1[0] + f(x1[0])</code>
<tr><th>②
-<td><code>lambda (x, y): x + f(y)</code>
-<td><code>lambda x_y: x_y[0] + f(x_y[1])</code>
+<td><code class=pp>lambda (x, y): x + f(y)</code>
+<td><code class=pp>lambda x_y: x_y[0] + f(x_y[1])</code>
<tr><th>③
-<td><code>lambda (x, (y, z)): x + y + z</code>
-<td><code>lambda x_y_z: x_y_z[0] + x_y_z[1][0] + x_y_z[1][1]</code>
+<td><code class=pp>lambda (x, (y, z)): x + y + z</code>
+<td><code class=pp>lambda x_y_z: x_y_z[0] + x_y_z[1][0] + x_y_z[1][1]</code>
<tr><th>④
-<td><code>lambda x, y, z: x + y + z</code>
+<td><code class=pp>lambda x, y, z: x + y + z</code>
<td><i>unchanged</i>
</table>
<ol>
@@ -799,14 +799,14 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>aClassInstance.aClassMethod.im_func</code>
-<td><code>aClassInstance.aClassMethod.__func__</code>
+<td><code class=pp>aClassInstance.aClassMethod.im_func</code>
+<td><code class=pp>aClassInstance.aClassMethod.__func__</code>
<tr><th>
-<td><code>aClassInstance.aClassMethod.im_self</code>
-<td><code>aClassInstance.aClassMethod.__self__</code>
+<td><code class=pp>aClassInstance.aClassMethod.im_self</code>
+<td><code class=pp>aClassInstance.aClassMethod.__self__</code>
<tr><th>
-<td><code>aClassInstance.aClassMethod.im_class</code>
-<td><code>aClassInstance.aClassMethod.__self__.__class__</code>
+<td><code class=pp>aClassInstance.aClassMethod.im_class</code>
+<td><code class=pp>aClassInstance.aClassMethod.__self__.__class__</code>
</table>
<h2 id=nonzero><code>__nonzero__</code> special method</h2>
<p>In Python 2, you could build your own classes that could be used in a boolean context. For example, you could instantiate the class and then use the instance in an <code>if</code> statement. To do this, you defined a special <code>__nonzero__()</code> method which returned <code>True</code> or <code>False</code>, and it was called whenever the instance was used in a boolean context. In Python 3, you can still do this, but the name of the method has changed to <code>__bool__()</code>.
@@ -815,14 +815,14 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def __nonzero__(self):
pass</code></pre>
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def __bool__(self):
pass</code></pre>
<tr><th>②
-<td><pre><code>class A:
+<td><pre><code class=pp>class A:
def __nonzero__(self, x, y):
pass</code></pre>
<td><i>no change</i>
@@ -838,8 +838,8 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>x = 0755</code>
-<td><code>x = 0o755</code>
+<td><code class=pp>x = 0755</code>
+<td><code class=pp>x = 0o755</code>
</table>
<h2 id=renames><code>sys.maxint</code></h2>
<p>Due to the <a href=#long>integration of the <code>long</code> and <code>int</code> types</a>, the <code>sys.maxint</code> constant is no longer accurate. Because the value may still be useful in determining platform-specific capabilities, it has been retained but renamed as <code>sys.maxsize</code>.
@@ -848,11 +848,11 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>from sys import maxint</code>
-<td><code>from sys import maxsize</code>
+<td><code class=pp>from sys import maxint</code>
+<td><code class=pp>from sys import maxsize</code>
<tr><th>②
-<td><code>a_function(sys.maxint)</code>
-<td><code>a_function(sys.maxsize)</code>
+<td><code class=pp>a_function(sys.maxint)</code>
+<td><code class=pp>a_function(sys.maxsize)</code>
</table>
<ol>
<li><code>maxint</code> becomes <code>maxsize</code>.
@@ -865,8 +865,8 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>callable(anything)</code>
-<td><code>hasattr(anything, '__call__')</code>
+<td><code class=pp>callable(anything)</code>
+<td><code class=pp>hasattr(anything, '__call__')</code>
</table>
<h2 id=zip><code>zip()</code> global function</h2>
<p>In Python 2, the global <code>zip()</code> function took any number of sequences and returned a list of tuples. The first tuple contained the first item from each sequence; the second tuple contained the second item from each sequence; and so on. In Python 3, <code>zip()</code> returns an iterator instead of a list.
@@ -875,10 +875,10 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>zip(a, b, c)</code>
-<td><code>list(zip(a, b, c))</code>
+<td><code class=pp>zip(a, b, c)</code>
+<td><code class=pp>list(zip(a, b, c))</code>
<tr><th>②
-<td><code>d.join(zip(a, b, c))</code>
+<td><code class=pp>d.join(zip(a, b, c))</code>
<td><i>no change</i>
</table>
<ol>
@@ -892,11 +892,11 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>x = StandardError()</code>
-<td><code>x = Exception()</code>
+<td><code class=pp>x = StandardError()</code>
+<td><code class=pp>x = Exception()</code>
<tr><th>
-<td><code>x = StandardError(a, b, c)</code>
-<td><code>x = Exception(a, b, c)</code>
+<td><code class=pp>x = StandardError(a, b, c)</code>
+<td><code class=pp>x = Exception(a, b, c)</code>
</table>
<h2 id=types><code>types</code> module constants</h2>
<p>The <code>types</code> module contains a variety of constants to help you determine the type of an object. In Python 2, it contained constants for all primitive types like <code>dict</code> and <code>int</code>. In Python 3, these constants have been eliminated; just use the primitive type name instead.
@@ -905,26 +905,26 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>types.StringType</code>
-<td><code>bytes</code>
+<td><code class=pp>types.StringType</code>
+<td><code class=pp>bytes</code>
<tr><th>
-<td><code>types.DictType</code>
-<td><code>dict</code>
+<td><code class=pp>types.DictType</code>
+<td><code class=pp>dict</code>
<tr><th>
-<td><code>types.IntType</code>
-<td><code>int</code>
+<td><code class=pp>types.IntType</code>
+<td><code class=pp>int</code>
<tr><th>
-<td><code>types.LongType</code>
-<td><code>int</code>
+<td><code class=pp>types.LongType</code>
+<td><code class=pp>int</code>
<tr><th>
-<td><code>types.ListType</code>
-<td><code>list</code>
+<td><code class=pp>types.ListType</code>
+<td><code class=pp>list</code>
<tr><th>
-<td><code>types.NoneType</code>
-<td><code>type(None)</code>
+<td><code class=pp>types.NoneType</code>
+<td><code class=pp>type(None)</code>
</table>
<blockquote class=note>
-<p><span>☞</span><code>types.StringType</code> gets mapped to <code>bytes</code> instead of <code>str</code> because a Python 2 “string” (not a Unicode string, just a regular string) is really just a sequence of bytes in a particular character encoding.
+<p><span class=u>☞</span><code>types.StringType</code> gets mapped to <code>bytes</code> instead of <code>str</code> because a Python 2 “string” (not a Unicode string, just a regular string) is really just a sequence of bytes in a particular character encoding.
</blockquote>
<h2 id=isinstance><code>isinstance()</code> global function (3.1+)</h2>
<p>The <code>isinstance()</code> function checks whether an object is an instance of a particular class or type. In Python 2, you could pass a tuple of types, and <code>isinstance()</code> would return <code>True</code> if the object was any of those types. In Python 3, you can still do this, but passing the same type twice is deprecated.
@@ -933,11 +933,11 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>isinstance(x, (int, float, int))</code>
-<td><code>isinstance(x, (int, float))</code>
+<td><code class=pp>isinstance(x, (int, float, int))</code>
+<td><code class=pp>isinstance(x, (int, float))</code>
</table>
<blockquote class=note>
-<p><span>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix these cases of <code>isinstance()</code> automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
+<p><span class=u>☞</span>The version of <code>2to3</code> that shipped with Python 3.0 would not fix these cases of <code>isinstance()</code> automatically. The fix first appeared in the <code>2to3</code> script that shipped with Python 3.1.
</blockquote>
<h2 id=basestring><code>basestring</code> datatype</h2>
<p>Python 2 had two string types: Unicode and non-Unicode. But there was also another type, <code>basestring</code>. It was an abstract type, a superclass for both the <code>str</code> and <code>unicode</code> types. It couldn’t be called or instantiated directly, but you could pass it to the global <code>isinstance()</code> function to check whether an object was either a Unicode or non-Unicode string. In Python 3, there is only one string type, so <code>basestring</code> has no reason to exist.
@@ -946,8 +946,8 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>isinstance(x, basestring)</code>
-<td><code>isinstance(x, str)</code>
+<td><code class=pp>isinstance(x, basestring)</code>
+<td><code class=pp>isinstance(x, str)</code>
</table>
<h2 id=itertools><code>itertools</code> module</h2>
<p>Python 2.3 introduced the <code>itertools</code> module, which defined variants of the global <code>zip()</code>, <code>map()</code>, and <code>filter()</code> functions that returned iterators instead of lists. In Python 3, those global functions return iterators, so those functions in the <code>itertools</code> module have been eliminated.
@@ -956,17 +956,17 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><code>itertools.izip(a, b)</code>
-<td><code>zip(a, b)</code>
+<td><code class=pp>itertools.izip(a, b)</code>
+<td><code class=pp>zip(a, b)</code>
<tr><th>②
-<td><code>itertools.imap(a, b)</code>
-<td><code>map(a, b)</code>
+<td><code class=pp>itertools.imap(a, b)</code>
+<td><code class=pp>map(a, b)</code>
<tr><th>③
-<td><code>itertools.ifilter(a, b)</code>
-<td><code>filter(a, b)</code>
+<td><code class=pp>itertools.ifilter(a, b)</code>
+<td><code class=pp>filter(a, b)</code>
<tr><th>④
-<td><code>from itertools import imap, izip, foo</code>
-<td><code>from itertools import foo</code>
+<td><code class=pp>from itertools import imap, izip, foo</code>
+<td><code class=pp>from itertools import foo</code>
</table>
<ol>
<li>Instead of <code>itertools.izip()</code>, just use the global <code>zip()</code> function.
@@ -981,14 +981,14 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>sys.exc_type</code>
-<td><code>sys.exc_info()[0]</code>
+<td><code class=pp>sys.exc_type</code>
+<td><code class=pp>sys.exc_info()[0]</code>
<tr><th>
-<td><code>sys.exc_value</code>
-<td><code>sys.exc_info()[1]</code>
+<td><code class=pp>sys.exc_value</code>
+<td><code class=pp>sys.exc_info()[1]</code>
<tr><th>
-<td><code>sys.exc_traceback</code>
-<td><code>sys.exc_info()[2]</code>
+<td><code class=pp>sys.exc_traceback</code>
+<td><code class=pp>sys.exc_info()[2]</code>
</table>
<h2 id=paren>List comprehensions over tuples</h2>
<p>In Python 2, if you wanted to code a list comprehension that iterated over a tuple, you did not need to put parentheses around the tuple values. In Python 3, explicit parentheses are required.
@@ -997,8 +997,8 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>[i for i in 1, 2]</code>
-<td><code>[i for i in (1, 2)]</code>
+<td><code class=pp>[i for i in 1, 2]</code>
+<td><code class=pp>[i for i in (1, 2)]</code>
</table>
<h2 id=getcwdu><code>os.getcwdu()</code> function</h2>
<p>Python 2 had a function named <code>os.getcwd()</code>, which returned the current working directory as a (non-Unicode) string. Because modern file systems can handle directory names in any character encoding, Python 2.3 introduced <code>os.getcwdu()</code>. The <code>os.getcwdu()</code> function returned the current working directory as a Unicode string. In Python 3, there is only one string type (Unicode), so <code>os.getcwd()</code> is all you need.
@@ -1007,8 +1007,8 @@ except:
<th>Python 2
<th>Python 3
<tr><th>
-<td><code>os.getcwdu()</code>
-<td><code>os.getcwd()</code>
+<td><code class=pp>os.getcwdu()</code>
+<td><code class=pp>os.getcwd()</code>
</table>
<h2 id=metaclass>Metaclasses</h2>
<p>In Python 2, you could create metaclasses either by defining the <code>metaclass</code> argument in the class declaration, or by defining a special class-level <code>__metaclass__</code> attribute. In Python 3, the class-level attribute has been eliminated.
@@ -1017,18 +1017,18 @@ except:
<th>Python 2
<th>Python 3
<tr><th>①
-<td><pre><code>class C(metaclass=PapayaMeta):
+<td><pre><code class=pp>class C(metaclass=PapayaMeta):
pass</code></pre>
<td><i>unchanged</i>
<tr><th>②
-<td><pre><code>class Whip:
+<td><pre><code class=pp>class Whip:
__metaclass__ = PapayaMeta</code></pre>
-<td><pre><code>class Whip(metaclass=PapayaMeta):
+<td><pre><code class=pp>class Whip(metaclass=PapayaMeta):
pass</code></pre>
<tr><th>③
-<td><pre><code>class C(Whipper, Beater):
+<td><pre><code class=pp>class C(Whipper, Beater):
__metaclass__ = PapayaMeta</code></pre>
-<td><pre><code>class C(Whipper, Beater, metaclass=PapayaMeta):
+<td><pre><code class=pp>class C(Whipper, Beater, metaclass=PapayaMeta):
pass</code></pre>
</table>
<ol>
@@ -1041,7 +1041,7 @@ except:
<h3 id=set_literal><code>set()</code> literals (explicit)</h3>
<p>In Python 2, the only way to define a literal set in your code was to call <code>set(a_sequence)</code>. This still works in Python 3, but a clearer way of doing it is to use the new set literal notation: curly braces. (Dictionaries are also defined with curly braces, which makes sense once you think about it, because dictionaries are just sets of key-value pairs.)
<blockquote class=note>
-<p><span>☞</span>The <code>2to3</code> script will not fix <code>set()</code> literals by default. To enable this fix, specify <kbd>-f set_literal</kbd> on the command line when you call <code>2to3</code>.
+<p><span class=u>☞</span>The <code>2to3</code> script will not fix <code>set()</code> literals by default. To enable this fix, specify <kbd>-f set_literal</kbd> on the command line when you call <code>2to3</code>.
</blockquote>
<table>
<tr><th>Notes
@@ -1049,19 +1049,19 @@ except:
<th>After
<tr><th>
-<td><code>set([1, 2, 3])</code>
-<td><code>{1, 2, 3}</code>
+<td><code class=pp>set([1, 2, 3])</code>
+<td><code class=pp>{1, 2, 3}</code>
<tr><th>
-<td><code>set((1, 2, 3))</code>
-<td><code>{1, 2, 3}</code>
+<td><code class=pp>set((1, 2, 3))</code>
+<td><code class=pp>{1, 2, 3}</code>
<tr><th>
-<td><code>set([i for i in a_sequence])</code>
-<td><code>{i for i in a_sequence}</code>
+<td><code class=pp>set([i for i in a_sequence])</code>
+<td><code class=pp>{i for i in a_sequence}</code>
</table>
<h3 id=buffer><code>buffer()</code> global function (explicit)</h3>
<p>Python objects implemented in C can export a “buffer interface,” which allows other Python code to directly read and write a block of memory. (That is exactly as powerful and scary as it sounds.) In Python 3, <code>buffer()</code> has been renamed to <code>memoryview()</code>. (It’s a little more complicated than that, but you can almost certainly ignore the differences.)
<blockquote class=note>
-<p><span>☞</span>The <code>2to3</code> script will not fix the <code>buffer()</code> function by default. To enable this fix, specify <kbd>-f buffer</kbd> on the command line when you call <code>2to3</code>.
+<p><span class=u>☞</span>The <code>2to3</code> script will not fix the <code>buffer()</code> function by default. To enable this fix, specify <kbd>-f buffer</kbd> on the command line when you call <code>2to3</code>.
</blockquote>
<table>
<tr><th>Notes
@@ -1069,13 +1069,13 @@ except:
<th>After
<tr><th>
-<td><code>x = buffer(y)</code>
-<td><code>x = memoryview(y)</code>
+<td><code class=pp>x = buffer(y)</code>
+<td><code class=pp>x = memoryview(y)</code>
</table>
<h3 id=wscomma>Whitespace around commas (explicit)</h3>
<p>Despite being draconian about whitespace for indenting and outdenting, Python is actually quite liberal about whitespace in other areas. Within lists, tuples, sets, and dictionaries, whitespace can appear before and after commas with no ill effects. However, the Python style guide states that commas should be preceded by zero spaces and followed by one. Although this is purely an aesthetic issue (the code works either way, in both Python 2 and Python 3), the <code>2to3</code> script can optionally fix this for you.
<blockquote class=note>
-<p><span>☞</span>The <code>2to3</code> script will not fix whitespace around commas by default. To enable this fix, specify <kbd>-f wscomma</kbd> on the command line when you call <code>2to3</code>.
+<p><span class=u>☞</span>The <code>2to3</code> script will not fix whitespace around commas by default. To enable this fix, specify <kbd>-f wscomma</kbd> on the command line when you call <code>2to3</code>.
</blockquote>
<table>
<tr><th>Notes
@@ -1083,16 +1083,16 @@ except:
<th>After
<tr><th>
-<td><code>a ,b</code>
-<td><code>a, b</code>
+<td><code class=pp>a ,b</code>
+<td><code class=pp>a, b</code>
<tr><th>
-<td><code>{a :b}</code>
-<td><code>{a: b}</code>
+<td><code class=pp>{a :b}</code>
+<td><code class=pp>{a: b}</code>
</table>
<h3 id=idioms>Common idioms (explicit)</h3>
<p>There were a number of common idioms built up in the Python community. Some, like the <code>while 1:</code> loop, date back to Python 1. (Python didn’t have a true boolean type until version 2.3, so developers used <code>1</code> and <code>0</code> instead.) Modern Python programmers should train their brains to use modern versions of these idioms instead.
<blockquote class=note>
-<p><span>☞</span>The <code>2to3</code> script will not fix common idioms by default. To enable this fix, specify <kbd>-f idioms</kbd> on the command line when you call <code>2to3</code>.
+<p><span class=u>☞</span>The <code>2to3</code> script will not fix common idioms by default. To enable this fix, specify <kbd>-f idioms</kbd> on the command line when you call <code>2to3</code>.
</blockquote>
<table>
<tr><th>Notes
@@ -1100,26 +1100,27 @@ except:
<th>After
<tr><th>
-<td><pre><code>while 1:
+<td><pre><code class=pp>while 1:
do_stuff()</code></pre>
-<td><pre><code>while True:
+<td><pre><code class=pp>while True:
do_stuff()</code></pre>
<tr><th>
-<td><code>type(x) == T</code>
-<td><code>isinstance(x, T)</code>
+<td><code class=pp>type(x) == T</code>
+<td><code class=pp>isinstance(x, T)</code>
<tr><th>
-<td><code>type(x) is T</code>
-<td><code>isinstance(x, T)</code>
+<td><code class=pp>type(x) is T</code>
+<td><code class=pp>isinstance(x, T)</code>
<tr><th>
-<td><pre><code>a_list = list(a_sequence)
+<td><pre><code class=pp>a_list = list(a_sequence)
a_list.sort()
do_stuff(a_list)</code></pre>
-<td><pre><code>a_list = sorted(a_sequence)
+<td><pre><code class=pp>a_list = sorted(a_sequence)
do_stuff(a_list)</code></pre>
</table>
<p>FIXME: once the rest of the book is written, this appendix should contain copious links back to any chapter or section that touches on these features.
-<p class=v><a href=where-to-go-from-here.html rel=prev title='back to “Where To Go From Here”'><span>☜</span></a> <a href=special-method-names.html rel=next title='onward to “Special Method Names”'><span>☞</span></a>
+<p class=v><a href=where-to-go-from-here.html rel=prev title='back to “Where To Go From Here”'><span class=u>☜</span></a> <a href=special-method-names.html rel=next title='onward to “Special Method Names”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/print.css b/print.css
index e949467..d1e640b 100644
--- a/print.css
+++ b/print.css
@@ -110,3 +110,16 @@ aside {
.w, .d, form, form + p, #level, #toc {
display: none !important;
}
+
+/* syntax highlighting */
+
+.str { color: #060; }
+.kwd { color: #006; font-weight: bold; }
+.com { color: #600; font-style: italic; }
+.typ { color: #404; font-weight: bold; }
+.lit { color: #044; }
+.pun { color: #440; }
+.pln { color: #000; }
+.tag { color: #006; font-weight: bold; }
+.atn { color: #404; }
+.atv { color: #060; }
diff --git a/publish b/publish
index be29cf9..cd9593c 100755
--- a/publish
+++ b/publish
@@ -31,12 +31,14 @@ done
# minimize JS and CSS
echo "minimizing JS"
revision=`hg log|grep changeset|cut -d":" -f3|head -1`
-java -jar util/yuicompressor-2.4.2.jar build/j/dip3.js > build/j/dip3-$revision.js
+java -jar util/yuicompressor-2.4.2.jar build/j/prettify.js > build/j/prettify.min.js
+java -jar util/yuicompressor-2.4.2.jar build/j/dip3.js > build/j/dip3.min.js
# combine jQuery and our script
echo "combining JS"
-cat build/j/jquery.min.js build/j/dip3-$revision.js > build/j/$revision.js
+cat build/j/jquery.min.js build/j/prettify.min.js build/j/dip3.min.js > build/j/$revision.js
sed -i -e "s|<script src=j/jquery.js></script>||g" build/*.html
+sed -i -e "s|<script src=j/prettify.js></script>||g" build/*.html
sed -i -e "s|<script src=j/dip3.js>|<script src=j/${revision}.js>|g" build/*.html
echo "minimizing CSS"
diff --git a/refactoring.html b/refactoring.html
index 4ea7b6e..d819acf 100644
--- a/refactoring.html
+++ b/refactoring.html
@@ -12,18 +12,18 @@ body{counter-reset:h1 10}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#refactoring>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#refactoring>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=advanced>♦♦♦♦♢</span>
<h1>Refactoring</h1>
<blockquote class=q>
-<p><span>❝</span> After one has played a vast quantity of notes and more notes, it is simplicity that emerges as the crowning reward of art. <span>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Fr%C3%A9d%C3%A9ric_Chopin>Frédéric Chopin</a>
+<p><span class=u>❝</span> After one has played a vast quantity of notes and more notes, it is simplicity that emerges as the crowning reward of art. <span class=u>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Fr%C3%A9d%C3%A9ric_Chopin>Frédéric Chopin</a>
</blockquote>
<p id=toc>
<h2 id=divingin>Diving In</h2>
<p class=f>Despite your best efforts to write comprehensive unit tests, bugs happen. What do I mean by “bug”? A bug is a test case you haven’t written yet.
<pre class=screen><samp class=p>>>> </samp><kbd>import roman7</kbd>
-<a><samp class=p>>>> </samp><kbd>roman7.from_roman('')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>roman7.from_roman('')</kbd> <span class=u>①</span></a>
<samp>0</samp></pre>
<ol>
<li>Remember in the [FIXME-xref] previous section when you kept seeing that an empty string would match the regular expression you were using to check for valid Roman numerals? Well, it turns out that this is still true for the final version of the regular expression. And that’s a bug; you want an empty string to raise an <code>InvalidRomanNumeralError</code> exception just like any other sequence of characters that don’t represent a valid Roman numeral.
@@ -31,13 +31,13 @@ body{counter-reset:h1 10}
<p>After reproducing the bug, and before fixing it, you should write a test case that fails, thus illustrating the bug.
-<pre><code>class FromRomanBadInput(unittest.TestCase):
+<pre><code class=pp>class FromRomanBadInput(unittest.TestCase):
.
.
.
def testBlank(self):
'''from_roman should fail with blank string'''
-<a> self.assertRaises(roman6.InvalidRomanNumeralError, roman6.from_roman, '') <span>①</span></a></code></pre>
+<a> self.assertRaises(roman6.InvalidRomanNumeralError, roman6.from_roman, '') <span class=u>①</span></a></code></pre>
<ol>
<li>Pretty simple stuff here. Call <code>from_roman()</code> with an empty string and make sure it raises an <code>InvalidRomanNumeralError</code> exception. The hard part was finding the bug; now that you know about it, testing for it is the easy part.
</ol>
@@ -72,9 +72,9 @@ FAILED (failures=1)</samp></pre>
<p><em>Now</em> you can fix the bug.
-<pre><code>def from_roman(s):
+<pre><code class=pp>def from_roman(s):
'''convert Roman numeral to integer'''
-<a> if not s: <span>①</span></a>
+<a> if not s: <span class=u>①</span></a>
raise InvalidRomanNumeralError, 'Input can not be blank'
if not re.search(romanNumeralPattern, s):
raise InvalidRomanNumeralError, 'Invalid Roman numeral: {0}'.format(s)
@@ -92,7 +92,7 @@ FAILED (failures=1)</samp></pre>
<pre class=screen>
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest8.py -v</kbd>
-<a><samp>from_roman should fail with blank string ... ok</samp> <span>①</span></a>
+<a><samp>from_roman should fail with blank string ... ok</samp> <span class=u>①</span></a>
<samp>from_roman should fail with malformed antecedents ... ok
from_roman should fail with repeated pairs of numerals ... ok
from_roman should fail with too many repeated numerals ... ok
@@ -107,7 +107,7 @@ to_roman should fail with 0 input ... ok
----------------------------------------------------------------------
Ran 11 tests in 0.156s
</samp>
-<a><samp>OK</samp> <span>②</span></a></pre>
+<a><samp>OK</samp> <span class=u>②</span></a></pre>
<ol>
<li>The blank string test case now passes, so the bug is fixed.
<li>All the other test cases still pass, which means that this bug fix didn’t break anything else. Stop coding.
@@ -123,14 +123,13 @@ Ran 11 tests in 0.156s
<p>Suppose, for instance, that you wanted to expand the range of the Roman numeral conversion functions. Remember [FIXME-xref] the rule that said that no character could be repeated more than three times? Well, the Romans were willing to make an exception to that rule by having 4 <code>M</code> characters in a row to represent <code>4000</code>. If you make this change, you’ll be able to expand the range of convertible numbers from <code>1..3999</code> to <code>1..4999</code>. But first, you need to make some changes to your test cases.
<p class=d>[<a href=examples/roman8.py>download <code>roman8.py</code></a>]
-<pre><code>
-class KnownValues(unittest.TestCase):
+<pre><code class=pp>class KnownValues(unittest.TestCase):
known_values = ( (1, 'I'),
.
.
.
(3999, 'MMMCMXCIX'),
-<a> (4000, 'MMMM'), <span>①</span></a>
+<a> (4000, 'MMMM'), <span class=u>①</span></a>
(4500, 'MMMMD'),
(4888, 'MMMMDCCCLXXXVIII'),
(4999, 'MMMMCMXCIX') )
@@ -138,7 +137,7 @@ class KnownValues(unittest.TestCase):
class ToRomanBadInput(unittest.TestCase):
def test_too_large(self):
'''to_roman should fail with large input'''
-<a> self.assertRaises(roman8.OutOfRangeError, roman8.to_roman, 5000) <span>②</span></a>
+<a> self.assertRaises(roman8.OutOfRangeError, roman8.to_roman, 5000) <span class=u>②</span></a>
.
.
@@ -147,7 +146,7 @@ class ToRomanBadInput(unittest.TestCase):
class FromRomanBadInput(unittest.TestCase):
def test_too_many_repeated_numerals(self):
'''from_roman should fail with too many repeated numerals'''
-<a> for s in ('MMMMM', 'DD', 'CCCC', 'LL', 'XXXX', 'VV', 'IIII'): <span>③</span></a>
+<a> for s in ('MMMMM', 'DD', 'CCCC', 'LL', 'XXXX', 'VV', 'IIII'): <span class=u>③</span></a>
self.assertRaises(roman8.InvalidRomanNumeralError, roman8.from_roman, s)
.
@@ -157,7 +156,7 @@ class FromRomanBadInput(unittest.TestCase):
class RoundtripCheck(unittest.TestCase):
def test_roundtrip(self):
'''from_roman(to_roman(n))==n for all n'''
-<a> for integer in range(1, 5000): <span>④</span></a>
+<a> for integer in range(1, 5000): <span class=u>④</span></a>
numeral = roman8.to_roman(integer)
result = roman8.from_roman(numeral)
self.assertEqual(integer, result)</code></pre>
@@ -177,9 +176,9 @@ from_roman should fail with malformed antecedents ... ok
from_roman should fail with non-string input ... ok
from_roman should fail with repeated pairs of numerals ... ok
from_roman should fail with too many repeated numerals ... ok
-<a>from_roman should give known result with known input ... ERROR <span>①</span></a>
-<a>to_roman should give known result with known input ... ERROR <span>②</span></a>
-<a>from_roman(to_roman(n))==n for all n ... ERROR <span>③</span></a>
+<a>from_roman should give known result with known input ... ERROR <span class=u>①</span></a>
+<a>to_roman should give known result with known input ... ERROR <span class=u>②</span></a>
+<a>from_roman(to_roman(n))==n for all n ... ERROR <span class=u>③</span></a>
to_roman should fail with negative input ... ok
to_roman should fail with non-integer input ... ok
to_roman should fail with large input ... ok
@@ -228,10 +227,9 @@ FAILED (errors=3)</samp></pre>
<p>Now that you have test cases that fail due to the new requirements, you can think about fixing the code to bring it in line with the test cases. (One thing that takes some getting used to when you first start coding unit tests is that the code being tested is never “ahead” of the test cases. While it’s behind, you still have some work to do, and as soon as it catches up to the test cases, you stop coding.)
<p class=d>[<a href=examples/roman9.py>download <code>roman9.py</code></a>]
-<pre><code>
-roman_numeral_pattern = re.compile('''
+<pre><code class=pp>roman_numeral_pattern = re.compile('''
^ # beginning of string
-<a> M{0,4} # thousands - 0 to 4 M's <span>①</span></a>
+<a> M{0,4} # thousands - 0 to 4 M's <span class=u>①</span></a>
(CM|CD|D?C{0,3}) # hundreds - 900 (CM), 400 (CD), 0-300 (0 to 3 C's),
# or 500-800 (D, followed by 0 to 3 C's)
(XC|XL|L?X{0,3}) # tens - 90 (XC), 40 (XL), 0-30 (0 to 3 X's),
@@ -243,7 +241,7 @@ roman_numeral_pattern = re.compile('''
def to_roman(n):
'''convert integer to Roman numeral'''
-<a> if not (0 < n < 5000): <span>②</span></a>
+<a> if not (0 < n < 5000): <span class=u>②</span></a>
raise OutOfRangeError('number out of range (must be 0..4999)')
if not isinstance(n, int):
raise NotIntegerError('non-integers can not be converted')
@@ -284,7 +282,7 @@ to_roman should fail with 0 input ... ok
----------------------------------------------------------------------
Ran 12 tests in 0.203s
-<a>OK <span>①</span></a></samp></pre>
+<a>OK <span class=u>①</span></a></samp></pre>
<ol>
<li>All the test cases pass. Stop coding.
</ol>
@@ -306,7 +304,7 @@ Ran 12 tests in 0.203s
<p>And best of all, you already have a complete set of unit tests. You can change over half the code in the module, but the unit tests will stay the same. That means you can prove — to yourself and to others — that the new code works just as well as the original.
<p class=d>[<a href=examples/roman10.py>download <code>roman10.py</code></a>]
-<pre><code>class OutOfRangeError(ValueError): pass
+<pre><code class=pp>class OutOfRangeError(ValueError): pass
class NotIntegerError(ValueError): pass
class InvalidRomanNumeralError(ValueError): pass
@@ -366,19 +364,19 @@ build_lookup_tables()</code></pre>
<p>Let’s break that down into digestable pieces. Arguably, the most important line is the last one:
-<pre><code>build_lookup_tables()</code></pre>
+<pre><code class=pp>build_lookup_tables()</code></pre>
<p>You will note that is a function call, but there’s no <code>if</code> statement around it. This is not an <code>if __name__ == '__main__'</code> block; it gets called <em>when the module is imported</em>. (It is important to understand that modules are only imported once, then cached. If you import an already-imported module, it does nothing. So this code will only get called the first time you import this module.)
<p>So what does the <code>build_lookup_tables()</code> function do? I’m glad you asked.
-<pre><code><a>to_roman_table = [ None ]
+<pre><code class=pp>to_roman_table = [ None ]
from_roman_table = {}
.
.
.
def build_lookup_tables():
-<a> def to_roman(n): <span>①</span></a>
+<a> def to_roman(n): <span class=u>①</span></a>
result = ''
for numeral, integer in roman_numeral_map:
if n >= integer:
@@ -390,8 +388,8 @@ def build_lookup_tables():
return result
for integer in range(1, 5000):
-<a> roman_numeral = to_roman(integer) <span>②</span></a>
-<a> to_roman_table.append(roman_numeral) <span>③</span></a>
+<a> roman_numeral = to_roman(integer) <span class=u>②</span></a>
+<a> to_roman_table.append(roman_numeral) <span class=u>③</span></a>
from_roman_table[roman_numeral] = integer</code></pre>
<ol>
<li>This is a clever bit of programming… perhaps too clever. The <code>to_roman()</code> function is defined above; it looks up values in the lookup table and returns them. But the <code>build_lookup_tables()</code> function redefines the <code>to_roman()</code> function to actually do work (like the previous examples did, before you added a lookup table). Within the <code>build_lookup_tables()</code> function, calling <code>to_roman()</code> will call this redefined version. Once the <code>build_lookup_tables()</code> function exits, the redefined version disappears — it is only defined in the local scope of the <code>build_lookup_tables()</code> function.
@@ -401,13 +399,13 @@ def build_lookup_tables():
<p>Once the lookup tables are built, the rest of the code is both easy and fast.
-<pre><code>def to_roman(n):
+<pre><code class=pp>def to_roman(n):
'''convert integer to Roman numeral'''
if not (0 < n < 5000):
raise OutOfRangeError('number out of range (must be 1..4999)')
if int(n) != n:
raise NotIntegerError('non-integers can not be converted')
-<a> return to_roman_table[n] <span>①</span></a>
+<a> return to_roman_table[n] <span class=u>①</span></a>
def from_roman(s):
'''convert Roman numeral to integer'''
@@ -417,7 +415,7 @@ def from_roman(s):
raise InvalidRomanNumeralError('Input can not be blank')
if s not in from_roman_table:
raise InvalidRomanNumeralError('Invalid Roman numeral: {0}'.format(s))
-<a> return from_roman_table[s] <span>②</span></a></code></pre>
+<a> return from_roman_table[s] <span class=u>②</span></a></code></pre>
<ol>
<li>After doing the same bounds checking as before, the <code>to_roman()</code> function simply finds the appropriate value in the lookup table and returns it.
<li>Similarly, the <code>from_roman()</code> function is reduced to some bounds checking and one line of code. No more regular expressions. No more looping. O(1) conversion to and from Roman numerals.
@@ -441,7 +439,7 @@ to_roman should fail with large input ... ok
to_roman should fail with 0 input ... ok
----------------------------------------------------------------------
-<a>Ran 12 tests in 0.031s <span>①</span></a>
+<a>Ran 12 tests in 0.031s <span class=u>①</span></a>
OK</samp></pre>
<ol>
@@ -473,7 +471,8 @@ OK</samp></pre>
<li>Refactoring mercilessly to improve performance, scalability, readability, maintainability, or whatever other -ility you’re lacking
</ul>
-<p class=v><a rel=prev class=todo><span>☜</span></a> <a rel=next class=todo><span>☞</span></a>
+<p class=v><a rel=prev class=todo><span class=u>☜</span></a> <a rel=next class=todo><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/regular-expressions.html b/regular-expressions.html
index 32c6e29..10937f6 100644
--- a/regular-expressions.html
+++ b/regular-expressions.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 4}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#regular-expressions>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#regular-expressions>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=intermediate>♦♦♦♢♢</span>
<h1>Regular Expressions</h1>
<blockquote class=q>
-<p><span>❝</span> Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems. <span>❞</span><br>— <a href=http://www.jwz.org/hacks/marginal.html>Jamie Zawinski</a>
+<p><span class=u>❝</span> Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems. <span class=u>❞</span><br>— <a href=http://www.jwz.org/hacks/marginal.html>Jamie Zawinski</a>
</blockquote>
<p id=toc>
<h2 id=divingin>Diving In</h2>
@@ -24,7 +24,7 @@ body{counter-reset:h1 4}
<p>If your goal can be accomplished with string methods, you should use them. They’re fast and simple and easy to read, and there’s a lot to be said for fast, simple, readable code. But if you find yourself using a lot of different string functions with <code>if</code> statements to handle special cases, or if you’re chaining calls to <code>split()</code> and <code>join()</code> to slice-and-dice your strings, you may need to move up to regular expressions.
<p>Regular expressions are a powerful and (mostly) standardized way of searching, replacing, and parsing text with complex patterns of characters. Although the regular expression syntax is tight and unlike normal code, the result can end up being <em>more</em> readable than a hand-rolled solution that uses a long chain of string functions. There are even ways of embedding comments within regular expressions, so you can include fine-grained documentation within them.
<blockquote class='note compare perl5'>
-<p><span>☞</span>If you’ve used regular expressions in other languages (like Perl 5), Python’s syntax will be very familiar. Read the summary of the <a href=http://docs.python.org/dev/library/re.html#module-contents><code>re</code> module</a> to get an overview of the available functions and their arguments.
+<p><span class=u>☞</span>If you’ve used regular expressions in other languages (like Perl 5), Python’s syntax will be very familiar. Read the summary of the <a href=http://docs.python.org/dev/library/re.html#module-contents><code>re</code> module</a> to get an overview of the available functions and their arguments.
</blockquote>
<p class=a>⁂
@@ -32,15 +32,15 @@ body{counter-reset:h1 4}
<p>This series of examples was inspired by a real-life problem I had in my day job several years ago, when I needed to scrub and standardize street addresses exported from a legacy system before importing them into a newer system. (See, I don’t just make this stuff up; it’s actually useful.) This example shows how I approached the problem.
<pre class=screen>
<samp class=p>>>> </samp><kbd>s = '100 NORTH MAIN ROAD'</kbd>
-<a><samp class=p>>>> </samp><kbd>s.replace('ROAD', 'RD.')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>s.replace('ROAD', 'RD.')</kbd> <span class=u>①</span></a>
<samp>'100 NORTH MAIN RD.'</samp>
<samp class=p>>>> </samp><kbd>s = '100 NORTH BROAD ROAD'</kbd>
-<a><samp class=p>>>> </samp><kbd>s.replace('ROAD', 'RD.')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>s.replace('ROAD', 'RD.')</kbd> <span class=u>②</span></a>
<samp>'100 NORTH BRD. RD.'</samp>
-<a><samp class=p>>>> </samp><kbd>s[:-4] + s[-4:].replace('ROAD', 'RD.')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>s[:-4] + s[-4:].replace('ROAD', 'RD.')</kbd> <span class=u>③</span></a>
<samp>'100 NORTH BROAD RD.'</samp>
-<a><samp class=p>>>> </samp><kbd>import re</kbd> <span>④</span></a>
-<a><samp class=p>>>> </samp><kbd>re.sub('ROAD$', 'RD.', s)</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>import re</kbd> <span class=u>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.sub('ROAD$', 'RD.', s)</kbd> <span class=u>⑤</span></a>
<samp>'100 NORTH BROAD RD.'</samp></pre>
<ol>
<li>My goal is to standardize a street address so that <code>'ROAD'</code> is always abbreviated as <code>'RD.'</code>. At first glance, I thought this was simple enough that I could just use the string method <code>replace()</code>. After all, all the data was already uppercase, so case mismatches would not be a problem. And the search string, <code>'ROAD'</code>, was a constant. And in this deceptively simple example, <code>s.replace()</code> does indeed work.
@@ -55,14 +55,14 @@ body{counter-reset:h1 4}
<samp class=p>>>> </samp><kbd>s = '100 BROAD'</kbd>
<samp class=p>>>> </samp><kbd>re.sub('ROAD$', 'RD.', s)</kbd>
<samp>'100 BRD.'</samp>
-<a><samp class=p>>>> </samp><kbd>re.sub('\\bROAD$', 'RD.', s)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.sub('\\bROAD$', 'RD.', s)</kbd> <span class=u>①</span></a>
<samp>'100 BROAD'</samp>
-<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD$', 'RD.', s)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD$', 'RD.', s)</kbd> <span class=u>②</span></a>
<samp>'100 BROAD'</samp>
<samp class=p>>>> </samp><kbd>s = '100 BROAD ROAD APT. 3'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD$', 'RD.', s)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD$', 'RD.', s)</kbd> <span class=u>③</span></a>
<samp>'100 BROAD ROAD APT. 3'</samp>
-<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD\b', 'RD.', s)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.sub(r'\bROAD\b', 'RD.', s)</kbd> <span class=u>④</span></a>
<samp>'100 BROAD RD. APT 3'</samp></pre>
<ol>
<li>What I <em>really</em> wanted was to match <code>'ROAD'</code> when it was at the end of the string <em>and</em> it was its own word (and not a part of some larger word). To express this in a regular expression, you use <code>\b</code>, which means “a word boundary must occur right here.” In Python, this is complicated by the fact that the <code>'\'</code> character in a string must itself be escaped. This is sometimes referred to as the backslash plague, and it is one reason why regular expressions are easier in Perl than in Python. On the down side, Perl mixes regular expressions with other syntax, so if you have a bug, it may be hard to tell whether it’s a bug in syntax or a bug in your regular expression.
@@ -96,15 +96,15 @@ body{counter-reset:h1 4}
<p>What would it take to validate that an arbitrary string is a valid Roman numeral? Let’s take it one digit at a time. Since Roman numerals are always written highest to lowest, let’s start with the highest: the thousands place. For numbers 1000 and higher, the thousands are represented by a series of <code>M</code> characters.
<pre class=screen>
<samp class=p>>>> </samp><kbd>import re</kbd>
-<a><samp class=p>>>> </samp><kbd>pattern = '^M?M?M?$'</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>pattern = '^M?M?M?$'</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span class=u>②</span></a>
<samp><SRE_Match object at 0106FB58></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span class=u>③</span></a>
<samp><SRE_Match object at 0106C290></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span class=u>④</span></a>
<samp><SRE_Match object at 0106AA38></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span>⑤</span></a>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, '')</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span class=u>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, '')</kbd> <span class=u>⑥</span></a>
<samp><SRE_Match object at 0106F4A8></samp></pre>
<ol>
<li>This pattern has three parts. <code>^</code> matches what follows only at the beginning of the string. If this were not specified, the pattern would match no matter where the <code>M</code> characters were, which is not what you want. You want to make sure that the <code>M</code> characters, if they’re there, are at the beginning of the string. <code>M?</code> optionally matches a single <code>M</code> character. Since this is repeated three times, you’re matching anywhere from zero to three <code>M</code> characters in a row. And <code>$</code> matches the end of the string. When combined with the <code>^</code> character at the beginning, this means that the pattern must match the entire string, with no other characters before or after the <code>M</code> characters.
@@ -142,15 +142,15 @@ body{counter-reset:h1 4}
<p>This example shows how to validate the hundreds place of a Roman numeral.
<pre class=screen>
<samp class=p>>>> </samp><kbd>import re</kbd>
-<a><samp class=p>>>> </samp><kbd>pattern = '^M?M?M?(CM|CD|D?C?C?C?)$'</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCM')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>pattern = '^M?M?M?(CM|CD|D?C?C?C?)$'</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCM')</kbd> <span class=u>②</span></a>
<samp><SRE_Match object at 01070390></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MD')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MD')</kbd> <span class=u>③</span></a>
<samp><SRE_Match object at 01073A50></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMCCC')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMCCC')</kbd> <span class=u>④</span></a>
<samp><SRE_Match object at 010748A8></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMC')</kbd> <span>⑤</span></a>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, '')</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMC')</kbd> <span class=u>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, '')</kbd> <span class=u>⑥</span></a>
<samp><SRE_Match object at 01071D98></samp></pre>
<ol>
<li>This pattern starts out the same as the previous one, checking for the beginning of the string (<code>^</code>), then the thousands place (<code>M?M?M?</code>). Then it has the new part, in parentheses, which defines a set of three mutually exclusive patterns, separated by vertical bars: <code>CM</code>, <code>CD</code>, and <code>D?C?C?C?</code> (which is an optional <code>D</code> followed by zero to three optional <code>C</code> characters). The regular expression parser checks for each of these patterns in order (from left to right), takes the first one that matches, and ignores the rest.
@@ -169,15 +169,15 @@ body{counter-reset:h1 4}
<pre class=screen>
<samp class=p>>>> </samp><kbd>import re</kbd>
<samp class=p>>>> </samp><kbd>pattern = '^M?M?M?$'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span class=u>①</span></a>
<samp><_sre.SRE_Match object at 0x008EE090></samp>
<samp class=p>>>> </samp><kbd>pattern = '^M?M?M?$'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span class=u>②</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
<samp class=p>>>> </samp><kbd>pattern = '^M?M?M?$'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span class=u>③</span></a>
<samp><_sre.SRE_Match object at 0x008EE090></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>This matches the start of the string, and then the first optional <code>M</code>, but not the second and third <code>M</code> (but that’s okay because they’re optional), and then the end of the string.
@@ -186,14 +186,14 @@ body{counter-reset:h1 4}
<li>This matches the start of the string, and then all three optional <code>M</code>, but then does not match the the end of the string (because there is still one unmatched <code>M</code>), so the pattern does not match and returns <code>None</code>.
</ol>
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>pattern = '^M{0,3}$'</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>pattern = '^M{0,3}$'</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span class=u>②</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MM')</kbd> <span class=u>③</span></a>
<samp><_sre.SRE_Match object at 0x008EE090></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMM')</kbd> <span class=u>④</span></a>
<samp><_sre.SRE_Match object at 0x008EEDA8></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMM')</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>This pattern says: “Match the start of the string, then anywhere from zero to three <code>M</code> characters, then the end of the string.” The 0 and 3 can be any numbers; if you want to match at least one but no more than three <code>M</code> characters, you could say <code>M{1,3}</code>.
@@ -206,15 +206,15 @@ body{counter-reset:h1 4}
<p>Now let’s expand the Roman numeral regular expression to cover the tens and ones place. This example shows the check for tens.
<pre class=screen>
<samp class=p>>>> </samp><kbd>pattern = '^M?M?M?(CM|CD|D?C?C?C?)(XC|XL|L?X?X?X?)$'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMXL')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMXL')</kbd> <span class=u>①</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCML')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCML')</kbd> <span class=u>②</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLX')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLX')</kbd> <span class=u>③</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXX')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXX')</kbd> <span class=u>④</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXXX')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXXX')</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>This matches the start of the string, then the first optional <code>M</code>, then <code>CM</code>, then <code>XL</code>, then the end of the string. Remember, the <code>(A|B|C)</code> syntax means “match exactly one of A, B, or C”. You match <code>XL</code>, so you ignore the <code>XC</code> and <code>L?X?X?X?</code> choices, and then move on to the end of the string. <code>MCML</code> is the Roman numeral representation of <code>1940</code>.
@@ -230,13 +230,13 @@ body{counter-reset:h1 4}
</pre><p>So what does that look like using this alternate <code>{n,m}</code> syntax? This example shows the new syntax.
<pre class=screen>
<samp class=p>>>> </samp><kbd>pattern = '^M{0,3}(CM|CD|D?C{0,3})(XC|XL|L?X{0,3})(IX|IV|V?I{0,3})$'</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MDLV')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MDLV')</kbd> <span class=u>①</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMDCLXVI')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMDCLXVI')</kbd> <span class=u>②</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMDCCCLXXXVIII')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMDCCCLXXXVIII')</kbd> <span class=u>③</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'I')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'I')</kbd> <span class=u>④</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp></pre>
<ol>
<li>This matches the start of the string, then one of a possible three <code>M</code> characters, then <code>D?C{0,3}</code>. Of that, it matches the optional <code>D</code> and zero of three possible <code>C</code> characters. Moving on, it matches <code>L?X{0,3}</code> by matching the optional <code>L</code> and zero of three possible <code>X</code> characters. Then it matches <code>V?I{0,3}</code> by matching the optional <code>V</code> and zero of three possible <code>I</code> characters, and finally the end of the string. <code>MDLV</code> is the Roman numeral representation of <code>1555</code>.
@@ -268,13 +268,13 @@ body{counter-reset:h1 4}
# or 5-8 (V, followed by 0 to 3 I's)
$ # end of string
'''</kbd>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M', re.VERBOSE)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M', re.VERBOSE)</kbd> <span class=u>①</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXXIX', re.VERBOSE)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MCMLXXXIX', re.VERBOSE)</kbd> <span class=u>②</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMDCCCLXXXVIII', re.VERBOSE)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'MMMDCCCLXXXVIII', re.VERBOSE)</kbd> <span class=u>③</span></a>
<samp><_sre.SRE_Match object at 0x008EEB48></samp>
-<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span>④</span></a></pre>
+<a><samp class=p>>>> </samp><kbd>re.search(pattern, 'M')</kbd> <span class=u>④</span></a></pre>
<ol>
<li>The most important thing to remember when using verbose regular expressions is that you need to pass an extra argument when working with them: <code>re.VERBOSE</code> is a constant defined in the <code>re</code> module that signals that the pattern should be treated as a verbose regular expression. As you can see, this pattern has quite a bit of whitespace (all of which is ignored), and several comments (all of which are ignored). Once you ignore the whitespace and the comments, this is exactly the same regular expression as you saw in the previous section, but it’s a lot more readable.
<li>This matches the start of the string, then one of a possible three <code>M</code>, then <code>CM</code>, then <code>L</code> and three of a possible three <code>X</code>, then <code>IX</code>, then the end of the string.
@@ -302,10 +302,10 @@ body{counter-reset:h1 4}
<p>Quite a variety! In each of these cases, I need to know that the area code was <code>800</code>, the trunk was <code>555</code>, and the rest of the phone number was <code>1212</code>. For those with an extension, I need to know that the extension was <code>1234</code>.
<p>Let’s work through developing a solution for phone number parsing. This example shows the first step.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})-(\d{3})-(\d{4})$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})-(\d{3})-(\d{4})$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234')</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>Always read regular expressions from left to right. This one matches the beginning of the string, and then <code>(\d{3})</code>. What’s <code>\d{3}</code>? Well, the <code>{3}</code> means “match exactly three numeric digits”; it’s a variation on the <a href=#nmsyntax><code>{n,m} syntax</code></a> you saw earlier. <code>\d</code> means “any numeric digit” (<code>0</code> through <code>9</code>). Putting it in parentheses means “match exactly three numeric digits, <em>and then remember them as a group that I can ask for later</em>”. Then match a literal hyphen. Then match another group of exactly three digits. Then another literal hyphen. Then another group of exactly four digits. Then match the end of the string.
@@ -313,12 +313,12 @@ body{counter-reset:h1 4}
<li>This regular expression is not the final answer, because it doesn’t handle a phone number with an extension on the end. For that, you’ll need to expand the regular expression.
</ol>
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})-(\d{3})-(\d{4})-(\d+)$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})-(\d{3})-(\d{4})-(\d+)$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800 555 1212 1234')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800 555 1212 1234')</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>This regular expression is almost identical to the previous one. Just as before, you match the beginning of the string, then a remembered group of three digits, then a hyphen, then a remembered group of three digits, then a hyphen, then a remembered group of four digits. What’s new is that you then match another hyphen, and a remembered group of one or more digits, then the end of the string.
@@ -328,14 +328,14 @@ body{counter-reset:h1 4}
</ol>
<p>The next example shows the regular expression to handle separators between the different parts of the phone number.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})\D+(\d{3})\D+(\d{4})\D+(\d+)$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800 555 1212 1234').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})\D+(\d{3})\D+(\d{4})\D+(\d+)$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800 555 1212 1234').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234').groups()</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212-1234').groups()</kbd> <span class=u>③</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>Hang on to your hat. You’re matching the beginning of the string, then a group of three digits, then <code>\D+</code>. What the heck is that? Well, <code>\D</code> matches any character <em>except</em> a numeric digit, and <code>+</code> means “1 or more”. So <code>\D+</code> matches one or more characters that are not digits. This is what you’re using instead of a literal hyphen, to try to match different separators.
@@ -346,14 +346,14 @@ body{counter-reset:h1 4}
</ol>
<p>The next example shows the regular expression for handling phone numbers <em>without</em> separators.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800.555.1212 x1234').groups()</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800.555.1212 x1234').groups()</kbd> <span class=u>③</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span class=u>④</span></a>
<samp>('800', '555', '1212', '')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('(800)5551212 x1234')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('(800)5551212 x1234')</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>The only change you’ve made since that last step is changing all the <code>+</code> to <code>*</code>. Instead of <code>\D+</code> between the parts of the phone number, you now match on <code>\D*</code>. Remember that <code>+</code> means “1 or more”? Well, <code>*</code> means “zero or more”. So now you should be able to parse phone numbers even when there is no separator character at all.
@@ -364,12 +364,12 @@ body{counter-reset:h1 4}
</ol>
<p>The next example shows how to handle leading characters in phone numbers.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^\D*(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('(800)5551212 ext. 1234').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'^\D*(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('(800)5551212 ext. 1234').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212').groups()</kbd> <span class=u>③</span></a>
<samp>('800', '555', '1212', '')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp></pre>
<ol>
<li>This is the same as in the previous example, except now you’re matching <code>\D*</code>, zero or more non-numeric characters, before the first remembered group (the area code). Notice that you’re not remembering these non-numeric characters (they’re not in parentheses). If you find them, you’ll just skip over them and then start remembering the area code whenever you get to it.
@@ -379,12 +379,12 @@ body{counter-reset:h1 4}
</ol>
<p>Let’s back up for a second. So far the regular expressions have all matched from the beginning of the string. But now you see that there may be an indeterminate amount of stuff at the beginning of the string that you want to ignore. Rather than trying to match it all just so you can skip over it, let’s take a different approach: don’t explicitly match the beginning of the string at all. This approach is shown in the next example.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234').groups()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern = re.compile(r'(\d{3})\D*(\d{3})\D*(\d{4})\D*(\d*)$')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234').groups()</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span class=u>③</span></a>
<samp>('800', '555', '1212', '')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('80055512121234')</kbd> <span class=u>④</span></a>
<samp>('800', '555', '1212', '1234')</samp></pre>
<ol>
<li>Note the lack of <code>^</code> in this regular expression. You are not matching the beginning of the string anymore. There’s nothing that says you need to match the entire input with your regular expression. The regular expression engine will do the hard work of figuring out where the input string starts to match, and go from there.
@@ -406,9 +406,9 @@ body{counter-reset:h1 4}
(\d*) # extension is optional and can be any number of digits
$ # end of string
''', re.VERBOSE)</kbd>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234').groups()</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('work 1-(800) 555.1212 #1234').groups()</kbd> <span class=u>①</span></a>
<samp>('800', '555', '1212', '1234')</samp>
-<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>phonePattern.search('800-555-1212')</kbd> <span class=u>②</span></a>
<samp>('800', '555', '1212', '')</samp></pre>
<ol>
<li>Other than being spread out over multiple lines, this is exactly the same regular expression as the last step, so it’s no surprise that it parses the same inputs.
@@ -433,7 +433,8 @@ body{counter-reset:h1 4}
<li><code>(x)</code> in general is a <em>remembered group</em>. You can get the value of what matched by using the <code>groups()</code> method of the object returned by <code>re.search</code>.
</ul>
<p>Regular expressions are extremely powerful, but they are not the correct solution for every problem. You should learn enough about them to know when they are appropriate, when they will solve your problems, and when they will cause more problems than they solve.
-<p class=v><a href=strings.html rel=prev title='back to “Strings”'><span>☜</span></a> <a href=generators.html rel=next title='onward to “Generators”'><span>☞</span></a>
+<p class=v><a href=strings.html rel=prev title='back to “Strings”'><span class=u>☜</span></a> <a href=generators.html rel=next title='onward to “Generators”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/special-method-names.html b/special-method-names.html
index 10394f1..87e5650 100644
--- a/special-method-names.html
+++ b/special-method-names.html
@@ -23,11 +23,11 @@ td a:link, td a:visited{border:0}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#special-method-names>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#special-method-names>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=pro>♦♦♦♦♦</span>
<h1>Special Method Names</h1>
<blockquote class=q>
-<p><span>❝</span> My specialty is being right when other people are wrong. <span>❞</span><br>— <a href=http://en.wikiquote.org/wiki/George_Bernard_Shaw>George Bernard Shaw</a>
+<p><span class=u>❝</span> My specialty is being right when other people are wrong. <span class=u>❞</span><br>— <a href=http://en.wikiquote.org/wiki/George_Bernard_Shaw>George Bernard Shaw</a>
</blockquote>
<p id=toc>
<h2 id=divingin>Diving in</h2>
@@ -44,24 +44,24 @@ td a:link, td a:visited{border:0}
<th>And Python Calls…
<tr><th>①
<td>to initialize an instance
-<td><code>x = MyClass()</code>
-<td><code>x.__init__()</code>
+<td><code class=pp>x = MyClass()</code>
+<td><code class=pp>x.__init__()</code>
<tr><th>②
<td>the “official” representation as a string
-<td><code>repr(x)</code>
-<td><code>x.__repr__()</code>
+<td><code class=pp>repr(x)</code>
+<td><code class=pp>x.__repr__()</code>
<tr><th>③
<td>the “informal” value as a string
-<td><code>str(x)</code>
-<td><code>x.__str__()</code>
+<td><code class=pp>str(x)</code>
+<td><code class=pp>x.__str__()</code>
<tr><th>④
<td>the “informal” value as a byte array
-<td><code>bytes(x)</code>
-<td><code>x.__bytes__()</code>
+<td><code class=pp>bytes(x)</code>
+<td><code class=pp>x.__bytes__()</code>
<tr><th>⑤
<td>the value as a formatted string
-<td><code>format(x)</code>
-<td><code>x.__format__(<var>format_spec</var>)</code>
+<td><code class=pp>format(x)</code>
+<td><code class=pp>x.__format__(<var>format_spec</var>)</code>
</table>
<ol>
<li>The <code>__init__()</code> method is called <em>after</em> the instance is created. If you want to control the actual creation process, use <a href=#esoterica>the <code>__new__()</code> method</a>.
@@ -82,15 +82,15 @@ td a:link, td a:visited{border:0}
<th>And Python Calls…
<tr><th>①
<td>to iterate through a sequence
-<td><code>iter(seq)</code>
+<td><code class=pp>iter(seq)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__iter__><code>seq.__iter__()</code></a>
<tr><th>②
<td>to get the next value from an iterator
-<td><code>next(seq)</code>
+<td><code class=pp>next(seq)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__next__><code>seq.__next__()</code></a>
<tr><th>③
<td>to create an iterator in reverse order
-<td><code>reversed(seq)</code>
+<td><code class=pp>reversed(seq)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__reversed__><code>seq.__reversed__()</code></a>
</table>
<ol>
@@ -110,23 +110,23 @@ td a:link, td a:visited{border:0}
<th>And Python Calls…
<tr><th>①
<td>to get a computed attribute (unconditionally)
-<td><code>x.my_property</code>
+<td><code class=pp>x.my_property</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__getattribute__><code>x.__getattribute__(<var>'my_property'</var>)</code></a>
<tr><th>②
<td>to get a computed attribute (fallback)
-<td><code>x.my_property</code>
+<td><code class=pp>x.my_property</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__getattr__><code>x.__getattr__(<var>'my_property'</var>)</code></a>
<tr><th>③
<td>to set an attribute
-<td><code>x.my_property = value</code>
+<td><code class=pp>x.my_property = value</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__setattr__><code>x.__setattr__(<var>'my_property'</var>, <var>value</var>)</code></a>
<tr><th>④
<td>to delete an attribute
-<td><code>del x.my_property</code>
+<td><code class=pp>del x.my_property</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__delattr__><code>x.__delattr__(<var>'my_property'</var>)</code></a>
<tr><th>⑤
<td>to list all attributes and methods
-<td><code>dir(x)</code>
+<td><code class=pp>dir(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__dir__><code>x.__dir__()</code></a>
</table>
<ol>
@@ -142,16 +142,16 @@ td a:link, td a:visited{border:0}
<pre class=screen>
<code>class Dynamo:
def __getattr__(self, key):
-<a> if key == 'color': <span>①</span></a>
+<a> if key == 'color': <span class=u>①</span></a>
return 'PapayaWhip'
else:
-<a> raise AttributeError <span>②</span></a></code>
+<a> raise AttributeError <span class=u>②</span></a></code>
<samp class=p>>>> </samp><kbd>dyn = Dynamo()</kbd>
-<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span class=u>③</span></a>
<samp>'PapayaWhip'</samp>
<samp class=p>>>> </samp><kbd>dyn.color = 'LemonChiffon'</kbd>
-<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span class=u>④</span></a>
<samp>'LemonChiffon'</samp></pre>
<ol>
<li>The attribute name is passed into the <code>__getattr()__</code> method as a string. If the name is <code>'color'</code>, the method returns a value. (In this case, it’s just a hard-coded string, but you would normally do some sort of computation and return the result.)
@@ -171,10 +171,10 @@ td a:link, td a:visited{border:0}
raise AttributeError</code>
<samp class=p>>>> </samp><kbd>dyn = SuperDynamo()</kbd>
-<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span class=u>①</span></a>
<samp>'PapayaWhip'</samp>
<samp class=p>>>> </samp><kbd>dyn.color = 'LemonChiffon'</kbd>
-<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>dyn.color</kbd> <span class=u>②</span></a>
<samp>'PapayaWhip'</samp></pre>
<ol>
<li>The <code>__getattribute__()</code> method is called to provide a value for <var>dyn.color</var>.
@@ -182,7 +182,7 @@ td a:link, td a:visited{border:0}
</ol>
<blockquote class=note>
-<p><span>☞</span>If your class defines a <code>__getattribute__()</code> method, you probably also want to define a <code>__setattr__()</code> method and coordinate between them to keep track of attribute values. Otherwise, any attributes you set after creating an instance will disappear into a black hole.
+<p><span class=u>☞</span>If your class defines a <code>__getattribute__()</code> method, you probably also want to define a <code>__setattr__()</code> method and coordinate between them to keep track of attribute values. Otherwise, any attributes you set after creating an instance will disappear into a black hole.
</blockquote>
<p>You need to be extra careful with the <code>__getattribute__()</code> method, because it is also called when Python looks up a method name on your class.
@@ -190,12 +190,12 @@ td a:link, td a:visited{border:0}
<pre class=screen>
<code>class Rastan:
def __getattribute__(self, key):
-<a> raise AttributeError <span>①</span></a>
+<a> raise AttributeError <span class=u>①</span></a>
def swim(self):
pass</code>
<samp class=p>>>> </samp><kbd>hero = Rastan()</kbd>
-<a><samp class=p>>>> </samp><kbd>hero.swim()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>hero.swim()</kbd> <span class=u>②</span></a>
<samp class=traceback>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __getattribute__
@@ -216,26 +216,25 @@ AttributeError</samp></pre>
<th>And Python Calls…
<tr><th>
<td>to “call” an instance like a function
-<td><code>my_instance()</code>
+<td><code class=pp>my_instance()</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__call__><code>my_instance.__call__()</code></a>
</table>
<p>The <a href=http://docs.python.org/3.0/library/zipfile.html><code>zipfile</code> module</a> uses this to define a class that can decrypt an encrypted zip file with a given password. The zip decryption algorithm requires you to store state during decryption. Defining the decryptor as a class allows you to maintain this state within a single instance of the decryptor class. The state is initialized in the <code>__init__()</code> method and updated as the file is decrypted. But since the class is also “callable” like a function, you can pass the instance as the first argument of the <code>map()</code> function, like so:
-<pre><code>
-# excerpt from zipfile.py
+<pre><code class=pp># excerpt from zipfile.py
class _ZipDecrypter:
.
.
.
def __init__(self, pwd):
-<a> self.key0 = 305419896 <span>①</span></a>
+<a> self.key0 = 305419896 <span class=u>①</span></a>
self.key1 = 591751049
self.key2 = 878082192
for p in pwd:
self._UpdateKeys(p)
-<a> def __call__(self, c): <span>②</span></a>
+<a> def __call__(self, c): <span class=u>②</span></a>
assert isinstance(c, int)
k = self.key2 | 2
c = c ^ (((k * (k^1)) >> 8) & 255)
@@ -244,9 +243,9 @@ class _ZipDecrypter:
.
.
.
-<a>zd = _ZipDecrypter(pwd) <span>③</span></a>
+<a>zd = _ZipDecrypter(pwd) <span class=u>③</span></a>
bytes = zef_file.read(12)
-<a>h = list(map(zd, bytes[0:12])) <span>④</span></a></code></pre>
+<a>h = list(map(zd, bytes[0:12])) <span class=u>④</span></a></code></pre>
<ol>
<li>The <code>_ZipDecryptor</code> class maintains state in the form of three rotating keys, which are later updated in the <code>_UpdateKeys()</code> method (not shown here).
<li>The class defines a <code>__call__()</code> method, which makes class instances callable like functions. In this case, the <code>__call__()</code> method decrypts a single byte of the zip file, then updates the rotating keys based on the byte that was decrypted.
@@ -265,21 +264,20 @@ bytes = zef_file.read(12)
<th>And Python Calls…
<tr><th>
<td>the length of a sequence
-<td><code>len(seq)</code>
+<td><code class=pp>len(seq)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__len__><code>seq.__len__()</code></a>
<tr><th>
<td>to know whether a sequence contains a specific value
-<td><code>x in seq</code>
+<td><code class=pp>x in seq</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__contains__><code>seq.__contains__(<var>x</var>)</code></a>
</table>
<p id=acts-like-list-example>The <a href=http://docs.python.org/3.0/library/cgi.html><code>cgi</code> module</a> uses these methods in its <code>FieldStorage</code> class, which represents all of the form fields or query parameters submitted to a dynamic web page.
-<pre><code>
-# A script which responds to http://example.com/search?q=cgi
+<pre><code class=pp># A script which responds to http://example.com/search?q=cgi
import cgi
fs = cgi.FieldStorage()
-<a>if 'q' in fs: <span>①</span></a>
+<a>if 'q' in fs: <span class=u>①</span></a>
do_search()
# An excerpt from cgi.py that explains how that works
@@ -287,12 +285,12 @@ class FieldStorage:
.
.
.
-<a> def __contains__(self, key): <span>②</span></a>
+<a> def __contains__(self, key): <span class=u>②</span></a>
if self.list is None:
raise TypeError('not indexable')
-<a> return any(item.name == key for item in self.list) <span>③</span></a>
+<a> return any(item.name == key for item in self.list) <span class=u>③</span></a>
-<a> def __len__(self): <span>④</span></a>
+<a> def __len__(self): <span class=u>④</span></a>
return len(self.keys())</code></pre>
<ol>
<li>Once you create an instance of the <code>cgi.FieldStorage</code> class, you can use the “<code>in</code>” operator to check whether a particular parameter was included in the query string.
@@ -312,37 +310,36 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>to get a value by its key
-<td><code>x[key]</code>
+<td><code class=pp>x[key]</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__getitem__><code>x.__getitem__(<var>'key'</var>)</code></a>
<tr><th>
<td>to set a value by its key
-<td><code>x[key] = value</code>
+<td><code class=pp>x[key] = value</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__setitem__><code>x.__setitem__(<var>'key'</var>, <var>value</var>)</code></a>
<tr><th>
<td>to delete a key-value pair
-<td><code>del x[key]</code>
+<td><code class=pp>del x[key]</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__delitem__><code>x.__delitem__(<var>'key'</var>)</code></a>
<tr><th>
<td>to provide a default value for missing keys
-<td><code>x[nonexistent_key]</code>
+<td><code class=pp>x[nonexistent_key]</code>
<td><a href=http://docs.python.org/3.0/library/collections.html#collections.defaultdict.__missing__><code>x.__missing__(<var>'nonexistent_key'</var>)</code></a>
</table>
<p>The <a href=#acts-like-list-example><code>FieldStorage</code> class</a> from the <a href=http://docs.python.org/3.0/library/cgi.html><code>cgi</code> module</a> also defines these special methods, which means you can do things like this:
-<pre><code>
-# A script which responds to http://example.com/search?q=cgi
+<pre><code class=pp># A script which responds to http://example.com/search?q=cgi
import cgi
fs = cgi.FieldStorage()
if 'q' in fs:
-<a> do_search(fs['q']) <span>①</span></a>
+<a> do_search(fs['q']) <span class=u>①</span></a>
# An excerpt from cgi.py that shows how it works
class FieldStorage:
.
.
.
-<a> def __getitem__(self, key): <span>②</span></a>
+<a> def __getitem__(self, key): <span class=u>②</span></a>
if self.list is None:
raise TypeError('not indexable')
found = []
@@ -378,55 +375,55 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>addition
-<td><code>x + y</code>
+<td><code class=pp>x + y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__add__><code>x.__add__(<var>y</var>)</code></a>
<tr><th>
<td>subtraction
-<td><code>x - y</code>
+<td><code class=pp>x - y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__sub__><code>x.__sub__(<var>y</var>)</code></a>
<tr><th>
<td>multiplication
-<td><code>x * y</code>
+<td><code class=pp>x * y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__mul__><code>x.__mul__(<var>y</var>)</code></a>
<tr><th>
<td>division
-<td><code>x / y</code>
+<td><code class=pp>x / y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__truediv__><code>x.__truediv__(<var>y</var>)</code></a>
<tr><th>
<td>floor division
-<td><code>x // y</code>
+<td><code class=pp>x // y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__floordiv__><code>x.__floordiv__(<var>y</var>)</code></a>
<tr><th>
<td>modulo (remainder)
-<td><code>x % y</code>
+<td><code class=pp>x % y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__mod__><code>x.__mod__(<var>y</var>)</code></a>
<tr><th>
<td>floor division <i class=baa>&</i> modulo
-<td><code>divmod(x, y)</code>
+<td><code class=pp>divmod(x, y)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__divmod__><code>x.__divmod__(<var>y</var>)</code></a>
<tr><th>
<td>raise to power
-<td><code>x ** y</code>
+<td><code class=pp>x ** y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__pow__><code>x.__pow__(<var>y</var>)</code></a>
<tr><th>
<td>left bit-shift
-<td><code>x << y</code>
+<td><code class=pp>x << y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__lshift__><code>x.__lshift__(<var>y</var>)</code></a>
<tr><th>
<td>right bit-shift
-<td><code>x >> y</code>
+<td><code class=pp>x >> y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rshift__><code>x.__rshift__(<var>y</var>)</code></a>
<tr><th>
<td>bitwise <code>and</code>
-<td><code>x & y</code>
+<td><code class=pp>x & y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__and__><code>x.__and__(<var>y</var>)</code></a>
<tr><th>
<td>bitwise <code>xor</code>
-<td><code>x ^ y</code>
+<td><code class=pp>x ^ y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__xor__><code>x.__xor__(<var>y</var>)</code></a>
<tr><th>
<td>bitwise <code>or</code>
-<td><code>x | y</code>
+<td><code class=pp>x | y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__or__><code>x.__or__(<var>y</var>)</code></a>
</table>
@@ -456,55 +453,55 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>addition
-<td><code>x + y</code>
+<td><code class=pp>x + y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__radd__><code>y.__radd__(<var>x</var>)</code></a>
<tr><th>
<td>subtraction
-<td><code>x - y</code>
+<td><code class=pp>x - y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rsub__><code>y.__rsub__(<var>x</var>)</code></a>
<tr><th>
<td>multiplication
-<td><code>x * y</code>
+<td><code class=pp>x * y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rmul__><code>y.__rmul__(<var>x</var>)</code></a>
<tr><th>
<td>division
-<td><code>x / y</code>
+<td><code class=pp>x / y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rtruediv__><code>y.__rtruediv__(<var>x</var>)</code></a>
<tr><th>
<td>floor division
-<td><code>x // y</code>
+<td><code class=pp>x // y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rfloordiv__><code>y.__rfloordiv__(<var>x</var>)</code></a>
<tr><th>
<td>modulo (remainder)
-<td><code>x % y</code>
+<td><code class=pp>x % y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rmod__><code>y.__rmod__(<var>x</var>)</code></a>
<tr><th>
<td>floor division <i class=baa>&</i> modulo
-<td><code>divmod(x, y)</code>
+<td><code class=pp>divmod(x, y)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rdivmod__><code>y.__rdivmod__(<var>x</var>)</code></a>
<tr><th>
<td>raise to power
-<td><code>x ** y</code>
+<td><code class=pp>x ** y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rpow__><code>y.__rpow__(<var>x</var>)</code></a>
<tr><th>
<td>left bit-shift
-<td><code>x << y</code>
+<td><code class=pp>x << y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rlshift__><code>y.__rlshift__(<var>x</var>)</code></a>
<tr><th>
<td>right bit-shift
-<td><code>x >> y</code>
+<td><code class=pp>x >> y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rrshift__><code>y.__rrshift__(<var>x</var>)</code></a>
<tr><th>
<td>bitwise <code>and</code>
-<td><code>x & y</code>
+<td><code class=pp>x & y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rand__><code>y.__rand__(<var>x</var>)</code></a>
<tr><th>
<td>bitwise <code>xor</code>
-<td><code>x ^ y</code>
+<td><code class=pp>x ^ y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__rxor__><code>y.__rxor__(<var>x</var>)</code></a>
<tr><th>
<td>bitwise <code>or</code>
-<td><code>x | y</code>
+<td><code class=pp>x | y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ror__><code>y.__ror__(<var>x</var>)</code></a>
</table>
@@ -517,51 +514,51 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>in-place addition
-<td><code>x += y</code>
+<td><code class=pp>x += y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__iadd__><code>x.__iadd__(<var>y</var>)</code></a>
<tr><th>
<td>in-place subtraction
-<td><code>x -= y</code>
+<td><code class=pp>x -= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__isub__><code>x.__isub__(<var>y</var>)</code></a>
<tr><th>
<td>in-place multiplication
-<td><code>x *= y</code>
+<td><code class=pp>x *= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__imul__><code>x.__imul__(<var>y</var>)</code></a>
<tr><th>
<td>in-place division
-<td><code>x /= y</code>
+<td><code class=pp>x /= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__itruediv__><code>x.__itruediv__(<var>y</var>)</code></a>
<tr><th>
<td>in-place floor division
-<td><code>x //= y</code>
+<td><code class=pp>x //= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ifloordiv__><code>x.__ifloordiv__(<var>y</var>)</code></a>
<tr><th>
<td>in-place modulo
-<td><code>x %= y</code>
+<td><code class=pp>x %= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__imod__><code>x.__imod__(<var>y</var>)</code></a>
<tr><th>
<td>in-place raise to power
-<td><code>x **= y</code>
+<td><code class=pp>x **= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ipow__><code>x.__ipow__(<var>y</var>)</code></a>
<tr><th>
<td>in-place left bit-shift
-<td><code>x <<= y</code>
+<td><code class=pp>x <<= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ilshift__><code>x.__ilshift__(<var>y</var>)</code></a>
<tr><th>
<td>in-place right bit-shift
-<td><code>x >>= y</code>
+<td><code class=pp>x >>= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__irshift__><code>x.__irshift__(<var>y</var>)</code></a>
<tr><th>
<td>in-place bitwise <code>and</code>
-<td><code>x &= y</code>
+<td><code class=pp>x &= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__iand__><code>x.__iand__(<var>y</var>)</code></a>
<tr><th>
<td>in-place bitwise <code>xor</code>
-<td><code>x ^= y</code>
+<td><code class=pp>x ^= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ixor__><code>x.__ixor__(<var>y</var>)</code></a>
<tr><th>
<td>in-place bitwise <code>or</code>
-<td><code>x |= y</code>
+<td><code class=pp>x |= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ior__><code>x.__ior__(<var>y</var>)</code></a>
</table>
@@ -584,55 +581,55 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>negative number
-<td><code>-x</code>
+<td><code class=pp>-x</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__neg__><code>x.__neg__()</code></a>
<tr><th>
<td>positive number
-<td><code>+x</code>
+<td><code class=pp>+x</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__pos__><code>x.__pos__()</code></a>
<tr><th>
<td>absolute value
-<td><code>abs(x)</code>
+<td><code class=pp>abs(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__abs__><code>x.__abs__()</code></a>
<tr><th>
<td>inverse
-<td><code>~x</code>
+<td><code class=pp>~x</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__invert__><code>x.__invert__()</code></a>
<tr><th>
<td>complex number
-<td><code>complex(x)</code>
+<td><code class=pp>complex(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__complex__><code>x.__complex__()</code></a>
<tr><th>
<td>integer
-<td><code>int(x)</code>
+<td><code class=pp>int(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__int__><code>x.__int__()</code></a>
<tr><th>
<td>floating point number
-<td><code>float(x)</code>
+<td><code class=pp>float(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__float__><code>x.__float__()</code></a>
<tr><th>
<td>number rounded to nearest integer
-<td><code>round(x)</code>
+<td><code class=pp>round(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__round__><code>x.__round__()</code></a>
<tr><th>
<td>number rounded to nearest <var>n</var> digits
-<td><code>round(x, n)</code>
+<td><code class=pp>round(x, n)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__round__><code>x.__round__(n)</code></a>
<tr><th>
<td>smallest integer <code>>= x</code>
-<td><code>math.ceil(x)</code>
+<td><code class=pp>math.ceil(x)</code>
<td><a href=http://docs.python.org/3.0/library/math.html#math.ceil><code>x.__ceil__()</code></a>
<tr><th>
<td>largest integer <code><= x</code>
-<td><code>math.floor(x)</code>
+<td><code class=pp>math.floor(x)</code>
<td><a href=http://docs.python.org/3.0/library/math.html#math.floor><code>x.__floor__()</code></a>
<tr><th>
<td>truncate <code>x</code> to nearest integer toward <code>0</code>
-<td><code>math.trunc(x)</code>
+<td><code class=pp>math.trunc(x)</code>
<td><a href=http://docs.python.org/3.0/library/math.html#math.trunc><code>x.__trunc__()</code></a>
<tr><th><a href=http://www.python.org/dev/peps/pep-0357/>PEP 357</a>
<td>number as a list index
-<td><code>a_list[<var>x</var>]</code>
+<td><code class=pp>a_list[<var>x</var>]</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__index__><code>x.__index__()</code></a>
</table>
@@ -647,31 +644,31 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>equality
-<td><code>x == y</code>
+<td><code class=pp>x == y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__eq__><code>x.__eq__(<var>y</var>)</code></a>
<tr><th>
<td>inequality
-<td><code>x != y</code>
+<td><code class=pp>x != y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ne__><code>x.__ne__(<var>y</var>)</code></a>
<tr><th>
<td>less than
-<td><code>x < y</code>
+<td><code class=pp>x < y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__lt__><code>x.__lt__(<var>y</var>)</code></a>
<tr><th>
<td>less than or equal to
-<td><code>x <= y</code>
+<td><code class=pp>x <= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__le__><code>x.__le__(<var>y</var>)</code></a>
<tr><th>
<td>greater than
-<td><code>x > y</code>
+<td><code class=pp>x > y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__gt__><code>x.__gt__(<var>y</var>)</code></a>
<tr><th>
<td>greater than or equal to
-<td><code>x >= y</code>
+<td><code class=pp>x >= y</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__ge__><code>x.__ge__(<var>y</var>)</code></a>
<tr><th>
<td>truth value in a boolean context
-<td><code>if x:</code>
+<td><code class=pp>if x:</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__bool__><code>x.__bool__()</code></a>
</table>
@@ -687,31 +684,31 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>a custom object copy
-<td><code>copy.copy(x)</code>
+<td><code class=pp>copy.copy(x)</code>
<td><a href=http://docs.python.org/3.0/library/copy.html><code>x.__copy__()</code></a>
<tr><th>
<td>a custom object deepcopy
-<td><code>copy.deepcopy(x)</code>
+<td><code class=pp>copy.deepcopy(x)</code>
<td><a href=http://docs.python.org/3.0/library/copy.html><code>x.__deepcopy__()</code></a>
<tr><th>
<td>to get an object’s state before pickling
-<td><code>pickle.dump(x, <var>file</var>)</code>
+<td><code class=pp>pickle.dump(x, <var>file</var>)</code>
<td><a href=http://docs.python.org/3.0/library/pickle.html#pickle-state><code>x.__getstate__()</code></a>
<tr><th>
<td>to serialize an object
-<td><code>pickle.dump(x, <var>file</var>)</code>
+<td><code class=pp>pickle.dump(x, <var>file</var>)</code>
<td><a href=http://docs.python.org/3.0/library/pickle.html#pickling-class-instances><code>x.__reduce__()</code></a>
<tr><th>
<td>to serialize an object (new pickling protocol)
-<td><code>pickle.dump(x, <var>file</var>, <var>protocol_version</var>)</code>
+<td><code class=pp>pickle.dump(x, <var>file</var>, <var>protocol_version</var>)</code>
<td><a href=http://docs.python.org/3.0/library/pickle.html#pickling-class-instances><code>x.__reduce_ex__(<var>protocol_version</var>)</code></a>
<tr><th>
<td>control over how an object is created during unpickling
-<td><code>x = pickle.load(<var>file</var>)</code>
+<td><code class=pp>x = pickle.load(<var>file</var>)</code>
<td><a href=http://docs.python.org/3.0/library/pickle.html#pickling-class-instances><code>x.__getnewargs__()</code></a>
<tr><th>
<td>to restore an object’s state after unpickling
-<td><code>x = pickle.load(<var>file</var>)</code>
+<td><code class=pp>x = pickle.load(<var>file</var>)</code>
<td><a href=http://docs.python.org/3.0/library/pickle.html#pickle-state><code>x.__setstate__()</code></a>
</table>
@@ -728,11 +725,11 @@ class FieldStorage:
<th>And Python Calls…
<tr><th>
<td>do something special when entering a <code>with</code> block
-<td><code>with x:</code>
+<td><code class=pp>with x:</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__enter__><code>x.__enter__()</code></a>
<tr><th>
<td>do something special when leaving a <code>with</code> block
-<td><code>with x:</code>
+<td><code class=pp>with x:</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__exit__><code>x.__exit__()</code></a>
</table>
@@ -740,7 +737,7 @@ class FieldStorage:
<p>FIXME-xref to as-yet-unwritten section on function annotations
-<pre><code># excerpt from io.py:
+<pre><code class=pp># excerpt from io.py:
def _checkClosed(self, msg=None):
'''Internal: raise an ValueError if file is closed
'''
@@ -750,12 +747,12 @@ def _checkClosed(self, msg=None):
def __enter__(self) -> 'IOBase':
'''Context management protocol. Returns self.'''
-<a> self._checkClosed() <span>①</span></a>
-<a> return self <span>②</span></a>
+<a> self._checkClosed() <span class=u>①</span></a>
+<a> return self <span class=u>②</span></a>
def __exit__(self, *args) -> None:
'''Context management protocol. Calls close()'''
-<a> self.close() <span>③</span></a></code></pre>
+<a> self.close() <span class=u>③</span></a></code></pre>
<ol>
<li>The file object defines both an <code>__enter__()</code> and an <code>__exit__()</code> method. The <code>__enter__()</code> method checks that the file is open; if it’s not, the <code>_checkClosed()</code> method raises an exception.
<li>The <code>__enter__()</code> method should almost always return <var>self</var> — this is the object that the <code>with</code> block will use to dispatch properties and methods.
@@ -763,7 +760,7 @@ def __exit__(self, *args) -> None:
</ol>
<blockquote class=note>
-<p><span>☞</span>The <code>__exit__()</code> method will always be called, even if an exception is raised inside the <code>with</code> block. In fact, if an exception is raises, the exception information will be passed to the <code>__exit__()</code> method. See <a href=http://www.python.org/doc/3.0/reference/datamodel.html#with-statement-context-managers>With Statement Context Managers</a> for more details.
+<p><span class=u>☞</span>The <code>__exit__()</code> method will always be called, even if an exception is raised inside the <code>with</code> block. In fact, if an exception is raises, the exception information will be passed to the <code>__exit__()</code> method. See <a href=http://www.python.org/doc/3.0/reference/datamodel.html#with-statement-context-managers>With Statement Context Managers</a> for more details.
</blockquote>
<h2 id=esoterica>Really Esoteric Stuff</h2>
@@ -777,11 +774,11 @@ def __exit__(self, *args) -> None:
<th>And Python Calls…
<tr><th>
<td>a class constructor
-<td><code>x = MyClass()</code>
+<td><code class=pp>x = MyClass()</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__new__><code>x.__new__()</code></a>
<tr><th>
<td>a class destructor
-<td><code>del x</code>
+<td><code class=pp>del x</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__del__><code>x.__del__()</code></a>
<tr><th>
<td>only a specific set of attributes to be defined
@@ -789,35 +786,36 @@ def __exit__(self, *args) -> None:
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__slots__><code>x.__slots__()</code></a>
<tr><th>
<td>a custom hash value
-<td><code>hash(x)</code>
+<td><code class=pp>hash(x)</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__hash__><code>x.__hash__()</code></a>
<tr><th>
<td>to get an attribute’s value
-<td><code>x.color</code>
+<td><code class=pp>x.color</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__get__><code>type(x).__dict__['color'].__get__(x, type(x))</code></a>
<tr><th>
<td>to set an attribute’s value
-<td><code>x.color = 'PapayaWhip'</code>
+<td><code class=pp>x.color = 'PapayaWhip'</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__set__><code>type(x).__dict__['color'].__set__(x, 'PapayaWhip')</code></a>
<tr><th>
<td>to delete an attribute
-<td><code>del x.color</code>
+<td><code class=pp>del x.color</code>
<td><a href=http://www.python.org/doc/3.0/reference/datamodel.html#object.__delete__><code>type(x).__dict__['color'].__del__(x)</code></a>
<tr><th>
<td>to control whether an object is an instance of your class
-<td><code>isinstance(x, MyClass)</code>
+<td><code class=pp>isinstance(x, MyClass)</code>
<td><a href=http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass><code>MyClass.__instancecheck__(x)</code></a>
<tr><th>
<td>to control whether a class is a subclass of your class
-<td><code>issubclass(C, MyClass)</code>
+<td><code class=pp>issubclass(C, MyClass)</code>
<td><a href=http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass><code>MyClass.__subclasscheck__(C)</code></a>
<tr><th>
<td>to control whether a class is a subclass of your abstract base class
-<td><code>issubclass(C, MyABC)</code>
+<td><code class=pp>issubclass(C, MyABC)</code>
<td><a href=http://docs.python.org/3.0/library/abc.html#abc.ABCMeta.__subclasshook__><code>MyABC.__subclasshook__(C)</code></a>
</table>
-<p class=v><a href=porting-code-to-python-3-with-2to3.html rel=prev title='back to “Porting code to Python 3 with 2to3”'><span>☜</span></a> <a rel=next class=todo><span>☞</span></a>
+<p class=v><a href=porting-code-to-python-3-with-2to3.html rel=prev title='back to “Porting code to Python 3 with 2to3”'><span class=u>☜</span></a> <a rel=next class=todo><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/strings.html b/strings.html
index 7e0c347..ba172cc 100644
--- a/strings.html
+++ b/strings.html
@@ -12,12 +12,12 @@ body{counter-reset:h1 3}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#strings>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#strings>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=intermediate>♦♦♦♢♢</span>
<h1>Strings</h1>
<blockquote class=q>
-<p><span>❝</span> I’m telling you this ’cause you’re one of my friends.<br>
-My alphabet starts where your alphabet ends! <span>❞</span><br>— Dr. Seuss, On Beyond Zebra!
+<p><span class=u>❝</span> I’m telling you this ’cause you’re one of my friends.<br>
+My alphabet starts where your alphabet ends! <span class=u>❞</span><br>— Dr. Seuss, On Beyond Zebra!
</blockquote>
<p id=toc>
<h2 id=boring-stuff>Some Boring Stuff You Need To Understand Before You Can Dive In</h2>
@@ -84,12 +84,12 @@ My alphabet starts where your alphabet ends! <span>❞</span><br>— Dr
<p>In Python 3, all strings are sequences of Unicode characters. There is no such thing as a Python string encoded in UTF-8, or a Python string encoded as CP-1252. “Is this string UTF-8?” is an invalid question. UTF-8 is a way of encoding characters as a sequence of bytes. If you want to take a string and turn it into a sequence of bytes in a particular character encoding, Python 3 can help you with that. If you want to take a sequence of bytes and turn it into a string, Python 3 can help you with that too. Bytes are not characters; bytes are bytes. Characters are an abstraction. A string is a sequence of those abstractions.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>s = '深入 Python'</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>len(s)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>s = '深入 Python'</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>len(s)</kbd> <span class=u>②</span></a>
<samp>9</samp>
-<a><samp class=p>>>> </samp><kbd>s[0]</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>s[0]</kbd> <span class=u>③</span></a>
<samp>'æ·±'</samp>
-<a><samp class=p>>>> </samp><kbd>s + ' 3'</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>s + ' 3'</kbd> <span class=u>④</span></a>
<samp>'深入 Python 3'</samp></pre>
<ol>
<li>To create a string, enclose it in quotes. Python strings can be defined with either single quotes (<code>'</code>) or double quotes (<code>"</code>).<!--"-->
@@ -106,12 +106,11 @@ My alphabet starts where your alphabet ends! <span>❞</span><br>— Dr
<p>Let’s take another look at <a href=your-first-python-program.html#divingin><code>humansize.py</code></a>:
<p class=d>[<a href=examples/humansize.py>download <code>humansize.py</code></a>]
-<pre><code>
-<a>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'], <span>①</span></a>
+<pre><code class=pp><a>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'], <span class=u>①</span></a>
1024: ['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']}
def approximate_size(size, a_kilobyte_is_1024_bytes=True):
-<a> '''Convert a file size to human-readable form. <span>②</span></a>
+<a> '''Convert a file size to human-readable form. <span class=u>②</span></a>
Keyword arguments:
size -- file size in bytes
@@ -120,15 +119,15 @@ def approximate_size(size, a_kilobyte_is_1024_bytes=True):
Returns: string
-<a> ''' <span>③</span></a>
+<a> ''' <span class=u>③</span></a>
if size < 0:
-<a> raise ValueError('number must be non-negative') <span>④</span></a>
+<a> raise ValueError('number must be non-negative') <span class=u>④</span></a>
multiple = 1024 if a_kilobyte_is_1024_bytes else 1000
for suffix in SUFFIXES[multiple]:
size /= multiple
if size < multiple:
-<a> return '{0:.1f} {1}'.format(size, suffix) <span>⑤</span></a>
+<a> return '{0:.1f} {1}'.format(size, suffix) <span class=u>⑤</span></a>
raise ValueError('number too large')</code></pre>
<ol>
@@ -143,8 +142,8 @@ def approximate_size(size, a_kilobyte_is_1024_bytes=True):
<pre class=screen>
<samp class=p>>>> </samp><kbd>username = 'mark'</kbd>
-<a><samp class=p>>>> </samp><kbd>password = 'PapayaWhip'</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>"{0}'s password is {1}".format(username, password)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>password = 'PapayaWhip'</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>"{0}'s password is {1}".format(username, password)</kbd> <span class=u>②</span></a>
<samp>"mark's password is PapayaWhip"</samp></pre>
<ol>
<li>No, my password is not really <kbd>PapayaWhip</kbd>.
@@ -157,10 +156,10 @@ def approximate_size(size, a_kilobyte_is_1024_bytes=True):
<pre class=screen>
<samp class=p>>>> </samp><kbd>import humansize</kbd>
-<a><samp class=p>>>> </samp><kbd>si_suffixes = humansize.SUFFIXES[1000]</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>si_suffixes = humansize.SUFFIXES[1000]</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>si_suffixes</kbd>
<samp>['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB']</samp>
-<a><samp class=p>>>> </samp><kbd>'1000{0[0]} = 1{0[1]}'.format(si_suffixes)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>'1000{0[0]} = 1{0[1]}'.format(si_suffixes)</kbd> <span class=u>②</span></a>
<samp>'1000KB = 1MB'</samp>
</pre>
<ol>
@@ -202,13 +201,13 @@ def approximate_size(size, a_kilobyte_is_1024_bytes=True):
<p>But wait! There’s more! Let’s take another look at that strange line of code from <code>humansize.py</code>:
-<pre><code>if size < multiple:
+<pre><code class=pp>if size < multiple:
return '{0:.1f} {1}'.format(size, suffix)</code></pre>
<p><code>{1}</code> is replaced with the second argument passed to the <code>format()</code> method, which is <var>suffix</var>. But what is <code>{0:.1f}</code>? It’s two things: <code>{0}</code>, which you recognize, and <code>:.1f</code>, which you don’t. The second half (including and after the colon) defines the <i>format specifier</i>, which further refines how the replaced variable should be formatted.
<blockquote class='note compare clang'>
-<p><span>☞</span>Format specifiers allow you to munge the replacement text in a variety of useful ways, like the <code>printf()</code> function in C. You can add zero- or space-padding, align strings, control decimal precision, and even convert numbers to hexadecimal.
+<p><span class=u>☞</span>Format specifiers allow you to munge the replacement text in a variety of useful ways, like the <code>printf()</code> function in C. You can add zero- or space-padding, align strings, control decimal precision, and even convert numbers to hexadecimal.
</blockquote>
<p>Within a replacement field, a colon (<code>:</code>) marks the start of the format specifier. The format specifier “<code>.1</code>” means “round to the nearest tenth” (<i>i.e.</i> display only one digit after the decimal point). The format specifier “<code>f</code>” means “fixed-point number” (as opposed to exponential notation or some other decimal representation). Thus, given a <var>size</var> of <code>698.25</code> and <var>suffix</var> of <code>'GB'</code>, the formatted string would be <code>'698.3 GB'</code>, because <code>698.25</code> gets rounded to one decimal place, then the suffix is appended after the number.
@@ -226,21 +225,21 @@ def approximate_size(size, a_kilobyte_is_1024_bytes=True):
<p>Besides formatting, strings can do a number of other useful tricks.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>s = '''Finished files are the re-</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>s = '''Finished files are the re-</kbd> <span class=u>①</span></a>
<samp class=p>... </samp><kbd>sult of years of scientif-</kbd>
<samp class=p>... </samp><kbd>ic study combined with the</kbd>
<samp class=p>... </samp><kbd>experience of years.'''</kbd>
-<a><samp class=p>>>> </samp><kbd>s.splitlines()</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>s.splitlines()</kbd> <span class=u>②</span></a>
<samp>['Finished files are the re-',
'sult of years of scientif-',
'ic study combined with the',
'experience of years.']</samp>
-<a><samp class=p>>>> </samp><kbd>print(s.lower())</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>print(s.lower())</kbd> <span class=u>③</span></a>
<samp>finished files are the re-
sult of years of scientif-
ic study combined with the
experience of years.</samp>
-<a><samp class=p>>>> </samp><kbd>s.lower().count('f')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>s.lower().count('f')</kbd> <span class=u>④</span></a>
<samp>6</samp></pre>
<ol>
<li>You can input multi-line strings in the Python interactive shell. Once you start a multi-line string with triple quotation marks, just hit <kbd>ENTER</kbd> and the interactive shell will prompt you to continue the string. Typing the closing triple quotation marks ends the string, and the next <kbd>ENTER</kbd> will execute the command (in this case, assigning the string to <var>s</var>).
@@ -253,13 +252,13 @@ experience of years.</samp>
<pre class=screen>
<samp class=p>>>> </samp><kbd>query = 'user=pilgrim&database=master&password=PapayaWhip'</kbd>
-<a><samp class=p>>>> </samp><kbd>a_list = query.split('&')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list = query.split('&')</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>a_list</kbd>
<samp>['user=pilgrim', 'database=master', 'password=PapayaWhip']</samp>
-<a><samp class=p>>>> </samp><kbd>a_list_of_lists = [v.split('=', 1) for v in a_list]</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>a_list_of_lists = [v.split('=', 1) for v in a_list]</kbd> <span class=u>②</span></a>
<samp class=p>>>> </samp><kbd>a_list_of_lists</kbd>
<samp>[['user', 'pilgrim'], ['database', 'master'], ['password', 'PapayaWhip']]</samp>
-<a><samp class=p>>>> </samp><kbd>a_dict = dict(a_list_of_lists)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>a_dict = dict(a_list_of_lists)</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp><kbd>a_dict</kbd>
<samp>{'password': 'PapayaWhip', 'user': 'pilgrim', 'database': 'master'}</samp></pre>
@@ -276,21 +275,21 @@ experience of years.</samp>
<p>Bytes are bytes; characters are an abstraction. An immutable sequence of Unicode characters is called a <i>string</i>. An immutable sequence of numbers-between-0-and-255 is called a <i>bytes</i> object.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>by = b'abcd\x65'</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>by = b'abcd\x65'</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>by</kbd>
<samp>b'abcde'</samp>
-<a><samp class=p>>>> </samp><kbd>type(by)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>type(by)</kbd> <span class=u>②</span></a>
<samp><class 'bytes'></samp>
-<a><samp class=p>>>> </samp><kbd>len(by)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>len(by)</kbd> <span class=u>③</span></a>
<samp>5</samp>
-<a><samp class=p>>>> </samp><kbd>by += b'\xff'</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>by += b'\xff'</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp><kbd>by</kbd>
<samp>b'abcde\xff'</samp>
-<a><samp class=p>>>> </samp><kbd>len(by)</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>len(by)</kbd> <span class=u>⑤</span></a>
<samp>6</samp>
-<a><samp class=p>>>> </samp><kbd>by[0]</kbd> <span>⑥</span></a>
+<a><samp class=p>>>> </samp><kbd>by[0]</kbd> <span class=u>⑥</span></a>
<samp>97</samp>
-<a><samp class=p>>>> </samp><kbd>by[0] = 102</kbd> <span>⑦</span></a>
+<a><samp class=p>>>> </samp><kbd>by[0] = 102</kbd> <span class=u>⑦</span></a>
<samp class=traceback>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'bytes' object does not support item assignment</samp></pre>
@@ -306,12 +305,12 @@ TypeError: 'bytes' object does not support item assignment</samp></pre>
<pre class=screen>
<samp class=p>>>> </samp><kbd>by = b'abcd\x65'</kbd>
-<a><samp class=p>>>> </samp><kbd>barr = bytearray(by)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>barr = bytearray(by)</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>barr</kbd>
<samp>bytearray(b'abcde')</samp>
-<a><samp class=p>>>> </samp><kbd>len(barr)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>len(barr)</kbd> <span class=u>②</span></a>
<samp>5</samp>
-<a><samp class=p>>>> </samp><kbd>barr[0] = 102</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>barr[0] = 102</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp><kbd>barr</kbd>
<samp>bytearray(b'fbcde')</samp></pre>
<ol>
@@ -325,15 +324,15 @@ TypeError: 'bytes' object does not support item assignment</samp></pre>
<pre class=screen>
<samp class=p>>>> </samp><kbd>by = b'd'</kbd>
<samp class=p>>>> </samp><kbd>s = 'abcde'</kbd>
-<a><samp class=p>>>> </samp><kbd>by + s</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>by + s</kbd> <span class=u>①</span></a>
<samp class=traceback>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't concat bytes to str</samp>
-<a><samp class=p>>>> </samp><kbd>s.count(by)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>s.count(by)</kbd> <span class=u>②</span></a>
<samp class=traceback>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't convert 'bytes' object to str implicitly</samp>
-<a><samp class=p>>>> </samp><kbd>s.count(by.decode('ascii'))</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>s.count(by.decode('ascii'))</kbd> <span class=u>③</span></a>
<samp>1</samp></pre>
<ol>
<li>You can’t concatenate bytes and strings. They are two different data types.
@@ -344,25 +343,25 @@ TypeError: Can't convert 'bytes' object to str implicitly</samp>
<p>And here is the link between strings and bytes: <code>bytes</code> objects have a <code>decode()</code> method that takes a character encoding and returns a string, and strings have an <code>encode()</code> method that takes a character encoding and returns a <code>bytes</code> object. In the previous example, the decoding was relatively straightforward — converting a sequence of bytes n the <abbr>ASCII</abbr> encoding into a string of characters. But the same process works with any encoding that supports the characters of the string — even legacy (non-Unicode) encodings.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>a_string = '深入 Python'</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>a_string = '深入 Python'</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>len(a_string)</kbd>
<samp>9</samp>
-<a><samp class=p>>>> </samp><kbd>by = a_string.encode('utf-8')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>by = a_string.encode('utf-8')</kbd> <span class=u>②</span></a>
<samp class=p>>>> </samp><kbd>by</kbd>
<samp>b'\xe6\xb7\xb1\xe5\x85\xa5 Python'</samp>
<samp class=p>>>> </samp><kbd>len(by)</kbd>
<samp>13</samp>
-<a><samp class=p>>>> </samp><kbd>by = a_string.encode('gb18030')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>by = a_string.encode('gb18030')</kbd> <span class=u>③</span></a>
<samp class=p>>>> </samp><kbd>by</kbd>
<samp>b'\xc9\xee\xc8\xeb Python'</samp>
<samp class=p>>>> </samp><kbd>len(by)</kbd>
<samp>11</samp>
-<a><samp class=p>>>> </samp><kbd>by = a_string.encode('big5')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>by = a_string.encode('big5')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp><kbd>by</kbd>
<samp>b'\xb2`\xa4J Python'</samp>
<samp class=p>>>> </samp><kbd>len(by)</kbd>
<samp>11</samp>
-<a><samp class=p>>>> </samp><kbd>roundtrip = by.decode('big5')</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>roundtrip = by.decode('big5')</kbd> <span class=u>⑤</span></a>
<samp class=p>>>> </samp><kbd>roundtrip</kbd>
<samp>'深入 Python'</samp>
<samp class=p>>>> </samp><kbd>a_string == roundtrip</kbd>
@@ -382,16 +381,16 @@ TypeError: Can't convert 'bytes' object to str implicitly</samp>
<p>Python 3 assumes that your source code — <i>i.e.</i> each <code>.py</code> file — is encoded in UTF-8.
<blockquote class='note compare python2'>
-<p><span>☞</span>In Python 2, the default encoding for <code>.py</code> files was <abbr>ASCII</abbr>. In Python 3, <a href=http://www.python.org/dev/peps/pep-3120/>the default encoding is UTF-8</a>.
+<p><span class=u>☞</span>In Python 2, the default encoding for <code>.py</code> files was <abbr>ASCII</abbr>. In Python 3, <a href=http://www.python.org/dev/peps/pep-3120/>the default encoding is UTF-8</a>.
</blockquote>
<p>If you would like to use a different encoding within your Python code, you can put an encoding declaration on the first line of each file. This declaration defines a <code>.py</code> file to be windows-1252:
-<pre><code># -*- coding: windows-1252 -*-</code></pre>
+<pre><code class=pp># -*- coding: windows-1252 -*-</code></pre>
<p>Technically, the character encoding override can also be on the second line, if the first line is a <abbr>UNIX</abbr>-like hash-bang command.
-<pre><code>#!/usr/bin/python3
+<pre><code class=pp>#!/usr/bin/python3
# -*- coding: windows-1252 -*-</code></pre>
<p>For more information, consult <a href=http://www.python.org/dev/peps/pep-0263/><abbr>PEP</abbr> 263: Defining Python Source Code Encodings</a>.
@@ -432,8 +431,9 @@ TypeError: Can't convert 'bytes' object to str implicitly</samp>
<li><a href=http://www.python.org/dev/peps/pep-3101/><abbr>PEP</abbr> 3101: Advanced String Formatting</a>
</ul>
-<p class=v><a href=native-datatypes.html rel=prev title='back to “Native Datatypes”'><span>☜</span></a> <a href=regular-expressions.html rel=next title='onward to “Regular Expressions”'><span>☞</span></a>
+<p class=v><a href=native-datatypes.html rel=prev title='back to “Native Datatypes”'><span class=u>☜</span></a> <a href=regular-expressions.html rel=next title='onward to “Regular Expressions”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/table-of-contents.html b/table-of-contents.html
index bdc315d..41fb7ad 100644
--- a/table-of-contents.html
+++ b/table-of-contents.html
@@ -15,7 +15,7 @@ ul li ol{margin:0;padding:0 0 0 2.5em}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8><input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> Dive Into Python 3 <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> Dive Into Python 3 <span class=u>‣</span>
<h1>Table of Contents</h1>
<ol start=-1>
<li id=whats-new><a href=whats-new.html>What’s New In “Dive Into Python 3”</a>
diff --git a/unit-testing.html b/unit-testing.html
index 67c4f12..fdf94a5 100644
--- a/unit-testing.html
+++ b/unit-testing.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 8}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#unit-testing>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#unit-testing>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=beginner>♦♦♢♢♢</span>
<h1>Unit Testing</h1>
<blockquote class=q>
-<p><span>❝</span> Certitude is not the test of certainty. We have been cocksure of many things that were not so. <span>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Oliver_Wendell_Holmes,_Jr.>Oliver Wendell Holmes, Jr.</a>
+<p><span class=u>❝</span> Certitude is not the test of certainty. We have been cocksure of many things that were not so. <span class=u>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Oliver_Wendell_Holmes,_Jr.>Oliver Wendell Holmes, Jr.</a>
</blockquote>
<p id=toc>
<h2 id=divingin>(Not) Diving In</h2>
@@ -57,10 +57,10 @@ body{counter-reset:h1 8}
</ol>
<p>It is not immediately obvious how this code does… well, <em>anything</em>. It defines a class which has no <code>__init__()</code> method. The class <em>does</em> have another method, but it is never called. The entire script has a <code>__main__</code> block, but it doesn’t reference the class or its method. But it does do something, I promise.
<p class=d>[<a href=examples/romantest1.py>download <code>romantest1.py</code></a>]
-<pre><code>import roman1
+<pre><code class=pp>import roman1
import unittest
-<a>class KnownValues(unittest.TestCase): <span>①</span></a>
+<a>class KnownValues(unittest.TestCase): <span class=u>①</span></a>
known_values = ( (1, 'I'),
(2, 'II'),
(3, 'III'),
@@ -116,13 +116,13 @@ import unittest
(3844, 'MMMDCCCXLIV'),
(3888, 'MMMDCCCLXXXVIII'),
(3940, 'MMMCMXL'),
-<a> (3999, 'MMMCMXCIX')) <span>②</span></a>
+<a> (3999, 'MMMCMXCIX')) <span class=u>②</span></a>
-<a> def test_to_roman_known_values(self): <span>③</span></a>
+<a> def test_to_roman_known_values(self): <span class=u>③</span></a>
'''to_roman should give known result with known input'''
for integer, numeral in self.known_values:
-<a> result = roman1.to_roman(integer) <span>④</span></a>
-<a> self.assertEqual(numeral, result) <span>⑤</span></a>
+<a> result = roman1.to_roman(integer) <span class=u>④</span></a>
+<a> self.assertEqual(numeral, result) <span class=u>⑤</span></a>
if __name__ == '__main__':
unittest.main()</code></pre>
@@ -135,18 +135,18 @@ if __name__ == '__main__':
</ol>
<aside>Write a test that fails, then code until it passes.</aside>
<p>Once you have a test case, you can start coding the <code>to_roman()</code> function. First, you should stub it out as an empty function and make sure the tests fail. If the tests succeed before you’ve written any code, you’re doing it wrong — your tests aren’t testing your code at all! Write a test that fails, then code until it passes.
-<pre><code># roman1.py
+<pre><code class=pp># roman1.py
function to_roman(n):
'''convert integer to Roman numeral'''
-<a> pass <span>①</span></a></code></pre>
+<a> pass <span class=u>①</span></a></code></pre>
<ol>
<li>At this stage, you want to define the <abbr>API</abbr> of the <code>to_roman()</code> function, but you don’t want to code it yet. (Your test needs to fail first.) To stub it out, use the Python reserved word <code>pass</code> [FIXME ref], which does precisely nothing.
</ol>
<p>Execute <code>romantest1.py</code> on the command line to run the test. If you call it with the <code>-v</code> command-line option, it will give more verbose output so you can see exactly what’s going on as each test case runs. With any luck, your output should look like this:
<pre class=screen>
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest1.py -v</kbd>
-<samp><a>to_roman should give known result with known input ... FAIL <span>①</span></a>
+<samp><a>to_roman should give known result with known input ... FAIL <span class=u>①</span></a>
======================================================================
FAIL: to_roman should give known result with known input
@@ -154,12 +154,12 @@ FAIL: to_roman should give known result with known input
Traceback (most recent call last):
File "romantest1.py", line 73, in test_to_roman_known_values
self.assertEqual(numeral, result)
-<a>AssertionError: 'I' != None <span>②</span></a>
+<a>AssertionError: 'I' != None <span class=u>②</span></a>
----------------------------------------------------------------------
-<a>Ran 1 test in 0.016s <span>③</span></a>
+<a>Ran 1 test in 0.016s <span class=u>③</span></a>
-<a>FAILED (failures=1) <span>④</span></a></samp></pre>
+<a>FAILED (failures=1) <span class=u>④</span></a></samp></pre>
<ol>
<li>Running the script runs <code>unittest.main()</code>, which runs each test case. Each test case is a method within each class in <code>romantest.py</code> that inherits from <code>unittest.TestCase</code>. For each test case, the <code>unittest</code> module will print out the <code>docstring</code> of the method and whether that test passed or failed. As expected, this test case fails.
<li>For each failed test case, <code>unittest</code> displays the trace information showing exactly what happened. In this case, the call to <code>assertEqual()</code> raised an <code>AssertionError</code> because it was expecting <code>to_roman(1)</code> to return <code>'I'</code>, but it didn’t. (Since there was no explicit return statement, the function returned <code>None</code>, the Python null value.)
@@ -168,7 +168,7 @@ Traceback (most recent call last):
</ol>
<p><em>Now</em>, finally, you can write the <code>to_roman()</code> function.
<p class=d>[<a href=examples/roman1.py>download <code>roman1.py</code></a>]
-<pre><code>roman_numeral_map = (('M', 1000),
+<pre><code class=pp>roman_numeral_map = (('M', 1000),
('CM', 900),
('D', 500),
('CD', 400),
@@ -180,13 +180,13 @@ Traceback (most recent call last):
('IX', 9),
('V', 5),
('IV', 4),
-<a> ('I', 1)) <span>①</span></a>
+<a> ('I', 1)) <span class=u>①</span></a>
def to_roman(n):
'''convert integer to Roman numeral'''
result = ''
for numeral, integer in roman_numeral_map:
-<a> while n >= integer: <span>②</span></a>
+<a> while n >= integer: <span class=u>②</span></a>
result += numeral
n -= integer
return result</code></pre>
@@ -195,7 +195,7 @@ def to_roman(n):
<li>Here’s where the rich data structure of <var>roman_numeral_map</var> pays off, because you don’t need any special logic to handle the subtraction rule. To convert to Roman numerals, simply iterate through <var>roman_numeral_map</var> looking for the largest integer value less than or equal to the input. Once found, add the Roman numeral representation to the end of the output, subtract the corresponding integer value from the input, lather, rinse, repeat.
</ol>
<p>If you’re still not clear how the <code>to_roman()</code> function works, add a <code>print()</code> call to the end of the <code>while</code> loop:
-<pre><code>
+<pre><code class=pp>
while n >= integer:
result += numeral
n -= integer
@@ -234,7 +234,7 @@ OK</samp></pre>
<samp>'MMMM'</samp>
<samp class=p>>>> </samp><kbd>roman1.to_roman(5000)</kbd>
<samp>'MMMMM'</samp>
-<a><samp class=p>>>> </samp><kbd>roman1.to_roman(9000)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>roman1.to_roman(9000)</kbd> <span class=u>①</span></a>
<samp>'MMMMMMMMM'</samp></pre>
<ol>
<li>That’s definitely not what you wanted — that’s not even a valid Roman numeral! In fact, each of these numbers is outside the range of acceptable input, but the function returns a bogus value anyway. Silently returning bad values is <em>baaaaaaad</em>; if a program is going to fail, it is far better that it fail quickly and noisily. “Halt and catch fire,” as the saying goes. The Pythonic way to halt and catch fire is to raise an exception.
@@ -245,11 +245,10 @@ OK</samp></pre>
</blockquote>
<p>What would that test look like?
<p class=d>[<a href=examples/romantest2.py>download <code>romantest2.py</code></a>]
-<pre><code>
-<a>class ToRomanBadInput(unittest.TestCase): <span>①</span></a>
-<a> def test_too_large(self): <span>②</span></a>
+<pre><code class=pp><a>class ToRomanBadInput(unittest.TestCase): <span class=u>①</span></a>
+<a> def test_too_large(self): <span class=u>②</span></a>
'''to_roman should fail with large input'''
-<a> self.assertRaises(roman2.OutOfRangeError, roman2.to_roman, 4000) <span>③</span></a></code></pre>
+<a> self.assertRaises(roman2.OutOfRangeError, roman2.to_roman, 4000) <span class=u>③</span></a></code></pre>
<ol>
<li>Like the previous test case, you create a class that inherits from <code>unittest.TestCase</code>. You can have more than one test per class (as you’ll see later in this chapter), but I chose to create a new class here because this test is something different than the last one. We’ll keep all the good input tests together in one class, and all the bad input tests together in another.
<li>Like the previous test case, the test itself is a method of the class, with a name starting with <code>test</code>.
@@ -261,7 +260,7 @@ OK</samp></pre>
<pre class=screen>
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest2.py -v</kbd>
<samp>to_roman should give known result with known input ... ok
-<a>to_roman should fail with large input ... ERROR <span>①</span></a>
+<a>to_roman should fail with large input ... ERROR <span class=u>①</span></a>
======================================================================
ERROR: to_roman should fail with large input
@@ -269,7 +268,7 @@ ERROR: to_roman should fail with large input
Traceback (most recent call last):
File "romantest2.py", line 78, in test_too_large
self.assertRaises(roman2.OutOfRangeError, roman2.to_roman, 4000)
-<a>AttributeError: 'module' object has no attribute 'OutOfRangeError' <span>②</span></a>
+<a>AttributeError: 'module' object has no attribute 'OutOfRangeError' <span class=u>②</span></a>
----------------------------------------------------------------------
Ran 2 tests in 0.000s
@@ -280,8 +279,8 @@ FAILED (errors=1)</samp></pre>
<li>Why didn’t the code execute properly? The traceback gives the answer: the module you’re testing doesn’t have an exception called <code>OutOfRangeError</code>. Remember, you passed this exception to the <code>assertRaises()</code> method, because it’s the exception you want the function to raise given an out-of-range input. But the exception doesn’t exist, so the call to the <code>assertRaises()</code> method failed. It never got a chance to test the <code>to_roman()</code> function; it didn’t get that far.
</ol>
<p>To solve this problem, you need to define the <code>OutOfRangeError</code> exception in <code>roman2.py</code>.
-<pre><code><a>class OutOfRangeError(ValueError): <span>①</span></a>
-<a> pass <span>②</span></a></code></pre>
+<pre><code class=pp><a>class OutOfRangeError(ValueError): <span class=u>①</span></a>
+<a> pass <span class=u>②</span></a></code></pre>
<ol>
<li>Exceptions are classes. An “out of range” error is a kind of value error — the argument value is out of its acceptable range. So this exception inherits from the built-in <code>ValueError</code> exception. This is not strictly necessary (it could just inherit from the base <code>Exception</code> class), but it feels right.
<li>Exceptions don’t actually do anything, but you need at least one line of code to make a class. Calling <code>pass</code> does precisely nothing, but it’s a line of Python code, so that makes it a class.
@@ -290,7 +289,7 @@ FAILED (errors=1)</samp></pre>
<pre class=screen>
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest2.py -v</kbd>
<samp>to_roman should give known result with known input ... ok
-<a>to_roman should fail with large input ... FAIL <span>①</span></a>
+<a>to_roman should fail with large input ... FAIL <span class=u>①</span></a>
======================================================================
FAIL: to_roman should fail with large input
@@ -298,7 +297,7 @@ FAIL: to_roman should fail with large input
Traceback (most recent call last):
File "romantest2.py", line 78, in test_too_large
self.assertRaises(roman2.OutOfRangeError, roman2.to_roman, 4000)
-<a>AssertionError: OutOfRangeError not raised by to_roman <span>②</span></a>
+<a>AssertionError: OutOfRangeError not raised by to_roman <span class=u>②</span></a>
----------------------------------------------------------------------
Ran 2 tests in 0.016s
@@ -310,10 +309,10 @@ FAILED (failures=1)</samp></pre>
</ol>
<p>Now you can write the code to make this test pass.
<p class=d>[<a href=examples/roman2.py>download <code>roman2.py</code></a>]
-<pre><code>def to_roman(n):
+<pre><code class=pp>def to_roman(n):
'''convert integer to Roman numeral'''
if n > 3999:
-<a> raise OutOfRangeError('number out of range (must be less than 3999)') <span>①</span></a>
+<a> raise OutOfRangeError('number out of range (must be less than 3999)') <span class=u>①</span></a>
result = ''
for numeral, integer in roman_numeral_map:
@@ -328,7 +327,7 @@ FAILED (failures=1)</samp></pre>
<pre class=screen>
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest2.py -v</kbd>
<samp>to_roman should give known result with known input ... ok
-<a>to_roman should fail with large input ... ok <span>①</span></a>
+<a>to_roman should fail with large input ... ok <span class=u>①</span></a>
----------------------------------------------------------------------
Ran 2 tests in 0.000s
@@ -354,19 +353,18 @@ OK</samp></pre>
<p>Well <em>that’s</em> not good. Let’s add tests for each of these conditions.
<p class=d>[<a href=examples/romantest3.py>download <code>romantest3.py</code></a>]
-<pre><code>
-class ToRomanBadInput(unittest.TestCase):
+<pre><code class=pp>class ToRomanBadInput(unittest.TestCase):
def test_too_large(self):
'''to_roman should fail with large input'''
-<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, 4000) <span>①</span></a>
+<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, 4000) <span class=u>①</span></a>
def test_zero(self):
'''to_roman should fail with 0 input'''
-<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, 0) <span>②</span></a>
+<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, 0) <span class=u>②</span></a>
def test_negative(self):
'''to_roman should fail with negative input'''
-<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, -1) <span>③</span></a></code></pre>
+<a> self.assertRaises(roman3.OutOfRangeError, roman3.to_roman, -1) <span class=u>③</span></a></code></pre>
<ol>
<li>The <code>test_too_large()</code> method has not changed since the previous step. I’m including it here to show where the new code fits.
<li>Here’s a new test: the <code>test_zero()</code> method. Like the <code>test_too_large()</code> method, it tells the <code>assertRaises()</code> method defined in <code>unittest.TestCase</code> to call our <code>to_roman()</code> function with a parameter of <code>0</code>, and check that it raises the appropriate exception, <code>OutOfRangeError</code>.
@@ -406,10 +404,10 @@ FAILED (failures=2)</samp></pre>
<p>Excellent. Both tests failed, as expected. Now let’s switch over to the code and see what we can do to make them pass.
<p class=d>[<a href=examples/roman3.py>download <code>roman3.py</code></a>]
-<pre><code>def to_roman(n):
+<pre><code class=pp>def to_roman(n):
'''convert integer to Roman numeral'''
-<a> if not (0 < n < 4000): <span>①</span></a>
-<a> raise OutOfRangeError('number out of range (must be 0..3999)') <span>②</span></a>
+<a> if not (0 < n < 4000): <span class=u>①</span></a>
+<a> raise OutOfRangeError('number out of range (must be 0..3999)') <span class=u>②</span></a>
result = ''
for numeral, integer in roman_numeral_map:
@@ -444,9 +442,9 @@ OK</samp></pre>
<pre class=screen>
<samp class=p>>>> </samp><kbd>import roman3</kbd>
-<a><samp class=p>>>> </samp><kbd>roman3.to_roman(0.5)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>roman3.to_roman(0.5)</kbd> <span class=u>①</span></a>
<samp>''</samp>
-<a><samp class=p>>>> </samp><kbd>roman3.to_roman(1.5)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>roman3.to_roman(1.5)</kbd> <span class=u>②</span></a>
<samp>'I'</samp></pre>
<ol>
<li>Oh, that’s bad.
@@ -455,13 +453,13 @@ OK</samp></pre>
<p>Testing for non-integers is not difficult. First, define a <code>NonIntegerError</code> exception.
-<pre><code># roman4.py
+<pre><code class=pp># roman4.py
class OutOfRangeError(ValueError): pass
<mark>class NotIntegerError(ValueError): pass</mark></code></pre>
<p>Next, write a test case that checks for the <code>NonIntegerError</code> exception.
-<pre><code>class ToRomanBadInput(unittest.TestCase):
+<pre><code class=pp>class ToRomanBadInput(unittest.TestCase):
.
.
.
@@ -494,12 +492,12 @@ FAILED (failures=1)</samp></pre>
<p>Write the code that makes the test pass.
-<pre><code>def to_roman(n):
+<pre><code class=pp>def to_roman(n):
'''convert integer to Roman numeral'''
if not (0 < n < 4000):
raise OutOfRangeError('number out of range (must be 0..3999)')
-<a> if not isinstance(n, int): <span>①</span></a>
-<a> raise NotIntegerError('non-integers can not be converted') <span>②</span></a>
+<a> if not isinstance(n, int): <span class=u>①</span></a>
+<a> raise NotIntegerError('non-integers can not be converted') <span class=u>②</span></a>
result = ''
for numeral, integer in roman_numeral_map:
@@ -529,7 +527,8 @@ OK</samp></pre>
<p>Now stop coding.
-<p class=v><a rel=prev class=todo><span>☜</span></a> <a rel=next class=todo><span>☞</span></a>
+<p class=v><a rel=prev class=todo><span class=u>☜</span></a> <a rel=next class=todo><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/util/lesscss.py b/util/lesscss.py
index a8f9db4..3d8cdb2 100644
--- a/util/lesscss.py
+++ b/util/lesscss.py
@@ -4,12 +4,20 @@ from pyquery import PyQuery as pq
import glob
import sys
+# These selectors are kept regardless of whether this script thinks they are used.
+# Most of these match nodes that are dynamically inserted or manipulated by script
+# after the page has loaded, which is why a static analysis thinks they're unused.
+SELECTOR_EXCEPTIONS = ('.w', '.b', '.str', '.kwd', '.com', '.typ', '.lit', '.pun', '.tag', '.atn', '.atv', '.dec', 'pre span.u', 'pre span.u span')
+
filename = sys.argv[1]
pqd = pq(filename=filename)
raw_data = open(filename, 'rb').read()
-if raw_data.count('<pre><code>') or raw_data.count('<pre class=screen>'):
+if raw_data.count('<pre><code') or raw_data.count('<pre class=screen>'):
def keep(s):
- return s == '.w' or s.startswith('.w ') or s == '.b' or s.startswith('.b ')
+ for selector in SELECTOR_EXCEPTIONS:
+ if s == selector: return True
+ if s.startswith(selector + ' '): return True
+ return False
else:
def keep(s):
return False
diff --git a/whats-new.html b/whats-new.html
index 54bf3b4..51b6646 100644
--- a/whats-new.html
+++ b/whats-new.html
@@ -13,10 +13,10 @@ h3:before{content:''}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#whats-new>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#whats-new>Dive Into Python 3</a> <span class=u>‣</span>
<h1>What’s New In “Dive Into Python 3”</h1>
<blockquote class=q>
-<p><span>❝</span> Isn’t this where we came in? <span>❞</span><br>— Pink Floyd, The Wall
+<p><span class=u>❝</span> Isn’t this where we came in? <span class=u>❞</span><br>— Pink Floyd, The Wall
</blockquote>
<p id=toc>
<h2 id=divingin><i>a.k.a.</i> “the minus level”</h2>
@@ -40,5 +40,3 @@ h3:before{content:''}
<p>That’s it for now; the book’s not finished yet! The file I/O subsystem is totally different now; I hope to write about that soon.
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
-<script src=j/jquery.js></script>
-<script src=j/dip3.js></script>
diff --git a/where-to-go-from-here.html b/where-to-go-from-here.html
index ad71b1a..2efb6b0 100644
--- a/where-to-go-from-here.html
+++ b/where-to-go-from-here.html
@@ -12,11 +12,11 @@ body{counter-reset:h1 20}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#where-to-go-from-here>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#where-to-go-from-here>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=pro>♦♦♦♦♦</span>
<h1>Where To Go From Here</h1>
<blockquote class=q>
-<p><span>❝</span> FIXME <span>❞</span><br>— FIXME
+<p><span class=u>❝</span> FIXME <span class=u>❞</span><br>— FIXME
</blockquote>
<p id=toc>
<h2 id=things-to-read>Things to Read</h2>
@@ -63,7 +63,7 @@ body{counter-reset:h1 20}
<li><a href='http://bitbucket.org/repo/all/?name=python3'>BitBucket: list of projects matching “python3”</a> (and <a href='http://bitbucket.org/repo/all/?name=python+3'>those matching “python 3”</a>)
</ul>
-<p class=v><a href=case-study-porting-chardet-to-python-3.html rel=prev title='back to “Case Study: Porting chardet to Python 3”'><span>☜</span></a> <a href=porting-code-to-python-3-with-2to3.html rel=next title='onward to “Porting Code to Python 3 with 2to3”'><span>☞</span></a>
+<p class=v><a href=case-study-porting-chardet-to-python-3.html rel=prev title='back to “Case Study: Porting chardet to Python 3”'><span class=u>☜</span></a> <a href=porting-code-to-python-3-with-2to3.html rel=next title='onward to “Porting Code to Python 3 with 2to3”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
diff --git a/xml.html b/xml.html
index 00e4bf5..8dd21a4 100644
--- a/xml.html
+++ b/xml.html
@@ -13,11 +13,11 @@ mark{display:inline}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=root value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#xml>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#xml>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=advanced>♦♦♦♦♢</span>
<h1>XML</h1>
<blockquote class=q>
-<p><span>❝</span> In the archonship of Aristaechmus, Draco enacted his ordinances. <span>❞</span><br>— <a href='http://www.perseus.tufts.edu/cgi-bin/ptext?doc=Perseus:text:1999.01.0046;query=chapter%3D%235;layout=;loc=3.1'>Aristotle</a>
+<p><span class=u>❝</span> In the archonship of Aristaechmus, Draco enacted his ordinances. <span class=u>❞</span><br>— <a href='http://www.perseus.tufts.edu/cgi-bin/ptext?doc=Perseus:text:1999.01.0046;query=chapter%3D%235;layout=;loc=3.1'>Aristotle</a>
</blockquote>
<p id=toc>
<h2 id=divingin>Diving In</h2>
@@ -26,7 +26,7 @@ mark{display:inline}
<p>Here, then, is the <abbr>XML</abbr> data we’ll be working with in this chapter. It’s a feed — specifically, an <a href=http://atompub.org/rfc4287.html>Atom syndication feed</a>.
<p class=d>[<a href=examples/feed.xml>download <code>feed.xml</code></a>]
-<pre><code><?xml version='1.0' encoding='utf-8'?>
+<pre><code class=pp><?xml version='1.0' encoding='utf-8'?>
<feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
<title>dive into mark</title>
<subtitle>currently between addictions</subtitle>
@@ -99,8 +99,8 @@ mark{display:inline}
<p><abbr>XML</abbr> is a generalized way of describing hierarchical structured data. An <abbr>XML</abbr> <i>document</i> contains one or more <i>elements</i>, which are delimited by <i>start and end tags</i>. This is a complete (albeit boring) <abbr>XML</abbr> document:
-<pre class=nd><code><a><foo> <span>①</span></a>
-<a></foo> <span>②</span></a></code></pre>
+<pre class=nd><code class=pp><a><foo> <span class=u>①</span></a>
+<a></foo> <span class=u>②</span></a></code></pre>
<ol>
<li>This is the <i>start tag</i> of the <code>foo</code> element.
<li>This is the matching <i>end tag</i> of the <code>foo</code> element. Like balancing parentheses in writing or mathematics or code, every start tag much be <i>closed</i> (matched) by a corresponding end tag.
@@ -108,20 +108,20 @@ mark{display:inline}
<p>Elements can be <i>nested</i> to any depth. An element <code>bar</code> inside an element <code>foo</code> is said to be a <i>subelement</i> or <i>child</i> of <code>foo</code>.
-<pre class=nd><code><foo>
+<pre class=nd><code class=pp><foo>
<mark><bar></bar></mark>
</foo>
</code></pre>
<p>The first element in every <abbr>XML</abbr> document is called the <i>root element</i>. An <abbr>XML</abbr> document can only have one root element. The following is <strong>not an <abbr>XML</abbr> document</strong>, because it has two root elements:
-<pre class=nd><code><foo></foo>
+<pre class=nd><code class=pp><foo></foo>
<bar></bar></code></pre>
<p>Elements can have <i>attributes</i>, which are name-value pairs. Attributes are listed within the start tag of an element and separated by whitespace. <i>Attribute names</i> can not be repeated within an element. <i>Attribute values</i> must be quoted.
-<pre class=nd><code><a><foo <mark>lang='en'</mark>> <span>①</span></a>
-<a> <bar <mark>lang='fr'</mark>></bar> <span>②</span></a>
+<pre class=nd><code class=pp><a><foo <mark>lang='en'</mark>> <span class=u>①</span></a>
+<a> <bar <mark>lang='fr'</mark>></bar> <span class=u>②</span></a>
</foo>
</code></pre>
<ol>
@@ -133,23 +133,23 @@ mark{display:inline}
<p>Elements can have <i>text content</i>.
-<pre class=nd><code><foo lang='en'>
+<pre class=nd><code class=pp><foo lang='en'>
<bar lang='fr'><mark>PapayaWhip</mark></bar>
</foo>
</code></pre>
<p>Elements that contain no text and no children are <i>empty</i>.
-<pre class=nd><code><foo></foo></code></pre>
+<pre class=nd><code class=pp><foo></foo></code></pre>
<p>There is a shorthand for writing empty elements. By putting a <code>/</code> character in the start tag, you can skip the end tag altogther. The <abbr>XML</abbr> document in the previous example could be written like this instead:
-<pre class=nd><code><foo<mark>/</mark>></code></pre>
+<pre class=nd><code class=pp><foo<mark>/</mark>></code></pre>
<p>Like Python functions can be declared in different <i>modules</i>, <abbr>XML</abbr> elements can be declared in different <i>namespaces</i>. Namespaces usually look like URLs. You use an <code>xmlns</code> declaration to define a <i>default namespace</i>. A namespace declaration looks similar to an attribute, but it has a different purpose.
-<pre class=nd><code><a><feed <mark>xmlns='http://www.w3.org/2005/Atom'</mark>> <span>①</span></a>
-<a> <title>dive into mark</title> <span>②</span></a>
+<pre class=nd><code class=pp><a><feed <mark>xmlns='http://www.w3.org/2005/Atom'</mark>> <span class=u>①</span></a>
+<a> <title>dive into mark</title> <span class=u>②</span></a>
</feed>
</code></pre>
<ol>
@@ -159,8 +159,8 @@ mark{display:inline}
<p>You can also use an <code>xmlns:<var>prefix</var></code> declaration to define a namespace and associate it with a <i>prefix</i>. Then each element in that namespace must be explicitly declared with the prefix.
-<pre class=nd><code><a><atom:feed <mark>xmlns:atom='http://www.w3.org/2005/Atom'</mark>> <span>①</span></a>
-<a> <atom:title>dive into mark</atom:title> <span>②</span></a>
+<pre class=nd><code class=pp><a><atom:feed <mark>xmlns:atom='http://www.w3.org/2005/Atom'</mark>> <span class=u>①</span></a>
+<a> <atom:title>dive into mark</atom:title> <span class=u>②</span></a>
</atom:feed></code></pre>
<ol>
<li>The <code>feed</code> element is in the <code>http://www.w3.org/2005/Atom</code> namespace.
@@ -171,7 +171,7 @@ mark{display:inline}
<p>Finally, <abbr>XML</abbr> documents can contain <a href=strings.html#one-ring-to-rule-them-all>character encoding information</a> on the first line, before the root element. (If you’re curious how a document can contain information which needs to be known before the document can be parsed, <a href=http://www.w3.org/TR/REC-xml/#sec-guessing-no-ext-info>Section F of the <abbr>XML</abbr> specification</a> details how to resolve this Catch-22.)
-<pre class=nd><code><?xml version='1.0' <mark>encoding='utf-8'</mark>?></code></pre>
+<pre class=nd><code class=pp><?xml version='1.0' <mark>encoding='utf-8'</mark>?></code></pre>
<p>And now you know just enough <abbr>XML</abbr> to be dangerous!
@@ -185,8 +185,8 @@ mark{display:inline}
<p>At the top level is the <i>root element</i>, which every Atom feed shares: the <code>feed</code> element in the <code>http://www.w3.org/2005/Atom</code> namespace.
-<pre><code><a><feed xmlns='http://www.w3.org/2005/Atom' <span>①</span></a>
-<a> xml:lang='en'> <span>②</span></a></code></pre>
+<pre><code class=pp><a><feed xmlns='http://www.w3.org/2005/Atom' <span class=u>①</span></a>
+<a> xml:lang='en'> <span class=u>②</span></a></code></pre>
<ol>
<li><code>http://www.w3.org/2005/Atom</code> is the Atom namespace.
<li>Any element can contain an <code>xml:lang</code> attribute, which declares the language of the element and its children. In this case, the <code>xml:lang</code> attribute is declared once on the root element, which means the entire feed is in English.
@@ -194,12 +194,12 @@ mark{display:inline}
<p>An Atom feed contains several pieces of information about the feed itself. These are declared as children of the root-level <code>feed</code> element.
-<pre><code><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
-<a> <title>dive into mark</title> <span>①</span></a>
-<a> <subtitle>currently between addictions</subtitle> <span>②</span></a>
-<a> <id>tag:diveintomark.org,2001-07-29:/</id> <span>③</span></a>
-<a> <updated>2009-03-27T21:56:07Z</updated> <span>④</span></a>
-<a> <link rel='alternate' type='text/html' href='http://diveintomark.org/'/> <span>⑤</span></a></code></pre>
+<pre><code class=pp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
+<a> <title>dive into mark</title> <span class=u>①</span></a>
+<a> <subtitle>currently between addictions</subtitle> <span class=u>②</span></a>
+<a> <id>tag:diveintomark.org,2001-07-29:/</id> <span class=u>③</span></a>
+<a> <updated>2009-03-27T21:56:07Z</updated> <span class=u>④</span></a>
+<a> <link rel='alternate' type='text/html' href='http://diveintomark.org/'/> <span class=u>⑤</span></a></code></pre>
<ol>
<li>The title of this feed is <code>dive into mark</code>.
<li>The subtitle of this feed is <code>currently between addictions</code>.
@@ -211,30 +211,30 @@ mark{display:inline}
<p>Now we know that this is a feed for a site named “dive into mark“ which is available at <a href=http://diveintomark.org/><code>http://diveintomark.org/</code></a> and was last updated on March 27, 2009.
<blockquote class=note>
-<p><span>☞</span>Although the order of elements can be relevant in some <abbr>XML</abbr> documents, it is not relevant in an Atom feed.
+<p><span class=u>☞</span>Although the order of elements can be relevant in some <abbr>XML</abbr> documents, it is not relevant in an Atom feed.
</blockquote>
<p>After the feed-level metadata is the list of the most recent articles. An article looks like this:
-<pre><code><entry>
-<a> <author> <span>①</span></a>
+<pre><code class=pp><entry>
+<a> <author> <span class=u>①</span></a>
<name>Mark</name>
<uri>http://diveintomark.org/</uri>
</author>
-<a> <title>Dive into history, 2009 edition</title> <span>②</span></a>
-<a> <link rel='alternate' type='text/html' <span>③</span></a>
+<a> <title>Dive into history, 2009 edition</title> <span class=u>②</span></a>
+<a> <link rel='alternate' type='text/html' <span class=u>③</span></a>
href='http://diveintomark.org/archives/2009/03/27/dive-into-history-2009-edition'/>
-<a> <id>tag:diveintomark.org,2009-03-27:/archives/20090327172042</id> <span>④</span></a>
-<a> <updated>2009-03-27T21:56:07Z</updated> <span>⑤</span></a>
+<a> <id>tag:diveintomark.org,2009-03-27:/archives/20090327172042</id> <span class=u>④</span></a>
+<a> <updated>2009-03-27T21:56:07Z</updated> <span class=u>⑤</span></a>
<published>2009-03-27T17:20:42Z</published>
-<a> <category scheme='http://diveintomark.org' term='diveintopython'/> <span>⑥</span></a>
+<a> <category scheme='http://diveintomark.org' term='diveintopython'/> <span class=u>⑥</span></a>
<category scheme='http://diveintomark.org' term='docbook'/>
<category scheme='http://diveintomark.org' term='html'/>
-<a> <summary type='html'>Putting an entire chapter on one page sounds <span>⑦</span></a>
+<a> <summary type='html'>Putting an entire chapter on one page sounds <span class=u>⑦</span></a>
bloated, but consider this &amp;mdash; my longest chapter so far
would be 75 printed pages, and it loads in under 5 seconds&amp;hellip;
On dialup.</summary>
-<a></entry> <span>⑧</span></a></code></pre>
+<a></entry> <span class=u>⑧</span></a></code></pre>
<ol>
<li>The <code>author</code> element tells who wrote this article: some guy named Mark, whom you can find loafing at <code>http://diveintomark.org/</code>. (This is the same as the alternate link in the feed metadata, but it doesn’t have to be. Many weblogs have multiple authors, each with their own personal website.)
<li>The <code>title</code> element gives the title of the article, “Dive into history, 2009 edition”.
@@ -254,10 +254,10 @@ mark{display:inline}
<p class=d>[<a href=examples/feed.xml>download <code>feed.xml</code></a>]
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>import xml.etree.ElementTree as etree</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>tree = etree.parse('examples/feed.xml')</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>root = tree.getroot()</kbd> <span>③</span></a>
-<a><samp class=p>>>> </samp><kbd>root</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>import xml.etree.ElementTree as etree</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>tree = etree.parse('examples/feed.xml')</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>root = tree.getroot()</kbd> <span class=u>③</span></a>
+<a><samp class=p>>>> </samp><kbd>root</kbd> <span class=u>④</span></a>
<samp><Element {http://www.w3.org/2005/Atom}feed at cd1eb0></samp></pre>
<ol>
<li>The ElementTree library is part of the Python standard library, in <code>xml.etree.ElementTree</code>.
@@ -267,7 +267,7 @@ mark{display:inline}
</ol>
<blockquote class=note>
-<p><span>☞</span>ElementTree represents <abbr>XML</abbr> elements as <code>{<var>namespace</var>}<var>localname</var></code>. You’ll see and use this format in multiple places in the ElementTree <abbr>API</abbr>.
+<p><span class=u>☞</span>ElementTree represents <abbr>XML</abbr> elements as <code>{<var>namespace</var>}<var>localname</var></code>. You’ll see and use this format in multiple places in the ElementTree <abbr>API</abbr>.
</blockquote>
<h3 id=xml-elements>Elements Are Lists</h3>
@@ -276,12 +276,12 @@ mark{display:inline}
<pre class=screen>
# continued from the previous example
-<a><samp class=p>>>> </samp><kbd>root.tag</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>root.tag</kbd> <span class=u>①</span></a>
<samp>'{http://www.w3.org/2005/Atom}feed'</samp>
-<a><samp class=p>>>> </samp><kbd>len(root)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>len(root)</kbd> <span class=u>②</span></a>
<samp>8</samp>
-<a><samp class=p>>>> </samp><kbd>for child in root:</kbd> <span>③</span></a>
-<a><samp class=p>... </samp><kbd> print(child)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>for child in root:</kbd> <span class=u>③</span></a>
+<a><samp class=p>... </samp><kbd> print(child)</kbd> <span class=u>④</span></a>
<samp class=p>... </samp>
<samp><Element {http://www.w3.org/2005/Atom}title at e2b5d0>
<Element {http://www.w3.org/2005/Atom}subtitle at e2b4e0>
@@ -306,17 +306,17 @@ mark{display:inline}
<pre class=screen>
# continuing from the previous example
-<a><samp class=p>>>> </samp><kbd>root.attrib</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>root.attrib</kbd> <span class=u>①</span></a>
<samp>{'{http://www.w3.org/XML/1998/namespace}lang': 'en'}</samp>
-<a><samp class=p>>>> </samp><kbd>root[4]</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>root[4]</kbd> <span class=u>②</span></a>
<samp><Element {http://www.w3.org/2005/Atom}link at e181b0></samp>
-<a><samp class=p>>>> </samp><kbd>root[4].attrib</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>root[4].attrib</kbd> <span class=u>③</span></a>
<samp>{'href': 'http://diveintomark.org/',
'type': 'text/html',
'rel': 'alternate'}</samp>
-<a><samp class=p>>>> </samp><kbd>root[3]</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>root[3]</kbd> <span class=u>④</span></a>
<samp><Element {http://www.w3.org/2005/Atom}updated at e2b4e0></samp>
-<a><samp class=p>>>> </samp><kbd>root[3].attrib</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>root[3].attrib</kbd> <span class=u>⑤</span></a>
<samp>{}</samp></pre>
<ol>
<li>The <code>attrib</code> property is a dictionary of the element’s attributes. The original markup here was <code><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'></code>. The <code>xml:</code> prefix refers to a built-in namespace that every <abbr>XML</abbr> document can use without declaring it.
@@ -336,15 +336,15 @@ mark{display:inline}
<samp class=p>>>> </samp><kbd>import xml.etree.ElementTree as etree</kbd>
<samp class=p>>>> </samp><kbd>tree = etree.parse('examples/feed.xml')</kbd>
<samp class=p>>>> </samp><kbd>root = tree.getroot()</kbd>
-<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span class=u>①</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}entry at e2b4e0>,
<Element {http://www.w3.org/2005/Atom}entry at e2b510>,
<Element {http://www.w3.org/2005/Atom}entry at e2b540>]</samp>
<samp class=p>>>> </samp><kbd>root.tag</kbd>
<samp>'{http://www.w3.org/2005/Atom}feed'</samp>
-<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}feed')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}feed')</kbd> <span class=u>②</span></a>
<samp>[]</samp>
-<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}author')</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}author')</kbd> <span class=u>③</span></a>
<samp>[]</samp></pre>
<ol>
<li>The <code>findall()</code> method finds child elements that match a specific query. (More on the query format in a minute.)
@@ -353,11 +353,11 @@ mark{display:inline}
</ol>
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>tree.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>tree.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span class=u>①</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}entry at e2b4e0>,
<Element {http://www.w3.org/2005/Atom}entry at e2b510>,
<Element {http://www.w3.org/2005/Atom}entry at e2b540>]</samp>
-<a><samp class=p>>>> </samp><kbd>tree.findall('{http://www.w3.org/2005/Atom}author')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>tree.findall('{http://www.w3.org/2005/Atom}author')</kbd> <span class=u>②</span></a>
<samp>[]</samp>
</pre>
<ol>
@@ -368,17 +368,17 @@ mark{display:inline}
<p>There <em>is</em> a way to search for <em>descendant</em> elements, <i>i.e.</i> children, grandchildren, and any element at any nesting level.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>all_links = tree.findall('//{http://www.w3.org/2005/Atom}link')</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>all_links = tree.findall('//{http://www.w3.org/2005/Atom}link')</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>all_links</kbd>
<samp>[<Element {http://www.w3.org/2005/Atom}link at e181b0>,
<Element {http://www.w3.org/2005/Atom}link at e2b570>,
<Element {http://www.w3.org/2005/Atom}link at e2b480>,
<Element {http://www.w3.org/2005/Atom}link at e2b5a0>]</samp>
-<a><samp class=p>>>> </samp><kbd>all_links[0].attrib</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>all_links[0].attrib</kbd> <span class=u>②</span></a>
<samp>{'href': 'http://diveintomark.org/',
'type': 'text/html',
'rel': 'alternate'}</samp>
-<a><samp class=p>>>> </samp><kbd>all_links[1].attrib</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>all_links[1].attrib</kbd> <span class=u>③</span></a>
<samp>{'href': 'http://diveintomark.org/archives/2009/03/27/dive-into-history-2009-edition',
'type': 'text/html',
'rel': 'alternate'}</samp>
@@ -400,8 +400,8 @@ mark{display:inline}
<pre class=screen>
# continuing from the previous example
-<a><samp class=p>>>> </samp><kbd>it = tree.getiterator('{http://www.w3.org/2005/Atom}link')</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>next(it)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>it = tree.getiterator('{http://www.w3.org/2005/Atom}link')</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>next(it)</kbd> <span class=u>②</span></a>
<Element {http://www.w3.org/2005/Atom}link at 122f1b0>
<samp class=p>>>> </samp><kbd>next(it)</kbd>
<Element {http://www.w3.org/2005/Atom}link at 122f1e0>
@@ -427,10 +427,10 @@ StopIteration</samp></pre>
<p><a href=http://codespeak.net/lxml/><code>lxml</code></a> is an open source third-party library that builds on the popular <a href=http://www.xmlsoft.org/>libxml2 parser</a>. It provides a 100% compatible ElementTree <abbr>API</abbr>, then extends it with full XPath support and a few other niceties. There are <a href=http://pypi.python.org/pypi/lxml/>installers available for Windows</a>; Linux users should always try to use distribution-specific tools like <code>yum</code> or <code>apt-get</code> to install precompiled binaries from their repositories. Otherwise you’ll need to <a href=http://codespeak.net/lxml/installation.html>install <code>lxml</code> manually</a>.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>from lxml import etree</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>tree = etree.parse('examples/feed.xml')</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>root = tree.getroot()</kbd> <span>③</span></a>
-<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>from lxml import etree</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>tree = etree.parse('examples/feed.xml')</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>root = tree.getroot()</kbd> <span class=u>③</span></a>
+<a><samp class=p>>>> </samp><kbd>root.findall('{http://www.w3.org/2005/Atom}entry')</kbd> <span class=u>④</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}entry at e2b4e0>,
<Element {http://www.w3.org/2005/Atom}entry at e2b510>,
<Element {http://www.w3.org/2005/Atom}entry at e2b540>]</samp></pre>
@@ -443,7 +443,7 @@ StopIteration</samp></pre>
<p>For large <abbr>XML</abbr> documents, <code>lxml</code> is significantly faster than the built-in ElementTree libary. If you’re only using the ElementTree <abbr>API</abbr> and want to use the fastest available implementation, you can try to import <code>lxml</code> and fall back to the built-in ElementTree.
-<pre><code>try:
+<pre><code class=pp>try:
from lxml import etree
except ImportError:
import xml.etree.ElementTree as etree</code></pre>
@@ -451,17 +451,17 @@ except ImportError:
<p>But <code>lxml</code> is more than just a faster ElementTree. Its <code>findall()</code> method includes support for more complicated expressions.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>import lxml.etree</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>import lxml.etree</kbd> <span class=u>①</span></a>
<samp class=p>>>> </samp><kbd>tree = lxml.etree.parse('examples/feed.xml')</kbd>
-<a><samp class=p>>>> </samp><kbd>tree.findall('//{http://www.w3.org/2005/Atom}*[@href]')</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>tree.findall('//{http://www.w3.org/2005/Atom}*[@href]')</kbd> <span class=u>②</span></a>
[<Element {http://www.w3.org/2005/Atom}link at eeb8a0>,
<Element {http://www.w3.org/2005/Atom}link at eeb990>,
<Element {http://www.w3.org/2005/Atom}link at eeb960>,
<Element {http://www.w3.org/2005/Atom}link at eeb9c0>]
-<a><samp class=p>>>> </samp><kbd>tree.findall("//{http://www.w3.org/2005/Atom}*[@href='http://diveintomark.org/']")</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>tree.findall("//{http://www.w3.org/2005/Atom}*[@href='http://diveintomark.org/']")</kbd> <span class=u>③</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}link at eeb930>]</samp>
<samp class=p>>>> </samp><kbd>NS = '{http://www.w3.org/2005/Atom}'</kbd>
-<a><samp class=p>>>> </samp><kbd>tree.findall('//{NS}author[{NS}uri]'.format(NS=NS))</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>tree.findall('//{NS}author[{NS}uri]'.format(NS=NS))</kbd> <span class=u>④</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}author at eeba80>,
<Element {http://www.w3.org/2005/Atom}author at eebba0>]</samp></pre>
<ol>
@@ -476,13 +476,13 @@ except ImportError:
<pre class=screen>
<samp class=p>>>> </samp><kbd>import lxml.etree</kbd>
<samp class=p>>>> </samp><kbd>tree = lxml.etree.parse('examples/feed.xml')</kbd>
-<a><samp class=p>>>> </samp><kbd>NSMAP = {'atom': 'http://www.w3.org/2005/Atom'}</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>entries = tree.xpath("//atom:category[@term='accessibility']/..",</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>NSMAP = {'atom': 'http://www.w3.org/2005/Atom'}</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>entries = tree.xpath("//atom:category[@term='accessibility']/..",</kbd> <span class=u>②</span></a>
<samp class=p>... </samp><kbd> namespaces=NSMAP)</kbd>
-<a><samp class=p>>>> </samp><kbd>entries</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>entries</kbd> <span class=u>③</span></a>
<samp>[<Element {http://www.w3.org/2005/Atom}entry at e2b630>]</samp>
<samp class=p>>>> </samp><kbd>entry = entries[0]</kbd>
-<a><samp class=p>>>> </samp><kbd>entry.xpath('./atom:title/text()', namespaces=nsmap)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>entry.xpath('./atom:title/text()', namespaces=nsmap)</kbd> <span class=u>④</span></a>
<samp>['Accessibility is a harsh mistress']</samp></pre>
<ol>
<li>To perform XPath queries on namespaced elements, you need to define a namespace prefix mapping. This is just a Python dictionary.
@@ -499,9 +499,9 @@ except ImportError:
<pre class=screen>
<samp class=p>>>> </samp><kbd>import xml.etree.ElementTree as etree</kbd>
-<a><samp class=p>>>> </samp><kbd>new_feed = etree.Element('{http://www.w3.org/2005/Atom}feed',</kbd> <span>①</span></a>
-<a><samp class=p>... </samp><kbd> attrib={'{http://www.w3.org/XML/1998/namespace}lang': 'en'})</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>print(etree.tostring(new_feed))</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>new_feed = etree.Element('{http://www.w3.org/2005/Atom}feed',</kbd> <span class=u>①</span></a>
+<a><samp class=p>... </samp><kbd> attrib={'{http://www.w3.org/XML/1998/namespace}lang': 'en'})</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>print(etree.tostring(new_feed))</kbd> <span class=u>③</span></a>
<samp><ns0:feed xmlns:ns0='http://www.w3.org/2005/Atom' xml:lang='en'/></samp></pre>
<ol>
<li>To create a new element, instantiate the <code>Element</code> class. You pass the element name (namespace + local name) as the first argument. This statement creates a <code>feed</code> element in the Atom namespace. This will be our new document’s root element.
@@ -513,11 +513,11 @@ except ImportError:
<p>An <abbr>XML</abbr> parser won’t “see” any difference between an <abbr>XML</abbr> document with a default namespace and an <abbr>XML</abbr> document with a prefixed namespace. The resulting <abbr>DOM</abbr> of this serialization:
-<pre class=nd><code><ns0:feed xmlns:ns0='http://www.w3.org/2005/Atom' xml:lang='en'/></code></pre>
+<pre class=nd><code class=pp><ns0:feed xmlns:ns0='http://www.w3.org/2005/Atom' xml:lang='en'/></code></pre>
<p>is identical to the <abbr>DOM</abbr> of this serialization:
-<pre class=nd><code><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'/></code></pre>
+<pre class=nd><code class=pp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'/></code></pre>
<p>The only practical difference is that the second serialization is several characters shorter. If we were to recast our entire sample feed with a <code>ns0:</code> prefix in every start and end tag, it would add 4 characters per start tag × 79 tags + 4 characters for the namespace declaration itself, for a total of 316 characters. Assuming <a href=strings.html#byte-arrays>UTF-8 encoding</a>, that’s 316 extra bytes. (After gzipping, the difference drops to 21 bytes, but still, 21 bytes is 21 bytes.) Maybe that doesn’t matter to you, but for something like an Atom feed, which may be downloaded several thousand times whenever it changes, saving a few bytes per request can quickly add up.
@@ -525,11 +525,11 @@ except ImportError:
<pre class=screen>
<samp class=p>>>> </samp><kbd>import lxml.etree</kbd>
-<a><samp class=p>>>> </samp><kbd>NSMAP = {None: 'http://www.w3.org/2005/Atom'}</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>new_feed = lxml.etree.Element('feed', nsmap=NSMAP)</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>NSMAP = {None: 'http://www.w3.org/2005/Atom'}</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>new_feed = lxml.etree.Element('feed', nsmap=NSMAP)</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd> <span class=u>③</span></a>
<samp><feed xmlns='http://www.w3.org/2005/Atom'/></samp>
-<a><samp class=p>>>> </samp><kbd>new_feed.set('{http://www.w3.org/XML/1998/namespace}lang', 'en')</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>new_feed.set('{http://www.w3.org/XML/1998/namespace}lang', 'en')</kbd> <span class=u>④</span></a>
<samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd>
<samp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'/></samp></pre>
<ol>
@@ -542,14 +542,14 @@ except ImportError:
<p>Are <abbr>XML</abbr> documents limited to one element per document? No, of course not. You can easily create child elements, too.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>title = lxml.etree.SubElement(new_feed, 'title',</kbd> <span>①</span></a>
-<a><samp class=p>... </samp><kbd> attrib={'type':'html'})</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>title = lxml.etree.SubElement(new_feed, 'title',</kbd> <span class=u>①</span></a>
+<a><samp class=p>... </samp><kbd> attrib={'type':'html'})</kbd> <span class=u>②</span></a>
<samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd>
<samp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'><title type='html'/></feed></samp>
-<a><samp class=p>>>> </samp><kbd>title.text = 'dive into &hellip;'</kbd> <span>③</span></a>
-<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>title.text = 'dive into &hellip;'</kbd> <span class=u>③</span></a>
+<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed))</kbd> <span class=u>④</span></a>
<samp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'><title type='html'>dive into &amp;hellip;</title></feed></samp>
-<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed, pretty_print=True))</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(new_feed, pretty_print=True))</kbd> <span class=u>⑤</span></a>
<samp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
<title type='html'>dive into&amp;hellip;</title>
</feed></samp></pre>
@@ -574,9 +574,9 @@ except ImportError:
<p>Here is a fragment of a broken <abbr>XML</abbr> document. I’ve highlighted the wellformedness error.
-<pre class=nd><code><?xml version='1.0' encoding='utf-8'?>
+<pre class=nd><code class=pp><?xml version='1.0' encoding='utf-8'?>
<feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
- <title>dive into <mark>…</mark></title>
+ <title>dive into <mark>&hellip;</mark></title>
...
</feed></code></pre>
@@ -600,16 +600,16 @@ lxml.etree.XMLSyntaxError: Entity 'hellip' not defined, line 3, column 28</samp>
<p>To parse this broken <abbr>XML</abbr> document, despite its wellformedness error, you need to create a custom <abbr>XML</abbr> parser.
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>parser = lxml.etree.XMLParser(recover=True)</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>tree = lxml.etree.parse('examples/feed-broken.xml', parser)</kbd> <span>②</span></a>
-<a><samp class=p>>>> </samp><kbd>parser.error_log</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>parser = lxml.etree.XMLParser(recover=True)</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>tree = lxml.etree.parse('examples/feed-broken.xml', parser)</kbd> <span class=u>②</span></a>
+<a><samp class=p>>>> </samp><kbd>parser.error_log</kbd> <span class=u>③</span></a>
<samp>examples/feed-broken.xml:3:28:FATAL:PARSER:ERR_UNDECLARED_ENTITY: Entity 'hellip' not defined</samp>
<samp class=p>>>> </samp><kbd>tree.findall('{http://www.w3.org/2005/Atom}title')</kbd>
<samp>[<Element {http://www.w3.org/2005/Atom}title at ead510>]</samp>
<samp class=p>>>> </samp><kbd>title = tree.findall('{http://www.w3.org/2005/Atom}title')[0]</kbd>
-<a><samp class=p>>>> </samp><kbd>title.text</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>title.text</kbd> <span class=u>④</span></a>
<samp>'dive into '</samp>
-<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(tree.getroot()))</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>print(lxml.etree.tounicode(tree.getroot()))</kbd> <span class=u>⑤</span></a>
<samp><feed xmlns='http://www.w3.org/2005/Atom' xml:lang='en'>
<title>dive into </title>
.
@@ -640,7 +640,8 @@ lxml.etree.XMLSyntaxError: Entity 'hellip' not defined, line 3, column 28</samp>
<li><a href=http://codespeak.net/lxml/1.3/xpathxslt.html>XPath and <abbr>XSLT</abbr> with <code>lxml</code></a>
</ul>
-<p class=v><a rel=prev class=todo><span>☜</span></a> <a rel=next class=todo><span>☞</span></a>
+<p class=v><a rel=prev class=todo><span class=u>☜</span></a> <a rel=next class=todo><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>
diff --git a/your-first-python-program.html b/your-first-python-program.html
index 56b399a..472bed9 100644
--- a/your-first-python-program.html
+++ b/your-first-python-program.html
@@ -4,6 +4,7 @@
<title>Your first Python program - Dive into Python 3</title>
<!--[if IE]><script src=j/html5.js></script><![endif]-->
<link rel=stylesheet href=dip3.css>
+<link rel=stylesheet href=prettify.css>
<style>
body{counter-reset:h1 1}
table{border:1px solid #bbb;border-collapse:collapse;margin:auto}
@@ -15,17 +16,17 @@ th{text-align:left}
<meta name=viewport content='initial-scale=1.0'>
</head>
<form action=http://www.google.com/cse><div><input type=hidden name=cx value=014021643941856155761:l5eihuescdw><input type=hidden name=ie value=UTF-8> <input name=q size=25> <input type=submit name=sa value=Search></div></form>
-<p>You are here: <a href=index.html>Home</a> <span>‣</span> <a href=table-of-contents.html#your-first-python-program>Dive Into Python 3</a> <span>‣</span>
+<p>You are here: <a href=index.html>Home</a> <span class=u>‣</span> <a href=table-of-contents.html#your-first-python-program>Dive Into Python 3</a> <span class=u>‣</span>
<p id=level>Difficulty level: <span title=novice>♦♢♢♢♢</span>
<h1>Your First Python Program</h1>
<blockquote class=q>
-<p><span>❝</span> Don’t bury your burden in saintly silence. You have a problem? Great. Rejoice, dive in, and investigate. <span>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Buddhism>Ven. Henepola Gunaratana</a>
+<p><span class=u>❝</span> Don’t bury your burden in saintly silence. You have a problem? Great. Rejoice, dive in, and investigate. <span class=u>❞</span><br>— <a href=http://en.wikiquote.org/wiki/Buddhism>Ven. Henepola Gunaratana</a>
</blockquote>
<p id=toc>
<h2 id=divingin>Diving In</h2>
<p class=f>Books about programming usually start with a bunch of boring chapters about fundamentals and eventually work up to building something useful. Let’s skip all that. Here is a complete, working Python program. It probably makes absolutely no sense to you. Don’t worry about that, because you’re going to dissect it line by line. But read through it first and see what, if anything, you can make of it.
<p class=d>[<a href=examples/humansize.py>download <code>humansize.py</code></a>]
-<pre><code>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],
+<pre><code class=pp>SUFFIXES = {1000: ['KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB'],
1024: ['KiB', 'MiB', 'GiB', 'TiB', 'PiB', 'EiB', 'ZiB', 'YiB']}
def approximate_size(size, a_kilobyte_is_1024_bytes=True):
@@ -74,16 +75,16 @@ if __name__ == '__main__':
<h2 id=declaringfunctions>Declaring Functions</h2>
<p>Python has functions like most other languages, but it does not have separate header files like <abbr>C++</abbr> or <code>interface</code>/<code>implementation</code> sections like Pascal. When you need a function, just declare it, like this:
-<pre><code>def approximate_size(size, a_kilobyte_is_1024_bytes=True):</code></pre>
+<pre><code class=pp>def approximate_size(size, a_kilobyte_is_1024_bytes=True):</code></pre>
<aside>When you need a function, just declare it.</aside>
<p>The keyword <code>def</code> starts the function declaration, followed by the function name, followed by the arguments in parentheses. Multiple arguments are separated with commas.
<p>Also note that the function doesn’t define a return datatype. Python functions do not specify the datatype of their return value; they don’t even specify whether or not they return a value. (In fact, every Python function returns a value; if the function ever executes a <code>return</code> statement, it will return that value, otherwise it will return <code>None</code>, the Python null value.)
<blockquote class=note>
-<p><span>☞</span>In some languages, functions (that return a value) start with <code>function</code>, and subroutines (that do not return a value) start with <code>sub</code>. There are no subroutines in Python. Everything is a function, all functions return a value (even if it’s <code>None</code>), and all functions start with <code>def</code>.
+<p><span class=u>☞</span>In some languages, functions (that return a value) start with <code>function</code>, and subroutines (that do not return a value) start with <code>sub</code>. There are no subroutines in Python. Everything is a function, all functions return a value (even if it’s <code>None</code>), and all functions start with <code>def</code>.
</blockquote>
<p>The <code>approximate_size()</code> function takes the two arguments — <var>size</var> and <var>a_kilobyte_is_1024_bytes</var> — but neither argument specifies a datatype. In Python, variables are never explicitly typed. Python figures out what type a variable is and keeps track of it internally.
<blockquote class='note compare java'>
-<p><span>☞</span>In Java and other statically-typed languages, you must specify the datatype of the function return value and each function argument. In Python, you never explicitly specify the datatype of anything. Based on what value you assign, Python keeps track of the datatype internally.
+<p><span class=u>☞</span>In Java and other statically-typed languages, you must specify the datatype of the function return value and each function argument. In Python, you never explicitly specify the datatype of anything. Based on what value you assign, Python keeps track of the datatype internally.
</blockquote>
<h3 id=optional-arguments>Optional and Named Arguments</h3>
@@ -92,16 +93,15 @@ if __name__ == '__main__':
<p>Let’s take another look at that <code>approximate_size()</code> function declaration:
-<pre><code>def approximate_size(size, a_kilobyte_is_1024_bytes=True):</code></pre>
+<pre><code class=pp>def approximate_size(size, a_kilobyte_is_1024_bytes=True):</code></pre>
<p>The second argument, <var>a_kilobyte_is_1024_bytes</var>, specifies a default value of <code>True</code>. This means the argument is <i>optional</i>; you can call the function without it, and Python will act as if you had called it with <code>True</code> as a second parameter.
<p>Now look at the bottom of the script:
-<pre><code>
-if __name__ == '__main__':
-<a> print(approximate_size(1000000000000, False)) <span>①</span></a>
-<a> print(approximate_size(1000000000000)) <span>②</span></a></code></pre>
+<pre><code class=pp>if __name__ == '__main__':
+<a> print(approximate_size(1000000000000, False)) <span class=u>①</span></a>
+<a> print(approximate_size(1000000000000)) <span class=u>②</span></a></code></pre>
<ol>
<li>This calls the <code>approximate_size()</code> function with two argument. Within the <code>approximate_size()</code> function, <var>a_kilobyte_is_1024_bytes</var> will be <code>False</code>, since you explicitly passed <code>False</code> as the second argument.
<li>This calls the <code>approximate_size()</code> function with only one argument. But that’s OK, because the second argument is optional! Since the caller doesn’t specify, the second argument defaults to <code>True</code>, as defined by the function declaration.
@@ -111,16 +111,16 @@ if __name__ == '__main__':
<pre class=screen>
<samp class=p>>>> </samp><kbd>from humansize import approximate_size</kbd>
-<a><samp class=p>>>> </samp><kbd>approximate_size(4000, a_kilobyte_is_1024_bytes=False)</kbd> <span>①</span></a>
+<a><samp class=p>>>> </samp><kbd>approximate_size(4000, a_kilobyte_is_1024_bytes=False)</kbd> <span class=u>①</span></a>
<samp>'4.0 KB'</samp>
-<a><samp class=p>>>> </samp><kbd>approximate_size(size=4000, a_kilobyte_is_1024_bytes=False)</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>approximate_size(size=4000, a_kilobyte_is_1024_bytes=False)</kbd> <span class=u>②</span></a>
<samp>'4.0 KB'</samp>
-<a><samp class=p>>>> </samp><kbd>approximate_size(a_kilobyte_is_1024_bytes=False, size=4000)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>approximate_size(a_kilobyte_is_1024_bytes=False, size=4000)</kbd> <span class=u>③</span></a>
<samp>'4.0 KB'</samp>
-<a><samp class=p>>>> </samp><kbd>approximate_size(a_kilobyte_is_1024_bytes=False, 4000)</kbd> <span>④</span></a>
+<a><samp class=p>>>> </samp><kbd>approximate_size(a_kilobyte_is_1024_bytes=False, 4000)</kbd> <span class=u>④</span></a>
<samp class=traceback> File "<stdin>", line 1
SyntaxError: non-keyword arg after keyword arg</samp>
-<a><samp class=p>>>> </samp><kbd>approximate_size(size=4000, False)</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>approximate_size(size=4000, False)</kbd> <span class=u>⑤</span></a>
<samp class=traceback> File "<stdin>", line 1
SyntaxError: non-keyword arg after keyword arg</samp></pre>
<ol>
@@ -137,7 +137,7 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
<p>I won’t bore you with a long finger-wagging speech about the importance of documenting your code. Just know that code is written once but read many times, and the most important audience for your code is yourself, six months after writing it (i.e. after you’ve forgotten everything but need to fix something). Python makes it easy to write readable code, so take advantage of it. You’ll thank me in six months.
<h3 id=docstrings>Documentation Strings</h3>
<p>You can document a Python function by giving it a documentation string (<code>docstring</code> for short). In this program, the <code>approximate_size()</code> function has a <code>docstring</code>:
-<pre><code>def approximate_size(size, a_kilobyte_is_1024_bytes=True):
+<pre><code class=pp>def approximate_size(size, a_kilobyte_is_1024_bytes=True):
'''Convert a file size to human-readable form.
Keyword arguments:
@@ -151,11 +151,11 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
<aside>Every function deserves a decent docstring.</aside>
<p>Triple quotes signify a multi-line string. Everything between the start and end quotes is part of a single string, including carriage returns, leading white space, and other quote characters. You can use them anywhere, but you’ll see them most often used when defining a <code>docstring</code>.
<blockquote class='note compare perl5'>
-<p><span>☞</span>Triple quotes are also an easy way to define a string with both single and double quotes, like <code>qq/.../</code> in Perl 5.
+<p><span class=u>☞</span>Triple quotes are also an easy way to define a string with both single and double quotes, like <code>qq/.../</code> in Perl 5.
</blockquote>
<p>Everything between the triple quotes is the function’s <code>docstring</code>, which documents what the function does. A <code>docstring</code>, if it exists, must be the first thing defined in a function (that is, on the next line after the function declaration). You don’t technically need to give your function a <code>docstring</code>, but you always should. I know you’ve heard this in every programming class you’ve ever taken, but Python gives you an added incentive: the <code>docstring</code> is available at runtime as an attribute of the function.
<blockquote class=note>
-<p><span>☞</span>Many Python <abbr>IDE</abbr>s use the <code>docstring</code> to provide context-sensitive documentation, so that when you type a function name, its <code>docstring</code> appears as a tooltip. This can be incredibly helpful, but it’s only as good as the <code>docstring</code>s you write.
+<p><span class=u>☞</span>Many Python <abbr>IDE</abbr>s use the <code>docstring</code> to provide context-sensitive documentation, so that when you type a function name, its <code>docstring</code> appears as a tooltip. This can be incredibly helpful, but it’s only as good as the <code>docstring</code>s you write.
</blockquote>
<p class=a>⁂
@@ -163,10 +163,10 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
<p>In case you missed it, I just said that Python functions have attributes, and that those attributes are available at runtime. A function, like everything else in Python, is an object.
<p>Run the interactive Python shell and follow along:
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>import humansize</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>print(humansize.approximate_size(4096, True))</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>import humansize</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>print(humansize.approximate_size(4096, True))</kbd> <span class=u>②</span></a>
<samp>4.0 KiB</samp>
-<a><samp class=p>>>> </samp><kbd>print(humansize.approximate_size.__doc__)</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>print(humansize.approximate_size.__doc__)</kbd> <span class=u>③</span></a>
<samp>Convert a file size to human-readable form.
Keyword arguments:
@@ -183,13 +183,13 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
<li>Instead of calling the function as you would expect to, you asked for one of the function’s attributes, <code>__doc__</code>.
</ol>
<blockquote class='note compare perl5'>
-<p><span>☞</span><code>import</code> in Python is like <code>require</code> in Perl. Once you <code>import</code> a Python module, you access its functions with <code><var>module</var>.<var>function</var></code>; once you <code>require</code> a Perl module, you access its functions with <code><var>module</var>::<var>function</var></code>.
+<p><span class=u>☞</span><code>import</code> in Python is like <code>require</code> in Perl. Once you <code>import</code> a Python module, you access its functions with <code><var>module</var>.<var>function</var></code>; once you <code>require</code> a Perl module, you access its functions with <code><var>module</var>::<var>function</var></code>.
</blockquote>
<h3 id=importsearchpath>The <code>import</code> Search Path</h3>
<p>Before this goes any further, I want to briefly mention the library search path. Python looks in several places when you try to import a module. Specifically, it looks in all the directories defined in <code>sys.path</code>. This is just a list, and you can easily view it or modify it with standard list methods. (You’ll learn more about lists in <a href=native-datatypes.html#lists>Native Datatypes</a>.)
<pre class=screen>
-<a><samp class=p>>>> </samp><kbd>import sys</kbd> <span>①</span></a>
-<a><samp class=p>>>> </samp><kbd>sys.path</kbd> <span>②</span></a>
+<a><samp class=p>>>> </samp><kbd>import sys</kbd> <span class=u>①</span></a>
+<a><samp class=p>>>> </samp><kbd>sys.path</kbd> <span class=u>②</span></a>
<samp>['',
'/usr/lib/python30.zip',
'/usr/lib/python3.0',
@@ -197,10 +197,10 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
'/usr/lib/python3.0/lib-dynload',
'/usr/lib/python3.0/dist-packages',
'/usr/local/lib/python3.0/dist-packages']</samp>
-<a><samp class=p>>>> </samp><kbd>sys</kbd> <span>③</span></a>
+<a><samp class=p>>>> </samp><kbd>sys</kbd> <span class=u>③</span></a>
<samp><module 'sys' (built-in)></samp>
-<a><samp class=p>>>> </samp><kbd>sys.path.insert(0, '/home/mark/py')</kbd> <span>④</span></a>
-<a><samp class=p>>>> </samp><kbd>sys.path</kbd> <span>⑤</span></a>
+<a><samp class=p>>>> </samp><kbd>sys.path.insert(0, '/home/mark/py')</kbd> <span class=u>④</span></a>
+<a><samp class=p>>>> </samp><kbd>sys.path</kbd> <span class=u>⑤</span></a>
<samp>['/home/mark/py',
'',
'/usr/lib/python30.zip',
@@ -225,13 +225,12 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
<h2 id=indentingcode>Indenting Code</h2>
<p>Python functions have no explicit <code>begin</code> or <code>end</code>, and no curly braces to mark where the function code starts and stops. The only delimiter is a colon (<code>:</code>) and the indentation of the code itself.
-<pre><code>
-<a>def approximate_size(size, a_kilobyte_is_1024_bytes=True): <span>①</span></a>
-<a> if size < 0: <span>②</span></a>
-<a> raise ValueError('number must be non-negative') <span>③</span></a>
-<a> <span>④</span></a>
+<pre><code class=pp><a>def approximate_size(size, a_kilobyte_is_1024_bytes=True): <span class=u>①</span></a>
+<a> if size < 0: <span class=u>②</span></a>
+<a> raise ValueError('number must be non-negative') <span class=u>③</span></a>
+<a> <span class=u>④</span></a>
multiple = 1024 if a_kilobyte_is_1024_bytes else 1000
-<a> for suffix in SUFFIXES[multiple]: <span>⑤</span></a>
+<a> for suffix in SUFFIXES[multiple]: <span class=u>⑤</span></a>
size /= multiple
if size < multiple:
return '{0:.1f} {1}'.format(size, suffix)
@@ -246,19 +245,19 @@ SyntaxError: non-keyword arg after keyword arg</samp></pre>
</ol>
<p>After some initial protests and several snide analogies to Fortran, you will make peace with this and start seeing its benefits. One major benefit is that all Python programs look similar, since indentation is a language requirement and not a matter of style. This makes it easier to read and understand other people’s Python code.
<blockquote class='note compare java'>
-<p><span>☞</span>Python uses carriage returns to separate statements and a colon and indentation to separate code blocks. <abbr>C++</abbr> and Java use semicolons to separate statements and curly braces to separate code blocks.
+<p><span class=u>☞</span>Python uses carriage returns to separate statements and a colon and indentation to separate code blocks. <abbr>C++</abbr> and Java use semicolons to separate statements and curly braces to separate code blocks.
</blockquote>
<p class=a>⁂
<h2 id=runningscripts>Running Scripts</h2>
<aside>Everything in Python is an object.</aside>
<p>Python modules are objects and have several useful attributes. You can use this to easily test your modules as you write them, by including a special block of code that executes when you run the Python file on the command line. Take the last few lines of <code>humansize.py</code>:
-<pre><code>
+<pre><code class=pp>
if __name__ == '__main__':
print(approximate_size(1000000000000, False))
print(approximate_size(1000000000000))</code></pre>
<blockquote class='note compare clang'>
-<p><span>☞</span>Like <abbr>C</abbr>, Python uses <code>==</code> for comparison and <code>=</code> for assignment. Unlike <abbr>C</abbr>, Python does not support in-line assignment, so there’s no chance of accidentally assigning the value you thought you were comparing.
+<p><span class=u>☞</span>Like <abbr>C</abbr>, Python uses <code>==</code> for comparison and <code>=</code> for assignment. Unlike <abbr>C</abbr>, Python does not support in-line assignment, so there’s no chance of accidentally assigning the value you thought you were comparing.
</blockquote>
<p>So what makes this <code>if</code> statement special? Well, modules are objects, and all modules have a built-in attribute <code>__name__</code>. A module’s <code>__name__</code> depends on how you’re using the module. If you <code>import</code> the module, then <code>__name__</code> is the module’s filename, without a directory path or file extension.
<pre class=screen>
@@ -280,7 +279,8 @@ if __name__ == '__main__':
<li><a href=http://www.python.org/dev/peps/pep-0008/>PEP 8: Style Guide for Python Code</a> discusses good indentation style.
<li><a href=http://docs.python.org/3.0/reference/><cite>Python Reference Manual</cite></a> explains what it means to say that <a href=http://docs.python.org/3.0/reference/datamodel.html#objects-values-and-types>everything in Python is an object</a>, because some people are <a href=http://www.douglasadams.com/dna/pedants.html>pedants</a> and like to discuss that sort of thing at great length.
</ul>
-<p class=v><a rel=prev class=todo><span>☜</span></a> <a rel=next href=native-datatypes.html title='onward to “Native Datatypes”'><span>☞</span></a>
+<p class=v><a rel=prev class=todo><span class=u>☜</span></a> <a rel=next href=native-datatypes.html title='onward to “Native Datatypes”'><span class=u>☞</span></a>
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
<script src=j/jquery.js></script>
+<script src=j/prettify.js></script>
<script src=j/dip3.js></script>