mirror of
https://github.com/kennethreitz/dive-into-python3.git
synced 2026-06-05 23:10:17 +00:00
whats-new, more special-method-names, typography fiddling
This commit is contained in:
Mark Pilgrim
+21
-21
@@ -17,13 +17,13 @@ body{counter-reset:h1 10}
|
||||
</blockquote>
|
||||
<p id=toc>
|
||||
<h2 id=divingin>Diving In</h2>
|
||||
<p class=f>Despite your best efforts to write comprehensive unit tests, bugs happen. What do I mean by “bug”? A bug is a test case you haven't written yet.
|
||||
<p class=f>Despite your best efforts to write comprehensive unit tests, bugs happen. What do I mean by “bug”? A bug is a test case you haven’t written yet.
|
||||
|
||||
<pre class=screen><samp class=p>>>> </samp><kbd>import roman7</kbd>
|
||||
<a><samp class=p>>>> </samp><kbd>roman7.from_roman("")</kbd> <span>①</span></a>
|
||||
<samp>0</samp></pre>
|
||||
<ol>
|
||||
<li>Remember in the [FIXME-xref] previous section when you kept seeing that an empty string would match the regular expression you were using to check for valid Roman numerals? Well, it turns out that this is still true for the final version of the regular expression. And that's a bug; you want an empty string to raise an <code>InvalidRomanNumeralError</code> exception just like any other sequence of characters that don't represent a valid Roman numeral.
|
||||
<li>Remember in the [FIXME-xref] previous section when you kept seeing that an empty string would match the regular expression you were using to check for valid Roman numerals? Well, it turns out that this is still true for the final version of the regular expression. And that’s a bug; you want an empty string to raise an <code>InvalidRomanNumeralError</code> exception just like any other sequence of characters that don’t represent a valid Roman numeral.
|
||||
</ol>
|
||||
|
||||
<p>After reproducing the bug, and before fixing it, you should write a test case that fails, thus illustrating the bug.
|
||||
@@ -107,15 +107,15 @@ Ran 11 tests in 0.156s
|
||||
<a><samp>OK</samp> <span>②</span></a></pre>
|
||||
<ol>
|
||||
<li>The blank string test case now passes, so the bug is fixed.
|
||||
<li>All the other test cases still pass, which means that this bug fix didn't break anything else. Stop coding.
|
||||
<li>All the other test cases still pass, which means that this bug fix didn’t break anything else. Stop coding.
|
||||
</ol>
|
||||
|
||||
<p>Coding this way does not make fixing bugs any easier. Simple bugs (like this one) require simple test cases; complex bugs will require complex test cases. In a testing-centric environment, it may <em>seem</em> like it takes longer to fix a bug, since you need to articulate in code exactly what the bug is (to write the test case), then fix the bug itself. Then if the test case doesn't pass right away, you need to figure out whether the fix was wrong, or whether the test case itself has a bug in it. However, in the long run, this back-and-forth between test code and code tested pays for itself, because it makes it more likely that bugs are fixed correctly the first time. Also, since you can easily re-run <em>all</em> the test cases along with your new one, you are much less likely to break old code when fixing new code. Today's unit test is tomorrow's regression test.
|
||||
<p>Coding this way does not make fixing bugs any easier. Simple bugs (like this one) require simple test cases; complex bugs will require complex test cases. In a testing-centric environment, it may <em>seem</em> like it takes longer to fix a bug, since you need to articulate in code exactly what the bug is (to write the test case), then fix the bug itself. Then if the test case doesn’t pass right away, you need to figure out whether the fix was wrong, or whether the test case itself has a bug in it. However, in the long run, this back-and-forth between test code and code tested pays for itself, because it makes it more likely that bugs are fixed correctly the first time. Also, since you can easily re-run <em>all</em> the test cases along with your new one, you are much less likely to break old code when fixing new code. Today’s unit test is tomorrow’s regression test.
|
||||
|
||||
<h2 id=changing-requirements>Handling Changing Requirements</h2>
|
||||
<p>Despite your best efforts to pin your customers to the ground and extract exact requirements from them on pain of horrible nasty things involving scissors and hot wax, requirements will change. Most customers don't know what they want until they see it, and even if they do, they aren't that good at articulating what they want precisely enough to be useful. And even if they do, they'll want more in the next release anyway. So be prepared to update your test cases as requirements change.
|
||||
<p>Despite your best efforts to pin your customers to the ground and extract exact requirements from them on pain of horrible nasty things involving scissors and hot wax, requirements will change. Most customers don’t know what they want until they see it, and even if they do, they aren’t that good at articulating what they want precisely enough to be useful. And even if they do, they’ll want more in the next release anyway. So be prepared to update your test cases as requirements change.
|
||||
|
||||
<p>Suppose, for instance, that you wanted to expand the range of the Roman numeral conversion functions. Remember [FIXME-xref] the rule that said that no character could be repeated more than three times? Well, the Romans were willing to make an exception to that rule by having 4 <code>M</code> characters in a row to represent <code>4000</code>. If you make this change, you'll be able to expand the range of convertible numbers from <code>1..3999</code> to <code>1..4999</code>. But first, you need to make some changes to your test cases.
|
||||
<p>Suppose, for instance, that you wanted to expand the range of the Roman numeral conversion functions. Remember [FIXME-xref] the rule that said that no character could be repeated more than three times? Well, the Romans were willing to make an exception to that rule by having 4 <code>M</code> characters in a row to represent <code>4000</code>. If you make this change, you’ll be able to expand the range of convertible numbers from <code>1..3999</code> to <code>1..4999</code>. But first, you need to make some changes to your test cases.
|
||||
|
||||
<p class=d>[<a href=examples/roman8.py>download <code>roman8.py</code></a>]
|
||||
<pre><code>
|
||||
@@ -157,7 +157,7 @@ class RoundtripCheck(unittest.TestCase):
|
||||
result = roman8.from_roman(numeral)
|
||||
self.assertEqual(integer, result)</code></pre>
|
||||
<ol>
|
||||
<li>The existing known values don't change (they're all still reasonable values to test), but you need to add a few more in the <code>4000</code> range. Here I've included <code>4000</code> (the shortest), <code>4500</code> (the second shortest), <code>4888</code> (the longest), and <code>4999</code> (the largest).
|
||||
<li>The existing known values don’t change (they’re all still reasonable values to test), but you need to add a few more in the <code>4000</code> range. Here I’ve included <code>4000</code> (the shortest), <code>4500</code> (the second shortest), <code>4888</code> (the longest), and <code>4999</code> (the largest).
|
||||
<li>The definition of “large input” has changed. This test used to call <code>to_roman()</code> with <code>4000</code> and expect an error; now that <code>4000-4999</code> are good values, you need to bump this up to <code>5000</code>.
|
||||
<li>The definition of “too many repeated numerals” has also changed. This test used to call <code>from_roman()</code> with <code>'MMMM'</code> and expect an error; now that <code>MMMM</code> is considered a valid Roman numeral, you need to bump this up to <code>'MMMMM'</code>.
|
||||
<li>The sanity check loops through every number in the range, from <code>1</code> to <code>3999</code>. Since the range has now expanded, this <code>for</code> loop need to be updated as well to go up to <code>4999</code>.
|
||||
@@ -220,7 +220,7 @@ FAILED (errors=3)</samp></pre>
|
||||
<li>The roundtrip check will also fail as soon as it hits <code>4000</code>, because <code>to_roman()</code> still thinks this is out of range.
|
||||
</ol>
|
||||
|
||||
<p>Now that you have test cases that fail due to the new requirements, you can think about fixing the code to bring it in line with the test cases. (One thing that takes some getting used to when you first start coding unit tests is that the code being tested is never “ahead” of the test cases. While it's behind, you still have some work to do, and as soon as it catches up to the test cases, you stop coding.)
|
||||
<p>Now that you have test cases that fail due to the new requirements, you can think about fixing the code to bring it in line with the test cases. (One thing that takes some getting used to when you first start coding unit tests is that the code being tested is never “ahead” of the test cases. While it’s behind, you still have some work to do, and as soon as it catches up to the test cases, you stop coding.)
|
||||
|
||||
<p class=d>[<a href=examples/roman9.py>download <code>roman9.py</code></a>]
|
||||
<pre><code>
|
||||
@@ -255,11 +255,11 @@ def from_roman(s):
|
||||
.
|
||||
.</code></pre>
|
||||
<ol>
|
||||
<li>You don't need to make any changes to the <code>from_roman()</code> function at all. The only change is to <var>roman_numeral_pattern</var>. If you look closely, you'll notice that I changed the maximum number of optional <code>M</code> characters from <code>3</code> to <code>4</code> in the first section of the regular expression. This will allow the Roman numeral equivalents of <code>4999</code> instead of <code>3999</code>. The actual <code>from_roman()</code> function is completely generic; it just looks for repeated Roman numeral characters and adds them up, without caring how many times they repeat. The only reason it didn't handle <code>'MMMM'</code> before is that you explicitly stopped it with the regular expression pattern matching.
|
||||
<li>The <code>to_roman()</code> function only needs one small change, in the range check. Where you used to check <code>0 < n < 4000</code>, you now check <code>0 < n < 5000</code>. And you change the error message that you <code>raise</code> to reflect the new acceptable range (<code>1..4999</code> instead of <code>1..3999</code>). You don't need to make any changes to the rest of the function; it handles the new cases already. (It merrily adds <code>'M'</code> for each thousand that it finds; given <code>4000</code>, it will spit out <code>'MMMM'</code>. The only reason it didn't do this before is that you explicitly stopped it with the range check.)
|
||||
<li>You don’t need to make any changes to the <code>from_roman()</code> function at all. The only change is to <var>roman_numeral_pattern</var>. If you look closely, you’ll notice that I changed the maximum number of optional <code>M</code> characters from <code>3</code> to <code>4</code> in the first section of the regular expression. This will allow the Roman numeral equivalents of <code>4999</code> instead of <code>3999</code>. The actual <code>from_roman()</code> function is completely generic; it just looks for repeated Roman numeral characters and adds them up, without caring how many times they repeat. The only reason it didn’t handle <code>'MMMM'</code> before is that you explicitly stopped it with the regular expression pattern matching.
|
||||
<li>The <code>to_roman()</code> function only needs one small change, in the range check. Where you used to check <code>0 < n < 4000</code>, you now check <code>0 < n < 5000</code>. And you change the error message that you <code>raise</code> to reflect the new acceptable range (<code>1..4999</code> instead of <code>1..3999</code>). You don’t need to make any changes to the rest of the function; it handles the new cases already. (It merrily adds <code>'M'</code> for each thousand that it finds; given <code>4000</code>, it will spit out <code>'MMMM'</code>. The only reason it didn’t do this before is that you explicitly stopped it with the range check.)
|
||||
</ol>
|
||||
|
||||
<p>You may be skeptical that these two small changes are all that you need. Hey, don't take my word for it; see for yourself.
|
||||
<p>You may be skeptical that these two small changes are all that you need. Hey, don’t take my word for it; see for yourself.
|
||||
|
||||
<pre class=screen>
|
||||
<samp class=p>you@localhost:~$ </samp><kbd>python3 romantest9.py -v</kbd>
|
||||
@@ -288,13 +288,13 @@ Ran 12 tests in 0.203s
|
||||
|
||||
<h2 id=refactoring>Refactoring</h2>
|
||||
|
||||
<p>The best thing about comprehensive unit testing is not the feeling you get when all your test cases finally pass, or even the feeling you get when someone else blames you for breaking their code and you can actually <em>prove</em> that you didn't. The best thing about unit testing is that it gives you the freedom to refactor mercilessly.
|
||||
<p>The best thing about comprehensive unit testing is not the feeling you get when all your test cases finally pass, or even the feeling you get when someone else blames you for breaking their code and you can actually <em>prove</em> that you didn’t. The best thing about unit testing is that it gives you the freedom to refactor mercilessly.
|
||||
|
||||
<p>Refactoring is the process of taking working code and making it work better. Usually, “better” means “faster”, although it can also mean “using less memory”, or “using less disk space”, or simply “more elegantly”. Whatever it means to you, to your project, in your environment, refactoring is important to the long-term health of any program.
|
||||
|
||||
<p>Here, “better” means both “faster” and “easier to maintain.” Specifically, the <code>from_roman()</code> function is slower and more complex than I'd like, because of that big nasty regular expression that you use to validate Roman numerals. Now, you might think, "Sure, the regular expression is big and hairy, but how else am I supposed to validate that an arbitrary string is a valid a Roman numeral?"
|
||||
<p>Here, “better” means both “faster” and “easier to maintain.” Specifically, the <code>from_roman()</code> function is slower and more complex than I’d like, because of that big nasty regular expression that you use to validate Roman numerals. Now, you might think, "Sure, the regular expression is big and hairy, but how else am I supposed to validate that an arbitrary string is a valid a Roman numeral?"
|
||||
|
||||
<p>Answer: there's only 5000 of them; why don't you just build a lookup table? This idea gets even better when you realize that <em>you don't need to use regular expressions at all</em>. As you build the lookup table for converting integers to Roman numerals, you can build the reverse lookup table to convert Roman numerals to integers. By the time you need to check whether an arbitrary string is a valid Roman numeral, you will have collected all the valid Roman numerals. “Validating” is reduced to a single dictionary lookup.
|
||||
<p>Answer: there’s only 5000 of them; why don’t you just build a lookup table? This idea gets even better when you realize that <em>you don’t need to use regular expressions at all</em>. As you build the lookup table for converting integers to Roman numerals, you can build the reverse lookup table to convert Roman numerals to integers. By the time you need to check whether an arbitrary string is a valid Roman numeral, you will have collected all the valid Roman numerals. “Validating” is reduced to a single dictionary lookup.
|
||||
|
||||
<p>And best of all, you already have a complete set of unit tests. You can change over half the code in the module, but the unit tests will stay the same. That means you can prove — to yourself and to others — that the new code works just as well as the original.
|
||||
|
||||
@@ -357,13 +357,13 @@ def build_lookup_tables():
|
||||
|
||||
build_lookup_tables()</code></pre>
|
||||
|
||||
<p>Let's break that down into digestable pieces. Arguably, the most important line is the last one:
|
||||
<p>Let’s break that down into digestable pieces. Arguably, the most important line is the last one:
|
||||
|
||||
<pre><code>build_lookup_tables()</code></pre>
|
||||
|
||||
<p>You will note that is a function call, but there's no <code>if</code> statement around it. This is not an <code>if __name__ == '__main__'</code> block; it gets called <em>when the module is imported</em>. (It is important to understand that modules are only imported once, then cached. If you import an already-imported module, it does nothing. So this code will only get called the first time you import this module.)
|
||||
<p>You will note that is a function call, but there’s no <code>if</code> statement around it. This is not an <code>if __name__ == '__main__'</code> block; it gets called <em>when the module is imported</em>. (It is important to understand that modules are only imported once, then cached. If you import an already-imported module, it does nothing. So this code will only get called the first time you import this module.)
|
||||
|
||||
<p>So what does the <code>build_lookup_tables()</code> function do? I'm glad you asked.
|
||||
<p>So what does the <code>build_lookup_tables()</code> function do? I’m glad you asked.
|
||||
|
||||
<pre><code><a>to_roman_table = [ None ]
|
||||
from_roman_table = {}
|
||||
@@ -438,7 +438,7 @@ to_roman should fail with 0 input ... ok
|
||||
|
||||
OK</samp></pre>
|
||||
<ol>
|
||||
<li>Not that you asked, but it's fast, too! Like, almost 10× as fast. Of course, it's not entirely a fair comparison, because this version takes longer to import (when it builds the lookup tables). But since the import is only done once, the startup cost is amortized over all the calls to the <code>to_roman()</code> and <code>from_roman()</code> functions. Since the tests make several thousand function calls (the roundtrip test alone makes 10,000), this savings adds up in a hurry!
|
||||
<li>Not that you asked, but it’s fast, too! Like, almost 10× as fast. Of course, it’s not entirely a fair comparison, because this version takes longer to import (when it builds the lookup tables). But since the import is only done once, the startup cost is amortized over all the calls to the <code>to_roman()</code> and <code>from_roman()</code> functions. Since the tests make several thousand function calls (the roundtrip test alone makes 10,000), this savings adds up in a hurry!
|
||||
</ol>
|
||||
|
||||
<p>The moral of the story?
|
||||
@@ -451,9 +451,9 @@ OK</samp></pre>
|
||||
|
||||
<h2 id=summary>Summary</h2>
|
||||
|
||||
<p>Unit testing is a powerful concept which, if properly implemented, can both reduce maintenance costs and increase flexibility in any long-term project. It is also important to understand that unit testing is not a panacea, a Magic Problem Solver, or a silver bullet. Writing good test cases is hard, and keeping them up to date takes discipline (especially when customers are screaming for critical bug fixes). Unit testing is not a replacement for other forms of testing, including functional testing, integration testing, and user acceptance testing. But it is feasible, and it does work, and once you've seen it work, you'll wonder how you ever got along without it.
|
||||
<p>Unit testing is a powerful concept which, if properly implemented, can both reduce maintenance costs and increase flexibility in any long-term project. It is also important to understand that unit testing is not a panacea, a Magic Problem Solver, or a silver bullet. Writing good test cases is hard, and keeping them up to date takes discipline (especially when customers are screaming for critical bug fixes). Unit testing is not a replacement for other forms of testing, including functional testing, integration testing, and user acceptance testing. But it is feasible, and it does work, and once you’ve seen it work, you’ll wonder how you ever got along without it.
|
||||
|
||||
<p>These few chapters have covered a lot of ground, and much of it wasn't even Python-specific. There are unit testing frameworks for many languages, all of which require you to understand the same basic concepts:
|
||||
<p>These few chapters have covered a lot of ground, and much of it wasn’t even Python-specific. There are unit testing frameworks for many languages, all of which require you to understand the same basic concepts:
|
||||
|
||||
<ul>
|
||||
<li>Designing test cases that are specific, automated, and independent
|
||||
@@ -461,7 +461,7 @@ OK</samp></pre>
|
||||
<li>Writing tests that test good input and check for proper results
|
||||
<li>Writing tests that test bad input and check for proper failure responses
|
||||
<li>Writing and updating test cases to reflect new requirements
|
||||
<li>Refactoring mercilessly to improve performance, scalability, readability, maintainability, or whatever other -ility you're lacking
|
||||
<li>Refactoring mercilessly to improve performance, scalability, readability, maintainability, or whatever other -ility you’re lacking
|
||||
</ul>
|
||||
|
||||
<p class=c>© 2001–9 <a href=about.html>Mark Pilgrim</a>
|
||||
|
||||
Reference in New Issue
Block a user