diff --git a/comprehensions.html b/comprehensions.html index 03550bc..fa3610d 100644 --- a/comprehensions.html +++ b/comprehensions.html @@ -63,7 +63,7 @@ body{counter-reset:h1 3}

Working With Filenames and Directory Names

-

While we’re on the subject of directories, I want to point out the os.path submodule. os.path contains functions for manipulating filenames and directory names. +

While we’re on the subject of directories, I want to point out the os.path module. os.path contains functions for manipulating filenames and directory names. 

 >>> import os
@@ -151,18 +151,19 @@ body{counter-reset:h1 3}
 >>> print(os.getcwd())                 ①
 c:\Users\pilgrim\diveintopython3\examples
 >>> metadata = os.stat('feed.xml')     ②
->>> metadata.st_mtime
+>>> metadata.st_mtime                  ③
 1247520344.9537716
->>> import time                        ③
->>> time.localtime(metadata.st_mtime)  ④
+>>> import time                        ④
+>>> time.localtime(metadata.st_mtime)  ⑤
 time.struct_time(tm_year=2009, tm_mon=7, tm_mday=13, tm_hour=17,
   tm_min=25, tm_sec=44, tm_wday=0, tm_yday=194, tm_isdst=1)
    -
  1. FIXME -
  2. FIXME -
  3. FIXME -
  4. FIXME +
  5. The current working directory is the examples folder. +
  6. feed.xml is a file in the examples folder. Calling the os.stat() function returns an object that contains several different types of metadata about the file. +
  7. st_mtime is the modification time, but it’s in a format that isn’t terribly useful. (Technically, it’s the number of seconds since the Epoch, which is defined as the first second of January 1st, 1970. Seriously.) +
  8. The time module is part of the Python standard library. It contains functions to convert between different time representations, format time values into strings, and fiddle with timezones. +
  9. The time.localtime() function converts a time value from seconds-since-the-Epoch (from the st_mtime property returned from the os.stat() function) into a more useful structure of year, month, day, hour, minute, second, and so on. This file was last modified on July 13, 2009, at around 5:25 PM. 
@@ -173,8 +174,8 @@ body{counter-reset:h1 3}
 >>> humansize.approximate_size(metadata.st_size)  ②
 '3.0 KiB'
    -
  1. FIXME -
  2. FIXME +
  3. The os.stat() function also returns the size of a file, in the st_size property. The file feed.xml is 3070 bytes. +
  4. You can pass the st_size property to the approximate_size() function.

Constructing Absolute Pathnames

@@ -188,7 +189,7 @@ body{counter-reset:h1 3} >>> print(os.path.abspath('feed.xml')) c:\Users\pilgrim\diveintopython3\examples\feed.xml -

List Comprehensions

+

List Comprehensions

A list comprehension provides a compact way of mapping a list into another list by applying a function to each of the elements of the list. @@ -211,19 +212,19 @@ body{counter-reset:h1 3} 

 >>> import os, glob
->>> glob.glob('*.xml')                                              ①
+>>> glob.glob('*.xml')                                ①
 ['feed-broken.xml', 'feed-ns0.xml', 'feed.xml']
->>> [os.path.abspath(filename) for filename in glob.glob('*.xml')]  ②
+>>> [os.path.abspath(f) for f in glob.glob('*.xml')]  ②
 ['c:\\Users\\pilgrim\\diveintopython3\\examples\\feed-broken.xml',
  'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed-ns0.xml',
  'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed.xml']
    -
  1. FIXME -
  2. +
  3. This returns a list of all the .xml files in the current working directory. +
  4. This list comprehension takes that list of .xml files and transforms it into a list of full pathnames.
-

List comprehensions can also filter items, producing a result that may be smaller than the original list. +

List comprehensions can also filter items, producing a result that can be smaller than the original list. 

 >>> import os, glob
@@ -243,76 +244,110 @@ body{counter-reset:h1 3}
 
 
 >>> import os, glob
->>> [(os.stat(f).st_size, os.path.abspath(f)) for f in glob.glob('*.xml')]  ①
+>>> [(os.stat(f).st_size, os.path.abspath(f)) for f in glob.glob('*.xml')]             ①
 [(3074, 'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed-broken.xml'),
  (3386, 'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed-ns0.xml'),
- (3070, 'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed.xml')]

+ (3070, 'c:\\Users\\pilgrim\\diveintopython3\\examples\\feed.xml')]
+>>> import humansize
+>>> [(humansize.approximate_size(os.stat(f).st_size), f) for f in glob.glob('*.xml')]  ②
+[('3.0 KiB', 'feed-broken.xml'),
+ ('3.3 KiB', 'feed-ns0.xml'),
+ ('3.0 KiB', 'feed.xml')]
    -
  1. This list comprehension finds all the .xml files in the current working directory, gets the size of each file, and returns a tuple of the file size and the absolute path of each file. +
  2. This list comprehension finds all the .xml files in the current working directory, gets the size of each file (by calling the os.stat() function), and constructs a tuple of the file size and the absolute path of each file (by calling the os.path.abspath() function). +
  3. This comprehension builds on the previous one to call the approximate_size() function with the file size of each .xml file.
-

FIXME +

⁂ -

->>> print("\n".join(["{0:>8} {1}".format(humansize.approximate_size(os.stat(f).st_size, False), os.path.abspath(f)) for f in glob.glob('*.py')]))
-  2.5 KB c:\Users\pilgrim\diveintopython3\examples\alphametics.py
-  2.5 KB c:\Users\pilgrim\diveintopython3\examples\alphameticstest.py
-  1.5 KB c:\Users\pilgrim\diveintopython3\examples\fibonacci.py
-  1.8 KB c:\Users\pilgrim\diveintopython3\examples\fibonacci2.py
-  2.5 KB c:\Users\pilgrim\diveintopython3\examples\humansize.py
-  0.2 KB c:\Users\pilgrim\diveintopython3\examples\oneline.py
-  1.9 KB c:\Users\pilgrim\diveintopython3\examples\plural1.py
-  2.3 KB c:\Users\pilgrim\diveintopython3\examples\plural2.py
-  2.3 KB c:\Users\pilgrim\diveintopython3\examples\plural3.py
-  2.3 KB c:\Users\pilgrim\diveintopython3\examples\plural4.py
-  2.4 KB c:\Users\pilgrim\diveintopython3\examples\plural5.py
-  2.8 KB c:\Users\pilgrim\diveintopython3\examples\plural6.py
-  3.0 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest1.py
-  3.0 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest2.py
-  3.0 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest3.py
-  3.0 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest4.py
-  3.0 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest5.py
-  6.1 KB c:\Users\pilgrim\diveintopython3\examples\pluraltest6.py
-  0.5 KB c:\Users\pilgrim\diveintopython3\examples\regression.py
-  2.2 KB c:\Users\pilgrim\diveintopython3\examples\roman1.py
-  3.4 KB c:\Users\pilgrim\diveintopython3\examples\roman10.py
-  2.3 KB c:\Users\pilgrim\diveintopython3\examples\roman2.py
-  2.3 KB c:\Users\pilgrim\diveintopython3\examples\roman3.py
-  2.5 KB c:\Users\pilgrim\diveintopython3\examples\roman4.py
-  2.7 KB c:\Users\pilgrim\diveintopython3\examples\roman5.py
-  3.6 KB c:\Users\pilgrim\diveintopython3\examples\roman6.py
-  3.7 KB c:\Users\pilgrim\diveintopython3\examples\roman7.py
-  3.7 KB c:\Users\pilgrim\diveintopython3\examples\roman8.py
-  3.7 KB c:\Users\pilgrim\diveintopython3\examples\roman9.py
-  4.0 KB c:\Users\pilgrim\diveintopython3\examples\romantest1.py
-  6.7 KB c:\Users\pilgrim\diveintopython3\examples\romantest10.py
-  4.2 KB c:\Users\pilgrim\diveintopython3\examples\romantest2.py
-  4.5 KB c:\Users\pilgrim\diveintopython3\examples\romantest3.py
-  4.7 KB c:\Users\pilgrim\diveintopython3\examples\romantest4.py
-  5.3 KB c:\Users\pilgrim\diveintopython3\examples\romantest5.py
-  6.1 KB c:\Users\pilgrim\diveintopython3\examples\romantest6.py
-  6.3 KB c:\Users\pilgrim\diveintopython3\examples\romantest7.py
-  6.5 KB c:\Users\pilgrim\diveintopython3\examples\romantest8.py
-  6.6 KB c:\Users\pilgrim\diveintopython3\examples\romantest9.py
-  0.4 KB c:\Users\pilgrim\diveintopython3\examples\stdout.py
+

Dictionary Comprehensions

+ +

A dictionary comprehension is like a list comprehension, but it constructs a dictionary instead of a list. + +

+>>> import os, glob
+>>> metadata = [(f, os.stat(f)) for f in glob.glob('*test*.py')]    ①
+>>> metadata[0]                                                     ②
+('alphameticstest.py', nt.stat_result(st_mode=33206, st_ino=0, st_dev=0,
+ st_nlink=0, st_uid=0, st_gid=0, st_size=2509, st_atime=1247520344,
+ st_mtime=1247520344, st_ctime=1247520344))
+>>> metadata_dict = {f:os.stat(f) for f in glob.glob('*test*.py')}  ③
+>>> type(metadata_dict)                                             ④
+<class 'dict'>
+>>> list(metadata_dict.keys())                                      ⑤
+['romantest8.py', 'pluraltest1.py', 'pluraltest2.py', 'pluraltest5.py',
+ 'pluraltest6.py', 'romantest7.py', 'romantest10.py', 'romantest4.py',
+ 'romantest9.py', 'pluraltest3.py', 'romantest1.py', 'romantest2.py',
+ 'romantest3.py', 'romantest5.py', 'romantest6.py', 'alphameticstest.py',
+ 'pluraltest4.py']
+>>> metadata_dict['alphameticstest.py'].st_size                     ⑥
+2509
+
    +
  1. This is not a dictionary comprehension; it’s a list comprehension. It finds all .py files with test in their name, then constructs a tuple of the filename and the file metadata (from calling the os.stat() function). +
  2. Each item of the resulting list is a tuple. +
  3. This is a dictionary comprehension. The syntax is similar to a list comprehension, with two differences. First, it is enclosed in curly braces instead of square brackets. Second, instead of a single expression for each item, it contains two expressions separated by a colon. The expression before the colon (f in this example) is the dictionary key; the expression after the colon (os.stat(f) in this example) is the value. +
  4. A dictionary comprehension returns a dictionary. +
  5. The keys of this particular dictionary are simply the filenames returned from the call to glob.glob('*test*.py'). +
  6. The value associated with each key is the return value from the os.stat() function. That means we can “look up” a file by name in this dictionary to get its file metadata. One of the pieces of metadata is st_size, the file size. The file alphameticstest.py is 2509 bytes long. +
+ +

Like list comprehensions, you can include an if clause in a dictionary comprehension to filter the input sequence based on an expression which is evaluated with each item. + +

+>>> import os, glob, humansize
+>>> humansize_dict = {os.path.splitext(f)[0]:humansize.approximate_size(os.stat(f).st_size) \     
+...                   for f in glob.glob('*') if os.stat(f).st_size > 6000}                     ①
+>>> list(humansize_dict.keys())                                                                 ②
+['romantest9', 'romantest8', 'romantest7', 'romantest6', 'romantest10', 'pluraltest6']
+>>> humansize_dict['romantest9']                                                                ③
+'6.5 KiB'
+
    +
  1. This dictionary comprehension constructs a list of all the files in the current working directory (glob.glob('*')), filters that list to include only those files larger than 6000 bytes (if os.stat(f).st_size > 6000), and uses that filtered list to construct a dictionary whose keys are the filename minus the extension (os.path.splitext(f)[0]) and whose values are the approximate size of each file (humansize.approximate_size(os.stat(f).st_size)). +
  2. As you saw in a previous example, there are six such files, thus there are six items in this dictionary. +
  3. The value of each key is the string returned from the approximate_size() function. +
+ +

Other Fun Stuff To Do With Dictionary Comprehensions

+ +

Here’s a trick with dictionary comprehensions that might be useful someday: swapping the keys and values of a dictionary. + +

+>>> a_dict = {'a': 1, 'b': 2, 'c': 3}
+>>> {value:key for key, value in a_dict.items()}
+{1: 'a', 2: 'b', 3: 'c'}

⁂ -

Set Comprehensions

+

Set Comprehensions

-

FIXME +

Not to be left out, sets have their own comprehension syntax as well. It is remarkably similar to the syntax for dictionary comprehensions. The only difference is that sets just have values instead of key:value pairs. -

⁂ - -

Dictionary Comprehensions

- -

FIXME +

+>>> a_set = set(range(10))
+>>> a_set
+{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
+>>> {x ** 2 for x in a_set}           ①
+{0, 1, 4, 81, 64, 9, 16, 49, 25, 36}
+>>> {x for x in a_set if x % 2 == 0}  ②
+{0, 8, 2, 4, 6}
+>>> {2**x for x in range(10)}         ③
+{32, 1, 2, 4, 8, 64, 128, 256, 16, 512}
+
+
    +
  1. Set comprehensions can take a set as input. This set comprehension calculates the squares of the set of numbers from 0 to 9. +
  2. Like list comprehensions and dictionary comprehensions, set comprehensions can contain an if clause to filter each item before returning it in the result set. +
  3. Set comprehensions do not need to take a set as input; they can take any sequence. +

⁂

Further Reading

© 2001–9 Mark Pilgrim