mirror of
https://github.com/kennethreitz/python-guide.git
synced 2026-06-05 23:00:18 +00:00
Merge pull request #242 from lambdadi/master
Clarified point about variable names, under Dynamic Typing
This commit is contained in:
Kenneth Reitz
+475
-475
@@ -1,475 +1,475 @@
|
|||||||
Structuring Your Project
|
Structuring Your Project
|
||||||
========================
|
========================
|
||||||
|
|
||||||
By "structure" we mean the decisions you make concerning
|
By "structure" we mean the decisions you make concerning
|
||||||
how your project best meets its objective. We need to consider how to
|
how your project best meets its objective. We need to consider how to
|
||||||
best leverage Python's features to create clean, effective code.
|
best leverage Python's features to create clean, effective code.
|
||||||
In practical terms, "structure" means making clean code whose logic and
|
In practical terms, "structure" means making clean code whose logic and
|
||||||
dependencies are clear as well as how the files and folders are organized
|
dependencies are clear as well as how the files and folders are organized
|
||||||
in the filesystem.
|
in the filesystem.
|
||||||
|
|
||||||
Which functions should go into which modules? How does data flow through
|
Which functions should go into which modules? How does data flow through
|
||||||
the project? What features and functions can be grouped together and
|
the project? What features and functions can be grouped together and
|
||||||
isolated? By answering questions like these you can begin to plan, in
|
isolated? By answering questions like these you can begin to plan, in
|
||||||
a broad sense, what your finished product will look like.
|
a broad sense, what your finished product will look like.
|
||||||
|
|
||||||
In this section we take a closer look at Python's module and import
|
In this section we take a closer look at Python's module and import
|
||||||
systems as they are the central element to enforcing structure in your
|
systems as they are the central element to enforcing structure in your
|
||||||
project. We then discuss various perspectives on how to build code which
|
project. We then discuss various perspectives on how to build code which
|
||||||
can be extended and tested reliably.
|
can be extended and tested reliably.
|
||||||
|
|
||||||
|
|
||||||
Structure is Key
|
Structure is Key
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
Thanks to the way imports and modules are handled in Python, it is
|
Thanks to the way imports and modules are handled in Python, it is
|
||||||
relatively easy to structure a Python project. Easy, here, means
|
relatively easy to structure a Python project. Easy, here, means
|
||||||
that you do not have many constraints and that the module
|
that you do not have many constraints and that the module
|
||||||
importing model is easy to grasp. Therefore, you are left with the
|
importing model is easy to grasp. Therefore, you are left with the
|
||||||
pure architectural task of crafting the different parts of your
|
pure architectural task of crafting the different parts of your
|
||||||
project and their interactions.
|
project and their interactions.
|
||||||
|
|
||||||
Easy structuring of a project means it is also easy
|
Easy structuring of a project means it is also easy
|
||||||
to do it poorly. Some signs of a poorly structured project
|
to do it poorly. Some signs of a poorly structured project
|
||||||
include:
|
include:
|
||||||
|
|
||||||
- Multiple and messy circular dependencies: if your classes
|
- Multiple and messy circular dependencies: if your classes
|
||||||
Table and Chair in furn.py need to import Carpenter from workers.py
|
Table and Chair in furn.py need to import Carpenter from workers.py
|
||||||
to answer a question such as table.isdoneby(),
|
to answer a question such as table.isdoneby(),
|
||||||
and if conversely the class Carpenter needs to import Table and Chair,
|
and if conversely the class Carpenter needs to import Table and Chair,
|
||||||
to answer the question carpenter.whatdo(), then you
|
to answer the question carpenter.whatdo(), then you
|
||||||
have a circular dependency. In this case you will have to resort to
|
have a circular dependency. In this case you will have to resort to
|
||||||
fragile hacks such as using import statements inside
|
fragile hacks such as using import statements inside
|
||||||
methods or functions.
|
methods or functions.
|
||||||
|
|
||||||
- Hidden coupling: each and every change in Table's implementation
|
- Hidden coupling: each and every change in Table's implementation
|
||||||
breaks 20 tests in unrelated test cases because it breaks Carpenter's code,
|
breaks 20 tests in unrelated test cases because it breaks Carpenter's code,
|
||||||
which requires very careful surgery to adapt the change. This means
|
which requires very careful surgery to adapt the change. This means
|
||||||
you have too many assumptions about Table in Carpenter's code or the
|
you have too many assumptions about Table in Carpenter's code or the
|
||||||
reverse.
|
reverse.
|
||||||
|
|
||||||
- Heavy usage of global state or context: instead of explicitly
|
- Heavy usage of global state or context: instead of explicitly
|
||||||
passing ``(height, width, type, wood)`` to each other, Table
|
passing ``(height, width, type, wood)`` to each other, Table
|
||||||
and Carpenter rely on global variables that can be modified
|
and Carpenter rely on global variables that can be modified
|
||||||
and are modified on the fly by different agents. You need to
|
and are modified on the fly by different agents. You need to
|
||||||
scrutinize all access to these global variables to understand why
|
scrutinize all access to these global variables to understand why
|
||||||
a rectangular table became a square, and discover that remote
|
a rectangular table became a square, and discover that remote
|
||||||
template code is also modifying this context, messing with
|
template code is also modifying this context, messing with
|
||||||
table dimensions.
|
table dimensions.
|
||||||
|
|
||||||
- Spaghetti code: multiple pages of nested if clauses and for loops
|
- Spaghetti code: multiple pages of nested if clauses and for loops
|
||||||
with a lot of copy-pasted procedural code and no
|
with a lot of copy-pasted procedural code and no
|
||||||
proper segmentation are known as spaghetti code. Python's
|
proper segmentation are known as spaghetti code. Python's
|
||||||
meaningful indentation (one of its most controversial features) make
|
meaningful indentation (one of its most controversial features) make
|
||||||
it very hard to maintain this kind of code. So the good news is that
|
it very hard to maintain this kind of code. So the good news is that
|
||||||
you might not see too much of it.
|
you might not see too much of it.
|
||||||
|
|
||||||
- Ravioli code is more likely in Python: it consists of hundreds of
|
- Ravioli code is more likely in Python: it consists of hundreds of
|
||||||
similar little pieces of logic, often classes or objects, without
|
similar little pieces of logic, often classes or objects, without
|
||||||
proper structure. If you never can remember if you have to use
|
proper structure. If you never can remember if you have to use
|
||||||
FurnitureTable, AssetTable or Table, or even TableNew for your
|
FurnitureTable, AssetTable or Table, or even TableNew for your
|
||||||
task at hand, you might be swimming in ravioli code.
|
task at hand, you might be swimming in ravioli code.
|
||||||
|
|
||||||
|
|
||||||
Modules
|
Modules
|
||||||
-------
|
-------
|
||||||
|
|
||||||
Python modules are one of the main abstraction layers available and probably the
|
Python modules are one of the main abstraction layers available and probably the
|
||||||
most natural one. Abstraction layers allow separating code into parts holding
|
most natural one. Abstraction layers allow separating code into parts holding
|
||||||
related data and functionality.
|
related data and functionality.
|
||||||
|
|
||||||
For example, a layer of a project can handle interfacing with user actions,
|
For example, a layer of a project can handle interfacing with user actions,
|
||||||
while another would handle low-level manipulation of data. The most natural way
|
while another would handle low-level manipulation of data. The most natural way
|
||||||
to separate these two layers is to regroup all interfacing functionality
|
to separate these two layers is to regroup all interfacing functionality
|
||||||
in one file, and all low-level operations in another file. In this case,
|
in one file, and all low-level operations in another file. In this case,
|
||||||
the interface file needs to import the low-level file. This is done with the
|
the interface file needs to import the low-level file. This is done with the
|
||||||
`import` and `from ... import` statements.
|
`import` and `from ... import` statements.
|
||||||
|
|
||||||
As soon as you use `import` statements you use modules. These can be either built-in
|
As soon as you use `import` statements you use modules. These can be either built-in
|
||||||
modules such as `os` and `sys`, third-party modules you have installed in your
|
modules such as `os` and `sys`, third-party modules you have installed in your
|
||||||
environment, or your project's internal modules.
|
environment, or your project's internal modules.
|
||||||
|
|
||||||
To keep in line with the style guide, keep module names short, lowercase, and
|
To keep in line with the style guide, keep module names short, lowercase, and
|
||||||
be sure to avoid using special symbols like the dot (.) or question mark (?).
|
be sure to avoid using special symbols like the dot (.) or question mark (?).
|
||||||
So a file name like `my.spam.py` is one you should try to avoid! Naming this way
|
So a file name like `my.spam.py` is one you should try to avoid! Naming this way
|
||||||
will interfere with the way python looks for modules.
|
will interfere with the way python looks for modules.
|
||||||
|
|
||||||
In this example python expects to find a "spam.py" file in a folder named "my"
|
In this example python expects to find a "spam.py" file in a folder named "my"
|
||||||
which is not the case. There is an
|
which is not the case. There is an
|
||||||
`example <http://docs.python.org/tutorial/modules.html#packages>`_ of how the
|
`example <http://docs.python.org/tutorial/modules.html#packages>`_ of how the
|
||||||
dot notation should be used in the python docs.
|
dot notation should be used in the python docs.
|
||||||
|
|
||||||
If you'd like you could name it as `my_spam.py` but even our friend the
|
If you'd like you could name it as `my_spam.py` but even our friend the
|
||||||
underscore should not be seen often in module names.
|
underscore should not be seen often in module names.
|
||||||
|
|
||||||
Aside for some naming restrictions, nothing special is required for a Python file
|
Aside for some naming restrictions, nothing special is required for a Python file
|
||||||
to be a module, but the import mechanism needs to be understood in order to use
|
to be a module, but the import mechanism needs to be understood in order to use
|
||||||
this concept properly and avoid some issues.
|
this concept properly and avoid some issues.
|
||||||
|
|
||||||
Concretely, the `import modu` statement will look for the proper file, which is
|
Concretely, the `import modu` statement will look for the proper file, which is
|
||||||
`modu.py` in the same directory as the caller if it exists. If it is not
|
`modu.py` in the same directory as the caller if it exists. If it is not
|
||||||
found, the Python interpreter will search for `modu.py` in the "path"
|
found, the Python interpreter will search for `modu.py` in the "path"
|
||||||
recursively and raise an ImportError exception if it is not found.
|
recursively and raise an ImportError exception if it is not found.
|
||||||
|
|
||||||
Once `modu.py` is found, the Python interpreter will execute the module in an
|
Once `modu.py` is found, the Python interpreter will execute the module in an
|
||||||
isolated scope. Any top-level statement in `modu.py` will be executed,
|
isolated scope. Any top-level statement in `modu.py` will be executed,
|
||||||
including other imports if any. Function and class definitions are stored in
|
including other imports if any. Function and class definitions are stored in
|
||||||
the module's dictionary.
|
the module's dictionary.
|
||||||
|
|
||||||
Then, the module's variables, functions, and classes will be available to the caller
|
Then, the module's variables, functions, and classes will be available to the caller
|
||||||
through the module's namespace, a central concept in programming that is
|
through the module's namespace, a central concept in programming that is
|
||||||
particularly helpful and powerful in Python.
|
particularly helpful and powerful in Python.
|
||||||
|
|
||||||
In many languages, an `include file` directive is used by the preprocessor to
|
In many languages, an `include file` directive is used by the preprocessor to
|
||||||
take all code found in the file and 'copy' it into the caller's code. It is
|
take all code found in the file and 'copy' it into the caller's code. It is
|
||||||
different in Python: the included code is isolated in a module namespace, which
|
different in Python: the included code is isolated in a module namespace, which
|
||||||
means that you generally don't have to worry that the included code could have
|
means that you generally don't have to worry that the included code could have
|
||||||
unwanted effects, e.g. override an existing function with the same name.
|
unwanted effects, e.g. override an existing function with the same name.
|
||||||
|
|
||||||
It is possible to simulate the more standard behavior by using a special syntax
|
It is possible to simulate the more standard behavior by using a special syntax
|
||||||
of the import statement: `from modu import *`. This is generally considered bad
|
of the import statement: `from modu import *`. This is generally considered bad
|
||||||
practice. **Using `import *` makes code harder to read and makes dependencies less
|
practice. **Using `import *` makes code harder to read and makes dependencies less
|
||||||
compartmentalized**.
|
compartmentalized**.
|
||||||
|
|
||||||
Using `from modu import func` is a way to pinpoint the function you want to
|
Using `from modu import func` is a way to pinpoint the function you want to
|
||||||
import and put it in the global namespace. While much less harmful than `import
|
import and put it in the global namespace. While much less harmful than `import
|
||||||
*` because it shows explicitly what is imported in the global namespace, its
|
*` because it shows explicitly what is imported in the global namespace, its
|
||||||
advantage over a simpler `import modu` is only that it will save some typing.
|
advantage over a simpler `import modu` is only that it will save some typing.
|
||||||
|
|
||||||
**Very bad**
|
**Very bad**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
[...]
|
[...]
|
||||||
from modu import *
|
from modu import *
|
||||||
[...]
|
[...]
|
||||||
x = sqrt(4) # Is sqrt part of modu? A builtin? Defined above?
|
x = sqrt(4) # Is sqrt part of modu? A builtin? Defined above?
|
||||||
|
|
||||||
**Better**
|
**Better**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
from modu import sqrt
|
from modu import sqrt
|
||||||
[...]
|
[...]
|
||||||
x = sqrt(4) # sqrt may be part of modu, if not redefined in between
|
x = sqrt(4) # sqrt may be part of modu, if not redefined in between
|
||||||
|
|
||||||
**Best**
|
**Best**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
import modu
|
import modu
|
||||||
[...]
|
[...]
|
||||||
x = modu.sqrt(4) # sqrt is visibly part of modu's namespace
|
x = modu.sqrt(4) # sqrt is visibly part of modu's namespace
|
||||||
|
|
||||||
As said in the section about style, readability is one of the main features of
|
As said in the section about style, readability is one of the main features of
|
||||||
Python. Readability means to avoid useless boilerplate text and clutter,
|
Python. Readability means to avoid useless boilerplate text and clutter,
|
||||||
therefore some efforts are spent trying to achieve a certain level of brevity.
|
therefore some efforts are spent trying to achieve a certain level of brevity.
|
||||||
But terseness and obscurity are the limits where brevity should stop. Being
|
But terseness and obscurity are the limits where brevity should stop. Being
|
||||||
able to tell immediately where a class or function comes from, as in the
|
able to tell immediately where a class or function comes from, as in the
|
||||||
`modu.func` idiom, greatly improves code readability and understandability in
|
`modu.func` idiom, greatly improves code readability and understandability in
|
||||||
all but the simplest single file projects.
|
all but the simplest single file projects.
|
||||||
|
|
||||||
|
|
||||||
Packages
|
Packages
|
||||||
--------
|
--------
|
||||||
|
|
||||||
Python provides a very straightforward packaging system, which is simply an
|
Python provides a very straightforward packaging system, which is simply an
|
||||||
extension of the module mechanism to a directory.
|
extension of the module mechanism to a directory.
|
||||||
|
|
||||||
Any directory with an __init__.py file is considered a Python package. The
|
Any directory with an __init__.py file is considered a Python package. The
|
||||||
different modules in the package are imported in a similar manner as plain
|
different modules in the package are imported in a similar manner as plain
|
||||||
modules, but with a special behavior for the __init__.py file, which is used to
|
modules, but with a special behavior for the __init__.py file, which is used to
|
||||||
gather all package-wide definitions.
|
gather all package-wide definitions.
|
||||||
|
|
||||||
A file modu.py in the directory pack/ is imported with the statement `import
|
A file modu.py in the directory pack/ is imported with the statement `import
|
||||||
pack.modu`. This statement will look for an __init__.py file in `pack`, execute
|
pack.modu`. This statement will look for an __init__.py file in `pack`, execute
|
||||||
all of its top-level statements. Then it will look for a file `pack/modu.py` and
|
all of its top-level statements. Then it will look for a file `pack/modu.py` and
|
||||||
execute all of its top-level statements. After these operations, any variable,
|
execute all of its top-level statements. After these operations, any variable,
|
||||||
function, or class defined in modu.py is available in the pack.modu namespace.
|
function, or class defined in modu.py is available in the pack.modu namespace.
|
||||||
|
|
||||||
A commonly seen issue is to add too much code to __init__.py
|
A commonly seen issue is to add too much code to __init__.py
|
||||||
files. When the project complexity grows, there may be sub-packages and
|
files. When the project complexity grows, there may be sub-packages and
|
||||||
sub-sub-packages in a deep directory structure, and then, importing a single item
|
sub-sub-packages in a deep directory structure, and then, importing a single item
|
||||||
from a sub-sub-package will require executing all __init__.py files met while
|
from a sub-sub-package will require executing all __init__.py files met while
|
||||||
traversing the tree.
|
traversing the tree.
|
||||||
|
|
||||||
Leaving an __init__.py file empty is considered normal and even a good practice,
|
Leaving an __init__.py file empty is considered normal and even a good practice,
|
||||||
if the package's modules and sub-packages do not need to share any code.
|
if the package's modules and sub-packages do not need to share any code.
|
||||||
|
|
||||||
Lastly, a convenient syntax is available for importing deeply nested packages:
|
Lastly, a convenient syntax is available for importing deeply nested packages:
|
||||||
`import very.deep.module as mod`. This allows you to use `mod` in place of the verbose
|
`import very.deep.module as mod`. This allows you to use `mod` in place of the verbose
|
||||||
repetition of `very.deep.module`.
|
repetition of `very.deep.module`.
|
||||||
|
|
||||||
Object-oriented programming
|
Object-oriented programming
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
Python is sometimes described as an object-oriented programming language. This
|
Python is sometimes described as an object-oriented programming language. This
|
||||||
can be somewhat misleading and needs to be clarified.
|
can be somewhat misleading and needs to be clarified.
|
||||||
|
|
||||||
In Python, everything is an object, and can be handled as such. This is what is
|
In Python, everything is an object, and can be handled as such. This is what is
|
||||||
meant when we say that, for example, functions are first-class objects.
|
meant when we say that, for example, functions are first-class objects.
|
||||||
Functions, classes, strings, and even types are objects in Python: like any
|
Functions, classes, strings, and even types are objects in Python: like any
|
||||||
objects, they have a type, they can be passed as function arguments, they may
|
objects, they have a type, they can be passed as function arguments, they may
|
||||||
have methods and properties. In this understanding, Python is an
|
have methods and properties. In this understanding, Python is an
|
||||||
object-oriented language.
|
object-oriented language.
|
||||||
|
|
||||||
However, unlike Java, Python does not impose object-oriented programming as the
|
However, unlike Java, Python does not impose object-oriented programming as the
|
||||||
main programming paradigm. It is perfectly viable for a Python project to not
|
main programming paradigm. It is perfectly viable for a Python project to not
|
||||||
be object-oriented, i.e. to use no or very few class definitions, class
|
be object-oriented, i.e. to use no or very few class definitions, class
|
||||||
inheritance, or any other mechanisms that are specific to object-oriented
|
inheritance, or any other mechanisms that are specific to object-oriented
|
||||||
programming.
|
programming.
|
||||||
|
|
||||||
Moreover, as seen in the modules_ section, the way Python handles modules and
|
Moreover, as seen in the modules_ section, the way Python handles modules and
|
||||||
namespaces gives the developer a natural way to ensure the
|
namespaces gives the developer a natural way to ensure the
|
||||||
encapsulation and separation of abstraction layers, both being the most common
|
encapsulation and separation of abstraction layers, both being the most common
|
||||||
reasons to use object-orientation. Therefore, Python programmers have more
|
reasons to use object-orientation. Therefore, Python programmers have more
|
||||||
latitude to not use object-orientation, when it is not required by the business
|
latitude to not use object-orientation, when it is not required by the business
|
||||||
model.
|
model.
|
||||||
|
|
||||||
There are some reasons to avoid unnecessary object-orientation. Defining
|
There are some reasons to avoid unnecessary object-orientation. Defining
|
||||||
custom classes is useful when we want to glue together some state and some
|
custom classes is useful when we want to glue together some state and some
|
||||||
functionality. The problem, as pointed out by the discussions about functional
|
functionality. The problem, as pointed out by the discussions about functional
|
||||||
programming, comes from the "state" part of the equation.
|
programming, comes from the "state" part of the equation.
|
||||||
|
|
||||||
In some architectures, typically web applications, multiple instances of Python
|
In some architectures, typically web applications, multiple instances of Python
|
||||||
processes are spawned to respond to external requests that can
|
processes are spawned to respond to external requests that can
|
||||||
happen at the same time. In this case, holding some state into instantiated
|
happen at the same time. In this case, holding some state into instantiated
|
||||||
objects, which means keeping some static information about the world, is prone
|
objects, which means keeping some static information about the world, is prone
|
||||||
to concurrency problems or race-conditions. Sometimes, between the initialization of
|
to concurrency problems or race-conditions. Sometimes, between the initialization of
|
||||||
the state of an object (usually done with the __init__() method) and the actual use
|
the state of an object (usually done with the __init__() method) and the actual use
|
||||||
of the object state through one of its methods, the world may have changed, and
|
of the object state through one of its methods, the world may have changed, and
|
||||||
the retained state may be outdated. For example, a request may load an item in
|
the retained state may be outdated. For example, a request may load an item in
|
||||||
memory and mark it as read by a user. If another request requires the deletion
|
memory and mark it as read by a user. If another request requires the deletion
|
||||||
of this item at the same, it may happen that the deletion actually occurs after
|
of this item at the same, it may happen that the deletion actually occurs after
|
||||||
the first process loaded the item, and then we have to mark as read a deleted
|
the first process loaded the item, and then we have to mark as read a deleted
|
||||||
object.
|
object.
|
||||||
|
|
||||||
This and other issues led to the idea that using stateless functions is a
|
This and other issues led to the idea that using stateless functions is a
|
||||||
better programming paradigm.
|
better programming paradigm.
|
||||||
|
|
||||||
Another way to say the same thing is to suggest using functions and procedures
|
Another way to say the same thing is to suggest using functions and procedures
|
||||||
with as few implicit contexts and side-effects as possible. A function's
|
with as few implicit contexts and side-effects as possible. A function's
|
||||||
implicit context is made up of any of the global variables or items in the persistence layer
|
implicit context is made up of any of the global variables or items in the persistence layer
|
||||||
that are accessed from within the function. Side-effects are the changes that a function makes
|
that are accessed from within the function. Side-effects are the changes that a function makes
|
||||||
to its implicit context. If a function saves or deletes data in a global variable or
|
to its implicit context. If a function saves or deletes data in a global variable or
|
||||||
in the persistence layer, it is said to have a side-effect.
|
in the persistence layer, it is said to have a side-effect.
|
||||||
|
|
||||||
Carefully isolating functions with context and side-effects from functions with
|
Carefully isolating functions with context and side-effects from functions with
|
||||||
logic (called pure functions) allow the following benefits:
|
logic (called pure functions) allow the following benefits:
|
||||||
|
|
||||||
- Pure functions are deterministic: given a fixed input,
|
- Pure functions are deterministic: given a fixed input,
|
||||||
the output will always be the same.
|
the output will always be the same.
|
||||||
|
|
||||||
- Pure functions are much easier to change or replace if they need to
|
- Pure functions are much easier to change or replace if they need to
|
||||||
be refactored or optimized.
|
be refactored or optimized.
|
||||||
|
|
||||||
- Pure functions are easier to test with unit-tests: There is less
|
- Pure functions are easier to test with unit-tests: There is less
|
||||||
need for complex context setup and data cleaning afterwards.
|
need for complex context setup and data cleaning afterwards.
|
||||||
|
|
||||||
- Pure functions are easier to manipulate, decorate, and pass-around.
|
- Pure functions are easier to manipulate, decorate, and pass-around.
|
||||||
|
|
||||||
In summary, pure functions, without any context or side-effects, are more
|
In summary, pure functions, without any context or side-effects, are more
|
||||||
efficient building blocks than classes and objects for some architectures.
|
efficient building blocks than classes and objects for some architectures.
|
||||||
|
|
||||||
Obviously, object-orientation is useful and even necessary in many cases, for
|
Obviously, object-orientation is useful and even necessary in many cases, for
|
||||||
example when developing graphical desktop applications or games, where the
|
example when developing graphical desktop applications or games, where the
|
||||||
things that are manipulated (windows, buttons, avatars, vehicles) have a
|
things that are manipulated (windows, buttons, avatars, vehicles) have a
|
||||||
relatively long life of their own in the computer's memory.
|
relatively long life of their own in the computer's memory.
|
||||||
|
|
||||||
|
|
||||||
Decorators
|
Decorators
|
||||||
----------
|
----------
|
||||||
|
|
||||||
The Python language provides a simple yet powerful syntax called 'decorators'.
|
The Python language provides a simple yet powerful syntax called 'decorators'.
|
||||||
A decorator is a function or a class that wraps (or decorate) a function
|
A decorator is a function or a class that wraps (or decorate) a function
|
||||||
or a method. The 'decorated' function or method will replace the original
|
or a method. The 'decorated' function or method will replace the original
|
||||||
'undecorated' function or method. Because functions are first-class objects
|
'undecorated' function or method. Because functions are first-class objects
|
||||||
in Python, it can be done 'manually', but using the @decorator syntax is
|
in Python, it can be done 'manually', but using the @decorator syntax is
|
||||||
clearer and thus preferred.
|
clearer and thus preferred.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
def foo():
|
def foo():
|
||||||
# do something
|
# do something
|
||||||
|
|
||||||
def decorator(func):
|
def decorator(func):
|
||||||
# manipulate func
|
# manipulate func
|
||||||
return func
|
return func
|
||||||
|
|
||||||
foo = decorator(foo) # Manually decorate
|
foo = decorator(foo) # Manually decorate
|
||||||
|
|
||||||
@decorator
|
@decorator
|
||||||
def bar():
|
def bar():
|
||||||
# Do something
|
# Do something
|
||||||
# bar() is decorated
|
# bar() is decorated
|
||||||
|
|
||||||
This mechanism is useful for separating concerns and avoiding
|
This mechanism is useful for separating concerns and avoiding
|
||||||
external un-related logic 'polluting' the core logic of the function
|
external un-related logic 'polluting' the core logic of the function
|
||||||
or method. A good example of a piece of functionality that is better handled
|
or method. A good example of a piece of functionality that is better handled
|
||||||
with decoration is memoization or caching: you want to store the results of an
|
with decoration is memoization or caching: you want to store the results of an
|
||||||
expensive function in a table and use them directly instead of recomputing
|
expensive function in a table and use them directly instead of recomputing
|
||||||
them when they have already been computed. This is clearly not part
|
them when they have already been computed. This is clearly not part
|
||||||
of the function logic.
|
of the function logic.
|
||||||
|
|
||||||
Dynamic typing
|
Dynamic typing
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
Python is said to be dynamically typed, which means that variables
|
Python is said to be dynamically typed, which means that variables
|
||||||
do not have a fixed type. In fact, in Python, variables are very
|
do not have a fixed type. In fact, in Python, variables are very
|
||||||
different from what they are in many other languages, specifically
|
different from what they are in many other languages, specifically
|
||||||
strongly-typed languages. Variables are not a segment of the computer's
|
strongly-typed languages. Variables are not a segment of the computer's
|
||||||
memory where some value is written, they are 'tags' or 'names' pointing
|
memory where some value is written, they are 'tags' or 'names' pointing
|
||||||
to objects. It is therefore possible for the variable 'a' to be set to
|
to objects. It is therefore possible for the variable 'a' to be set to
|
||||||
the value 1, then to the value 'a string', then to a function.
|
the value 1, then to the value 'a string', then to a function.
|
||||||
|
|
||||||
The dynamic typing of Python is often considered to be a weakness, and indeed
|
The dynamic typing of Python is often considered to be a weakness, and indeed
|
||||||
it can lead to complexities and hard-to-debug code. Something
|
it can lead to complexities and hard-to-debug code. Something
|
||||||
named 'a' can be set to many different things, and the developer or the
|
named 'a' can be set to many different things, and the developer or the
|
||||||
maintainer needs to track this name in the code to make sure it has not
|
maintainer needs to track this name in the code to make sure it has not
|
||||||
been set to a completely unrelated object.
|
been set to a completely unrelated object.
|
||||||
|
|
||||||
Some guidelines help to avoid this issue:
|
Some guidelines help to avoid this issue:
|
||||||
|
|
||||||
- Avoid using variables for different things.
|
- Avoid using the same variable name for different things.
|
||||||
|
|
||||||
**Bad**
|
**Bad**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
a = 1
|
a = 1
|
||||||
a = 'a string'
|
a = 'a string'
|
||||||
def a():
|
def a():
|
||||||
pass # Do something
|
pass # Do something
|
||||||
|
|
||||||
**Good**
|
**Good**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
count = 1
|
count = 1
|
||||||
msg = 'a string'
|
msg = 'a string'
|
||||||
def func():
|
def func():
|
||||||
pass # Do something
|
pass # Do something
|
||||||
|
|
||||||
Using short functions or methods helps reduce the risk
|
Using short functions or methods helps reduce the risk
|
||||||
of using the same name for two unrelated things.
|
of using the same name for two unrelated things.
|
||||||
|
|
||||||
It is better to use different names even for things that are related,
|
It is better to use different names even for things that are related,
|
||||||
when they have a different type:
|
when they have a different type:
|
||||||
|
|
||||||
**Bad**
|
**Bad**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
items = 'a b c d' # This is a string...
|
items = 'a b c d' # This is a string...
|
||||||
items = items.split(' ') # ...becoming a list
|
items = items.split(' ') # ...becoming a list
|
||||||
items = set(items) # ...and then a set
|
items = set(items) # ...and then a set
|
||||||
|
|
||||||
There is no efficiency gain when reusing names: the assignments
|
There is no efficiency gain when reusing names: the assignments
|
||||||
will have to create new objects anyway. However, when the complexity
|
will have to create new objects anyway. However, when the complexity
|
||||||
grows and each assignment is separated by other lines of code, including
|
grows and each assignment is separated by other lines of code, including
|
||||||
'if' branches and loops, it becomes harder to ascertain what a given
|
'if' branches and loops, it becomes harder to ascertain what a given
|
||||||
variable's type is.
|
variable's type is.
|
||||||
|
|
||||||
Some coding practices, like functional programming, recommend never reassigning a variable.
|
Some coding practices, like functional programming, recommend never reassigning a variable.
|
||||||
In Java this is done with the `final` keyword. Python does not have a `final` keyword
|
In Java this is done with the `final` keyword. Python does not have a `final` keyword
|
||||||
and it would be against its philosophy anyway. However, it may be a good
|
and it would be against its philosophy anyway. However, it may be a good
|
||||||
discipline to avoid assigning to a variable more than once, and it helps
|
discipline to avoid assigning to a variable more than once, and it helps
|
||||||
in grasping the concept of mutable and immutable types.
|
in grasping the concept of mutable and immutable types.
|
||||||
|
|
||||||
Mutable and immutable types
|
Mutable and immutable types
|
||||||
---------------------------
|
---------------------------
|
||||||
|
|
||||||
Python has two kinds of built-in or user-defined types.
|
Python has two kinds of built-in or user-defined types.
|
||||||
|
|
||||||
Mutable types are those that allow in-place modification
|
Mutable types are those that allow in-place modification
|
||||||
of the content. Typical mutables are lists and dictionaries:
|
of the content. Typical mutables are lists and dictionaries:
|
||||||
All lists have mutating methods, like append() or pop(), and
|
All lists have mutating methods, like append() or pop(), and
|
||||||
can be modified in place. The same goes for dictionaries.
|
can be modified in place. The same goes for dictionaries.
|
||||||
|
|
||||||
Immutable types provide no method for changing their content.
|
Immutable types provide no method for changing their content.
|
||||||
For instance, the variable x set to the integer 6 has no "increment" method. If you
|
For instance, the variable x set to the integer 6 has no "increment" method. If you
|
||||||
want to compute x + 1, you have to create another integer and give it
|
want to compute x + 1, you have to create another integer and give it
|
||||||
a name.
|
a name.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
my_list = [1, 2, 3]
|
my_list = [1, 2, 3]
|
||||||
my_list[0] = 4
|
my_list[0] = 4
|
||||||
print my_list # [4, 2, 3] <- The same list as changed
|
print my_list # [4, 2, 3] <- The same list as changed
|
||||||
|
|
||||||
x = 6
|
x = 6
|
||||||
x = x + 1 # The new x is another object
|
x = x + 1 # The new x is another object
|
||||||
|
|
||||||
One consequence of this difference in behavior is that mutable
|
One consequence of this difference in behavior is that mutable
|
||||||
types are not "stable", and therefore cannot be used as dictionary
|
types are not "stable", and therefore cannot be used as dictionary
|
||||||
keys.
|
keys.
|
||||||
|
|
||||||
Using properly mutable types for things that are mutable in nature
|
Using properly mutable types for things that are mutable in nature
|
||||||
and immutable types for things that are fixed in nature
|
and immutable types for things that are fixed in nature
|
||||||
helps to clarify the intent of the code.
|
helps to clarify the intent of the code.
|
||||||
|
|
||||||
For example, the immutable equivalent of a list is the tuple, created
|
For example, the immutable equivalent of a list is the tuple, created
|
||||||
with ``(1, 2)``. This tuple is a pair that cannot be changed in-place,
|
with ``(1, 2)``. This tuple is a pair that cannot be changed in-place,
|
||||||
and can be used as a key for a dictionary.
|
and can be used as a key for a dictionary.
|
||||||
|
|
||||||
One peculiarity of Python that can surprise beginners is that
|
One peculiarity of Python that can surprise beginners is that
|
||||||
strings are immutable. This means that when constructing a string from
|
strings are immutable. This means that when constructing a string from
|
||||||
its parts, it is much more efficient to accumulate the parts in a list,
|
its parts, it is much more efficient to accumulate the parts in a list,
|
||||||
which is mutable, and then glue ('join') the parts together when the
|
which is mutable, and then glue ('join') the parts together when the
|
||||||
full string is needed. One thing to notice, however, is that list
|
full string is needed. One thing to notice, however, is that list
|
||||||
comprehensions are better and faster than constructing a list in a loop
|
comprehensions are better and faster than constructing a list in a loop
|
||||||
with calls to append().
|
with calls to append().
|
||||||
|
|
||||||
**Bad**
|
**Bad**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
||||||
nums = ""
|
nums = ""
|
||||||
for n in range(20):
|
for n in range(20):
|
||||||
nums += str(n) # slow and inefficient
|
nums += str(n) # slow and inefficient
|
||||||
print nums
|
print nums
|
||||||
|
|
||||||
**Good**
|
**Good**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
||||||
nums = []
|
nums = []
|
||||||
for n in range(20):
|
for n in range(20):
|
||||||
nums.append(str(n))
|
nums.append(str(n))
|
||||||
print "".join(nums) # much more efficient
|
print "".join(nums) # much more efficient
|
||||||
|
|
||||||
**Best**
|
**Best**
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
# create a concatenated string from 0 to 19 (e.g. "012..1819")
|
||||||
print "".join([str(n) for n in range(20)])
|
print "".join([str(n) for n in range(20)])
|
||||||
|
|
||||||
One final thing to mention about strings is that using join() is not always
|
One final thing to mention about strings is that using join() is not always
|
||||||
best. In the instances where you are creating a new string from a pre-determined
|
best. In the instances where you are creating a new string from a pre-determined
|
||||||
number of strings, using the addition operator is actually faster, but in cases
|
number of strings, using the addition operator is actually faster, but in cases
|
||||||
like above or in cases where you are adding to an existing string, using join()
|
like above or in cases where you are adding to an existing string, using join()
|
||||||
should be your preferred method.
|
should be your preferred method.
|
||||||
|
|
||||||
.. code-block:: python
|
.. code-block:: python
|
||||||
|
|
||||||
foo = 'foo'
|
foo = 'foo'
|
||||||
bar = 'bar'
|
bar = 'bar'
|
||||||
|
|
||||||
foobar = foo + bar # This is good
|
foobar = foo + bar # This is good
|
||||||
foo += 'ooo' # This is bad, instead you should do:
|
foo += 'ooo' # This is bad, instead you should do:
|
||||||
foo = ''.join([foo, 'ooo'])
|
foo = ''.join([foo, 'ooo'])
|
||||||
|
|
||||||
Vendorizing Dependencies
|
Vendorizing Dependencies
|
||||||
------------------------
|
------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Runners
|
Runners
|
||||||
-------
|
-------
|
||||||
|
|
||||||
|
|
||||||
Further Reading
|
Further Reading
|
||||||
---------------
|
---------------
|
||||||
|
|||||||
Reference in New Issue
Block a user