mirror of
https://github.com/kennethreitz/python-guide.git
synced 2026-06-05 23:00:18 +00:00
Merge pull request #146 from GloryFish/structuring-edits
Structuring edits
This commit is contained in:
Kenneth Reitz
+101
-102
@@ -8,49 +8,49 @@ Structuring your project properly is extremely important.
|
||||
Structure is Key
|
||||
----------------
|
||||
|
||||
Thanks to the way imports and module are handled in Python, it is
|
||||
Thanks to the way imports and modules are handled in Python, it is
|
||||
relatively easy to structure a python project. Easy, here, means
|
||||
actually that you have not many constraints and that the module
|
||||
importing model is easy grasp. Therefore, you are left with the
|
||||
pure architectural task of drawing the different parts of your
|
||||
that you do not have many constraints and that the module
|
||||
importing model is easy to grasp. Therefore, you are left with the
|
||||
pure architectural task of crafting the different parts of your
|
||||
project and their interactions.
|
||||
|
||||
Easy structuration of a project means it is also easy
|
||||
to do it poorly. Some signs of a poorly structured projects
|
||||
Easy structuring of a project means it is also easy
|
||||
to do it poorly. Some signs of a poorly structured project
|
||||
include:
|
||||
|
||||
- Multiple and messy circular dependencies: if your classes
|
||||
- Multiple and messy circular dependencies: If your classes
|
||||
Table and Chair in furn.py need to import Carpenter from workers.py
|
||||
to answer to a question such as table.isdoneby(),
|
||||
and if convertly the class Carpenter need to import Table and Chair,
|
||||
for example to answer to carpenter.whatdo(), then you
|
||||
have a circular dependency, and will have to resort to
|
||||
to answer a question such as table.isdoneby(),
|
||||
and if conversely the class Carpenter needs to import Table and Chair,
|
||||
to answer the question carpenter.whatdo(), then you
|
||||
have a circular dependency. In this case you will have to resort to
|
||||
fragile hacks such has using import statements inside
|
||||
methods or functions.
|
||||
|
||||
- Hidden coupling. Each and every change in Table implementation
|
||||
- Hidden coupling: Each and every change in Table's implementation
|
||||
breaks 20 tests in unrelated test cases because it breaks Carpenter's code,
|
||||
which requires very careful surgery to adapt the change. This means
|
||||
you have too many assumptions about Table in Carpenter's code or the
|
||||
reverse.
|
||||
|
||||
- Heavy usage of global state or context: Instead of explicitely
|
||||
- Heavy usage of global state or context: Instead of explicitly
|
||||
passing ``(height, width, type, wood)`` to each other, Table
|
||||
and Carpenter rely on global variables that can be modified
|
||||
and are modified on the fly by different agent. You need to
|
||||
scrutinize all access to this global variables to understand why
|
||||
a rectangular table became a sqaure, and discover that a remote
|
||||
and are modified on the fly by different agents. You need to
|
||||
scrutinize all access to these global variables to understand why
|
||||
a rectangular table became a square, and discover that remote
|
||||
template code is also modifying this context, messing with
|
||||
table dimensions.
|
||||
|
||||
- Spaghetti code: Multiple pages of nested if clauses and for loops
|
||||
with a lot of copy-pasted procedural code and no
|
||||
proper segmentation are known as spaghetti code. Python's
|
||||
meaningful indentation (one of its most controversial feature) make
|
||||
meaningful indentation (one of its most controversial features) make
|
||||
it very hard to maintain this kind of code. So the good news is that
|
||||
you might not see too much of it.
|
||||
|
||||
- Ravioli code is more likely in Python: it consists of hundreds of
|
||||
- Ravioli code is more likely in Python: It consists of hundreds of
|
||||
similar little pieces of logic, often classes or objects, without
|
||||
proper structure. If you never can remember if you have to use
|
||||
FurnitureTable, AssetTable or Table, or even TableNew for your
|
||||
@@ -60,53 +60,53 @@ include:
|
||||
Modules
|
||||
-------
|
||||
|
||||
Python modules are one of the main abstraction layer available and probably the
|
||||
Python modules are one of the main abstraction layers available and probably the
|
||||
most natural one. Abstraction layers allow separating code into parts holding
|
||||
related data and functionalities.
|
||||
related data and functionality.
|
||||
|
||||
For example, a layer of a project can handle interfacing with user actions,
|
||||
while another would handle low-level manipulation of data. The most natural way
|
||||
to separate these two layers is to regroup all interfacing functionalities
|
||||
to separate these two layers is to regroup all interfacing functionality
|
||||
in one file, and all low-level operations in another file. In this case,
|
||||
the interface file need to import the low-level file. This is done with the
|
||||
the interface file needs to import the low-level file. This is done with the
|
||||
`import` and `from ... import` statements.
|
||||
|
||||
As soon as you use `import` statements you use modules, either builtin modules
|
||||
such as `os` and `sys`, or third-party modules you have installed in your
|
||||
environment, or project's internal modules.
|
||||
As soon as you use `import` statements you use modules. These can be either built-in
|
||||
modules such as `os` and `sys`, third-party modules you have installed in your
|
||||
environment, or your project's internal modules.
|
||||
|
||||
Nothing special is required for a Python file to be a module, but the import
|
||||
mechanism need to be understood in order to use this concept properly and avoid
|
||||
mechanism needs to be understood in order to use this concept properly and avoid
|
||||
some issues.
|
||||
|
||||
Concretely, the `import modu` statement will look for the proper file, which is
|
||||
`modu.py` in the same directory as the caller if it exists. If it is not
|
||||
found, the Python interpreter with search for `modu.py` in the "path"
|
||||
found, the Python interpreter will search for `modu.py` in the "path"
|
||||
recursively and raise an ImportError exception if it is not found.
|
||||
|
||||
Once `modu.py` is found, the Python interpreter will execute the module in an
|
||||
isolated scope. Any top-level statement in `modu.py` will be executed,
|
||||
including other imports if any. Function and classes definitions are stored in
|
||||
including other imports if any. Function and class definitions are stored in
|
||||
the module's dictionary.
|
||||
|
||||
Then modules variables, functions and classes will be available to the caller
|
||||
Then, the module's variables, functions, and classes will be available to the caller
|
||||
through the module's namespace, a central concept in programming that is
|
||||
particularly helpful and powerful in Python.
|
||||
|
||||
In many languages, a `include file` directive is used by the preprocessor to
|
||||
take all code found in the file and 'copy' it in the caller's code. It is
|
||||
In many languages, an `include file` directive is used by the preprocessor to
|
||||
take all code found in the file and 'copy' it into the caller's code. It is
|
||||
different in Python: the included code is isolated in a module namespace, which
|
||||
means that you generally don't have to worry that the included code could have
|
||||
unwanted effect, eg override an existing function with the same name.
|
||||
unwanted effects, e.g. override an existing function with the same name.
|
||||
|
||||
It is possible to simulate the more standard behavior by using a special syntax
|
||||
of the import statement: `from modu import *`. This is generally considered bad
|
||||
practice, **using import * makes code harder to read and dependencies less
|
||||
compartimented**.
|
||||
practice. **Using `import *` makes code harder to read and makes dependencies less
|
||||
compartmentalized**.
|
||||
|
||||
Using `from modu import func` is a way to pinpoint the function you want to
|
||||
import and put it is the global namespace. While much less harmful than `import
|
||||
*` because it shows explicitely what is imported in the global namespace, it's
|
||||
import and put it in the global namespace. While much less harmful than `import
|
||||
*` because it shows explicitly what is imported in the global namespace, its
|
||||
advantage over a simpler `import modu` is only that it will save some typing.
|
||||
|
||||
**Very bad**
|
||||
@@ -134,13 +134,13 @@ advantage over a simpler `import modu` is only that it will save some typing.
|
||||
[...]
|
||||
x = modu.sqrt(4) # sqrt is visibly part of modu's namespace
|
||||
|
||||
As said in the section about style, readability is one of the main feature of
|
||||
As said in the section about style, readability is one of the main features of
|
||||
Python. Readability means to avoid useless boilerplate text and clutter,
|
||||
therefore some efforts are spent trying to achieve a certain level of brevity.
|
||||
But terseness and obscurity are the limits where brevity should stop: being
|
||||
able to tell immediately from where comes a class or a function, as in the
|
||||
`modu.func` idiom, improves greatly code readability and understandability in
|
||||
most cases but the simplest single file projects.
|
||||
But terseness and obscurity are the limits where brevity should stop. Being
|
||||
able to tell immediately where a class or function comes from, as in the
|
||||
`modu.func` idiom, greatly improves code readability and understandability in
|
||||
all but the simplest single file projects.
|
||||
|
||||
|
||||
Packages
|
||||
@@ -149,35 +149,35 @@ Packages
|
||||
Python provides a very straightforward packaging system, which is simply an
|
||||
extension of the module mechanism to a directory.
|
||||
|
||||
Any directory with a __init__.py file is considered a Python package. The
|
||||
Any directory with an __init__.py file is considered a Python package. The
|
||||
different modules in the package are imported in a similar manner as plain
|
||||
modules, will a special behavior for the __init__.py file, that is used to
|
||||
modules, but with a special behavior for the __init__.py file, which is used to
|
||||
gather all package-wide definitions.
|
||||
|
||||
A file modu.py in the directory pack/ is imported with the statement `import
|
||||
pack.modu`. This statement will look for a __init__.py file in `pack`, execute
|
||||
all its top-level statements. Then it will look for a file `pack/modu.py` and
|
||||
execute all its top-level statements. After these operations, any variable,
|
||||
function or class defined in modu.py is available in pack.modu namespace.
|
||||
pack.modu`. This statement will look for an __init__.py file in `pack`, execute
|
||||
all of its top-level statements. Then it will look for a file `pack/modu.py` and
|
||||
execute all of its top-level statements. After these operations, any variable,
|
||||
function, or class defined in modu.py is available in the pack.modu namespace.
|
||||
|
||||
A commonly seen issue is to add too many code and functions in __init__.py
|
||||
A commonly seen issue is to add too much code to __init__.py
|
||||
files. When the project complexity grows, there may be sub-packages and
|
||||
sub-sub-packages in a deep directory structure, and then, import a single item
|
||||
from a sub-sub-package will require to execute all __init__.py file met while
|
||||
descending the tree.
|
||||
sub-sub-packages in a deep directory structure, and then, importing a single item
|
||||
from a sub-sub-package will require executing all __init__.py files met while
|
||||
traversing the tree.
|
||||
|
||||
Leaving a __init__.py file empty is considered normal and even a good pratice,
|
||||
Leaving an __init__.py file empty is considered normal and even a good practice,
|
||||
if the package's modules and sub-packages do not need to share any code.
|
||||
|
||||
Lastly, a convenient syntax is available for importing deeply nested packages:
|
||||
`import very.deep.module as mod` allow to use `mod` in place of the verbose
|
||||
repetition of `very.deep.module` in front of each calls to module items.
|
||||
`import very.deep.module as mod`. This allows you to use `mod` in place of the verbose
|
||||
repetition of `very.deep.module`.
|
||||
|
||||
Object-oriented programming
|
||||
---------------------------
|
||||
|
||||
Python is sometime described as an object-oriented programming language. This
|
||||
can be somewhat misleading and need to be clarified.
|
||||
Python is sometimes described as an object-oriented programming language. This
|
||||
can be somewhat misleading and needs to be clarified.
|
||||
|
||||
In Python, everything is an object, and can be handled as such. This is what is
|
||||
meant when we say that, for example, functions are first-class objects.
|
||||
@@ -188,46 +188,46 @@ object-oriented language.
|
||||
|
||||
However, unlike Java, Python do not impose object-oriented programming as the
|
||||
main programming paradigm. It is perfectly viable for a Python project to not
|
||||
be object-oriented, ie. to use no or very few class definitions, class
|
||||
inheritance, and any other mechanism that are specific to object-oriented
|
||||
be object-oriented, i.e. to use no or very few class definitions, class
|
||||
inheritance, or any other mechanisms that are specific to object-oriented
|
||||
programming.
|
||||
|
||||
Moreover, as seen in the modules_ section, the way Python handles modules and
|
||||
namespaces gives directly to the developer a natural way to ensure
|
||||
namespaces gives the developer a natural way to ensure
|
||||
encapsulation and separation of abstraction layers, both being the most common
|
||||
reasons to use object-orientation. Therefore, Python programmers have more
|
||||
latitude to not use object-orientation, when it is not required by the business
|
||||
model to be constructed.
|
||||
model.
|
||||
|
||||
There are some reasons to avoid unnecessary object-orientation. Definining
|
||||
There are some reasons to avoid unnecessary object-orientation. Defining
|
||||
custom classes is useful when we want to glue together some state and some
|
||||
functionality. The problem, as pointed out by the discussions about functional
|
||||
functionality. The problem, as pointed out by the discussions about functional
|
||||
programming, comes from the "state" part of the equation.
|
||||
|
||||
In some architectures, typically web applications, instances of Python
|
||||
processes are spawned simultaneously to answer to external requests that can
|
||||
happen at the same time. In this case, holding some state into instanciated
|
||||
In some architectures, typically web applications, multiple instances of Python
|
||||
processes are spawned to respond to external requests that can
|
||||
happen at the same time. In this case, holding some state into instantiated
|
||||
objects, which means keeping some static information about the world, is prone
|
||||
to concurrency problems or race-conditions: between the initialization of the
|
||||
to concurrency problems or race-conditions. Sometime between the initialization of the
|
||||
state of an object, usually done with the __init__() method, and the actual use
|
||||
of the object state through one of its method, the world may have changed, and
|
||||
of the object state through one of its methods, the world may have changed, and
|
||||
the retained state may be outdated. For example, a request may load an item in
|
||||
memory and mark it as read by a user. If another request requires the deletion
|
||||
of this item at the same, it may happen that the deletion actually occur after
|
||||
of this item at the same, it may happen that the deletion actually occurs after
|
||||
the first process loaded the item, and then we have to mark as read a deleted
|
||||
object.
|
||||
|
||||
This and other issues led to the idea that using stateless functions is a
|
||||
better programming paradigm.
|
||||
|
||||
Another way to say the same thing is to propose to use functions and procedures
|
||||
with as few implicit context and side-effects as possible. A function's
|
||||
implicit context is decelable when the function body refers to some global
|
||||
variables or fetches data from the persistence layer. Side-effects are the
|
||||
opposite: if a function body modifies the global context or save or delete data
|
||||
on the persistence layer, it is said to have side-effect.
|
||||
Another way to say the same thing is to suggest using functions and procedures
|
||||
with as few implicit contexts and side-effects as possible. A function's
|
||||
implicit context is made up of any of the global variables or items in the persistence layer
|
||||
that are accessed from within the function. Side-effects are the changes that a function makes
|
||||
to it's implicit context. If a function saves or deletes data in a global variable or
|
||||
in the persistence layer, it is said to have a side-effect.
|
||||
|
||||
Isolating carefully functions with context and side-effects from functions with
|
||||
Carefully isolating functions with context and side-effects from functions with
|
||||
logic (called pure functions) allow the following benefits:
|
||||
|
||||
- Pure functions are more likely to be deterministic: given a fixed input,
|
||||
@@ -239,7 +239,7 @@ logic (called pure functions) allow the following benefits:
|
||||
- Pure functions are easier to test with unit-tests: There is less
|
||||
need for complex context setup and data cleaning afterwards.
|
||||
|
||||
- Pure functions are easier to manipulate, decorate_, pass-around.
|
||||
- Pure functions are easier to manipulate, decorate_, and pass-around.
|
||||
|
||||
In summary, pure functions, without any context or side-effects, are more
|
||||
efficient building blocks than classes and objects for some architectures.
|
||||
@@ -253,12 +253,12 @@ relatively long life of their own in the computer's memory.
|
||||
Decorators
|
||||
----------
|
||||
|
||||
Python language provides a simple yet powerful syntax called 'decorators'.
|
||||
The Python language provides a simple yet powerful syntax called 'decorators'.
|
||||
A decorator is a function or a class that wraps (or decorate) a function
|
||||
or a method. The 'decorated' function or method will replace the original
|
||||
'undecorated' function or method. Because function are first-class objects
|
||||
'undecorated' function or method. Because functions are first-class objects
|
||||
in Python it can be done 'manually' but using the @decorator syntax is
|
||||
clearer and thus prefered.
|
||||
clearer and thus preferred.
|
||||
|
||||
.. code-block:: Python
|
||||
|
||||
@@ -277,8 +277,8 @@ clearer and thus prefered.
|
||||
# bar() is decorated
|
||||
|
||||
Using this mechanism is useful for separating concerns and avoiding
|
||||
external un-related logic to 'pollute' the core logic of the function
|
||||
or method. A good example of a functionality that is better handled
|
||||
external un-related logic 'polluting' the core logic of the function
|
||||
or method. A good example of a piece of functionality that is better handled
|
||||
with decoration is memoization or caching: you want to store the results of an
|
||||
expensive function in a table and use them directly instead of recomputing
|
||||
them when they have already been computed. This is clearly not part
|
||||
@@ -290,18 +290,18 @@ Dynamic typing
|
||||
Python is said to be dynamically typed, which means that variables
|
||||
do not have a fixed type. In fact, in Python, variables are very
|
||||
different from what they are in many other languages, specifically
|
||||
strongly-typed languages: variables are not a segment of the computer's
|
||||
memory where some value ir written, they are 'tags' or 'names' pointing
|
||||
strongly-typed languages. Variables are not a segment of the computer's
|
||||
memory where some value is written, they are 'tags' or 'names' pointing
|
||||
to objects. It is therefore possible for the variable 'a' to be set to
|
||||
the value 1, then to the value 'a string', then to a function.
|
||||
|
||||
The dynanic typing of Python is often considered as a weakness, and indeed
|
||||
it can lead to complexities and to hard-to-debug code, where something
|
||||
The dynamic typing of Python is often considered to be a weakness, and indeed
|
||||
it can lead to complexities and hard-to-debug code. Something
|
||||
named 'a' can be set to many different things, and the developer or the
|
||||
maintainer need to track this name in the code to make sure it has not
|
||||
maintainer needs to track this name in the code to make sure it has not
|
||||
been set to a completely unrelated object.
|
||||
|
||||
Some guidelines allow to avoid this issue:
|
||||
Some guidelines help to avoid this issue:
|
||||
|
||||
- Avoid using variables for different things.
|
||||
|
||||
@@ -323,9 +323,8 @@ Some guidelines allow to avoid this issue:
|
||||
def func()
|
||||
pass # Do something
|
||||
|
||||
Using short functions or methods helps writing good code for many
|
||||
reasons, one being that their local scope is clearer, and the risk
|
||||
of using the same name for two unrelated things is lowered.
|
||||
Using short functions or methods helps reduce the risk
|
||||
of using the same name for two unrelated things.
|
||||
|
||||
It is better to use different names even for things that are related,
|
||||
when they have a different type:
|
||||
@@ -340,14 +339,14 @@ when they have a different type:
|
||||
|
||||
There is no efficiency gain when reusing names: the assignments
|
||||
will have to create new objects anyway. However, when the complexity
|
||||
grows are each assignment are separated by other lines of code, including
|
||||
'if' branches and loops, it becomes harder to acertain which type is the
|
||||
variable at hand.
|
||||
grows and each assignment is separated by other lines of code, including
|
||||
'if' branches and loops, it becomes harder to ascertain what a given
|
||||
variable's type is.
|
||||
|
||||
Some coding practices, like functional programming, even recommend to never re-assign a variable, which
|
||||
is done in Java with the keyword final. Python do not have such a keyword,
|
||||
and it would be against its philosophy anyway, but it may be a good
|
||||
discipline to avoid setting more than once any variable, and it helps
|
||||
Some coding practices, like functional programming, recommend never reassigning a variable.
|
||||
In Java this is done with the `final` keyword. Python does not have a `final` keyword
|
||||
and it would be against its philosophy anyway. However, it may be a good
|
||||
discipline to avoid assigning to a variable more than once, and it helps
|
||||
in grasping the concept of mutable and immutable types.
|
||||
|
||||
Mutable and immutable types
|
||||
@@ -357,12 +356,12 @@ Python has two kinds of built-in or user-defined types.
|
||||
|
||||
Mutable types are those that allow in-place modification
|
||||
of the content. Typical mutables are lists and dictionaries:
|
||||
All lists have muting methods, like append() or pop(), and
|
||||
can be modified in place. Same for dictionaries.
|
||||
All lists have mutating methods, like append() or pop(), and
|
||||
can be modified in place. The same goes for dictionaries.
|
||||
|
||||
Immutable types provide no method for changing their content.
|
||||
For instance, the variable x set to the integer 6 has no "increment" method. If you
|
||||
want to computed x + 1, you have to create another integer and give it
|
||||
want to compute x + 1, you have to create another integer and give it
|
||||
a name.
|
||||
|
||||
.. code-block:: python
|
||||
@@ -386,8 +385,8 @@ For example, the immutable equivalent of a list is the tuple, created
|
||||
with ``(1, 2)``. This tuple is a pair that cannot be changed in-place,
|
||||
and can be used as a key for a dictionary.
|
||||
|
||||
One particularity of Python that can surprise in the beginning is that
|
||||
string are immutable. This means that when constructing a string from
|
||||
One peculiarity of Python that can surprise beginners is that
|
||||
strings are immutable. This means that when constructing a string from
|
||||
its parts, it is much more efficient to accumulate the parts in a list,
|
||||
which is mutable, and then glue ('join') the parts together when the
|
||||
full string is needed.
|
||||
|
||||
Reference in New Issue
Block a user