Merge remote-tracking branch 'upstream/master'

This commit is contained in:
Adam Nelson
2016-11-14 14:25:47 -05:00
35 changed files with 638 additions and 104 deletions
+22
View File
@@ -6,6 +6,28 @@ see:
http://docs.python-guide.org/en/latest/notes/contribute/ http://docs.python-guide.org/en/latest/notes/contribute/
How to test your changes
------------------------
The html version of this guide is built with [sphinx](http://www.sphinx-doc.org/en/stable/). You may test your revisions locally by having sphinx installed. You can do this easily with pip (as described in the link).
``` bash
pip install --user sphinx
```
Then navigate to the directory of the makefile and ```make build``` or ```make html```. Sphinx will then generate the html in a folder called _build/html
After navigating to this folder, you can then use python's built in webserver to view your changes locally:
``` bash
python3 -m http.server
```
By default, http.server listens on every ip address bound on your host on port 8000. To bind to a specific one, say, localhost on port 8005:
``` bash
python3 -m http.server 8005 --bind 127.0.0.1
```
Style Guide Style Guide
----------- -----------
+11 -3
View File
@@ -20,6 +20,13 @@
<p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p> <p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p>
<h3>O'Reilly Book</h3>
<p>This guide is now available in tangible book form!</p>
<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
<h3>Other Projects</h3> <h3>Other Projects</h3>
@@ -53,7 +60,8 @@
<h3>Translations</h3> <h3>Translations</h3>
<ul> <ul>
<li><a href="http://docs.python-guide.org/en/latest/">English</a></li> <li><a href="http://docs.python-guide.org/en/latest/">English</a></li>
<li><a href="http://pythonguidecn.readthedocs.org/zh/latest/">Chinese</a></li> <li><a href="https://python-guide-fr.readthedocs.io/fr/latest/">French</a></li>
<li><a href="http://python-guide-ja.readthedocs.org/en/latest/">Japanese</a></li> <li><a href="https://pythonguidecn.readthedocs.io/zh/latest/">Chinese</a></li>
<li><a href="http://python-guide-kr.readthedocs.org/ko/latest/">Korean</a></li> <li><a href="https://python-guide-ja.readthedocs.io/en/latest/">Japanese</a></li>
<li><a href="https://python-guide-kr.readthedocs.io/ko/latest/">Korean</a></li>
</ul> </ul>
+10 -3
View File
@@ -19,7 +19,13 @@
<p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p> <p><a href="http://tinyletter.com/kennethreitz">Join Mailing List</a>.</p>
<h3>O'Reilly Book</h3>
<p>This guide is now available in tangible book form!</p>
<a href="https://www.amazon.com/Hitchhikers-Guide-Python-Practices-Development/dp/1491933178/ref=as_li_ss_il?ie=UTF8&linkCode=li2&tag=bookforkind-20&linkId=804806ebdacaf3b56567347f3afbdbca" target="_blank"><img border="0" src="https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&ASIN=1491933178&Format=_SL160_&ID=AsinImage&MarketPlace=US&ServiceVersion=20070822&WS=1&tag=bookforkind-20" ></a><img src="//ir-na.amazon-adsystem.com/e/ir?t=bookforkind-20&l=li2&o=1&a=1491933178" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
<p>All proceeds are being directly donated to the <a href="https://djangogirls.org">DjangoGirls</a> organization.</p>
<h3>Other Projects</h3> <h3>Other Projects</h3>
@@ -36,7 +42,8 @@
<h3>Translations</h3> <h3>Translations</h3>
<ul> <ul>
<li><a href="http://docs.python-guide.org/en/latest/">English</a></li> <li><a href="http://docs.python-guide.org/en/latest/">English</a></li>
<li><a href="http://pythonguidecn.readthedocs.org/zh/latest/">Chinese</a></li> <li><a href="https://python-guide-fr.readthedocs.io/fr/latest/">French</a></li>
<li><a href="http://python-guide-ja.readthedocs.org/en/latest/">Japanese</a></li> <li><a href="https://pythonguidecn.readthedocs.io/zh/latest/">Chinese</a></li>
<li><a href="http://python-guide-kr.readthedocs.org/ko/latest/">Korean</a></li> <li><a href="https://python-guide-ja.readthedocs.io/en/latest/">Japanese</a></li>
<li><a href="https://python-guide-kr.readthedocs.io/ko/latest/">Korean</a></li>
</ul> </ul>
+1
View File
@@ -64,6 +64,7 @@ different scenarios.
scenarios/xml scenarios/xml
scenarios/json scenarios/json
scenarios/crypto scenarios/crypto
scenarios/ml
scenarios/clibs scenarios/clibs
+9
View File
@@ -150,6 +150,15 @@ features can be brought to IntelliJ with the free
versions of PyCharm: Professional Edition (Free 30-day trial) and Community versions of PyCharm: Professional Edition (Free 30-day trial) and Community
Edition(Apache 2.0 License) with fewer features. Edition(Apache 2.0 License) with fewer features.
Python (on Visual Studio Code)
-----------------------------
`Python for Visual Studio <https://marketplace.visualstudio.com/items?itemName=donjayamanne.python>`_ is an extension for the `Visual Studio Code IDE <https://code.visualstudio.com>`_.
This is a free, light weight, open source IDE, with support for Mac, Windows, and Linux.
Built using open source technologies such as Node.js and Python, with compelling features such as Intellisense (autocompletion), local and remote debugging, linting, and the like.
MIT licensed.
Enthought Canopy Enthought Canopy
---------------- ----------------
`Enthought Canopy <https://www.enthought.com/products/canopy/>`_ is a Python `Enthought Canopy <https://www.enthought.com/products/canopy/>`_ is a Python
+7 -3
View File
@@ -8,7 +8,11 @@ projects in separate places, by creating virtual Python environments for them.
It solves the "Project X depends on version 1.x but, Project Y needs 4.x" It solves the "Project X depends on version 1.x but, Project Y needs 4.x"
dilemma, and keeps your global site-packages directory clean and manageable. dilemma, and keeps your global site-packages directory clean and manageable.
<<<<<<< HEAD
For example, you can work on a project which requires Django 1.9 while also For example, you can work on a project which requires Django 1.9 while also
=======
For example, you can work on a project which requires Django 1.10 while also
>>>>>>> upstream/master
maintaining a project which requires Django 1.8. maintaining a project which requires Django 1.8.
virtualenv virtualenv
@@ -127,7 +131,7 @@ control by adding it to the ignore list.
virtualenvwrapper virtualenvwrapper
----------------- -----------------
`virtualenvwrapper <http://virtualenvwrapper.readthedocs.org/en/latest/index.html>`_ `virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/index.html>`_
provides a set of commands which makes working with virtual environments much provides a set of commands which makes working with virtual environments much
more pleasant. It also places all your virtual environments in one place. more pleasant. It also places all your virtual environments in one place.
@@ -139,7 +143,7 @@ To install (make sure **virtualenv** is already installed):
$ export WORKON_HOME=~/Envs $ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh $ source /usr/local/bin/virtualenvwrapper.sh
(`Full virtualenvwrapper install instructions <http://virtualenvwrapper.readthedocs.org/en/latest/install.html>`_.) (`Full virtualenvwrapper install instructions <https://virtualenvwrapper.readthedocs.io/en/latest/install.html>`_.)
For Windows, you can use the `virtualenvwrapper-win <https://github.com/davidmarble/virtualenvwrapper-win/>`_. For Windows, you can use the `virtualenvwrapper-win <https://github.com/davidmarble/virtualenvwrapper-win/>`_.
@@ -211,7 +215,7 @@ Other useful commands
``lssitepackages`` ``lssitepackages``
Shows contents of :file:`site-packages` directory. Shows contents of :file:`site-packages` directory.
`Full list of virtualenvwrapper commands <http://virtualenvwrapper.readthedocs.org/en/latest/command_ref.html>`_. `Full list of virtualenvwrapper commands <https://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html>`_.
virtualenv-burrito virtualenv-burrito
------------------ ------------------
+1 -1
View File
@@ -38,7 +38,7 @@ without having to install Python locally.
If you want a more traditional book, *Python For You and Me* is an excellent If you want a more traditional book, *Python For You and Me* is an excellent
resource for learning all aspects of the language. resource for learning all aspects of the language.
`Python for You and Me <http://pymbook.readthedocs.org/>`_ `Python for You and Me <https://pymbook.readthedocs.io/>`_
Online Python Tutor Online Python Tutor
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
+7
View File
@@ -47,3 +47,10 @@ Weekly Python Newsletter containing Python Articles, Projects, Videos, Tweets
delivered in your inbox. Keep Your Python Programming Skills Updated. delivered in your inbox. Keep Your Python Programming Skills Updated.
`Import Python Weekly Newsletter <http://www.importpython.com/newsletter/>`_ `Import Python Weekly Newsletter <http://www.importpython.com/newsletter/>`_
Awesome Python Newsletter
~~~~~~~~~~~~~~~~~~~~
A weekly overview of the most popular Python news, articles and packages.
`Awesome Python Newsletter <https://python.libhunt.com/newsletter>`_
+12 -3
View File
@@ -190,7 +190,7 @@ Ansible
`Ansible <http://ansible.com/>`_ is an open source system automation tool. `Ansible <http://ansible.com/>`_ is an open source system automation tool.
The biggest advantage over Puppet or Chef is it does not require an agent on The biggest advantage over Puppet or Chef is it does not require an agent on
the client machine. Playbooks are Ansibles configuration, deployment, and the client machine. Playbooks are Ansibles configuration, deployment, and
orchestration language and are written in in YAML with Jinja2 for templating. orchestration language and are written in YAML with Jinja2 for templating.
Ansible supports Python versions 2.6 and 2.7 and can be installed via pip: Ansible supports Python versions 2.6 and 2.7 and can be installed via pip:
@@ -262,7 +262,7 @@ To create a simple cookbook the `knife <https://docs.chef.io/knife.html>`_ comma
`Getting started with Chef <http://gettingstartedwithchef.com/first-steps-with-chef.html>`_ `Getting started with Chef <http://gettingstartedwithchef.com/first-steps-with-chef.html>`_
is a good starting point for Chef Beginners and many community maintained cookbooks that can is a good starting point for Chef Beginners and many community maintained cookbooks that can
serve as a good reference or tweaked to serve your infrustructure configuration needs can be serve as a good reference or tweaked to serve your infrastructure configuration needs can be
found on the `Chef Supermarket <https://supermarket.chef.io/cookbooks>`_. found on the `Chef Supermarket <https://supermarket.chef.io/cookbooks>`_.
- `Chef Documentation <https://docs.chef.io/>`_ - `Chef Documentation <https://docs.chef.io/>`_
@@ -378,5 +378,14 @@ Buildout is primarily used to download and set up dependencies in Python eggs
format of the software being developed or deployed. Recipes for build tasks in any format of the software being developed or deployed. Recipes for build tasks in any
environment can be created, and many are already available. environment can be created, and many are already available.
Buidout is written in Python. Shinken
-------
`Shinken <http://www.shinken-monitoring.org/>`_ is a modern, Nagios compatible
monitoring framework written in Python. Its main goal is to give users a flexible
architecture for their monitoring system that is designed to scale to large
environments.
Shinken is backwards-compatible with the Nagios configuration standard, and
plugins.It works on any operating system, and architecture that supports Python
which includes Windows, GNU/Linux, and FreeBSD.
+1 -1
View File
@@ -35,7 +35,7 @@ automate the compile/test cycle to validate code changes.
Tox Tox
--- ---
`tox <http://tox.readthedocs.org/en/latest/>`_ is an automation tool providing `tox <https://tox.readthedocs.io/en/latest/>`_ is an automation tool providing
packaging, testing and deployment of Python software right from the console or packaging, testing and deployment of Python software right from the console or
CI server. It is a generic virtualenv management and test command line tool CI server. It is a generic virtualenv management and test command line tool
which provides the following features: which provides the following features:
+11 -1
View File
@@ -16,7 +16,7 @@ Some popular command-line applications include:
* `httpie <https://github.com/jakubroztocil/httpie>`_ - A command line HTTP * `httpie <https://github.com/jakubroztocil/httpie>`_ - A command line HTTP
client, a user-friendly cURL replacement client, a user-friendly cURL replacement
* `git <http://git-scm.com/>`_ - A distributed version control system * `git <http://git-scm.com/>`_ - A distributed version control system
* `mercurial <http://mercurial.selenic.com/>`_ - A distributed version control * `mercurial <https://www.mercurial-scm.org/>`_ - A distributed version control
system primarily written in Python system primarily written in Python
Clint Clint
@@ -62,3 +62,13 @@ subcommands, output formatters, and other extensions. The framework is meant
to be used to create multi-level commands such as subversion and git, where to be used to create multi-level commands such as subversion and git, where
the main program handles some basic argument parsing and then invokes a the main program handles some basic argument parsing and then invokes a
sub-command to do the work. sub-command to do the work.
Cement
------
`Cement <http://builtoncement.com/>`_ is an advanced CLI Application Framework.
Its goal is to introduce a standard, and feature-full platform
for both simple and complex command line applications as well
as support rapid development needs without sacrificing quality.
Cement is flexible, and it's use cases span from the simplicity of a micro-framework
to the complexity of a meg-framework.
+2 -2
View File
@@ -4,7 +4,7 @@ Interfacing with C/C++ Libraries
C Foreign Function Interface C Foreign Function Interface
---------------------------- ----------------------------
`CFFI <https://cffi.readthedocs.org/en/latest/>`_ provides a simple to use `CFFI <https://cffi.readthedocs.io/en/latest/>`_ provides a simple to use
mechanism for interfacing with C from both CPython and PyPy. It supports two mechanism for interfacing with C from both CPython and PyPy. It supports two
modes: an inline ABI compatibility mode (example provided below), which allows modes: an inline ABI compatibility mode (example provided below), which allows
you to dynamically load and run functions from executable modules (essentially you to dynamically load and run functions from executable modules (essentially
@@ -39,7 +39,7 @@ types, such as structs and unions, and allows you to modify things such as
padding and alignment, if needed. It can be a bit crufty to use, but in padding and alignment, if needed. It can be a bit crufty to use, but in
conjunction with the `struct <https://docs.python.org/3.5/library/struct.html>`_ conjunction with the `struct <https://docs.python.org/3.5/library/struct.html>`_
module, you are essentially provided full control over how your data types get module, you are essentially provided full control over how your data types get
translated into something something usable by a pure C(++) method. translated into something usable by a pure C(++) method.
Struct Equivalents Struct Equivalents
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~
+1 -1
View File
@@ -6,7 +6,7 @@ Cryptography
`Cryptography <https://cryptography.io/en/latest/>`_ is an actively developed `Cryptography <https://cryptography.io/en/latest/>`_ is an actively developed
library that provides cryptographic recipes and primitives. It supports library that provides cryptographic recipes and primitives. It supports
Python 2.6-2.7, Python 3.2+ and PyPy. Python 2.6-2.7, Python 3.3+ and PyPy.
Cryptography is divided into two layers of recipes and hazardous materials Cryptography is divided into two layers of recipes and hazardous materials
+4 -4
View File
@@ -29,7 +29,7 @@ Records
`Records <https://github.com/kennethreitz/records>`_ is minimalist SQL library, `Records <https://github.com/kennethreitz/records>`_ is minimalist SQL library,
designed for sending raw SQL queries to various databases. Data can be used designed for sending raw SQL queries to various databases. Data can be used
programatically, or exported to a number of useful data formats. programmatically, or exported to a number of useful data formats.
.. code-block:: console .. code-block:: console
@@ -60,11 +60,11 @@ peewee
`peewee <http://docs.peewee-orm.com/en/latest/>`_ is another ORM with a focus `peewee <http://docs.peewee-orm.com/en/latest/>`_ is another ORM with a focus
on being lightweight with support for Python 2.6+ and 3.2+ which supports on being lightweight with support for Python 2.6+ and 3.2+ which supports
SQLite, MySQL and Postgres by default. The SQLite, MySQL and Postgres by default. The
`model layer <https://peewee.readthedocs.org/en/latest/peewee/quickstart.html#model-definition>`_ `model layer <https://peewee.readthedocs.io/en/latest/peewee/quickstart.html#model-definition>`_
is similar to that of the Django ORM and it has is similar to that of the Django ORM and it has
`SQL-like methods <https://peewee.readthedocs.org/en/latest/peewee/quickstart.html#retrieving-data>`_ `SQL-like methods <https://peewee.readthedocs.io/en/latest/peewee/quickstart.html#retrieving-data>`_
to query data. While SQLite, MySQL and Postgres are supported out-of-the-box, to query data. While SQLite, MySQL and Postgres are supported out-of-the-box,
there is a `collection of add-ons <https://peewee.readthedocs.org/en/latest/peewee/playhouse.html#playhouse>`_ there is a `collection of add-ons <https://peewee.readthedocs.io/en/latest/peewee/playhouse.html#playhouse>`_
available. available.
PonyORM PonyORM
+1 -1
View File
@@ -28,7 +28,7 @@ applications be ported from PyGTK to PyGObject.
PyGObject aka (PyGi) PyGObject aka (PyGi)
-------------------- --------------------
`PyGObject <https://wiki.gnome.org/Projects/PyGObject>`_ provides Python bindings, which gives access to the entire GNOME software platform. `PyGObject <https://wiki.gnome.org/Projects/PyGObject>`_ provides Python bindings, which gives access to the entire GNOME software platform.
It is fully compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3 Tutorial <http://python-gtk-3-tutorial.readthedocs.org/en/latest/>`_. It is fully compatible with GTK+ 3. Here is a tutorial to get started with `Python GTK+ 3 Tutorial <https://python-gtk-3-tutorial.readthedocs.io/en/latest/>`_.
`API Reference <http://lazka.github.io/pgi-docs/>`_ `API Reference <http://lazka.github.io/pgi-docs/>`_
+3 -3
View File
@@ -24,7 +24,7 @@ Installation
Before installing Pillow, you'll have to install Pillow's prerequisites. Find Before installing Pillow, you'll have to install Pillow's prerequisites. Find
the instructions for your platform in the the instructions for your platform in the
`Pillow installation instructions <https://pillow.readthedocs.org/en/3.0.0/installation.html>`_. `Pillow installation instructions <https://pillow.readthedocs.io/en/3.0.0/installation.html>`_.
After that, it's straightforward: After that, it's straightforward:
@@ -57,7 +57,7 @@ Example
exif_data exif_data
There are more examples of the Pillow library in the There are more examples of the Pillow library in the
`Pillow tutorial <http://pillow.readthedocs.org/en/3.0.x/handbook/tutorial.html>`_. `Pillow tutorial <https://pillow.readthedocs.io/en/3.0.x/handbook/tutorial.html>`_.
OpenSource Computer Vision OpenSource Computer Vision
@@ -104,4 +104,4 @@ Example
There are more Python-implemented examples of OpenCV in this `collection of There are more Python-implemented examples of OpenCV in this `collection of
tutorials tutorials
<http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_tutorials.html>`_. <https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html>`_.
+1 -1
View File
@@ -47,7 +47,7 @@ simplejson
The JSON library was added to Python in version 2.6. The JSON library was added to Python in version 2.6.
If you're using an earlier version of Python, the If you're using an earlier version of Python, the
`simplejson <https://simplejson.readthedocs.org/en/latest/>`_ library is `simplejson <https://simplejson.readthedocs.io/en/latest/>`_ library is
available via PyPI. available via PyPI.
simplejson mimics the json standard library. It is available so that developers simplejson mimics the json standard library. It is available so that developers
+117
View File
@@ -0,0 +1,117 @@
================
Machine Learning
================
Python has a vast number of libraries for data analysis, statistics and Machine Learning itself, making it a language of choice for many data scientists.
Some widely used packages for Machine Learning and other Data Science applications are enlisted below.
Scipy Stack
-----------
The Scipy stack consists of a bunch of core helper packages used in data science, for statistical analysis and visualising data. Because of its huge number of functionalities and ease of use, the Stack is considered a must-have for most data science applications.
The Stack consists of the following packages (link to documentation given):
1. `NumPy <http://www.numpy.org/>`_
2. `SciPy library <https://www.scipy.org/>`_
3. `Matplotlib <http://matplotlib.org/>`_
4. `IPython <https://ipython.org/>`_
5. `pandas <http://pandas.pydata.org/>`_
6. `Sympy <http://www.sympy.org/en/index.html>`_
7. `nose <http://nose.readthedocs.io/en/latest/>`_
The stack also comes with Python bundled in, but has been excluded from the above list.
Installation
~~~~~~~~~~~~
For installing the full stack, or individual packages, you can refer to the instructions given `here <https://www.scipy.org/install.html>`_.
**NB:** `Anaconda <https://www.continuum.io/anaconda-overview>`_ is highly preferred and recommended for installing and maintaining data science packages seamlessly.
scikit-learn
------------
Scikit is a free and open-source machine learning library for Python. It offers off-the-shelf functions to implement many algorithms like linear regression, classifiers, SVMs, k-means, Neural Networks etc. It also has a few sample datasets which can be directly used for training and testing.
Because of its speed, robustness and easiness to use, it's one of the most widely-used libraries for many Machine Learning applications.
Installation
~~~~~~~~~~~~
Through PyPI:
.. code-block:: python
pip install -U scikit-learn
Through conda:
.. code-block:: python
conda install scikit-learn
scikit-learn also comes in shipped with Anaconda (mentioned above). For more installation instructions, refer to `this link <http://scikit-learn.org/stable/install.html>`_.
Example
~~~~~~~
For this example, we train a simple classifier on the `Iris dataset <http://en.wikipedia.org/wiki/Iris_flower_data_set>`_, which comes bundled in with scikit-learn.
The dataset takes four features of flowers: sepal length, sepal width, petal length and petal width, and classifies them into three flower species (labels): setosa, versicolor or virginica. The labels have been represented as numbers in the dataset: 0 (setosa), 1 (versicolor) and 2 (virginica).
We shuffle the Iris dataset, and divide it into separate training and testing sets: keeping the last 10 data points for testing and rest for training. We then train the classifier on the training set, and predict on the testing set.
.. code-block:: python
from sklearn.datasets import load_iris
from sklearn import tree
from sklearn.metrics import accuracy_score
import numpy as np
#loading the iris dataset
iris = load_iris()
x = iris.data #array of the data
y = iris.target #array of labels (i.e answers) of each data entry
#getting label names i.e the three flower species
y_names = iris.target_names
#taking random indices to split the dataset into train and test
test_ids = np.random.permutation(len(x))
#splitting data and labels into train and test
#keeping last 10 entries for testing, rest for training
x_train = x[test_ids[:-10]]
x_test = x[test_ids[-10:]]
y_train = y[test_ids[:-10]]
y_test = y[test_ids[-10:]]
#classifying using decision tree
clf = tree.DecisionTreeClassifier()
#training (fitting) the classifier with the training set
clf.fit(x_train, y_train)
#predictions on the test dataset
pred = clf.predict(x_test)
print pred #predicted labels i.e flower species
print y_test #actual labels
print (accuracy_score(pred, y_test))*100 #prediction accuracy
Since we're splitting randomly and the classifier trains on every iteration, the accuracy may vary. Running the above code gives:
.. code-block:: python
[0 1 1 1 0 2 0 2 2 2]
[0 1 1 1 0 2 0 2 2 2]
100.0
The first line contains the labels (i.e flower species) of the testing data as predicted by our classifier, and the second line contains the actual flower species as given in the dataset. We thus get an accuracy of 100% this time.
More on scikit-learn can be read in the `documentation <http://scikit-learn.org/stable/user_guide.html>`_.
+1 -1
View File
@@ -94,7 +94,7 @@ auto-alignment of data.
Rpy2 Rpy2
---- ----
`Rpy2 <http://rpy.sourceforge.net/rpy2.html>`_ is a Python binding for the R `Rpy2 <http://rpy2.bitbucket.org>`_ is a Python binding for the R
statistical package allowing the execution of R functions from Python and statistical package allowing the execution of R functions from Python and
passing data back and forth between the two environments. Rpy2 is the object passing data back and forth between the two environments. Rpy2 is the object
oriented implementation of the `Rpy <http://rpy.sourceforge.net/rpy.html>`_ oriented implementation of the `Rpy <http://rpy.sourceforge.net/rpy.html>`_
+207 -7
View File
@@ -108,7 +108,7 @@ keywords compared to the next one, which is implemented in pure Python:
def primes(kmax): def primes(kmax):
"""Calculation of prime numbers in standard Python syntax""" """Calculation of prime numbers in standard Python syntax"""
p= range(1000) p = range(1000)
result = [] result = []
if kmax > 1000: if kmax > 1000:
kmax = 1000 kmax = 1000
@@ -145,7 +145,7 @@ to be compiled into C types while also creating a Python list:
def primes(kmax): def primes(kmax):
"""Calculation of prime numbers in standard Python syntax""" """Calculation of prime numbers in standard Python syntax"""
p= range(1000) p = range(1000)
result = [] result = []
What is the difference? In the upper Cython version you can see the What is the difference? In the upper Cython version you can see the
@@ -222,17 +222,212 @@ Pyrex
Shedskin? Shedskin?
--------- ---------
Numba Concurrency
----- :::::::::::
.. todo:: Write about Numba and the autojit compiler for NumPy
Threading
:::::::::
Concurrent.futures
------------------
The `concurrent.futures`_ module is a module in the standard library that
provides a "high-level interface for asynchronously executing callables". It
abstracts away a lot of the more complicated details about using multiple
threads or processes for concurrency, and allows the user to focus on
accomplishing the task at hand.
The `concurrent.futures`_ module exposes two main classes, the
`ThreadPoolExecutor` and the `ProcessPoolExecutor`. The ThreadPoolExecutor
will create a pool of worker threads that a user can submit jobs to. These jobs
will then be executed in another thread when the next worker thread becomes
available.
The ProcessPoolExecutor works in the same way, except instead of using multiple
threads for its workers, it will use multiple processes. This makes it possible
to side-step the GIL, however because of the way things are passed to worker
processes, only picklable objects can be executed and returned.
Because of the way the GIL works, a good rule of thumb is to use a
ThreadPoolExecutor when the task being executed involves a lot of blocking
(i.e. making requests over the network) and to use a ProcessPoolExecutor
executor when the task is computationally expensive.
There are two main ways of executing things in parallel using the two
Executors. One way is with the `map(func, iterables)` method. This works
almost exactly like the builtin `map()` function, except it will execute
everything in parallel. :
.. code-block:: python
from concurrent.futures import ThreadPoolExecutor
import requests
def get_webpage(url):
page = requests.get(url)
return page
pool = ThreadPoolExecutor(max_workers=5)
my_urls = ['http://google.com/']*10 # Create a list of urls
for page in pool.map(get_webpage, my_urls):
# Do something with the result
print(page.text)
For even more control, the `submit(func, *args, **kwargs)` method will schedule
a callable to be executed ( as `func(*args, **kwargs)`) and returns a `Future`_
object that represents the execution of the callable.
The Future object provides various methods that can be used to check on the
progress of the scheduled callable. These include:
cancel()
Attempt to cancel the call.
cancelled()
Return True if the call was successfully cancelled.
running()
Return True if the call is currently being executed and cannot be
cancelled.
done()
Return True if the call was successfully cancelled or finished running.
result()
Return the value returned by the call. Note that this call will block until
the scheduled callable returns by default.
exception()
Return the exception raised by the call. If no exception was raised then
this returns `None`. Note that this will block just like `result()`.
add_done_callback(fn)
Attach a callback function that will be executed (as `fn(future)`) when the
scheduled callable returns.
.. code-block:: python
from concurrent.futures import ProcessPoolExecutor, as_completed
def is_prime(n):
if n % 2 == 0:
return n, False
sqrt_n = int(n**0.5)
for i in range(3, sqrt_n + 1, 2):
if n % i == 0:
return n, False
return n, True
PRIMES = [
112272535095293,
112582705942171,
112272535095293,
115280095190773,
115797848077099,
1099726899285419]
futures = []
with ProcessPoolExecutor(max_workers=4) as pool:
# Schedule the ProcessPoolExecutor to check if a number is prime
# and add the returned Future to our list of futures
for p in PRIMES:
fut = pool.submit(is_prime, p)
futures.append(fut)
# As the jobs are completed, print out the results
for number, result in as_completed(futures):
if result:
print("{} is prime".format(number))
else:
print("{} is not prime".format(number))
The `concurrent.futures`_ module contains two helper functions for working with
Futures. The `as_completed(futures)` function returns an iterator over the list
of futures, yielding the futures as they complete.
The `wait(futures)` function will simply block until all futures in the list of
futures provided have completed.
For more information, on using the `concurrent.futures`_ module, consult the
official documentation.
Threading Threading
--------- ---------
The standard library comes with a `threading`_ module that allows a user to
work with multiple threads manually.
Running a function in another thread is as simple as passing a callable and
it's arguments to `Thread`'s constructor and then calling `start()`:
.. code-block:: python
from threading import Thread
import requests
def get_webpage(url):
page = requests.get(url)
return page
some_thread = Thread(get_webpage, 'http://google.com/')
some_thread.start()
To wait until the thread has terminated, call `join()`:
.. code-block:: python
some_thread.join()
After calling `join()`, it is always a good idea to check whether the thread is
still alive (because the join call timed out):
.. code-block:: python
if some_thread.is_alive():
print("join() must have timed out.")
else:
print("Our thread has terminated.")
Because multiple threads have access to the same section of memory, sometimes
there might be situations where two or more threads are trying to write to the
same resource at the same time or where the output is dependent on the sequence
or timing of certain events. This is called a `data race`_ or race condition.
When this happens, the output will be garbled or you may encounter problems
which are difficult to debug. A good example is this `stackoverflow post`_.
The way this can be avoided is by using a `Lock`_ that each thread needs to
acquire before writing to a shared resource. Locks can be acquired and released
through either the contextmanager protocol (`with` statement), or by using
`acquire()` and `release()` directly. Here is a (rather contrived) example:
.. code-block:: python
from threading import Lock, Thread
file_lock = Lock()
def log(msg):
with file_lock:
open('website_changes.log', 'w') as f:
f.write(changes)
def monitor_website(some_website):
"""
Monitor a website and then if there are any changes,
log them to disk.
"""
while True:
changes = check_for_changes(some_website)
if changes:
log(changes)
websites = ['http://google.com/', ... ]
for website in websites:
t = Thread(monitor_website, website)
t.start()
Here, we have a bunch of threads checking for changes on a list of sites and
whenever there are any changes, they attempt to write those changes to a file
by calling `log(changes)`. When `log()` is called, it will wait to acquire
the lock with `with file_lock:`. This ensures that at any one time, only one
thread is writing to the file.
Spawning Processes Spawning Processes
------------------ ------------------
@@ -248,3 +443,8 @@ Multiprocessing
.. _`New GIL`: http://www.dabeaz.com/python/NewGIL.pdf .. _`New GIL`: http://www.dabeaz.com/python/NewGIL.pdf
.. _`Special care`: http://docs.python.org/c-api/init.html#threads .. _`Special care`: http://docs.python.org/c-api/init.html#threads
.. _`David Beazley's`: http://www.dabeaz.com/GIL/gilvis/measure2.py .. _`David Beazley's`: http://www.dabeaz.com/GIL/gilvis/measure2.py
.. _`concurrent.futures`: https://docs.python.org/3/library/concurrent.futures.html
.. _`Future`: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
.. _`threading`: https://docs.python.org/3/library/threading.html
.. _`stackoverflow post`: http://stackoverflow.com/questions/26688424/python-threads-are-printing-at-the-same-time-messing-up-the-text-output
.. _`data race`: https://en.wikipedia.org/wiki/Race_condition
+10 -11
View File
@@ -102,10 +102,9 @@ I do not recommend using Tornado unless you think you need it.
Pyramid Pyramid
-------- --------
`Pyramid <http://www.pylonsproject.org/>`_ is a very flexible `Pyramid <https://trypyramid.com/>`_ is a very flexible framework with a heavy
framework with a heavy focus on modularity. It comes with a small number focus on modularity. It comes with a small number of libraries ("batteries")
of libraries ("batteries") built-in, and encourages users to extend its built-in, and encourages users to extend its base functionality.
base functionality.
Pyramid does not have a large user base, unlike Django and Flask. It's a Pyramid does not have a large user base, unlike Django and Flask. It's a
capable framework, but not a very popular choice for new Python web capable framework, but not a very popular choice for new Python web
@@ -155,7 +154,7 @@ Gunicorn is the recommended choice for new Python web applications today.
Waitress Waitress
-------- --------
`Waitress <http://waitress.readthedocs.org>`_ is a pure-python WSGI server `Waitress <https://waitress.readthedocs.io>`_ is a pure-python WSGI server
that claims "very acceptable performance". Its documentation is not very that claims "very acceptable performance". Its documentation is not very
detailed, but it does offer some nice functionality that Gunicorn doesn't have detailed, but it does offer some nice functionality that Gunicorn doesn't have
(e.g. HTTP request buffering). (e.g. HTTP request buffering).
@@ -167,18 +166,18 @@ Waitress is gaining popularity within the Python web development community.
uWSGI uWSGI
----- -----
`uWSGI <https://uwsgi-docs.readthedocs.org>`_ is a full stack for building `uWSGI <https://uwsgi-docs.readthedocs.io>`_ is a full stack for building
hosting services. In addition to process management, process monitoring, hosting services. In addition to process management, process monitoring,
and other functionality, uWSGI acts as an application server for various and other functionality, uWSGI acts as an application server for various
programming languages and protocols - including Python and WSGI. uWSGI can programming languages and protocols - including Python and WSGI. uWSGI can
either be run as a stand-alone web router, or be run behind a full web either be run as a stand-alone web router, or be run behind a full web
server (such as Nginx or Apache). In the latter case, a web server can server (such as Nginx or Apache). In the latter case, a web server can
configure uWSGI and an application's operation over the configure uWSGI and an application's operation over the
`uwsgi protocol <https://uwsgi-docs.readthedocs.org/en/latest/Protocol.html>`_. `uwsgi protocol <https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html>`_.
uWSGI's web server support allows for dynamically configuring uWSGI's web server support allows for dynamically configuring
Python, passing environment variables and further tuning. For full details, Python, passing environment variables and further tuning. For full details,
see `uWSGI magic see `uWSGI magic
variables <https://uwsgi-docs.readthedocs.org/en/latest/Vars.html>`_. variables <https://uwsgi-docs.readthedocs.io/en/latest/Vars.html>`_.
I do not recommend using uWSGI unless you know why you need it. I do not recommend using uWSGI unless you know why you need it.
@@ -394,10 +393,10 @@ Jinja2 is the recommended templating library for new Python web applications.
Chameleon Chameleon
--------- ---------
`Chameleon <https://chameleon.readthedocs.org/>`_ Page Templates are an HTML/XML template `Chameleon <https://chameleon.readthedocs.io/>`_ Page Templates are an HTML/XML template
engine implementation of the `Template Attribute Language (TAL) <http://en.wikipedia.org/wiki/Template_Attribute_Language>`_, engine implementation of the `Template Attribute Language (TAL) <http://en.wikipedia.org/wiki/Template_Attribute_Language>`_,
`TAL Expression Syntax (TALES) <http://chameleon.readthedocs.org/en/latest/reference.html#expressions-tales>`_, `TAL Expression Syntax (TALES) <https://chameleon.readthedocs.io/en/latest/reference.html#expressions-tales>`_,
and `Macro Expansion TAL (Metal) <http://chameleon.readthedocs.org/en/latest/reference.html#macros-metal>`_ syntaxes. and `Macro Expansion TAL (Metal) <https://chameleon.readthedocs.io/en/latest/reference.html#macros-metal>`_ syntaxes.
Chameleon is available for Python 2.5 and up (including 3.x and pypy), and Chameleon is available for Python 2.5 and up (including 3.x and pypy), and
is commonly used by the `Pyramid Framework <http://trypyramid.com>`_. is commonly used by the `Pyramid Framework <http://trypyramid.com>`_.
+1 -1
View File
@@ -8,7 +8,7 @@ Freezing Your Code
to end-users, that contains all of your application code as well as the to end-users, that contains all of your application code as well as the
Python interpreter. Python interpreter.
Applications such as 'Dropbox', 'Eve Online', 'Civilisation IV', and Applications such as 'Dropbox', 'Eve Online', 'Civilization IV', and
BitTorrent clients do this. BitTorrent clients do this.
The advantage of distributing this way is that your application will "just work", The advantage of distributing this way is that your application will "just work",
+2 -1
View File
@@ -25,7 +25,7 @@ and being able and willing to use tools such as pip to install your code's
other dependencies. This is fine when distributing to other developers, but other dependencies. This is fine when distributing to other developers, but
makes this method unsuitable for distributing applications to end-users. makes this method unsuitable for distributing applications to end-users.
The `Python Packaging Guide <https://python-packaging-user-guide.readthedocs.org/en/latest/>`_ The `Python Packaging Guide <https://python-packaging-user-guide.readthedocs.io/>`_
provides an extensive guide on creating and maintaining Python packages. provides an extensive guide on creating and maintaining Python packages.
Alternatives to Packaging Alternatives to Packaging
@@ -184,3 +184,4 @@ Useful Tools
- `fpm <https://github.com/jordansissel/fpm>`_ - `fpm <https://github.com/jordansissel/fpm>`_
- `alien <http://joeyh.name/code/alien/>`_ - `alien <http://joeyh.name/code/alien/>`_
- `dh-virtualenv <https://dh-virtualenv.readthedocs.io/en/latest/info.html>`_ (for APT/DEB omnibus packaging)
+2 -2
View File
@@ -55,8 +55,8 @@ in separate places, by creating virtual Python environments for them. It solves
"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps "Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
your global site-packages directory clean and manageable. your global site-packages directory clean and manageable.
For example, you can work on a project which requires Django 1.3 while also For example, you can work on a project which requires Django 1.10 while also
maintaining a project which requires Django 1.0. maintaining a project which requires Django 1.8.
To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs. To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs.
+37 -4
View File
@@ -24,13 +24,16 @@ Doing it Right
Let's install a real version of Python. Let's install a real version of Python.
Before installing Python, you'll need to install GCC. GCC can be obtained Before installing Python, you'll need to install a C compiler. The fastest way
by downloading `XCode <http://developer.apple.com/xcode/>`_, the smaller is to install the Xcode Command Line Tools by running
`Command Line Tools <https://developer.apple.com/downloads/>`_ (must have an ``xcode-select --install``. You can also download the full version of
Apple account) or the even smaller `OSX-GCC-Installer <https://github.com/kennethreitz/osx-gcc-installer#readme>`_ `Xcode <http://developer.apple.com/xcode/>`_ from the Mac App Store, or the
minimal but unofficial
`OSX-GCC-Installer <https://github.com/kennethreitz/osx-gcc-installer#readme>`_
package. package.
.. note:: .. note::
<<<<<<< HEAD
If you already have XCode installed, do not install OSX-GCC-Installer. If you already have XCode installed, do not install OSX-GCC-Installer.
In combination, the software can cause issues that are difficult to In combination, the software can cause issues that are difficult to
diagnose. diagnose.
@@ -38,6 +41,11 @@ package.
.. note:: .. note::
If you perform a fresh install of XCode, you will also need to add the If you perform a fresh install of XCode, you will also need to add the
commandline tools by running ``xcode-select --install`` on the terminal. commandline tools by running ``xcode-select --install`` on the terminal.
=======
If you already have Xcode installed or plan to use Homebrew, do not install
OSX-GCC-Installer. In combination, the software can cause issues that are
difficult to diagnose.
>>>>>>> upstream/master
While OS X comes with a large number of UNIX utilities, those familiar with While OS X comes with a large number of UNIX utilities, those familiar with
Linux systems will notice one key component missing: a decent package manager. Linux systems will notice one key component missing: a decent package manager.
@@ -66,7 +74,17 @@ Now, we can install Python 2.7:
$ brew install python $ brew install python
<<<<<<< HEAD
This will take a minute or two. This will take a minute or two.
=======
or Python 3:
.. code-block:: console
$ brew install python3
This will take a minute or two.
>>>>>>> upstream/master
Setuptools and Pip Setuptools and Pip
@@ -75,6 +93,7 @@ Setuptools and Pip
Homebrew installs Setuptools and ``pip`` for you. Homebrew installs Setuptools and ``pip`` for you.
Setuptools enables you to download and install any compliant Python Setuptools enables you to download and install any compliant Python
<<<<<<< HEAD
-software over a network (usually the Internet) with a single command -software over a network (usually the Internet) with a single command
-(``easy_install``). It also enables you to add this network installation -(``easy_install``). It also enables you to add this network installation
-capability to your own Python software with very little work. -capability to your own Python software with very little work.
@@ -83,6 +102,14 @@ Setuptools enables you to download and install any compliant Python
recommended over ``easy_install``. It is superior to recommended over ``easy_install``. It is superior to
``easy_install`` in `several ways ``easy_install`` in `several ways
<https://python-packaging-user-guide.readthedocs.org/en/latest/pip_easy_install/#pip-vs-easy-install>`_, <https://python-packaging-user-guide.readthedocs.org/en/latest/pip_easy_install/#pip-vs-easy-install>`_,
=======
software over a network (usually the Internet) with a single command
(``easy_install``). It also enables you to add this network installation
capability to your own Python software with very little work.
``pip`` is a tool for easily installing and managing Python packages,
that is recommended over ``easy_install``. It is superior to ``easy_install`` in `several ways <https://python-packaging-user-guide.readthedocs.io/pip_easy_install/#pip-vs-easy-install>`_,
>>>>>>> upstream/master
and is actively maintained. and is actively maintained.
@@ -95,7 +122,13 @@ in separate places, by creating virtual Python environments for them. It solves
your global site-packages directory clean and manageable. your global site-packages directory clean and manageable.
For example, you can work on a project which requires Django 1.10 while also For example, you can work on a project which requires Django 1.10 while also
<<<<<<< HEAD
maintaining a project which requires Django 1.7. maintaining a project which requires Django 1.7.
=======
maintaining a project which requires Django 1.8.
To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs.
>>>>>>> upstream/master
To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs. To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs.
+3 -3
View File
@@ -3,7 +3,7 @@
Installing Python on Windows Installing Python on Windows
============================ ============================
First, download the `latest version <https://www.python.org/ftp/python/2.7.10/python-2.7.10.msi>`_ First, download the `latest version <https://www.python.org/ftp/python/2.7.12/python-2.7.12.msi>`_
of Python 2.7 from the official Website. If you want to be sure you are installing a fully of Python 2.7 from the official Website. If you want to be sure you are installing a fully
up-to-date version, click the Downloads > Windows link from the home page of the up-to-date version, click the Downloads > Windows link from the home page of the
`Python.org web site <http://python.org>`_ . `Python.org web site <http://python.org>`_ .
@@ -74,8 +74,8 @@ in separate places, by creating virtual Python environments for them. It solves
"Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps "Project X depends on version 1.x but, Project Y needs 4.x" dilemma, and keeps
your global site-packages directory clean and manageable. your global site-packages directory clean and manageable.
For example, you can work on a project which requires Django 1.3 while also For example, you can work on a project which requires Django 1.10 while also
maintaining a project which requires Django 1.0. maintaining a project which requires Django 1.8.
To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs. To start using this and see more information: :ref:`Virtual Environments <virtualenvironments-ref>` docs.
+2 -2
View File
@@ -16,7 +16,7 @@ Installation Guides
These guides go over the proper installation of :ref:`Python <which-python>` These guides go over the proper installation of :ref:`Python <which-python>`
for development purposes, as well as setuptools, pip and virtualenv. for development purposes, as well as setuptools, pip and virtualenv.
- :ref:`Python 3 on Mac OS X <install3-osx>`. - :ref:`Python 3 on MacOS <install3-osx>`.
- :ref:`Python 2 on Mac OS X <install-osx>`. - :ref:`Python 2 on MacOS <install-osx>`.
- :ref:`Python 2 on Microsoft Windows <install-windows>`. - :ref:`Python 2 on Microsoft Windows <install-windows>`.
- :ref:`Python 2 on Ubuntu Linux <install-linux>`. - :ref:`Python 2 on Ubuntu Linux <install-linux>`.
+5 -3
View File
@@ -137,12 +137,12 @@ installation with the .NET Common Language Runtime (CLR). This is the
inverse approach to that taken by IronPython (see above), to which it inverse approach to that taken by IronPython (see above), to which it
is more complementary than competing with. is more complementary than competing with.
In conjunction with Mono, PythonNet enables native Python In conjunction with Mono, pythonnet enables native Python
installations on non-Windows operating systems, such as OS X and installations on non-Windows operating systems, such as OS X and
Linux, to operate within the .NET framework. It can be run in Linux, to operate within the .NET framework. It can be run in
addition to IronPython without conflict. addition to IronPython without conflict.
PythonNet supports from Python 2.3 up to Python 2.7. [#pythonnet_ver]_ Pythonnet supports from Python 2.6 up to Python 3.5. [#pythonnet_ver1]_ [#pythonnet_ver2]_
.. [#pypy_ver] http://pypy.org/compat.html .. [#pypy_ver] http://pypy.org/compat.html
@@ -150,6 +150,8 @@ PythonNet supports from Python 2.3 up to Python 2.7. [#pythonnet_ver]_
.. [#iron_ver] http://ironpython.codeplex.com/releases/view/81726 .. [#iron_ver] http://ironpython.codeplex.com/releases/view/81726
.. [#pythonnet_ver] http://pythonnet.github.io/readme.html .. [#pythonnet_ver1] https://travis-ci.org/pythonnet/pythonnet
.. [#pythonnet_ver2] https://ci.appveyor.com/project/TonyRoberts/pythonnet-480xs
.. [#pep373_eol] https://www.python.org/dev/peps/pep-0373/#id2 .. [#pep373_eol] https://www.python.org/dev/peps/pep-0373/#id2
+106 -3
View File
@@ -65,6 +65,11 @@ There is also **great**, **free** hosting for your Sphinx_ docs:
your source repository so that rebuilding your documentation will your source repository so that rebuilding your documentation will
happen automatically. happen automatically.
When run, Sphinx_ will import your code and using Python's introspection
features it will extract all function, method and class signatures. It will
also extract the accompanying docstrings, and compile it all into well
structured and easily readable documentation for your project.
.. note:: .. note::
Sphinx is famous for its API generation, but it also works well Sphinx is famous for its API generation, but it also works well
@@ -104,11 +109,11 @@ In Python, *docstrings* describe modules, classes, and functions:
.. code-block:: python .. code-block:: python
def square_and_rooter(x): def square_and_rooter(x):
"""Returns the square root of self times self.""" """Return the square root of self times self."""
... ...
In general, follow the comment section of :pep:`8#comments` (the "Python Style In general, follow the comment section of :pep:`8#comments` (the "Python Style
Guide"). Guide"). More information about docstrings can be found at :pep:`0257#specification` (The Docstring Conventions Guide).
Commenting Sections of Code Commenting Sections of Code
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -127,6 +132,30 @@ Some tools use docstrings to embed more-than-documentation behavior,
such as unit test logic. Those can be nice, but you won't ever go such as unit test logic. Those can be nice, but you won't ever go
wrong with vanilla "here's what this does." wrong with vanilla "here's what this does."
Tools like Sphinx_ will parse your docstrings as reStructuredText and render it
correctly as HTML. This makes it very easy to embed snippets of example code in
a project's documentation.
Additionally, Doctest_ will read all embedded docstrings that look like input
from the Python commandline (prefixed with ">>>") and run them, checking to see
if the output of the command matches the text on the following line. This
allows developers to embed real examples and usage of functions alongside
their source code, and as a side effect, it also ensures that their code is
tested and works.
::
def my_function(a, b):
"""
>>> my_function(2, 3)
6
>>> my_function('a', 3)
'aaa'
"""
return a * b
.. _Doctest: https://docs.python.org/3/library/doctest.html
Docstrings versus Block comments Docstrings versus Block comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -141,8 +170,82 @@ comment block is a programmer's note. The docstring describes the
"""Returns the square root of self times self.""" """Returns the square root of self times self."""
... ...
.. see also:: Further reading on docstrings: :pep:`257` Unlike block comments, docstrings are built into the Python language itself.
This means you can use all of Python's powerful introspection capabilities to
access docstrings at runtime, compared with comments which are optimised out.
Docstrings are accessible from both the `__doc__` dunder attribute for almost
every Python object, as well as with the built in `help()` function.
While block comments are usually used to explain *what* a section of code is
doing, or the specifics of an algorithm, docstrings are more intended for
explaining to other users of your code (or you in 6 months time) *how* a
particular function can be used and the general purpose of a function, class,
or module.
Writing Docstrings
~~~~~~~~~~~~~~~~~~
Depending on the complexity of the function, method, or class being written, a
one-line docstring may be perfectly appropriate. These are generally used for
really obvious cases, such as::
def add(a, b):
"""Add two numbers and return the result."""
return a + b
The docstring should describe the function in a way that is easy to understand.
For simple cases like trivial functions and classes, simply embedding the
function's signature (i.e. `add(a, b) -> result`) in the docstring is
unnecessary. This is because with Python's `inspect` module, it is already
quite easy to find this information if needed, and it is also readily available
by reading the source code.
In larger or more complex projects however, it is often a good idea to give
more information about a function, what it does, any exceptions it may raise,
what it returns, or relevant details about the parameters.
For more detailed documentation of code a popular style is the one used for the
Numpy project, often called `Numpy style`_ docstrings. While it can take up a
few more lines the previous example, it allows the developer to include a lot
more information about a method, function, or class. ::
def random_number_generator(arg1, arg2):
"""
Summary line.
Extended description of function.
Parameters
----------
arg1 : int
Description of arg1
arg2 : str
Description of arg2
Returns
-------
int
Description of return value
"""
return 42
The `sphinx.ext.napoleon`_ plugin allows Sphinx to parse this style of
docstrings, making it easy to incorporate NumPy style docstrings into your
project.
At the end of the day, it doesn't really matter what style is used for writing
docstrings, their purpose is to serve as documentation for anyone who may need
to read or make changes to your code. As long as it is correct, understandable
and gets the relevant points across then it has done the job it was designed to
do.
For further reading on docstrings, feel free to consult :pep:`257`
.. _thomas-cokelaer.info: http://thomas-cokelaer.info/tutorials/sphinx/docstring_python.html
.. _sphinx.ext.napoleon: https://sphinxcontrib-napoleon.readthedocs.io/
.. _`NumPy style`: http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html
Other Tools Other Tools
----------- -----------
+1 -1
View File
@@ -224,7 +224,7 @@ Removing Bytecode (.pyc) Files
Here's nice trick for removing all of these files, if they already exist:: Here's nice trick for removing all of these files, if they already exist::
$ find . -name "*.pyc" -exec rm -rf {} \; $ find . -type f -name "*.py[co]" -delete -or -type d -name "__pycache__" -delete
Run that from the root directory of your project, and all ``.pyc`` files Run that from the root directory of your project, and all ``.pyc`` files
will suddenly vanish. Much better. will suddenly vanish. Much better.
+4 -4
View File
@@ -163,10 +163,10 @@ the configuration dictionary.
'formatter': 'f', 'formatter': 'f',
'level': logging.DEBUG} 'level': logging.DEBUG}
}, },
loggers = { root = {
'root': {'handlers': ['h'], 'handlers': ['h'],
'level': logging.DEBUG} 'level': logging.DEBUG,
} },
) )
dictConfig(logging_config) dictConfig(logging_config)
+1 -1
View File
@@ -1,7 +1,7 @@
Reading Great Code Reading Great Code
================== ==================
One of the core tenants behind the design of Python is creating One of the core tenets behind the design of Python is creating
readable code. The motivation behind this design is simple: The number readable code. The motivation behind this design is simple: The number
one thing that Python programmers do is read code. one thing that Python programmers do is read code.
+5 -3
View File
@@ -254,6 +254,8 @@ your project.
test: test:
py.test tests py.test tests
.PHONY: init test
Other generic management scripts (e.g. ``manage.py`` Other generic management scripts (e.g. ``manage.py``
or ``fabfile.py``) belong at the root of the repository as well. or ``fabfile.py``) belong at the root of the repository as well.
@@ -269,7 +271,7 @@ following, as they always have:
:: ::
$ django-admin.py start-project samplesite $ django-admin.py startproject samplesite
The resulting repository structure looks like this: The resulting repository structure looks like this:
@@ -291,7 +293,7 @@ Let's do it properly:
:: ::
$ django-admin.py start-project samplesite . $ django-admin.py startproject samplesite .
Note the "``.``". Note the "``.``".
@@ -600,7 +602,7 @@ clearer and thus preferred.
This mechanism is useful for separating concerns and avoiding This mechanism is useful for separating concerns and avoiding
external un-related logic 'polluting' the core logic of the function external un-related logic 'polluting' the core logic of the function
or method. A good example of a piece of functionality that is better handled or method. A good example of a piece of functionality that is better handled
with decoration is memoization or caching: you want to store the results of an with decoration is `memoization <https://en.wikipedia.org/wiki/Memoization#Overview>`__ or caching: you want to store the results of an
expensive function in a table and use them directly instead of recomputing expensive function in a table and use them directly instead of recomputing
them when they have already been computed. This is clearly not part them when they have already been computed. This is clearly not part
of the function logic. of the function logic.
+16 -16
View File
@@ -3,17 +3,17 @@
Code Style Code Style
========== ==========
If you ask Python programmers what they like most in Python, they will If you ask Python programmers what they like most about Python, they will
often say its high readability. Indeed, a high level of readability often cite its high readability. Indeed, a high level of readability
is at the heart of the design of the Python language, following the is at the heart of the design of the Python language, following the
recognized fact that code is read much more often than it is written. recognized fact that code is read much more often than it is written.
One reason for Python code to be easily read and understood is its relatively One reason for the high readability of Python code is its relatively
complete set of Code Style guidelines and "Pythonic" idioms. complete set of Code Style guidelines and "Pythonic" idioms.
Moreover, when a veteran Python developer (a Pythonista) points to portions of When a veteran Python developer (a Pythonista) calls portions of
code and says they are not "Pythonic", it usually means that these lines code not "Pythonic", they usually mean that these lines
of code do not follow the common guidelines and fail to express the intent in of code do not follow the common guidelines and fail to express its intent in
what is considered the best (hear: most readable) way. what is considered the best (hear: most readable) way.
On some border cases, no best way has been agreed upon on how to express On some border cases, no best way has been agreed upon on how to express
@@ -53,7 +53,7 @@ One statement per line
While some compound statements such as list comprehensions are While some compound statements such as list comprehensions are
allowed and appreciated for their brevity and their expressiveness, allowed and appreciated for their brevity and their expressiveness,
it is bad practice to have two disjoint statements on the same line of code. it is bad practice to have two disjointed statements on the same line of code.
**Bad** **Bad**
@@ -124,13 +124,13 @@ inside the function) that was added "just in case" and is seemingly never used,
than to add a new optional argument and its logic when needed. than to add a new optional argument and its logic when needed.
3. The **arbitrary argument list** is the third way to pass arguments to a 3. The **arbitrary argument list** is the third way to pass arguments to a
function. If the function intention is better expressed by a signature with an function. If the function intention is better expressed by a signature with
extensible number of positional arguments, it can be defined with the ``*args`` an extensible number of positional arguments, it can be defined with the
constructs. In the function body, ``args`` will be a tuple of all the ``*args`` constructs. In the function body, ``args`` will be a tuple of all
remaining positional arguments. For example, ``send(message, *args)`` can be the remaining positional arguments. For example, ``send(message, *args)``
called with each recipient as an argument: ``send('Hello', 'God', 'Mom', can be called with each recipient as an argument:``send('Hello', 'God',
'Cthulhu')``, and in the function body ``args`` will be equal to ``('God', 'Mom', 'Cthulhu')``, and in the function body ``args`` will be equal to
'Mom', 'Cthulhu')``. ``('God', 'Mom', 'Cthulhu')``.
However, this construct has some drawbacks and should be used with caution. If a However, this construct has some drawbacks and should be used with caution. If a
function receives a list of arguments of the same nature, it is often more function receives a list of arguments of the same nature, it is often more
@@ -455,9 +455,9 @@ PEP 8
easy-to-read version of PEP 8 is also available at `pep8.org <http://pep8.org/>`_. easy-to-read version of PEP 8 is also available at `pep8.org <http://pep8.org/>`_.
This is highly recommended reading. The entire Python community does their This is highly recommended reading. The entire Python community does their
best to adhere to the guidelines laidout within this document. Some project best to adhere to the guidelines laid out within this document. Some project
may sway from it from time to time, while others may may sway from it from time to time, while others may
`ammend its recommendations <http://docs.python-requests.org/en/master/dev/contributing/#kenneth-reitz-s-code-style>`_. `amend its recommendations <http://docs.python-requests.org/en/master/dev/contributing/#kenneth-reitz-s-code-style>`_.
That being said, conforming your Python code to PEP 8 is generally a good That being said, conforming your Python code to PEP 8 is generally a good
idea and helps make code more consistent when working on projects with other idea and helps make code more consistent when working on projects with other
+12 -12
View File
@@ -3,8 +3,8 @@ Testing Your Code
Testing your code is very important. Testing your code is very important.
Getting used to writing the testing code and the running code in parallel is Getting used to writing testing code and running this code in parallel is now
now considered a good habit. Used wisely, this method helps you define more considered a good habit. Used wisely, this method helps you define more
precisely your code's intent and have a more decoupled architecture. precisely your code's intent and have a more decoupled architecture.
Some general rules of testing: Some general rules of testing:
@@ -12,8 +12,8 @@ Some general rules of testing:
- A testing unit should focus on one tiny bit of functionality and prove it - A testing unit should focus on one tiny bit of functionality and prove it
correct. correct.
- Each test unit must be fully independent. Each of them must be able to run - Each test unit must be fully independent. Each test must be able to run
alone, and also within the test suite, regardless of the order they are alone, and also within the test suite, regardless of the order that they are
called. The implication of this rule is that each test must be loaded with called. The implication of this rule is that each test must be loaded with
a fresh dataset and may have to do some cleanup afterwards. This is a fresh dataset and may have to do some cleanup afterwards. This is
usually handled by :meth:`setUp()` and :meth:`tearDown()` methods. usually handled by :meth:`setUp()` and :meth:`tearDown()` methods.
@@ -27,8 +27,8 @@ Some general rules of testing:
tests as often as needed. tests as often as needed.
- Learn your tools and learn how to run a single test or a test case. Then, - Learn your tools and learn how to run a single test or a test case. Then,
when developing a function inside a module, run this function's tests very when developing a function inside a module, run this function's tests
often, ideally automatically when you save the code. frequently, ideally automatically when you save the code.
- Always run the full test suite before a coding session, and run it again - Always run the full test suite before a coding session, and run it again
after. This will give you more confidence that you did not break anything after. This will give you more confidence that you did not break anything
@@ -63,11 +63,11 @@ Some general rules of testing:
- Another use of the testing code is as an introduction to new developers. When - Another use of the testing code is as an introduction to new developers. When
someone will have to work on the code base, running and reading the related someone will have to work on the code base, running and reading the related
testing code is often the best they can do. They will or should discover the testing code is often the best thing that they can do to start. They will
hot spots, where most difficulties arise, and the corner cases. If they have or should discover the hot spots, where most difficulties arise, and the
to add some functionality, the first step should be to add a test and, by this corner cases. If they have to add some functionality, the first step should
means, ensure the new functionality is not already a working path that has not be to add a test to ensure that the new functionality is not already a
been plugged into the interface. working path that has not been plugged into the interface.
The Basics The Basics
:::::::::: ::::::::::
@@ -202,7 +202,7 @@ nose provides automatic test discovery to save you the hassle of manually
creating test suites. It also provides numerous plugins for features such as creating test suites. It also provides numerous plugins for features such as
xUnit-compatible test output, coverage reporting, and test selection. xUnit-compatible test output, coverage reporting, and test selection.
`nose <http://readthedocs.org/docs/nose/en/latest/>`_ `nose <https://nose.readthedocs.io/en/latest/>`_
tox tox