.. _basic: Basic Usage of Pipenv ===================== .. image:: https://farm4.staticflickr.com/3931/33173826122_b7ee8f1a26_k_d.jpg This document covers some of Pipenv's more basic features. ☤ Example Pipfile & Pipfile.lock -------------------------------- Pipfiles contain information for the dependencies of the project, and supersedes the requirements.txt file used in most Python projects. You should add a Pipfile in the Git repository. The only thing required for users who clone the repository would be installing Pipenv in the machine and typing ``pipenv install``. Pipenv is a reference implementation for using Pipfile. .. _example_files: Here is a simple example of a ``Pipfile`` and the resulting ``Pipfile.lock``. Example Pipfile /////////////// :: [[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi" [packages] requests = "*" [dev-packages] pytest = "*" Example Pipfile.lock //////////////////// :: { "_meta": { "hash": { "sha256": "8d14434df45e0ef884d6c3f6e8048ba72335637a8631cc44792f52fd20b6f97a" }, "host-environment-markers": { "implementation_name": "cpython", "implementation_version": "3.6.1", "os_name": "posix", "platform_machine": "x86_64", "platform_python_implementation": "CPython", "platform_release": "16.7.0", "platform_system": "Darwin", "platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64", "python_full_version": "3.6.1", "python_version": "3.6", "sys_platform": "darwin" }, "pipfile-spec": 5, "requires": {}, "sources": [ { "name": "pypi", "url": "https://pypi.python.org/simple", "verify_ssl": true } ] }, "default": { "certifi": { "hashes": [ "sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704", "sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5" ], "version": "==2017.7.27.1" }, "chardet": { "hashes": [ "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691", "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae" ], "version": "==3.0.4" }, "idna": { "hashes": [ "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4", "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f" ], "version": "==2.6" }, "requests": { "hashes": [ "sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b", "sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e" ], "version": "==2.18.4" }, "urllib3": { "hashes": [ "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b", "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f" ], "version": "==1.22" } }, "develop": { "py": { "hashes": [ "sha256:2ccb79b01769d99115aa600d7eed99f524bf752bba8f041dc1c184853514655a", "sha256:0f2d585d22050e90c7d293b6451c83db097df77871974d90efd5a30dc12fcde3" ], "version": "==1.4.34" }, "pytest": { "hashes": [ "sha256:b84f554f8ddc23add65c411bf112b2d88e2489fd45f753b1cae5936358bdf314", "sha256:f46e49e0340a532764991c498244a60e3a37d7424a532b3ff1a6a7653f1a403a" ], "version": "==3.2.2" } } } ☤ General Recommendations & Version Control ------------------------------------------- - Generally, keep both ``Pipfile`` and ``Pipfile.lock`` in version control. - Do not keep ``Pipfile.lock`` in version control if multiple versions of Python are being targeted. - Specify your target Python version in your ``Pipfile``'s ``[requires]`` section. Ideally, you should only have one target Python version, as this is a deployment tool. ``python_version`` should be in the format ``X.Y`` (or ``X``) and ``python_full_version`` should be in ``X.Y.Z`` format. - ``pipenv install`` is fully compatible with ``pip install`` syntax, for which the full documentation can be found `here `__. - Note that the ``Pipfile`` uses the `TOML Spec `_. ☤ Example Pipenv Workflow ------------------------- Clone / create project repository:: $ cd myproject Install from Pipfile, if there is one:: $ pipenv install Or, add a package to your new project:: $ pipenv install This will create a ``Pipfile`` if one doesn't exist. If one does exist, it will automatically be edited with the new package you provided. Next, activate the Pipenv shell:: $ pipenv shell $ python --version This will spawn a new shell subprocess, which can be deactivated by using ``exit``. .. _initialization: ☤ Example Pipenv Upgrade Workflow --------------------------------- - Find out what's changed upstream: ``$ pipenv update --outdated``. - Upgrade packages, two options: a. Want to upgrade everything? Just do ``$ pipenv update``. b. Want to upgrade packages one-at-a-time? ``$ pipenv update `` for each outdated package. ☤ Importing from requirements.txt --------------------------------- If you only have a ``requirements.txt`` file available when running ``pipenv install``, pipenv will automatically import the contents of this file and create a ``Pipfile`` for you. You can also specify ``$ pipenv install -r path/to/requirements.txt`` to import a requirements file. If your requirements file has version numbers pinned, you'll likely want to edit the new ``Pipfile`` to remove those, and let ``pipenv`` keep track of pinning. If you want to keep the pinned versions in your ``Pipfile.lock`` for now, run ``pipenv lock --keep-outdated``. Make sure to `upgrade <#initialization>`_ soon! .. _specifying_versions: ☤ Specifying Versions of a Package ---------------------------------- You can specify versions of a package using the `Semantic Versioning scheme `_ (i.e. ``major.minor.micro``). For example, to install requests you can use: :: $ pipenv install requests~=1.2 Pipenv will install version ``1.2`` and any minor update, but not ``2.0``. This will update your ``Pipfile`` to reflect this requirement, automatically. In general, Pipenv uses the same specifier format as pip. However, note that according to `PEP 440`_ , you can't use versions containing a hyphen or a plus sign. .. _`PEP 440`: https://www.python.org/dev/peps/pep-0440/ To make inclusive or exclusive version comparisons you can use: :: $ pipenv install "requests>=1.4" # will install a version equal or larger than 1.4.0 $ pipenv install "requests<=2.13" # will install a version equal or lower than 2.13.0 $ pipenv install "requests>2.19" # will install 2.19.1 but not 2.19.0 .. note:: The use of double quotes around the package and version specification (i.e. ``"requests>2.19"``) is highly recommended to avoid issues with `Input and output redirection `_ in Unix-based operating systems. The use of ``~=`` is preferred over the ``==`` identifier as the latter prevents pipenv from updating the packages: :: $ pipenv install "requests~=2.2" # locks the major version of the package (this is equivalent to using >=2.2, ==2.*) To avoid installing a specific version you can use the ``!=`` identifier. For an in depth explanation of the valid identifiers and more complex use cases check `the relevant section of PEP-440`_. .. _`the relevant section of PEP-440`: https://www.python.org/dev/peps/pep-0440/#version-specifiers ☤ Specifying Versions of Python ------------------------------- To create a new virtualenv, using a specific version of Python you have installed (and on your ``PATH``), use the ``--python VERSION`` flag, like so: Use Python 3:: $ pipenv --python 3 Use Python3.6:: $ pipenv --python 3.6 Use Python 2.7.14:: $ pipenv --python 2.7.14 When given a Python version, like this, Pipenv will automatically scan your system for a Python that matches that given version. If a ``Pipfile`` hasn't been created yet, one will be created for you, that looks like this:: [[source]] url = "https://pypi.python.org/simple" verify_ssl = true [dev-packages] [packages] [requires] python_version = "3.6" .. note:: The inclusion of ``[requires] python_version = "3.6"`` specifies that your application requires this version of Python, and will be used automatically when running ``pipenv install`` against this ``Pipfile`` in the future (e.g. on other machines). If this is not true, feel free to simply remove this section. If you don't specify a Python version on the command–line, either the ``[requires]`` ``python_full_version`` or ``python_version`` will be selected automatically, falling back to whatever your system's default ``python`` installation is, at time of execution. ☤ Editable Dependencies (e.g. ``-e .`` ) ---------------------------------------- You can tell Pipenv to install a path as editable — often this is useful for the current working directory when working on packages:: $ pipenv install --dev -e . $ cat Pipfile ... [dev-packages] "e1839a8" = {path = ".", editable = true} ... .. note:: All sub-dependencies will get added to the ``Pipfile.lock`` as well. Sub-dependencies are **not** added to the ``Pipfile.lock`` if you leave the ``-e`` option out. .. _environment_management: ☤ Environment Management with Pipenv ------------------------------------ The three primary commands you'll use in managing your pipenv environment are ``$ pipenv install``, ``$ pipenv uninstall``, and ``$ pipenv lock``. .. _pipenv_install: $ pipenv install //////////////// ``$ pipenv install`` is used for installing packages into the pipenv virtual environment and updating your Pipfile. Along with the basic install command, which takes the form:: $ pipenv install [package names] The user can provide these additional parameters: - ``--three`` — Performs the installation in a virtualenv using the system ``python3`` link. - ``--python`` — Performs the installation in a virtualenv using the provided Python interpreter. .. warning:: None of the above commands should be used together. They are also **destructive** and will delete your current virtualenv before replacing it with an appropriately versioned one. - ``--dev`` — Install both ``develop`` and ``default`` packages from ``Pipfile``. - ``--system`` — Use the system ``pip`` command rather than the one from your virtualenv. - ``--deploy`` — Make sure the packages are properly locked in Pipfile.lock, and abort if the lock file is out-of-date. - ``--ignore-pipfile`` — Ignore the ``Pipfile`` and install from the ``Pipfile.lock``. - ``--skip-lock`` — Ignore the ``Pipfile.lock`` and install from the ``Pipfile``. In addition, do not write out a ``Pipfile.lock`` reflecting changes to the ``Pipfile``. .. _pipenv_uninstall: $ pipenv uninstall ////////////////// ``$ pipenv uninstall`` supports all of the parameters in `pipenv install <#pipenv-install>`_, as well as two additional options, ``--all`` and ``--all-dev``. - ``--all`` — This parameter will purge all files from the virtual environment, but leave the Pipfile untouched. - ``--all-dev`` — This parameter will remove all of the development packages from the virtual environment, and remove them from the Pipfile. .. _pipenv_lock: $ pipenv lock ///////////// ``$ pipenv lock`` is used to create a ``Pipfile.lock``, which declares **all** dependencies (and sub-dependencies) of your project, their latest available versions, and the current hashes for the downloaded files. This ensures repeatable, and most importantly *deterministic*, builds. ☤ About Shell Configuration --------------------------- Shells are typically misconfigured for subshell use, so ``$ pipenv shell --fancy`` may produce unexpected results. If this is the case, try ``$ pipenv shell``, which uses "compatibility mode", and will attempt to spawn a subshell despite misconfiguration. A proper shell configuration only sets environment variables like ``PATH`` during a login session, not during every subshell spawn (as they are typically configured to do). In fish, this looks like this:: if status --is-login set -gx PATH /usr/local/bin $PATH end You should do this for your shell too, in your ``~/.profile`` or ``~/.bashrc`` or wherever appropriate. .. note:: The shell launched in interactive mode. This means that if your shell reads its configuration from a specific file for interactive mode (e.g. bash by default looks for a ``~/.bashrc`` configuration file for interactive mode), then you'll need to modify (or create) this file. If you experience issues with ``$ pipenv shell``, just check the ``PIPENV_SHELL`` environment variable, which ``$ pipenv shell`` will use if available. For detail, see :ref:`configuration-with-environment-variables`. ☤ A Note about VCS Dependencies ------------------------------- You can install packages with pipenv from git and other version control systems using URLs formatted according to the following rule:: +:////@#egg= The only optional section is the ``@`` section. When using git over SSH, you may use the shorthand vcs and scheme alias ``git+git@:/@#egg=``. Note that this is translated to ``git+ssh://git@`` when parsed. Note that it is **strongly recommended** that you install any version-controlled dependencies in editable mode, using ``pipenv install -e``, in order to ensure that dependency resolution can be performed with an up-to-date copy of the repository each time it is performed, and that it includes all known dependencies. Below is an example usage which installs the git repository located at ``https://github.com/requests/requests.git`` from tag ``v2.20.1`` as package name ``requests``:: $ pipenv install -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests Creating a Pipfile for this project... Installing -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests... [...snipped...] Adding -e git+https://github.com/requests/requests.git@v2.20.1#egg=requests to Pipfile's [packages]... [...] $ cat Pipfile [packages] requests = {git = "https://github.com/requests/requests.git", editable = true, ref = "v2.20.1"} Valid values for ```` include ``git``, ``bzr``, ``svn``, and ``hg``. Valid values for ```` include ``http``, ``https``, ``ssh``, and ``file``. In specific cases you also have access to other schemes: ``svn`` may be combined with ``svn`` as a scheme, and ``bzr`` can be combined with ``sftp`` and ``lp``. You can read more about pip's implementation of VCS support `here `__. For more information about other options available when specifying VCS dependencies, please check the `Pipfile spec `_. ☤ Pipfile.lock Security Features -------------------------------- ``Pipfile.lock`` takes advantage of some great new security improvements in ``pip``. By default, the ``Pipfile.lock`` will be generated with the sha256 hashes of each downloaded package. This will allow ``pip`` to guarantee you're installing what you intend to when on a compromised network, or downloading dependencies from an untrusted PyPI endpoint. We highly recommend approaching deployments with promoting projects from a development environment into production. You can use ``pipenv lock`` to compile your dependencies on your development environment and deploy the compiled ``Pipfile.lock`` to all of your production environments for reproducible builds. .. note:: If you'd like a ``requirements.txt`` output of the lockfile, run ``$ pipenv requirements``. ☤ Pipenv and Docker Containers ------------------------------ In general, you should not have Pipenv inside a linux container image, since it is a build tool. If you want to use it to build, and install the run time dependencies for your application, you can use a multistage build for creating a virtual environment with your dependencies. In this approach, Pipenv in installed in the base layer, it is then used to create the virtual environment. In a later stage, in a ``runtime`` layer the virtual environment is copied from the base layer, the layer containing pipenv and other build dependencies is discarded. This results in a smaller image, which can still run your application. Here is an example ``Dockerfile``, which you can use as a starting point for doing a multistage build for your application:: FROM docker.io/python:3.9 AS builder RUN pip install --user pipenv # Tell pipenv to create venv in the current directory ENV PIPENV_VENV_IN_PROJECT=1 # Pipfile contains requests ADD Pipfile.lock Pipfile /usr/src/ WORKDIR /usr/src # NOTE: If you install binary packages required for a python module, you need # to install them again in the runtime. For example, if you need to install pycurl # you need to have pycurl build dependencies libcurl4-gnutls-dev and libcurl3-gnutls # In the runtime container you need only libcurl3-gnutls # RUN apt install -y libcurl3-gnutls libcurl4-gnutls-dev RUN /root/.local/bin/pipenv sync RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)" FROM docker.io/python:3.9 AS runtime RUN mkdir -v /usr/src/.venv COPY --from=builder /usr/src/.venv/ /usr/src/.venv/ RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)" # HERE GOES ANY CODE YOU NEED TO ADD TO CREATE YOUR APPLICATION'S IMAGE # For example # RUN apt install -y libcurl3-gnutls # RUN adduser --uid 123123 coolio # ADD run.py /usr/src/ WORKDIR /usr/src/ USER coolio CMD ["./.venv/bin/python", "-m", "run.py"] .. Note:: Pipenv is not meant to run as root. However, in the multistage build above it is done nevertheless. A calculated risk, since the intermediate image is discarded. The runtime image later shows that you should create a user and user it to run your application. **Once again, you should not run pipenv as root (or Admin on Windows) normally. This could lead to breakage of your Python installation, or even your complete OS.** When you build an image with this example (assuming requests is found in Pipfile), you will see that ``requests`` is installed in the ``runtime`` image:: $ sudo docker build --no-cache -t oz/123:0.1 . Sending build context to Docker daemon 1.122MB Step 1/12 : FROM docker.io/python:3.9 AS builder ---> 81f391f1a7d7 Step 2/12 : RUN pip install --user pipenv ---> Running in b83ed3c28448 ... trimmed ... ---> 848743eb8c65 Step 4/12 : ENV PIPENV_VENV_IN_PROJECT=1 ---> Running in 814e6f5fec5b Removing intermediate container 814e6f5fec5b ---> 20167b4a13e1 Step 5/12 : ADD Pipfile.lock Pipfile /usr/src/ ---> c7632cb3d5bd Step 6/12 : WORKDIR /usr/src ---> Running in 1d75c6cfce10 Removing intermediate container 1d75c6cfce10 ---> 2dcae54cc2e5 Step 7/12 : RUN /root/.local/bin/pipenv sync ---> Running in 1a00b326b1ee Creating a virtualenv for this project... ... trimmed ... ✔ Successfully created virtual environment! Virtualenv location: /usr/src/.venv Installing dependencies from Pipfile.lock (fe5a22)... ... trimmed ... Step 8/12 : RUN /usr/src/.venv/bin/python -c "import requests; print(requests.__version__)" ---> Running in 3a66e3ce4a11 2.27.1 Removing intermediate container 3a66e3ce4a11 ---> 1db657d0ac17 Step 9/12 : FROM docker.io/python:3.9 AS runtime ... trimmed ... Step 12/12 : RUN /usr/src/venv/bin/python -c "import requests; print(requests.__version__)" ---> Running in fa39ba4080c5 2.27.1 Removing intermediate container fa39ba4080c5 ---> 2b1c90fd414e Successfully built 2b1c90fd414e Successfully tagged oz/123:0.1