There exists a problem (maybe a bug?) in pip when using a locale like
`LC_ALL=C` (which is commonly used by CI environments and system
configuration tools such as Puppet), where the PKG-INFO file is decoded
using ascii, raising a UnicodeDecodeError when PKG-INFO contains
non-ASCII characters (such as the n-dash removed by this commit):
> Downloading/unpacking requests from https://pypi.python.org/packages/source/r/requests/requests-2.5.1.tar.gz
> Downloading requests-2.5.1.tar.gz (443Kb): 443Kb downloaded
> Running setup.py egg_info for package requests
>
> Exception:
> Traceback (most recent call last):
> File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 104, in main
> status = self.run(options, args)
> File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 245, in run
> requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
> File "/usr/lib/python3/dist-packages/pip/req.py", line 1014, in prepare_files
> req_to_install.assert_source_matches_version()
> File "/usr/lib/python3/dist-packages/pip/req.py", line 359, in assert_source_matches_version
> version = self.installed_version
> File "/usr/lib/python3/dist-packages/pip/req.py", line 351, in installed_version
> return self.pkg_info()['version']
> File "/usr/lib/python3/dist-packages/pip/req.py", line 318, in pkg_info
> data = self.egg_info_data('PKG-INFO')
> File "/usr/lib/python3/dist-packages/pip/req.py", line 261, in egg_info_data
> data = fp.read()
> File "/usr/lib/python3.2/encodings/ascii.py", line 26, in decode
> return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 8161: ordinal not in range(128)
I'm not able to reproduce this by installing the package directly, only by
installing another package which depends on it (via `install_requires`).
The command I used was `LC_ALL=C /usr/bin/pip-3.2 install --upgrade ocflib`,
with "pip 1.1 from /usr/lib/python3/dist-packages (python 3.2)" (which
is the packaged version for Debian 7).
Although this should probably be addressed ultimately in pip, replacing
this character in HISTORY is a pain-free and pragmatic way to help
developers and system administrators who may be stuck with old versions
of pip for many years to come.
Instead of only checking one or another type of string-like object that
we accept, let's be able to check both. Previously, we only checked if
the filename was an instance of the native str type which on Python 2
excluded unicode filenames and bytes-like filenames on Python 3.
Fixes#2411
Rasmus Scholer (TimelineX)
kennethreitz