flake8-encodings

A Flake8 plugin to identify incorrect use of encodings.

See also

PEP 597 – Add optional EncodingWarning

Docs

Documentation Build Status Docs Check Status

Tests

Linux Test Status Windows Test Status macOS Test Status Coverage

PyPI

PyPI - Package Version PyPI - Supported Python Versions PyPI - Supported Implementations PyPI - Wheel

Anaconda

Conda - Package Version Conda - Platform

Activity

GitHub last commit GitHub commits since tagged version Maintenance PyPI - Downloads

QA

CodeFactor Grade Flake8 Status mypy status

Other

License GitHub top language Requirements Status

Installation

python3 -m pip install flake8-encodings --user

In version 0.5.1 and above the functionality for checking classes (ConfigParser and Path for now) requires the classes extra to be installed:

python3 -m pip install flake8-encodings[classes]

The checks for classes are slower and CPU intensive, so only enable them if you use the classes in question.

Motivation

Developers using macOS or Linux may forget that the default encoding is not always UTF-8.

For example, long_description = open("README.md").read() in setup.py is a common mistake. Many Windows users cannot install the package if there is at least one non-ASCII character (e.g. emoji) in the README.md file which is encoded in UTF-8.

For example, 489 packages of the 4000 most downloaded packages from PyPI used non-ASCII characters in README. And 82 packages of them cannot be installed from source package when the locale encoding is ASCII. 1 They used the default encoding to read README or TOML file.

Even Python experts assume that default encoding is UTF-8. It creates bugs that happen only on Windows. See 2, 3, 4, and 5 for example.

PEP 597 proposed adding a new EncodingWarning to Python, which can be used in conjunction with this tool to identify issues at runtime.

Footnotes

1

“Packages can’t be installed when encoding is not UTF-8” (https://github.com/methane/pep597-pypi-ascii)

2

Packaging tutorial in packaging.python.org didn’t specify encoding to read a README.md (https://github.com/pypa/packaging.python.org/pull/682)

3

json.tool had used locale encoding to read JSON files. (https://bugs.python.org/issue33684)

4

site: Potential UnicodeDecodeError when handling pth file (https://bugs.python.org/issue33684)

5

pypa/pip: “Installing packages fails if Python 3 installed into path with non-ASCII characters” (https://github.com/pypa/pip/issues/9054)