SlideShare a Scribd company logo

Princeton Wintersession: Software Quality Assurance Tooling

Henry Schreiner, profile picture
Henry Schreiner

The document discusses software quality assurance tooling, focusing on Python tools such as pipx for package management and nox for task automation, including code quality checks and formatters. It emphasizes the importance of code quality in improving readability, finding errors early, and reducing merge conflicts, while also detailing various pre-commit hooks for maintaining quality and addressing specific issues in code. Additionally, it introduces several code formatting and linting tools to enhance Python projects and demonstrates how to configure and use them effectively.

Technology
1 of 61
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
Software Quality Assurance Tooling After today you will know SQAT Henry Schreiner January 23, 2023 ISciNumPy https://iscinumpy.dev Scikit-HEP Developer https://scikit-hep.org/developer https://github.com/henryiii/sqat-example
Language We’ll focus on Python (especially at fi rst) But the general concepts are around in most other languages You just need to fi nd the matching tool(s)
Packaging aside: pipx 3 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: build: make SDists and wheels twine: upload SDists and wheels cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder black: Python code formatter pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system Packages can con fl ict Updates get slower over time Lose track of why things are installed Manual updates are painful Hates Python being replaced $ pipx install <application> $ <application> Better! Automatic venv for each package No con fl icts ever Everything updatable / replaceable Doesn’t like Python being replaced $ pipx run <application> Best! Automatic venv caching Never more than a week old No pre-install or setup No maintenance Replace Python at will pipx run --spec git+https://github.com/henryiii/rich-cli@patch-1 rich pipx has fi rst class support on GHA & Azure!
Task runner aside: Nox 4 Make fi les Custom language Painful to write Painful to maintain Looks like garbage OS dependent No Python environments Everywhere Tox Custom language Concise to write Tricky to read Ties you to tox OS independent Python environments Python package Nox Python, mimics pytest Simple but verbose Easy to read Teaches commands OS independent Python environments Python package Other task runners available for other purposes, like Rake (Ruby)
Writing a nox fi le.py 5 import nox @nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11"]) def tests(session: nox.Session) -> None: """ Run the unit and regular tests. """ session.install(".[test]") session.run("pytest", *session.posargs)
Running nox 6 ~/g/s/uproot-browser   henryiii/feat/logo *$  nox -s tests-3.9 nox > Running session tests-3.9 nox > Creating virtual environment (virtualenv) using python3.9 in .nox/tests-3-9 nox > python -m pip install '.[test]' nox > pytest =========================================== test session starts =========================================== platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0 rootdir: /Users/henryschreiner/git/scikit-hep/uproot-browser, configfile: pyproject.toml, testpaths: tests collected 3 items tests/test_dirs.py .. [ 66%] tests/test_package.py . [100%] =========================================== 3 passed in 0.01s ============================================= nox > Session tests-3.9 was successful.
Features of nox 7 Full control over environments Easy fl y-by contributions Transparent, simple .nox directory Conda support Trade speed for reproducibility Some ideas for sessions lint tests docs build bump pylint regenerate update_pins check_manifest make_changelog update_python_dependencies See pypa/cibuildwheel pypa/manylinux scikit-hep/hist scikit-hep/boost-histogram pybind/pybind11 scikit-hep/cookie scikit-hep/scikit-hep.github.io Optional environment reuse
Python launcher for Unix 8 Rust implementation of “py” for UNIX But also automatically picks up .venv folder! Meant for lazy experts Launcher $ py -m pytest Classic $ . .venv/bin/activate (.venv) $ python -m pytest (.venv) $ deactivate Classic, take 2 $ .venv/bin/python -m pytest
Cookiecutter 9 Quickly set up a project Takes options Scikit-hep/cookie is a great cookiecutter for Python! How to run pipx run cookiecutter gh:scikit-hep/cookie
Part 0: Intro 10
Code Quality 11 Why does code quality matter? Improve readability Find errors before they happen Avoid historical baggage Reduce merge con fl icts Warm fuzzy feelings How to run Discussion of checks (Opinionated) Mostly focusing on Python today
pre-commit 12 Poorly named? Has a pre-commit hook mode You don’t have to use it that way! Generic check runner conda coursier dart docker docker_image dotnet fail golang lua node perl python python_venv r ruby rust swift pygrep script system Written in Python pipx, nox, homebrew, etc. Designed for speed & reproducibility Ultra fast environment caching Locked environments Easy autoupdate command pre-commit.ci Automatic updates Automatic fi xes for PRs Large library of hooks https://pre-commit.com/hooks.html Custom hooks are simple
Con fi guring pre-commit 13 Design A hook is just a YAML dict Fields can be overridden Environments globally cached by git tag Supports checks and fi xers # .pre-commit-config.yaml hooks: - repo: https://github.com/psf/black rev: "22.12.0" hooks: - id: black # Black’s .pre-commit-hooks.yaml - id: black name: black description: "Black: The uncompromising code formatter" entry: black language: python minimum_pre_commit_version: 2.9.2 require_serial: true types_or: [python, pyi] - id: black-jupyter name: black-jupyter description: "Black (with Jupyter Notebook support)" entry: black language: python minimum_pre_commit_version: 2.9.2 require_serial: true types_or: [python, pyi, jupyter] additional_dependencies: [".[jupyter]"] You write this Formatter author writes this
Options for pre-commit 14 Selected options fi les: explicit include regex exclude: explicit exclude regex types_or/types/exclude_types: fi le types args: control arguments additional_dependencies: extra things to install stages: select the git stage (like manual)
Running pre-commit 15 Run all checks pre-commit run -a Update all hooks pre-commit autoupdate Install as a pre-commit hook pre-commit install (Skip with git commit -n) Skip checks SKIP=… <run> Run one check pre-commit run -a <id> Run manual stage pre-commit run --hook-stage manual
Examples of pre-commit checks 16 Almost everything following in this talk - repo: local hooks: - id: disallow-caps name: Disallow improper capitalization language: pygrep entry: PyBind|Numpy|Cmake|CCache|Github|PyTest exclude: .pre-commit-config.yaml Don’t grep the fi le this is in! “Entry” is the grep, in this case Using pygrep “language” Custom hook
pre-commit/pre-commit-hooks 17 - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.4.0 hooks: - id: check-added-large-files - id: check-case-conflict - id: check-merge-conflict - id: check-symlinks - id: check-yaml - id: debug-statements - id: end-of-file-fixer - id: mixed-line-ending - id: requirements-txt-fixer - id: trailing-whitespace Small common checks Some Python leaning Some pre-commit hook specialization
pre-commit/pygrep-hooks 18 Small common pygreps - repo: https://github.com/pre-commit/pygrep-hooks rev: "v1.10.0" hooks: - id: python-check-blanket-noqa - id: python-check-blanket-type-ignore - id: python-no-eval - id: python-use-type-annotations - id: rst-backticks - id: rst-directive-colons - id: rst-inline-touching-normal Opinion: blanket CQ ignores are bad. Ignores should be speci fi c Better readability/searchability - avoid hiding unrelated issue Optional, some fi les might need to be excluded
CI (GitHub Actions) 19 on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-python@v3 - uses: pre-commit/action@v3.0.0 Great, fast caching, but maintenance only - replaced by pre-commit.ci on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - run: pipx run nox -s lint @nox.session def lint(session: nox.Session) -> None: session.install("pre-commit") session.run("pre-commit", "run", "--all-files", "--show-diff-on-failure", *session.posargs)
Useful GitHub Actions 20 actions/checkout actions/setup-python actions/cache actions/upload-artifact actions/download-artifact ilammy/msvc-dev-cmd jwlawson/actions-setup-cmake excitedleigh/setup-nox pypa/gh-action-pypi-publish pre-commit/action conda-incubator/setup-miniconda peaceiris/actions-gh-pages ruby/setup-miniconda Writing your own composite action is really easy!
Part 1: Formatters 21
Using code formatters 22 Existing projects Apply all-at-once, not spread out over time Add the format commit to .git-blame-ignore-revs
Black 23 hooks: - repo: https://github.com/psf/black rev: "22.12.0" hooks: - id: black-jupyter Python code formatter Close to the one true format for Python Almost not con fi gurable (this is a feature) A good standard is better than perfection Designed to reduce merge con fl icts Reading blacked code is fast Write your code to produce nice formatting You can disable line/lines if you have to Workaround for single quotes (use double) Magic trailing comma Online version https://black.vercel.app
Write for good format 24 raise RuntimeError( "This was not a valid value for some_value: {}".format(repr(some_value)) ) Bad: msg = f"This was not a valid value for some_value: {some_value!r}" raise RuntimeError(msg) Good: Better stacktrace More readable Two lines instead of three Faster (f-string)
Notebook cleaner 25 hooks: - repo: https://github.com/kynan/nbstripout rev: "0.6.0" hooks: - id: nbstripout Remove outputs from notebooks Best if not stored in VCS You can render outputs in JupyterBook, etc. Use Binder or JupyterLite
NBQA: Notebook adaptor 26 - repo: https://github.com/nbQA-dev/nbQA rev: 1.6.1 hooks: - id: nbqa-pyupgrade args: ["--py37-plus"] - id: nbqa-isort args: ["--float-to-top"] Run Python formatters on notebooks Lots of formatters supported Black natively supports notebooks these days, FYI
isort 27 hooks: - repo: https://github.com/PyCQA/isort rev: "5.11.4" hooks: - id: isort Sort your Python imports Very con fi gurable Reduces merge con fl icts Grouping imports helps readers Can inject future imports # pyproject.toml [tool.isort] profile = "black" args: ["-a", "from __future__ import annotations"] Default groupings Future imports Stdlib imports Third party packages Local imports
pyupgrade 28 hooks: - repo: https://github.com/asottile/pyupgrade rev: "v3.3.1" hooks: - id: pyupgrade args: [--py37-plus] Update Python syntax Avoid deprecated or obsolete code Fairly cautious Can target a speci fi c Python 3 min (Mostly) not con fi gurable Remove static if sys.version_info blocks Python 2.7 Set literals Dictionary comprehensions Generators in functions Format speci fi er & .format ⚙ Comparison for const literals (3.8 warn) Invalid escapes Python 3 Unicode literals Long literals, octal literals Modern super() New style classes Future import removal yield from Remove six compatibility code io.open -> open Remove error aliases Python 3.x f-strings (partial) (3.6) ⚙ NamedTuple/TypedDict (3.6) subprocess.run updates (3.7) lru_cache parens (3.8) lru_cache(None) -> cache (3.9) Typing & annotation rewrites (various) abspath(__file__) removal (3.9)
pyupgrade examples 29 Before After for a, b in c: yield (a, b) yield from c "{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}" dict([(a, b) for a, b in y]) {a: b for a, b in y}
pyupgrade limits 30 PyUpgrade does not over modernize isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10) No match statement conversions (3.10) Nothing converts to using walrus := (3.8) (probably a good thing!) Except for a bit of typing Optional[int] -> int | None (I like this one now, though) ❌
setup-cfg-fmt 31 hooks: - repo: https://github.com/asottile/setup-cfg-fmt rev: "v2.2.0" hooks: - id: setup-cfg-fmt args: [--max-py-version=3.11, --include-version-classifiers] Maintain setup.cfg fi le Can add and fi x trove classi fi ers Sorts and cleans A bit opinionated, will not fi x some bugs What about pyproject.toml? Three projects (at least) popping up Very young Best process unclear ini2toml useful for conversion Or hatch new --init
Part 2: Linters 32
Using code linters 33 Existing projects Feel free to build a long ignore list Work on one or a few at a time You don’t have to have every check
hooks: - repo: https://gitlab.com/pycqa/flake8 rev: "6.0.0" hooks: - id: flake8 additional_dependencies: [flake8-bugbear] Flake8 34 Fast simple extendable linter Very con fi gurable: setup.cfg or . fl ake8 Many plugins, local plugins easy No auto- fi xers like rubocop (Ruby) Opinion: fl ake8-bugbear is great Example: fl ake8-print, avoid all prints # .flake8 [flake8] max-complexity = 12 extend-ignore = E203, E501, E722, B950 extend-select = B902, B903, B904
Flake8 example checks 35 Bugbear Do not use bare except No mutable argument defaults getattr(x, "const") should be x.const No assert False, use raise AssertionError Pointless comparison ❤ pytest PyFlakes (default) Unused modules & variables String formatting mistakes No placeholders in f-string Dictionary key repetition Assert a tuple (it’s always true) Various syntax errors Unde fi ned names Rede fi nition of unused var ❤ pytest McCabe (default) Complexity checks PyCodeStyle (default) Style checks Flake8-print Avoid leaking debugging print statements
Custom local fl ake8 plugin 36 import ast import sys from typing import NamedTuple, Iterator class Flake8ASTErrorInfo(NamedTuple): line_number: int offset: int msg: str cls: type # unused
Custom local fl ake8 plugin 37 class Visitor(ast.NodeVisitor): msg = "AK101 exception must be wrapped in ak._v2._util.*error" def __init__(self) -> None: self.errors: list[Flake8ASTErrorInfo] = [] def visit_Raise(self, node: ast.Node) -> None: if isinstance(node.exc, ast.Call): if isinstance(node.exc.func, ast.Attribute): if node.exc.func.attr in {"error", "indexerror"}: return if node.exc.func.id in {"ImportError"}: return self.errors.append( Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self)) )
Custom local fl ake8 plugin 38 class AwkwardASTPlugin: name = "flake8_awkward" version = "0.0.0" def __init__(self, tree: ast.AST) -> None: self._tree = tree def run(self) -> Iterator[Flake8ASTErrorInfo]: visitor = Visitor() visitor.visit(self._tree) yield from visitor.errors
Custom local fl ake8 plugin 39 [flake8:local-plugins] extension = AK1 = flake8_awkward:AwkwardASTPlugin paths = ./dev/ def main(path: str) -> None: with open(path) as f: code = f.read() node = ast.parse(code) plugin = AwkwardASTPlugin(node) for err in plugin.run(): print(f"{path}:{err.line_number}:{err.offset} {err.msg}") if __name__ == "__main__": for item in sys.argv[1:]: main(item)
Flake8 helpers 40 hooks: - repo: https://github.com/hadialqattan/pycln rev: "v2.1.3" hooks: - id: pycln args: [--config=pyproject.toml] stages: [manual] hooks: - repo: https://github.com/asottile/yesqa rev: "v1.4.0" hooks: - id: yesqa additional_dependencies: &flake8-dependencies - flake8-bugbear - repo: https://gitlab.com/pycqa/flake8 rev: "6.0.0" hooks: - id: flake8 additional_dependencies: *flake8-dependencies # pyproject.toml [tool.pycln] all = true PyCln Quickly remove unused imports Great as a manual hook YesQA Remove unused NoQA’s
PyLint 41 PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it) It’s also a bit slow, so a good candidate for nox @nox.session def pylint(session: nox.Session) -> None: session.install("-e", ".") session.install("pylint") session.run("pylint", "src", *session.posargs) # pyproject.toml [tool.pylint] master.py-version = "3.7" master.jobs = "0" reports.output-format = "colorized" similarities.ignore-imports = "yes" messages_control.enable = ["useless-suppression"] messages_control.disable = [ "design", "fixme", "line-too-long", "wrong-import-position", ] Code linter Can be very opinionated Signal to noise ratio poor You will need to disable checks - that’s okay! A bit more advanced / less static than fl ake8 But can catch hard to fi nd bugs! For an example of lots of suppressions: https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml
Example PyLint rules 42 Duplicate code Finds large repeated code patterns Attribute de fi ned outside init Only __init__ should de fi ne attributes No self use Can be @classmethod or @staticmethod Unnecessary code Lambdas, comprehensions, etc. Unreachable code Finds things that can’t be reached Consider using in x in {stuff} vs chaining or’s Arguments di ff er Subclass should have matching arguments Consider iterating dictionary Better use of dictionary iteration Consider merging isinstance You can use a tuple in isinstance Useless else on loop They are bad enough when useful :) Consider using enumerate Avoid temp variables, idiomatic Global variable not assigned You should only declare global to assign
Controversial PyLint rules 43 No else after control- fl ow Guard-style only Can simply complex control fl ow Removes useless indentation if x: return x else: return None # Should be: if x: return x return None # Or: return x if x else None # Or: return x or None Design Too many various things Too few methods Can just silence “design” (I’m on the in-favor side)
Ruff: one linter to rule them all? 44 A very new entry in the Python linting space, but gaining traction like wild fi re 100x faster than existing Python linters Has support for fi xers! Implements all of (modern) fl ake8’s checks Implements dozens of fl ake8 plugins Implements most of pyupgrade (as rules!) Can replace simple usage of isort Can remove unused noqa’s Can remove unused imports Some of PyLint 0 dependencies Con fi gured with pyproject.toml Only binary platforms (Rust compiled) Doesn’t support user plugins Examples https://github.com/scikit-build/scikit-build-core https://github.com/pypa/build
Ruff con fi g 45 [tool.ruff] select = [ "E", "F", "W", # flake8 "B", "B904", # flake8-bugbear "I", # isort "ARG", # flake8-unused-arguments "C4", # flake8-comprehensions "EM", # flake8-errmsg "ICN", # flake8-import-conventions "ISC", # flake8-implicit-str-concat "PGH", # pygrep-hooks "PIE", # flake8-pie "PLC", "PLE", "PLR", "PLW", # pylint "PT", # flake8-pytest-style "RET", # flake8-return "RUF", # Ruff-specific "SIM", # flake8-simplify "T20", # flake8-print "UP", # pyupgrade "YTT", # flake8-2020 ] ignore = ["PLR2004", "E501"] target-version = "py37" typing-modules = ["scikit_build_core._compat.typing"] src = ["src"] unfixable = ["T20"] [tool.ruff.per-file-ignores] "tests/**" = ["T20"] "noxfile.py" = ["T20"] - repo: https://github.com/charliermarsh/ruff-pre-commit rev: v0.0.229 hooks: - id: ruff args: ["--fix"] Flake8 con fi g? Try: pipx run flake8-to-ruff .flake8
Static type checking: MyPy 46 hooks: - repo: https://gitlab.com/pre-commit/mirrors-mypy rev: "v0.991" hooks: - id: mypy files: src args: [--show-error-codes] Like a linter on steroids Uses Python typing Enforces correct type annotations Designed to be iteratively enabled Should be in a controlled environment (pre-commit or nox) Always specify args (bad hook defaults) Almost always need additional_dependencies Con fi gure in pyproject.toml Pros Can catch many things tests normally catch, without writing tests Therefore it can catch things not covered by tests (yet, hopefully) Code is more readable with types Sort of works without types initially Cons Lots of work to add all types Typing can be tricky in Python Active development area for Python
Con fi guring MyPy 47 [tool.mypy] files = "src" python_version = "3.7" warn_unused_configs = true strict = true [[tool.mypy.overrides]] module = [ "numpy.*" ] ignore_missing_imports = true Start small Start without strictness Add a check at a time Extra libraries Try adding them to your environment You can ignore untyped or slow libraries You can provide stubs for untyped libraries if you want Tests? Adding pytest is rather slow I prefer to avoid tests, or keep them mostly untyped
Typing tricks 48 Protocols Better than ABCs, great for duck typing @typing.runtime_checkable class Duck(Protocol): def quack() -> str: ... def f(x: Duck) -> str: return x.quack() class MyDuck: def quack() -> str: return "quack" if typing.TYPE_CHECKING: _: Duck = typing.cast(MyDuck, None) Type Narrowing Integral to how mypy works x: Union[A, B] if isinstance(x, A): reveal_type(x) # A else: reveal_type(x) # B Make a typed package Must include py.typed marker fi le Always use sys.version_info Better for readers than try/except, and static Also sys.platform instead of os.name
Future annotations 49 Classic code (3.5+) from typing import Union, List def f(x: int) -> List[int]: return list(range(x)) def g(x: Union[str, int]) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Modern code (3.7+) from __future__ import annotations def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Ultramodern code (3.10+) def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) With the future import, you get all the bene fi ts of future code in 3.7+ annotations Typing is already extra code, simpler is better
Part 3: Other languages
Clang-format 51 hooks: - repo: https://github.com/pre-commit/mirrors-clang-format rev: "v15.0.7" hooks: - id: clang-format types_or: [c++, c, cuda] C++ and more code formatter Very con fi gurable: .clang-format fi le Opinion: stay close to llvm style PyPI clang-format wheels, under 2MB No more issues with mismatched LLVM!
CMake-format 52 hooks: - repo: https://github.com/cheshirekow/cmake-format-precommit rev: "v0.6.13" hooks: - id: cmake-format additional_dependencies: [pyyaml] CMake code formatter Very con fi gurable: .cmake-format.yaml fi le Anything that helps with CMake!
Markdown & YAML with Prettier 53 hooks: - repo: https://github.com/pre-commit/mirrors-prettier rev: "v3.0.0-alpha.4" hooks: - id: prettier types_or: [yaml, markdown, html, css, scss, javascript, json] args: [--prose-wrap=always] exclude: "^tests" JavaScript Linter Lots of formats supported A few customization points
ShellCheck 54 hooks: - repo: https://github.com/shellcheck-py/shellcheck-py rev: "v0.9.0.2" hooks: - id: shellcheck Linter for bash scripts Can locally disable Prioritizes correctness over terseness
CodeSpell 55 hooks: - repo: https://github.com/codespell-project/codespell rev: "v2.2.2" hooks: - id: shellcheck args: ["-L", "sur,nd"] Find common misspellings Inverted spell checker - looks for misspellings Can con fi gure or provide wordlist Actually can catch bugs!
See how your repo measures! 56 https://scikit-hep.org/developer/reporeview
Bonus part: pytest tips 57
pytest tips 58 Spend time learning pytest Full of amazing things that really make testing fun! Tests are code too Or for C++: Catch2 or doctest, etc. Also maybe learn Hypothesis for pytest [tool.pytest.ini_options] minversion = "6.0" addopts = [ "-ra", "--showlocals", "--strict-markers", "--strict-config", ] xfail_strict = true filterwarnings = [ "error", ] log_cli_level = "info" testpaths = [ "tests", ] Don’t let warnings slip by! Makes logging more useful Strictness is good Useful summary Print out locals on errors Use pytest.approx Even works on numpy arrays Remember to test for failures If you expect a failure, test it! Test your installed package That’s how users will get it, not from a directory
pytest Tricks 59 Mock and Monkeypatch This is how you make tricky tests “unit” tests Fixtures This keeps tests simple and scalable @pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True) def platform_system(request, monkeypatch): monkeypatch.setattr(platform, "system", lambda _: request.param) Parametrize Directly or in a fi xture for reuse Use conftest.py Fixtures available in same and nested directories
Running pytest 60 Show locals on failure --showlocals/-l Jump into a debugger on failure --pdb Start with last failing test --lf Jump into a debugger immediately --trace or use breakpoint() Run matching tests -k <expression> Run speci fi c test filename.py::testname Run speci fi c marker -m <marker> Control traceback style --tb=<style>
In conclusion 61 Code quality tools can help a lot with Readability Reducing bugs Boosting developer productivity Consistency Refactoring Teaching others good practice too Hopefully we have had some helpful discussions! It’s okay to disable a check Try to understand why it’s there Remember there are multiple concerns involved in decisions

More Related Content

PDF
Software Quality Assurance Tooling 2023
Henry Schreiner
 
PDF
Digital RSE: automated code quality checks - RSE group meeting
Henry Schreiner
 
PDF
Software Quality Assurance Tooling - Wintersession 2024
Henry Schreiner
 
PDF
Tools That Help You Write Better Code - 2025 Princeton Software Engineering S...
Henry Schreiner
 
PDF
Tools to help you write better code - Princeton Wintersession
Henry Schreiner
 
PDF
Helpful pre commit hooks for Python and Django
roskakori
 
PDF
Princeton RSE: Building Python Packages (+binary)
Henry Schreiner
 
PDF
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 
Software Quality Assurance Tooling 2023
Henry Schreiner
 
Digital RSE: automated code quality checks - RSE group meeting
Henry Schreiner
 
Software Quality Assurance Tooling - Wintersession 2024
Henry Schreiner
 
Tools That Help You Write Better Code - 2025 Princeton Software Engineering S...
Henry Schreiner
 
Tools to help you write better code - Princeton Wintersession
Henry Schreiner
 
Helpful pre commit hooks for Python and Django
roskakori
 
Princeton RSE: Building Python Packages (+binary)
Henry Schreiner
 
SciPy 2025 - Packaging a Scientific Python Project
Henry Schreiner
 

Similar to Princeton Wintersession: Software Quality Assurance Tooling (20)

PDF
SciPy22 - Building binary extensions with pybind11, scikit build, and cibuild...
Henry Schreiner
 
PDF
Everything you didn't know you needed
Henry Schreiner
 
PDF
SciPy 2022 Scikit-HEP
Henry Schreiner
 
PDF
Python testing like a pro by Keith Yang
PYCON MY PLT
 
PDF
PyBCN - pipenv - python dev workflow for humans
Andreu Vallbona Plazas
 
PDF
Python Linters at Scale.pdf
Jimmy Lai
 
PDF
Princeton RSE Peer network first meeting
Henry Schreiner
 
PPTX
Complete python toolbox for modern developers
Jan Giacomelli
 
PDF
Tools for maintaining an open source project
All Things Open
 
PDF
PyCon JP 2024 Streamlining Testing in a Large Python Codebase .pdf
Jimmy Lai
 
PPTX
The New York Times: Sustainable Systems, Powered by Python
All Things Open
 
PDF
Isolated development in python
Andrés J. Díaz
 
PDF
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
PDF
Py Day Mallorca - Pipenv - Python Dev Workflow for Humans
Andreu Vallbona Plazas
 
PDF
Pipenv python dev workflow for humans
Andreu Vallbona Plazas
 
PPTX
¡El mejor lenguaje para automatizar pruebas!
Antonio Robres Turon
 
PDF
Pipfile, pipenv, pip… what?!
Ivan Chernoff
 
PDF
PyCon2022 - Building Python Extensions
Henry Schreiner
 
ODP
biopython, doctest and makefiles
Giovanni Marco Dall'Olio
 
PDF
Effectively using Open Source with conda
Travis Oliphant
 
SciPy22 - Building binary extensions with pybind11, scikit build, and cibuild...
Henry Schreiner
 
Everything you didn't know you needed
Henry Schreiner
 
SciPy 2022 Scikit-HEP
Henry Schreiner
 
Python testing like a pro by Keith Yang
PYCON MY PLT
 
PyBCN - pipenv - python dev workflow for humans
Andreu Vallbona Plazas
 
Python Linters at Scale.pdf
Jimmy Lai
 
Princeton RSE Peer network first meeting
Henry Schreiner
 
Complete python toolbox for modern developers
Jan Giacomelli
 
Tools for maintaining an open source project
All Things Open
 
PyCon JP 2024 Streamlining Testing in a Large Python Codebase .pdf
Jimmy Lai
 
The New York Times: Sustainable Systems, Powered by Python
All Things Open
 
Isolated development in python
Andrés J. Díaz
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Py Day Mallorca - Pipenv - Python Dev Workflow for Humans
Andreu Vallbona Plazas
 
Pipenv python dev workflow for humans
Andreu Vallbona Plazas
 
¡El mejor lenguaje para automatizar pruebas!
Antonio Robres Turon
 
Pipfile, pipenv, pip… what?!
Ivan Chernoff
 
PyCon2022 - Building Python Extensions
Henry Schreiner
 
biopython, doctest and makefiles
Giovanni Marco Dall'Olio
 
Effectively using Open Source with conda
Travis Oliphant
 
Ad

More from Henry Schreiner (18)

PDF
Learning Rust with Advent of Code 2023 - Princeton
Henry Schreiner
 
PDF
The two flavors of Python 3.13 - PyHEP 2024
Henry Schreiner
 
PDF
Modern binary build systems - PyCon 2024
Henry Schreiner
 
PDF
What's new in Python 3.11
Henry Schreiner
 
PDF
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
Henry Schreiner
 
PDF
boost-histogram / Hist: PyHEP Topical meeting
Henry Schreiner
 
PDF
CMake best practices
Henry Schreiner
 
PDF
Pybind11 - SciPy 2021
Henry Schreiner
 
PDF
RDM 2020: Python, Numpy, and Pandas
Henry Schreiner
 
PDF
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
Henry Schreiner
 
PDF
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
Henry Schreiner
 
PDF
ACAT 2019: A hybrid deep learning approach to vertexing
Henry Schreiner
 
PDF
2019 CtD: A hybrid deep learning approach to vertexing
Henry Schreiner
 
PDF
2019 IRIS-HEP AS workshop: Boost-histogram and hist
Henry Schreiner
 
PDF
IRIS-HEP: Boost-histogram and Hist
Henry Schreiner
 
PDF
2019 IRIS-HEP AS workshop: Particles and decays
Henry Schreiner
 
PDF
IRIS-HEP Retreat: Boost-Histogram Roadmap
Henry Schreiner
 
PDF
PyHEP 2019: Python 3.8
Henry Schreiner
 
Learning Rust with Advent of Code 2023 - Princeton
Henry Schreiner
 
The two flavors of Python 3.13 - PyHEP 2024
Henry Schreiner
 
Modern binary build systems - PyCon 2024
Henry Schreiner
 
What's new in Python 3.11
Henry Schreiner
 
PyCon 2022 -Scikit-HEP Developer Pages: Guidelines for modern packaging
Henry Schreiner
 
boost-histogram / Hist: PyHEP Topical meeting
Henry Schreiner
 
CMake best practices
Henry Schreiner
 
Pybind11 - SciPy 2021
Henry Schreiner
 
RDM 2020: Python, Numpy, and Pandas
Henry Schreiner
 
HOW 2019: Machine Learning for the Primary Vertex Reconstruction
Henry Schreiner
 
HOW 2019: A complete reproducible ROOT environment in under 5 minutes
Henry Schreiner
 
ACAT 2019: A hybrid deep learning approach to vertexing
Henry Schreiner
 
2019 CtD: A hybrid deep learning approach to vertexing
Henry Schreiner
 
2019 IRIS-HEP AS workshop: Boost-histogram and hist
Henry Schreiner
 
IRIS-HEP: Boost-histogram and Hist
Henry Schreiner
 
2019 IRIS-HEP AS workshop: Particles and decays
Henry Schreiner
 
IRIS-HEP Retreat: Boost-Histogram Roadmap
Henry Schreiner
 
PyHEP 2019: Python 3.8
Henry Schreiner
 
Ad

Recently uploaded (20)

PPTX
TLE Review Electricity (Electricity).pptx
JayPolicarpio2
 
PPTX
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
PPTX
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
PDF
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
PPTX
A Presentation on Artificial Intelligence
memembruh336
 
PDF
NewMind AI Weekly Chronicles - August'25-Week II
NewMind AI
 
PPTX
cloud_computing_Infrastucture_as_cloud_p
mainakbhattacharya20
 
PDF
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
PDF
August Patch Tuesday
Ivanti
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PPTX
Chapter 5: Probability Theory and Statistics
RandyFin
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
PDF
project resource management chapter-09.pdf
ssuser00e6e21
 
PDF
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PPTX
SOPHOS-XG Firewall Administrator PPT.pptx
AgnesMunisi
 
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 
TLE Review Electricity (Electricity).pptx
JayPolicarpio2
 
Programs and apps: productivity, graphics, security and other tools
4mqw9zch22
 
1. Introduction to Computer Programming.pptx
bonakiduza1024
 
Encapsulation_ Review paper, used for researhc scholars
gurumoop
 
A Presentation on Artificial Intelligence
memembruh336
 
NewMind AI Weekly Chronicles - August'25-Week II
NewMind AI
 
cloud_computing_Infrastucture_as_cloud_p
mainakbhattacharya20
 
Assigned Numbers - 2025 - Bluetooth® Document
Bloombase
 
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
ssuser28425f
 
August Patch Tuesday
Ivanti
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
Chapter 5: Probability Theory and Statistics
RandyFin
 
Encapsulation theory and applications.pdf
gurumoop
 
A comparative analysis of optical character recognition models for extracting...
IAESIJAI
 
Univ-Connecticut-ChatGPT-Presentaion.pdf
ry7r52mzb4
 
project resource management chapter-09.pdf
ssuser00e6e21
 
WOOl fibre morphology and structure.pdf for textiles
Rajendrakumar868651
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
SOPHOS-XG Firewall Administrator PPT.pptx
AgnesMunisi
 
Digital-Transformation-Roadmap-for-Companies.pptx
David Malick Dieng
 

Princeton Wintersession: Software Quality Assurance Tooling

  • 1. Software Quality Assurance Tooling After today you will know SQAT Henry Schreiner January 23, 2023 ISciNumPy https://iscinumpy.dev Scikit-HEP Developer https://scikit-hep.org/developer https://github.com/henryiii/sqat-example
  • 2. Language We’ll focus on Python (especially at fi rst) But the general concepts are around in most other languages You just need to fi nd the matching tool(s)
  • 3. Packaging aside: pipx 3 $ pip install <application> $ <application> I’m sure you’ve seen this: Examples of applications: build: make SDists and wheels twine: upload SDists and wheels cibuildwheel: make redistributable wheels nox/tox: Python task runners jupylite: WebAssembly Python site builder black: Python code formatter pypi-command-line: query PyPI uproot-browser: ROOT file browser (HEP) tiptop: fancy top-style monitor rich-cli: pretty print files cookiecutter: template packages clang-format: format C/C++/CUDA code pre-commit: general CQA tool cmake: build system generator meson: another build system generator ninja: build system Packages can con fl ict Updates get slower over time Lose track of why things are installed Manual updates are painful Hates Python being replaced $ pipx install <application> $ <application> Better! Automatic venv for each package No con fl icts ever Everything updatable / replaceable Doesn’t like Python being replaced $ pipx run <application> Best! Automatic venv caching Never more than a week old No pre-install or setup No maintenance Replace Python at will pipx run --spec git+https://github.com/henryiii/rich-cli@patch-1 rich pipx has fi rst class support on GHA & Azure!
  • 4. Task runner aside: Nox 4 Make fi les Custom language Painful to write Painful to maintain Looks like garbage OS dependent No Python environments Everywhere Tox Custom language Concise to write Tricky to read Ties you to tox OS independent Python environments Python package Nox Python, mimics pytest Simple but verbose Easy to read Teaches commands OS independent Python environments Python package Other task runners available for other purposes, like Rake (Ruby)
  • 5. Writing a nox fi le.py 5 import nox @nox.session(python=["3.7", "3.8", "3.9", "3.10", "3.11"]) def tests(session: nox.Session) -> None: """ Run the unit and regular tests. """ session.install(".[test]") session.run("pytest", *session.posargs)
  • 6. Running nox 6 ~/g/s/uproot-browser   henryiii/feat/logo *$  nox -s tests-3.9 nox > Running session tests-3.9 nox > Creating virtual environment (virtualenv) using python3.9 in .nox/tests-3-9 nox > python -m pip install '.[test]' nox > pytest =========================================== test session starts =========================================== platform darwin -- Python 3.9.10, pytest-7.0.1, pluggy-1.0.0 rootdir: /Users/henryschreiner/git/scikit-hep/uproot-browser, configfile: pyproject.toml, testpaths: tests collected 3 items tests/test_dirs.py .. [ 66%] tests/test_package.py . [100%] =========================================== 3 passed in 0.01s ============================================= nox > Session tests-3.9 was successful.
  • 7. Features of nox 7 Full control over environments Easy fl y-by contributions Transparent, simple .nox directory Conda support Trade speed for reproducibility Some ideas for sessions lint tests docs build bump pylint regenerate update_pins check_manifest make_changelog update_python_dependencies See pypa/cibuildwheel pypa/manylinux scikit-hep/hist scikit-hep/boost-histogram pybind/pybind11 scikit-hep/cookie scikit-hep/scikit-hep.github.io Optional environment reuse
  • 8. Python launcher for Unix 8 Rust implementation of “py” for UNIX But also automatically picks up .venv folder! Meant for lazy experts Launcher $ py -m pytest Classic $ . .venv/bin/activate (.venv) $ python -m pytest (.venv) $ deactivate Classic, take 2 $ .venv/bin/python -m pytest
  • 9. Cookiecutter 9 Quickly set up a project Takes options Scikit-hep/cookie is a great cookiecutter for Python! How to run pipx run cookiecutter gh:scikit-hep/cookie
  • 11. Code Quality 11 Why does code quality matter? Improve readability Find errors before they happen Avoid historical baggage Reduce merge con fl icts Warm fuzzy feelings How to run Discussion of checks (Opinionated) Mostly focusing on Python today
  • 12. pre-commit 12 Poorly named? Has a pre-commit hook mode You don’t have to use it that way! Generic check runner conda coursier dart docker docker_image dotnet fail golang lua node perl python python_venv r ruby rust swift pygrep script system Written in Python pipx, nox, homebrew, etc. Designed for speed & reproducibility Ultra fast environment caching Locked environments Easy autoupdate command pre-commit.ci Automatic updates Automatic fi xes for PRs Large library of hooks https://pre-commit.com/hooks.html Custom hooks are simple
  • 13. Con fi guring pre-commit 13 Design A hook is just a YAML dict Fields can be overridden Environments globally cached by git tag Supports checks and fi xers # .pre-commit-config.yaml hooks: - repo: https://github.com/psf/black rev: "22.12.0" hooks: - id: black # Black’s .pre-commit-hooks.yaml - id: black name: black description: "Black: The uncompromising code formatter" entry: black language: python minimum_pre_commit_version: 2.9.2 require_serial: true types_or: [python, pyi] - id: black-jupyter name: black-jupyter description: "Black (with Jupyter Notebook support)" entry: black language: python minimum_pre_commit_version: 2.9.2 require_serial: true types_or: [python, pyi, jupyter] additional_dependencies: [".[jupyter]"] You write this Formatter author writes this
  • 14. Options for pre-commit 14 Selected options fi les: explicit include regex exclude: explicit exclude regex types_or/types/exclude_types: fi le types args: control arguments additional_dependencies: extra things to install stages: select the git stage (like manual)
  • 15. Running pre-commit 15 Run all checks pre-commit run -a Update all hooks pre-commit autoupdate Install as a pre-commit hook pre-commit install (Skip with git commit -n) Skip checks SKIP=… <run> Run one check pre-commit run -a <id> Run manual stage pre-commit run --hook-stage manual
  • 16. Examples of pre-commit checks 16 Almost everything following in this talk - repo: local hooks: - id: disallow-caps name: Disallow improper capitalization language: pygrep entry: PyBind|Numpy|Cmake|CCache|Github|PyTest exclude: .pre-commit-config.yaml Don’t grep the fi le this is in! “Entry” is the grep, in this case Using pygrep “language” Custom hook
  • 17. pre-commit/pre-commit-hooks 17 - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.4.0 hooks: - id: check-added-large-files - id: check-case-conflict - id: check-merge-conflict - id: check-symlinks - id: check-yaml - id: debug-statements - id: end-of-file-fixer - id: mixed-line-ending - id: requirements-txt-fixer - id: trailing-whitespace Small common checks Some Python leaning Some pre-commit hook specialization
  • 18. pre-commit/pygrep-hooks 18 Small common pygreps - repo: https://github.com/pre-commit/pygrep-hooks rev: "v1.10.0" hooks: - id: python-check-blanket-noqa - id: python-check-blanket-type-ignore - id: python-no-eval - id: python-use-type-annotations - id: rst-backticks - id: rst-directive-colons - id: rst-inline-touching-normal Opinion: blanket CQ ignores are bad. Ignores should be speci fi c Better readability/searchability - avoid hiding unrelated issue Optional, some fi les might need to be excluded
  • 19. CI (GitHub Actions) 19 on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-python@v3 - uses: pre-commit/[email protected] Great, fast caching, but maintenance only - replaced by pre-commit.ci on: pull_request: push: branches: - main jobs: lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - run: pipx run nox -s lint @nox.session def lint(session: nox.Session) -> None: session.install("pre-commit") session.run("pre-commit", "run", "--all-files", "--show-diff-on-failure", *session.posargs)
  • 22. Using code formatters 22 Existing projects Apply all-at-once, not spread out over time Add the format commit to .git-blame-ignore-revs
  • 23. Black 23 hooks: - repo: https://github.com/psf/black rev: "22.12.0" hooks: - id: black-jupyter Python code formatter Close to the one true format for Python Almost not con fi gurable (this is a feature) A good standard is better than perfection Designed to reduce merge con fl icts Reading blacked code is fast Write your code to produce nice formatting You can disable line/lines if you have to Workaround for single quotes (use double) Magic trailing comma Online version https://black.vercel.app
  • 24. Write for good format 24 raise RuntimeError( "This was not a valid value for some_value: {}".format(repr(some_value)) ) Bad: msg = f"This was not a valid value for some_value: {some_value!r}" raise RuntimeError(msg) Good: Better stacktrace More readable Two lines instead of three Faster (f-string)
  • 25. Notebook cleaner 25 hooks: - repo: https://github.com/kynan/nbstripout rev: "0.6.0" hooks: - id: nbstripout Remove outputs from notebooks Best if not stored in VCS You can render outputs in JupyterBook, etc. Use Binder or JupyterLite
  • 26. NBQA: Notebook adaptor 26 - repo: https://github.com/nbQA-dev/nbQA rev: 1.6.1 hooks: - id: nbqa-pyupgrade args: ["--py37-plus"] - id: nbqa-isort args: ["--float-to-top"] Run Python formatters on notebooks Lots of formatters supported Black natively supports notebooks these days, FYI
  • 27. isort 27 hooks: - repo: https://github.com/PyCQA/isort rev: "5.11.4" hooks: - id: isort Sort your Python imports Very con fi gurable Reduces merge con fl icts Grouping imports helps readers Can inject future imports # pyproject.toml [tool.isort] profile = "black" args: ["-a", "from __future__ import annotations"] Default groupings Future imports Stdlib imports Third party packages Local imports
  • 28. pyupgrade 28 hooks: - repo: https://github.com/asottile/pyupgrade rev: "v3.3.1" hooks: - id: pyupgrade args: [--py37-plus] Update Python syntax Avoid deprecated or obsolete code Fairly cautious Can target a speci fi c Python 3 min (Mostly) not con fi gurable Remove static if sys.version_info blocks Python 2.7 Set literals Dictionary comprehensions Generators in functions Format speci fi er & .format ⚙ Comparison for const literals (3.8 warn) Invalid escapes Python 3 Unicode literals Long literals, octal literals Modern super() New style classes Future import removal yield from Remove six compatibility code io.open -> open Remove error aliases Python 3.x f-strings (partial) (3.6) ⚙ NamedTuple/TypedDict (3.6) subprocess.run updates (3.7) lru_cache parens (3.8) lru_cache(None) -> cache (3.9) Typing & annotation rewrites (various) abspath(__file__) removal (3.9)
  • 29. pyupgrade examples 29 Before After for a, b in c: yield (a, b) yield from c "{foo} {bar}".format(foo=foo, bar=bar) f"{foo} {bar}" dict([(a, b) for a, b in y]) {a: b for a, b in y}
  • 30. pyupgrade limits 30 PyUpgrade does not over modernize isinstance(x, (int, str)) -> isinstance(x, int | str) (3.10) No match statement conversions (3.10) Nothing converts to using walrus := (3.8) (probably a good thing!) Except for a bit of typing Optional[int] -> int | None (I like this one now, though) ❌
  • 31. setup-cfg-fmt 31 hooks: - repo: https://github.com/asottile/setup-cfg-fmt rev: "v2.2.0" hooks: - id: setup-cfg-fmt args: [--max-py-version=3.11, --include-version-classifiers] Maintain setup.cfg fi le Can add and fi x trove classi fi ers Sorts and cleans A bit opinionated, will not fi x some bugs What about pyproject.toml? Three projects (at least) popping up Very young Best process unclear ini2toml useful for conversion Or hatch new --init
  • 33. Using code linters 33 Existing projects Feel free to build a long ignore list Work on one or a few at a time You don’t have to have every check
  • 34. hooks: - repo: https://gitlab.com/pycqa/flake8 rev: "6.0.0" hooks: - id: flake8 additional_dependencies: [flake8-bugbear] Flake8 34 Fast simple extendable linter Very con fi gurable: setup.cfg or . fl ake8 Many plugins, local plugins easy No auto- fi xers like rubocop (Ruby) Opinion: fl ake8-bugbear is great Example: fl ake8-print, avoid all prints # .flake8 [flake8] max-complexity = 12 extend-ignore = E203, E501, E722, B950 extend-select = B902, B903, B904
  • 35. Flake8 example checks 35 Bugbear Do not use bare except No mutable argument defaults getattr(x, "const") should be x.const No assert False, use raise AssertionError Pointless comparison ❤ pytest PyFlakes (default) Unused modules & variables String formatting mistakes No placeholders in f-string Dictionary key repetition Assert a tuple (it’s always true) Various syntax errors Unde fi ned names Rede fi nition of unused var ❤ pytest McCabe (default) Complexity checks PyCodeStyle (default) Style checks Flake8-print Avoid leaking debugging print statements
  • 36. Custom local fl ake8 plugin 36 import ast import sys from typing import NamedTuple, Iterator class Flake8ASTErrorInfo(NamedTuple): line_number: int offset: int msg: str cls: type # unused
  • 37. Custom local fl ake8 plugin 37 class Visitor(ast.NodeVisitor): msg = "AK101 exception must be wrapped in ak._v2._util.*error" def __init__(self) -> None: self.errors: list[Flake8ASTErrorInfo] = [] def visit_Raise(self, node: ast.Node) -> None: if isinstance(node.exc, ast.Call): if isinstance(node.exc.func, ast.Attribute): if node.exc.func.attr in {"error", "indexerror"}: return if node.exc.func.id in {"ImportError"}: return self.errors.append( Flake8ASTErrorInfo(node.lineno, node.col_offset, self.msg, type(self)) )
  • 38. Custom local fl ake8 plugin 38 class AwkwardASTPlugin: name = "flake8_awkward" version = "0.0.0" def __init__(self, tree: ast.AST) -> None: self._tree = tree def run(self) -> Iterator[Flake8ASTErrorInfo]: visitor = Visitor() visitor.visit(self._tree) yield from visitor.errors
  • 39. Custom local fl ake8 plugin 39 [flake8:local-plugins] extension = AK1 = flake8_awkward:AwkwardASTPlugin paths = ./dev/ def main(path: str) -> None: with open(path) as f: code = f.read() node = ast.parse(code) plugin = AwkwardASTPlugin(node) for err in plugin.run(): print(f"{path}:{err.line_number}:{err.offset} {err.msg}") if __name__ == "__main__": for item in sys.argv[1:]: main(item)
  • 40. Flake8 helpers 40 hooks: - repo: https://github.com/hadialqattan/pycln rev: "v2.1.3" hooks: - id: pycln args: [--config=pyproject.toml] stages: [manual] hooks: - repo: https://github.com/asottile/yesqa rev: "v1.4.0" hooks: - id: yesqa additional_dependencies: &flake8-dependencies - flake8-bugbear - repo: https://gitlab.com/pycqa/flake8 rev: "6.0.0" hooks: - id: flake8 additional_dependencies: *flake8-dependencies # pyproject.toml [tool.pycln] all = true PyCln Quickly remove unused imports Great as a manual hook YesQA Remove unused NoQA’s
  • 41. PyLint 41 PyLint recommends having your project installed, so it is not a good pre-commit hook (though you can do it) It’s also a bit slow, so a good candidate for nox @nox.session def pylint(session: nox.Session) -> None: session.install("-e", ".") session.install("pylint") session.run("pylint", "src", *session.posargs) # pyproject.toml [tool.pylint] master.py-version = "3.7" master.jobs = "0" reports.output-format = "colorized" similarities.ignore-imports = "yes" messages_control.enable = ["useless-suppression"] messages_control.disable = [ "design", "fixme", "line-too-long", "wrong-import-position", ] Code linter Can be very opinionated Signal to noise ratio poor You will need to disable checks - that’s okay! A bit more advanced / less static than fl ake8 But can catch hard to fi nd bugs! For an example of lots of suppressions: https://github.com/scikit-hep/awkward-1.0/blob/1.8.0/pyproject.toml
  • 42. Example PyLint rules 42 Duplicate code Finds large repeated code patterns Attribute de fi ned outside init Only __init__ should de fi ne attributes No self use Can be @classmethod or @staticmethod Unnecessary code Lambdas, comprehensions, etc. Unreachable code Finds things that can’t be reached Consider using in x in {stuff} vs chaining or’s Arguments di ff er Subclass should have matching arguments Consider iterating dictionary Better use of dictionary iteration Consider merging isinstance You can use a tuple in isinstance Useless else on loop They are bad enough when useful :) Consider using enumerate Avoid temp variables, idiomatic Global variable not assigned You should only declare global to assign
  • 43. Controversial PyLint rules 43 No else after control- fl ow Guard-style only Can simply complex control fl ow Removes useless indentation if x: return x else: return None # Should be: if x: return x return None # Or: return x if x else None # Or: return x or None Design Too many various things Too few methods Can just silence “design” (I’m on the in-favor side)
  • 44. Ruff: one linter to rule them all? 44 A very new entry in the Python linting space, but gaining traction like wild fi re 100x faster than existing Python linters Has support for fi xers! Implements all of (modern) fl ake8’s checks Implements dozens of fl ake8 plugins Implements most of pyupgrade (as rules!) Can replace simple usage of isort Can remove unused noqa’s Can remove unused imports Some of PyLint 0 dependencies Con fi gured with pyproject.toml Only binary platforms (Rust compiled) Doesn’t support user plugins Examples https://github.com/scikit-build/scikit-build-core https://github.com/pypa/build
  • 45. Ruff con fi g 45 [tool.ruff] select = [ "E", "F", "W", # flake8 "B", "B904", # flake8-bugbear "I", # isort "ARG", # flake8-unused-arguments "C4", # flake8-comprehensions "EM", # flake8-errmsg "ICN", # flake8-import-conventions "ISC", # flake8-implicit-str-concat "PGH", # pygrep-hooks "PIE", # flake8-pie "PLC", "PLE", "PLR", "PLW", # pylint "PT", # flake8-pytest-style "RET", # flake8-return "RUF", # Ruff-specific "SIM", # flake8-simplify "T20", # flake8-print "UP", # pyupgrade "YTT", # flake8-2020 ] ignore = ["PLR2004", "E501"] target-version = "py37" typing-modules = ["scikit_build_core._compat.typing"] src = ["src"] unfixable = ["T20"] [tool.ruff.per-file-ignores] "tests/**" = ["T20"] "noxfile.py" = ["T20"] - repo: https://github.com/charliermarsh/ruff-pre-commit rev: v0.0.229 hooks: - id: ruff args: ["--fix"] Flake8 con fi g? Try: pipx run flake8-to-ruff .flake8
  • 46. Static type checking: MyPy 46 hooks: - repo: https://gitlab.com/pre-commit/mirrors-mypy rev: "v0.991" hooks: - id: mypy files: src args: [--show-error-codes] Like a linter on steroids Uses Python typing Enforces correct type annotations Designed to be iteratively enabled Should be in a controlled environment (pre-commit or nox) Always specify args (bad hook defaults) Almost always need additional_dependencies Con fi gure in pyproject.toml Pros Can catch many things tests normally catch, without writing tests Therefore it can catch things not covered by tests (yet, hopefully) Code is more readable with types Sort of works without types initially Cons Lots of work to add all types Typing can be tricky in Python Active development area for Python
  • 47. Con fi guring MyPy 47 [tool.mypy] files = "src" python_version = "3.7" warn_unused_configs = true strict = true [[tool.mypy.overrides]] module = [ "numpy.*" ] ignore_missing_imports = true Start small Start without strictness Add a check at a time Extra libraries Try adding them to your environment You can ignore untyped or slow libraries You can provide stubs for untyped libraries if you want Tests? Adding pytest is rather slow I prefer to avoid tests, or keep them mostly untyped
  • 48. Typing tricks 48 Protocols Better than ABCs, great for duck typing @typing.runtime_checkable class Duck(Protocol): def quack() -> str: ... def f(x: Duck) -> str: return x.quack() class MyDuck: def quack() -> str: return "quack" if typing.TYPE_CHECKING: _: Duck = typing.cast(MyDuck, None) Type Narrowing Integral to how mypy works x: Union[A, B] if isinstance(x, A): reveal_type(x) # A else: reveal_type(x) # B Make a typed package Must include py.typed marker fi le Always use sys.version_info Better for readers than try/except, and static Also sys.platform instead of os.name
  • 49. Future annotations 49 Classic code (3.5+) from typing import Union, List def f(x: int) -> List[int]: return list(range(x)) def g(x: Union[str, int]) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Modern code (3.7+) from __future__ import annotations def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) Ultramodern code (3.10+) def f(x: int) -> list[int]: return list(range(x)) def g(x: str | int) -> None: if isinstance(x, str): print("string", x.lower()) else: print("int", x) With the future import, you get all the bene fi ts of future code in 3.7+ annotations Typing is already extra code, simpler is better
  • 50. Part 3: Other languages
  • 51. Clang-format 51 hooks: - repo: https://github.com/pre-commit/mirrors-clang-format rev: "v15.0.7" hooks: - id: clang-format types_or: [c++, c, cuda] C++ and more code formatter Very con fi gurable: .clang-format fi le Opinion: stay close to llvm style PyPI clang-format wheels, under 2MB No more issues with mismatched LLVM!
  • 52. CMake-format 52 hooks: - repo: https://github.com/cheshirekow/cmake-format-precommit rev: "v0.6.13" hooks: - id: cmake-format additional_dependencies: [pyyaml] CMake code formatter Very con fi gurable: .cmake-format.yaml fi le Anything that helps with CMake!
  • 53. Markdown & YAML with Prettier 53 hooks: - repo: https://github.com/pre-commit/mirrors-prettier rev: "v3.0.0-alpha.4" hooks: - id: prettier types_or: [yaml, markdown, html, css, scss, javascript, json] args: [--prose-wrap=always] exclude: "^tests" JavaScript Linter Lots of formats supported A few customization points
  • 54. ShellCheck 54 hooks: - repo: https://github.com/shellcheck-py/shellcheck-py rev: "v0.9.0.2" hooks: - id: shellcheck Linter for bash scripts Can locally disable Prioritizes correctness over terseness
  • 55. CodeSpell 55 hooks: - repo: https://github.com/codespell-project/codespell rev: "v2.2.2" hooks: - id: shellcheck args: ["-L", "sur,nd"] Find common misspellings Inverted spell checker - looks for misspellings Can con fi gure or provide wordlist Actually can catch bugs!
  • 56. See how your repo measures! 56 https://scikit-hep.org/developer/reporeview
  • 58. pytest tips 58 Spend time learning pytest Full of amazing things that really make testing fun! Tests are code too Or for C++: Catch2 or doctest, etc. Also maybe learn Hypothesis for pytest [tool.pytest.ini_options] minversion = "6.0" addopts = [ "-ra", "--showlocals", "--strict-markers", "--strict-config", ] xfail_strict = true filterwarnings = [ "error", ] log_cli_level = "info" testpaths = [ "tests", ] Don’t let warnings slip by! Makes logging more useful Strictness is good Useful summary Print out locals on errors Use pytest.approx Even works on numpy arrays Remember to test for failures If you expect a failure, test it! Test your installed package That’s how users will get it, not from a directory
  • 59. pytest Tricks 59 Mock and Monkeypatch This is how you make tricky tests “unit” tests Fixtures This keeps tests simple and scalable @pytest.fixture(params=["Linux", "Darwin", "Windows"], autouse=True) def platform_system(request, monkeypatch): monkeypatch.setattr(platform, "system", lambda _: request.param) Parametrize Directly or in a fi xture for reuse Use conftest.py Fixtures available in same and nested directories
  • 60. Running pytest 60 Show locals on failure --showlocals/-l Jump into a debugger on failure --pdb Start with last failing test --lf Jump into a debugger immediately --trace or use breakpoint() Run matching tests -k <expression> Run speci fi c test filename.py::testname Run speci fi c marker -m <marker> Control traceback style --tb=<style>
  • 61. In conclusion 61 Code quality tools can help a lot with Readability Reducing bugs Boosting developer productivity Consistency Refactoring Teaching others good practice too Hopefully we have had some helpful discussions! It’s okay to disable a check Try to understand why it’s there Remember there are multiple concerns involved in decisions