---
name: python
description: >-
  Use this skill when writing, linting, type-checking, testing, or packaging Python —
  including async code, CLI tools, MCP servers, Red-DiscordBot cogs, Home Assistant custom
  components, and library packages. Covers strict typing with mypy, ruff configuration,
  virtualenv discipline, async patterns, framework conventions, and packaging.
---

# Python — Rules & Preferences

Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Python specifics.
For the surrounding build and deploy scripts, see [`bash.md`](skills-public/code-languages/bash.md).

§1–§9 are the general rules. §10–§12 are the three frameworks most of this Python is written
against, each of which overrides a general rule somewhere — and says so.

---

## 1. The Gate

```bash
ruff format --check .
ruff check .
mypy --strict .
pytest
```

All four clean before anything is called done. Configure them in `pyproject.toml` so the
editor, the CLI, and the hooks all read the same settings.

**The gate runs locally and through `pre-commit`, not through CI** — workflows do not run under
this account (see the GitHub Actions section of the agent profile), so a green repository with no
local run means nothing was checked. A `.pre-commit-config.yaml` with `ruff-pre-commit`
(check + format) and `mirrors-mypy` scoped `files: src` is where enforcement actually lives.

Scale the gate to what the code is: **a new owner-authored project gets all four**. A vendored
fork gets lint only — match the surrounding style, keep the patch minimal and reviewable, and
do not open a mass reformat of someone else's package (`coding-dev.md` §8 scope discipline).

---

## 2. Typing — Strict

**Every function gets annotated**, including `-> None`. Type hints are not optional
documentation; they are the checker's input.

```toml
[tool.mypy]
strict = true
warn_unreachable = true
warn_return_any = true
disallow_untyped_defs = true
```

- **Never bare `Any`.** Use a `Protocol`, a `TypeVar`, or `object` + narrowing.
- `X | None` over `Optional[X]`; `list[str]` over `List[str]` (modern syntax, no `typing` import).
- `TypedDict` or a dataclass for structured dicts — not `dict[str, Any]`.
- `Literal` and `Enum` for closed sets of values, not bare strings.
- `@dataclass(frozen=True, slots=True)` for value objects.
- Third-party library missing stubs: install `types-*` if it exists, otherwise a narrowly
  scoped `# type: ignore[import-untyped]` with a comment — never a blanket module-level ignore.
  For a **framework with no stubs at all**, the per-import ignore does not scale; use a named
  `[[tool.mypy.overrides]]` block listing those modules with `ignore_missing_imports = true`
  and a comment saying why, so the exemption is one reviewable list rather than scattered.
- Worth adding beyond `strict`: `no_implicit_optional`, `warn_redundant_casts`,
  `warn_unused_ignores`, `warn_unused_configs`, `show_error_codes`, `allow_redefinition = false`.

### Value types over primitives

The general rule is in [`coding-dev.md` §7](skills-public/coding-dev.md); Python's concrete
choices are:

- **`yarl.URL` for URLs**, not f-string assembly. It ships with `aiohttp`, so any async HTTP
  project already has it. `URL(base) / "api" / "v4" / "users"` joins without double slashes,
  and `.with_query({...})` escapes parameters that string formatting would corrupt.
- **`pathlib.Path` for paths**, never `os.path` + `str`. Ruff's `PTH` rules enforce this;
  keep `os.fdopen` where a descriptor rather than a name is being adopted.
- `enum.StrEnum` (3.11+) where a closed set must also serialize as a string, so the wire
  format stays a string without the code comparing bare literals.
- `typing.NewType` for identifiers that should not be interchangeable — a `UserId` and a
  `RepoId` are both `int` and nothing otherwise stops them being swapped.
- Parse at the boundary: take `str` in the public signature if callers supply one, convert
  immediately, and keep the real type internal.

---

## 3. Linting

`ruff` is the linter and formatter (it replaces flake8, isort, black, pyupgrade, and most
plugins). The preferred shape is **everything on, minus a documented ignore list**:

```toml
[tool.ruff.lint]
select = ["ALL"]
ignore = [
  "COM812",  # conflicts with the formatter
  "D203", "D213",  # mutually exclusive with the docstring style chosen below
  "E501",  # the formatter owns line length
]
```

The explicit set below is the floor, for a codebase where `ALL` would bury real findings:

```toml
select = ["E", "F", "W", "I", "N", "UP", "B", "A", "C4", "SIM", "RUF", "ASYNC", "S", "PTH"]
```

- `S` (bandit security rules) and `ASYNC` are worth the noise. `PTH` keeps filesystem access on
  `pathlib`. Add `PLR0915` and `C901` as the function-size proxies (§8).
- **Comment the ignore list too.** Every entry gets the reason beside it, exactly as a `# noqa`
  does — an uncommented ignore list is a set of rules nobody can tell were disabled on purpose.
- As always: fix, don't suppress — and when you must suppress, use a specific code
  (`# noqa: S603`) with a reason.
- Do not add `black` alongside ruff. Where both appear it is an inherited upstream toolchain,
  not a choice; ruff's formatter is the one to keep in new work.

---

## 4. Environments

- **Always a virtualenv or project-managed environment** (`uv`, `poetry`, `pdm`, `venv`).
  Never install into the system interpreter, and never `pip install --user` as a workaround
  for an immutable OS — use a container or a venv.
- `uv` is the preferred installer/resolver where free choice exists; it is dramatically faster
  and lockfile-native. **Check it exists before relying on it** (`command -v uv`) — a committed
  `uv.lock` is not evidence that the tool is installed on this machine.

> [!CAUTION]
> **Bare `python` and bare `pip` can be different interpreters.** On this host `python` is the
> system 3.14 while `pip`/`pip3` on PATH are bridge wrappers into the development container's
> 3.13 — so `pip install x` followed by `python x.py` produces a `ModuleNotFoundError` that
> looks like a bad install. Always go through a project venv, or `python -m pip` with the
> interpreter named explicitly. See [`distrobox-buildbox-dev.md`](skills-private/distrobox-buildbox-dev.md).

- **Lockfile by artifact type**: an application commits its lock, a library does not. A
  throwaway script directory legitimately stays on a `requirements.txt` with floor pins and no
  lock at all.
- Dev dependencies go in PEP 735 `[dependency-groups] dev = [...]`. `[tool.uv]
  dev-dependencies` is the older uv-specific form — fine to leave in a fork, not what new work
  uses.
- Pin the Python version in `pyproject.toml` (`requires-python`). Where there is no `[project]`
  table at all — a plugin or custom component rather than a package — the pin lives in ruff's
  `target-version` and mypy's `python_version` instead, and **those two will drift apart unless
  something checks them**; set them in the same edit.
- `pathlib.Path` over `os.path` everywhere (ruff's `PTH` rules enforce this).

---

## 5. Errors & Logging

- **Specific exceptions only.** `except Exception:` needs a reason; bare `except:` is never
  acceptable (it catches `KeyboardInterrupt` and `SystemExit`).
- **Never swallow silently.** If you catch and continue, log at `warning` or above.
- `raise ... from err` to preserve the chain.
- Custom exception types deriving from a single package-level base, so callers can catch the
  whole family.
- **`logging` module, never `print`,** in anything that isn't a CLI writing to stdout as its
  actual output. One `logger = logging.getLogger(__name__)` per module, at module scope — unless
  the framework owns the logger tree, in which case its prefix wins (§11).
- **Never log secrets** — tokens, auth headers, message content, personal data.
- A blanket `contextlib.suppress(Exception)` is permitted in exactly one situation: a
  development-ergonomics escape hatch whose every suppression carries a written paragraph saying
  what staleness or platform quirk it absorbs. The rule is not "never suppress", it is
  **suppression is allowed when the comment carries the argument**.

---

## 6. Async

- `asyncio` with `async`/`await`. **Never block the event loop** — no `time.sleep`, no
  synchronous `requests`, no blocking file I/O in a coroutine. Use `asyncio.to_thread` for
  genuinely blocking calls.
- `httpx` (or the framework's own client) over `requests` in async code.
- **`asyncio.TaskGroup`** (3.11+) over bare `create_task` **where one failure should cancel the
  rest** — it propagates exceptions instead of losing them to an unawaited task.
- **For fan-out over N external calls, TaskGroup is the wrong default**: cancel-on-first-error
  means one dead provider takes down the other nine. Use a bounded helper instead — an
  `asyncio.Semaphore(limit)` around each worker plus `asyncio.gather(..., return_exceptions=True)`
  — and keep it in one shared function rather than re-deriving it per caller.
- **Every `create_task` result must be held in a reference**, or the garbage collector can
  cancel it mid-flight. Clear it in `finally`.
- Cancellation is cooperative: clean up in `finally`, and never swallow `CancelledError` —
  but **distinguish a cancellation you asked for from a framework shutdown**. Re-raise unless a
  stop was requested; converting a real shutdown into an application error strands it.
- Where a blocking library runs **its own** background thread, don't wrap it in `to_thread` —
  bridge back onto the loop with `loop.call_soon_threadsafe(future.set_result, …)`.
- **Events over polling.** Subscribe to the library's event/callback API; do not loop on a
  timer. Where polling is genuinely unavoidable (no event source exists), use a **staggered
  backoff** rather than a fixed interval, and say why polling was necessary.

---

## 7. Rate Limiting & External Services

When talking to a rate-limited API (a chat platform, a store, a tracker):

- **Funnel outbound calls through a single queue/worker** rather than calling from arbitrary
  call sites. It is the only place a limit can actually be enforced.
- **Backoff with jitter** on retry; never a tight retry loop.
- Respect `Retry-After` and documented limits; assume undocumented ones exist.
- **Be extremely careful scraping real consumer services.** Conservative delays, realistic
  headers, plenty of failsafes, and abort fast when a response looks unexpected. Assume you
  are one mistake away from a ban.

---

## 8. Structure

- Package layout: `src/<package>/`, tests in `tests/`. A single-binary stdlib-only CLI may stay
  flat; a plugin or component follows its framework's layout instead.
- A tool that has to run on an immutable host is worth keeping **dependency-free** where the
  stdlib covers it (`argparse`, `subprocess`, `shlex`, `json`, `pathlib`) — nothing to provision
  is the strongest form of "it works on that machine".
- Entry points are `[project.scripts]`. `typer` for a CLI with subcommands, stdlib `argparse`
  for everything else; do not add a CLI framework to a two-flag script.
- **Shared behavior goes in a real shared module**, imported by each consumer — not
  copy-pasted between sibling packages. When three components grow the same queue/retry/
  formatting logic, that is the signal to extract it.
- Reusable, generic functionality (a format parser, a protocol client) becomes **its own
  installable package** with its own repo and version, consumed as a dependency.
- Keep persistent state in the framework's own config/storage system so it survives restarts;
  do not invent a parallel JSON file beside it.
- File and function size limits from the general rules apply. Ruff has **no line-based**
  function-length rule — `PLR0915` (`too-many-statements`, default 50) and `C901`
  (cyclomatic complexity) are the closest proxies and are worth enabling, but neither
  measures lines. Put **both** the function-length and file-length limits in a pytest
  case (see [`coding-dev.md` §4](skills-public/coding-dev.md)): file 1000 lines max, function
  200 lines max measured by walking the `ast` rather than counting lines of text. **Guard the guard** with a test asserting the glob matched anything at
  all — a pattern that matches nothing makes every limit vacuous and still passes.

---

## 9. Testing

- `pytest`, with `pytest-asyncio`. Set `asyncio_mode = "auto"` in
  `[tool.pytest.ini_options]` — it removes the per-test marker entirely — plus
  `asyncio_default_fixture_loop_scope = "function"`.
- Fixtures over setup/teardown methods; `tmp_path` for anything touching the filesystem.
- **Regression test every fixed bug**, and give them a home: a `test_regressions.py` is where
  they stop being scattered and start being a list.
- **Mock at a boundary, not inside your own code.** A test that patches the code under test
  proves nothing. The boundary can be a network mock (`respx`, `responses`), a **real local fake
  service** the test drives end to end, or **recorded payloads fed to the provider seam** — the
  last two are usually better evidence than a mocked HTTP layer, because they exercise the
  parsing too.
- **Never exercise destructive operations against real accounts or real data.** Temporary
  fixtures only.
- `pytest -p no:randomly` only when order genuinely matters, and note why.

---

## 10. MCP Servers

> [!CAUTION]
> **On the stdio transport, stdout is the protocol.** A stray `print`, a `logging`
> `StreamHandler` pointed at stdout, or a library's progress bar corrupts the frame and the
> client sees a parse error rather than your message. Log to **stderr** only, and keep §5's
> no-`print` rule absolute here.

- Make the transport an explicit option (`--transport stdio|http`) rather than hardcoding one.
- **Tool schemas are derived from the signature and the docstring**, so §2's "every function
  annotated" rule has a second payoff: an unannotated parameter produces a useless client-facing
  schema. Never hand-write JSON schema.
- Set the behavioural annotations on every tool — `readOnlyHint`, `openWorldHint`, a human
  `title`.
- **Errors are raised, not returned.** Raise (FastMCP serialises it) or return
  `isError=True` with a `TextContent` on the low-level SDK. A tool that returns the string
  `"ERROR: …"` as its normal value is indistinguishable from data to the client and defeats
  its retry logic.
- `pydantic-settings` for layered config is good, but note what using a settings value as a
  **tool parameter default** means: it is frozen at import time *and* published in the advertised
  schema. Fine for a hostname, a secret leak for anything else.

---

## 11. Red-DiscordBot Cogs

- `self.config = Config.get_conf(self, identifier=<int>, force_registration=True)`, then
  `register_global`/`register_guild` for every key. **Never a side-car JSON file** beside the cog.
- Read-modify-write goes through the context manager — `async with
  self.config.guild(ctx.guild).channels() as channels:` — because a plain get/set around a
  mutable list is a lost-update bug.
- Lifecycle: `async def setup(bot)` in `__init__.py`; `async def cog_load(self)` for I/O-bearing
  startup; `@commands.Cog.listener()` for gateway events.
- **`cog_unload` is synchronous.** It cannot await, so it cancels held tasks and fires teardown
  off into `self.bot.loop.create_task(self.api.close())`. Start long-lived tasks the same way,
  on the bot's loop.
- `async def red_delete_data_for_user(self, *, _requester: str, _user_id: int) -> None` is a
  mandatory contract, not optional boilerplate.
- **The logger name overrides §5**: `logging.getLogger("red.<owner>.<cog>")`, not `__name__`.
  Under `__name__` the cog sits outside the `red.*` tree where Red's own log-level controls
  apply, so the generic rule produces a logger nobody can turn up.
- Red loads each cog directory independently, so a cross-cog shared module is awkward by design.
  The answer is a per-cog `utils/` package, not a copy-pasted `lib.py` — §8's shared-module rule
  still applies *within* a cog.

---

## 12. Home Assistant Custom Components

- `from __future__ import annotations` at the top of every module.
- The `async_setup_entry` / `async_unload_entry` / `_async_update_listener` triple, with state on
  **`entry.runtime_data`**, not `hass.data`.
- **Never create your own `aiohttp.ClientSession`** — take `async_get_clientsession(hass)`, or
  you leak a session per reload and HA logs it.
- A `DataUpdateCoordinator[T]` subclass whose `_async_update_data` raises `UpdateFailed`. **No
  provider exception may escape it.**
- Declare dependencies in `manifest.json` `requirements`, not in `pyproject.toml` — the component
  is not a package.
- Fire events in size-bounded chunks with a run id in the envelope; a single huge event is not
  a thing the bus handles gracefully.
- PEP 695 syntax is current here: `type MyConfigEntry = ConfigEntry[MyCoordinator]`, and
  `def f[T](…)` for generics.