315 lines
17 KiB
Markdown
315 lines
17 KiB
Markdown
---
|
|
name: python
|
|
description: >-
|
|
Use this skill when writing, linting, type-checking, testing, or packaging Python —
|
|
including async code, CLI tools, MCP servers, Red-DiscordBot cogs, Home Assistant custom
|
|
components, and library packages. Covers strict typing with mypy, ruff configuration,
|
|
virtualenv discipline, async patterns, framework conventions, and packaging.
|
|
---
|
|
|
|
# Python — Rules & Preferences
|
|
|
|
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Python specifics.
|
|
For the surrounding build and deploy scripts, see [`bash.md`](skills-public/code-languages/bash.md).
|
|
|
|
§1–§9 are the general rules. §10–§12 are the three frameworks most of this Python is written
|
|
against, each of which overrides a general rule somewhere — and says so.
|
|
|
|
---
|
|
|
|
## 1. The Gate
|
|
|
|
```bash
|
|
ruff format --check .
|
|
ruff check .
|
|
mypy --strict .
|
|
pytest
|
|
```
|
|
|
|
All four clean before anything is called done. Configure them in `pyproject.toml` so the
|
|
editor, the CLI, and the hooks all read the same settings.
|
|
|
|
**The gate runs locally and through `pre-commit`, not through CI** — workflows do not run under
|
|
this account (see the GitHub Actions section of the agent profile), so a green repository with no
|
|
local run means nothing was checked. A `.pre-commit-config.yaml` with `ruff-pre-commit`
|
|
(check + format) and `mirrors-mypy` scoped `files: src` is where enforcement actually lives.
|
|
|
|
Scale the gate to what the code is: **a new owner-authored project gets all four**. A vendored
|
|
fork gets lint only — match the surrounding style, keep the patch minimal and reviewable, and
|
|
do not open a mass reformat of someone else's package (`coding-dev.md` §8 scope discipline).
|
|
|
|
---
|
|
|
|
## 2. Typing — Strict
|
|
|
|
**Every function gets annotated**, including `-> None`. Type hints are not optional
|
|
documentation; they are the checker's input.
|
|
|
|
```toml
|
|
[tool.mypy]
|
|
strict = true
|
|
warn_unreachable = true
|
|
warn_return_any = true
|
|
disallow_untyped_defs = true
|
|
```
|
|
|
|
- **Never bare `Any`.** Use a `Protocol`, a `TypeVar`, or `object` + narrowing.
|
|
- `X | None` over `Optional[X]`; `list[str]` over `List[str]` (modern syntax, no `typing` import).
|
|
- `TypedDict` or a dataclass for structured dicts — not `dict[str, Any]`.
|
|
- `Literal` and `Enum` for closed sets of values, not bare strings.
|
|
- `@dataclass(frozen=True, slots=True)` for value objects.
|
|
- Third-party library missing stubs: install `types-*` if it exists, otherwise a narrowly
|
|
scoped `# type: ignore[import-untyped]` with a comment — never a blanket module-level ignore.
|
|
For a **framework with no stubs at all**, the per-import ignore does not scale; use a named
|
|
`[[tool.mypy.overrides]]` block listing those modules with `ignore_missing_imports = true`
|
|
and a comment saying why, so the exemption is one reviewable list rather than scattered.
|
|
- Worth adding beyond `strict`: `no_implicit_optional`, `warn_redundant_casts`,
|
|
`warn_unused_ignores`, `warn_unused_configs`, `show_error_codes`, `allow_redefinition = false`.
|
|
|
|
### Value types over primitives
|
|
|
|
The general rule is in [`coding-dev.md` §7](skills-public/coding-dev.md); Python's concrete
|
|
choices are:
|
|
|
|
- **`yarl.URL` for URLs**, not f-string assembly. It ships with `aiohttp`, so any async HTTP
|
|
project already has it. `URL(base) / "api" / "v4" / "users"` joins without double slashes,
|
|
and `.with_query({...})` escapes parameters that string formatting would corrupt.
|
|
- **`pathlib.Path` for paths**, never `os.path` + `str`. Ruff's `PTH` rules enforce this;
|
|
keep `os.fdopen` where a descriptor rather than a name is being adopted.
|
|
- `enum.StrEnum` (3.11+) where a closed set must also serialize as a string, so the wire
|
|
format stays a string without the code comparing bare literals.
|
|
- `typing.NewType` for identifiers that should not be interchangeable — a `UserId` and a
|
|
`RepoId` are both `int` and nothing otherwise stops them being swapped.
|
|
- Parse at the boundary: take `str` in the public signature if callers supply one, convert
|
|
immediately, and keep the real type internal.
|
|
|
|
---
|
|
|
|
## 3. Linting
|
|
|
|
`ruff` is the linter and formatter (it replaces flake8, isort, black, pyupgrade, and most
|
|
plugins). The preferred shape is **everything on, minus a documented ignore list**:
|
|
|
|
```toml
|
|
[tool.ruff.lint]
|
|
select = ["ALL"]
|
|
ignore = [
|
|
"COM812", # conflicts with the formatter
|
|
"D203", "D213", # mutually exclusive with the docstring style chosen below
|
|
"E501", # the formatter owns line length
|
|
]
|
|
```
|
|
|
|
The explicit set below is the floor, for a codebase where `ALL` would bury real findings:
|
|
|
|
```toml
|
|
select = ["E", "F", "W", "I", "N", "UP", "B", "A", "C4", "SIM", "RUF", "ASYNC", "S", "PTH"]
|
|
```
|
|
|
|
- `S` (bandit security rules) and `ASYNC` are worth the noise. `PTH` keeps filesystem access on
|
|
`pathlib`. Add `PLR0915` and `C901` as the function-size proxies (§8).
|
|
- **Comment the ignore list too.** Every entry gets the reason beside it, exactly as a `# noqa`
|
|
does — an uncommented ignore list is a set of rules nobody can tell were disabled on purpose.
|
|
- As always: fix, don't suppress — and when you must suppress, use a specific code
|
|
(`# noqa: S603`) with a reason.
|
|
- Do not add `black` alongside ruff. Where both appear it is an inherited upstream toolchain,
|
|
not a choice; ruff's formatter is the one to keep in new work.
|
|
|
|
---
|
|
|
|
## 4. Environments
|
|
|
|
- **Always a virtualenv or project-managed environment** (`uv`, `poetry`, `pdm`, `venv`).
|
|
Never install into the system interpreter, and never `pip install --user` as a workaround
|
|
for an immutable OS — use a container or a venv.
|
|
- `uv` is the preferred installer/resolver where free choice exists; it is dramatically faster
|
|
and lockfile-native. **Check it exists before relying on it** (`command -v uv`) — a committed
|
|
`uv.lock` is not evidence that the tool is installed on this machine.
|
|
|
|
> [!CAUTION]
|
|
> **Bare `python` and bare `pip` can be different interpreters.** On this host `python` is the
|
|
> system 3.14 while `pip`/`pip3` on PATH are bridge wrappers into the development container's
|
|
> 3.13 — so `pip install x` followed by `python x.py` produces a `ModuleNotFoundError` that
|
|
> looks like a bad install. Always go through a project venv, or `python -m pip` with the
|
|
> interpreter named explicitly. See [`distrobox-buildbox-dev.md`](skills-private/distrobox-buildbox-dev.md).
|
|
|
|
- **Lockfile by artifact type**: an application commits its lock, a library does not. A
|
|
throwaway script directory legitimately stays on a `requirements.txt` with floor pins and no
|
|
lock at all.
|
|
- Dev dependencies go in PEP 735 `[dependency-groups] dev = [...]`. `[tool.uv]
|
|
dev-dependencies` is the older uv-specific form — fine to leave in a fork, not what new work
|
|
uses.
|
|
- Pin the Python version in `pyproject.toml` (`requires-python`). Where there is no `[project]`
|
|
table at all — a plugin or custom component rather than a package — the pin lives in ruff's
|
|
`target-version` and mypy's `python_version` instead, and **those two will drift apart unless
|
|
something checks them**; set them in the same edit.
|
|
- `pathlib.Path` over `os.path` everywhere (ruff's `PTH` rules enforce this).
|
|
|
|
---
|
|
|
|
## 5. Errors & Logging
|
|
|
|
- **Specific exceptions only.** `except Exception:` needs a reason; bare `except:` is never
|
|
acceptable (it catches `KeyboardInterrupt` and `SystemExit`).
|
|
- **Never swallow silently.** If you catch and continue, log at `warning` or above.
|
|
- `raise ... from err` to preserve the chain.
|
|
- Custom exception types deriving from a single package-level base, so callers can catch the
|
|
whole family.
|
|
- **`logging` module, never `print`,** in anything that isn't a CLI writing to stdout as its
|
|
actual output. One `logger = logging.getLogger(__name__)` per module, at module scope — unless
|
|
the framework owns the logger tree, in which case its prefix wins (§11).
|
|
- **Never log secrets** — tokens, auth headers, message content, personal data.
|
|
- A blanket `contextlib.suppress(Exception)` is permitted in exactly one situation: a
|
|
development-ergonomics escape hatch whose every suppression carries a written paragraph saying
|
|
what staleness or platform quirk it absorbs. The rule is not "never suppress", it is
|
|
**suppression is allowed when the comment carries the argument**.
|
|
|
|
---
|
|
|
|
## 6. Async
|
|
|
|
- `asyncio` with `async`/`await`. **Never block the event loop** — no `time.sleep`, no
|
|
synchronous `requests`, no blocking file I/O in a coroutine. Use `asyncio.to_thread` for
|
|
genuinely blocking calls.
|
|
- `httpx` (or the framework's own client) over `requests` in async code.
|
|
- **`asyncio.TaskGroup`** (3.11+) over bare `create_task` **where one failure should cancel the
|
|
rest** — it propagates exceptions instead of losing them to an unawaited task.
|
|
- **For fan-out over N external calls, TaskGroup is the wrong default**: cancel-on-first-error
|
|
means one dead provider takes down the other nine. Use a bounded helper instead — an
|
|
`asyncio.Semaphore(limit)` around each worker plus `asyncio.gather(..., return_exceptions=True)`
|
|
— and keep it in one shared function rather than re-deriving it per caller.
|
|
- **Every `create_task` result must be held in a reference**, or the garbage collector can
|
|
cancel it mid-flight. Clear it in `finally`.
|
|
- Cancellation is cooperative: clean up in `finally`, and never swallow `CancelledError` —
|
|
but **distinguish a cancellation you asked for from a framework shutdown**. Re-raise unless a
|
|
stop was requested; converting a real shutdown into an application error strands it.
|
|
- Where a blocking library runs **its own** background thread, don't wrap it in `to_thread` —
|
|
bridge back onto the loop with `loop.call_soon_threadsafe(future.set_result, …)`.
|
|
- **Events over polling.** Subscribe to the library's event/callback API; do not loop on a
|
|
timer. Where polling is genuinely unavoidable (no event source exists), use a **staggered
|
|
backoff** rather than a fixed interval, and say why polling was necessary.
|
|
|
|
---
|
|
|
|
## 7. Rate Limiting & External Services
|
|
|
|
When talking to a rate-limited API (a chat platform, a store, a tracker):
|
|
|
|
- **Funnel outbound calls through a single queue/worker** rather than calling from arbitrary
|
|
call sites. It is the only place a limit can actually be enforced.
|
|
- **Backoff with jitter** on retry; never a tight retry loop.
|
|
- Respect `Retry-After` and documented limits; assume undocumented ones exist.
|
|
- **Be extremely careful scraping real consumer services.** Conservative delays, realistic
|
|
headers, plenty of failsafes, and abort fast when a response looks unexpected. Assume you
|
|
are one mistake away from a ban.
|
|
|
|
---
|
|
|
|
## 8. Structure
|
|
|
|
- Package layout: `src/<package>/`, tests in `tests/`. A single-binary stdlib-only CLI may stay
|
|
flat; a plugin or component follows its framework's layout instead.
|
|
- A tool that has to run on an immutable host is worth keeping **dependency-free** where the
|
|
stdlib covers it (`argparse`, `subprocess`, `shlex`, `json`, `pathlib`) — nothing to provision
|
|
is the strongest form of "it works on that machine".
|
|
- Entry points are `[project.scripts]`. `typer` for a CLI with subcommands, stdlib `argparse`
|
|
for everything else; do not add a CLI framework to a two-flag script.
|
|
- **Shared behavior goes in a real shared module**, imported by each consumer — not
|
|
copy-pasted between sibling packages. When three components grow the same queue/retry/
|
|
formatting logic, that is the signal to extract it.
|
|
- Reusable, generic functionality (a format parser, a protocol client) becomes **its own
|
|
installable package** with its own repo and version, consumed as a dependency.
|
|
- Keep persistent state in the framework's own config/storage system so it survives restarts;
|
|
do not invent a parallel JSON file beside it.
|
|
- File and function size limits from the general rules apply. Ruff has **no line-based**
|
|
function-length rule — `PLR0915` (`too-many-statements`, default 50) and `C901`
|
|
(cyclomatic complexity) are the closest proxies and are worth enabling, but neither
|
|
measures lines. Put **both** the function-length and file-length limits in a pytest
|
|
case (see [`coding-dev.md` §4](skills-public/coding-dev.md)): file 1000 lines max, function
|
|
200 lines max measured by walking the `ast` rather than counting lines of text. **Guard the guard** with a test asserting the glob matched anything at
|
|
all — a pattern that matches nothing makes every limit vacuous and still passes.
|
|
|
|
---
|
|
|
|
## 9. Testing
|
|
|
|
- `pytest`, with `pytest-asyncio`. Set `asyncio_mode = "auto"` in
|
|
`[tool.pytest.ini_options]` — it removes the per-test marker entirely — plus
|
|
`asyncio_default_fixture_loop_scope = "function"`.
|
|
- Fixtures over setup/teardown methods; `tmp_path` for anything touching the filesystem.
|
|
- **Regression test every fixed bug**, and give them a home: a `test_regressions.py` is where
|
|
they stop being scattered and start being a list.
|
|
- **Mock at a boundary, not inside your own code.** A test that patches the code under test
|
|
proves nothing. The boundary can be a network mock (`respx`, `responses`), a **real local fake
|
|
service** the test drives end to end, or **recorded payloads fed to the provider seam** — the
|
|
last two are usually better evidence than a mocked HTTP layer, because they exercise the
|
|
parsing too.
|
|
- **Never exercise destructive operations against real accounts or real data.** Temporary
|
|
fixtures only.
|
|
- `pytest -p no:randomly` only when order genuinely matters, and note why.
|
|
|
|
---
|
|
|
|
## 10. MCP Servers
|
|
|
|
> [!CAUTION]
|
|
> **On the stdio transport, stdout is the protocol.** A stray `print`, a `logging`
|
|
> `StreamHandler` pointed at stdout, or a library's progress bar corrupts the frame and the
|
|
> client sees a parse error rather than your message. Log to **stderr** only, and keep §5's
|
|
> no-`print` rule absolute here.
|
|
|
|
- Make the transport an explicit option (`--transport stdio|http`) rather than hardcoding one.
|
|
- **Tool schemas are derived from the signature and the docstring**, so §2's "every function
|
|
annotated" rule has a second payoff: an unannotated parameter produces a useless client-facing
|
|
schema. Never hand-write JSON schema.
|
|
- Set the behavioural annotations on every tool — `readOnlyHint`, `openWorldHint`, a human
|
|
`title`.
|
|
- **Errors are raised, not returned.** Raise (FastMCP serialises it) or return
|
|
`isError=True` with a `TextContent` on the low-level SDK. A tool that returns the string
|
|
`"ERROR: …"` as its normal value is indistinguishable from data to the client and defeats
|
|
its retry logic.
|
|
- `pydantic-settings` for layered config is good, but note what using a settings value as a
|
|
**tool parameter default** means: it is frozen at import time *and* published in the advertised
|
|
schema. Fine for a hostname, a secret leak for anything else.
|
|
|
|
---
|
|
|
|
## 11. Red-DiscordBot Cogs
|
|
|
|
- `self.config = Config.get_conf(self, identifier=<int>, force_registration=True)`, then
|
|
`register_global`/`register_guild` for every key. **Never a side-car JSON file** beside the cog.
|
|
- Read-modify-write goes through the context manager — `async with
|
|
self.config.guild(ctx.guild).channels() as channels:` — because a plain get/set around a
|
|
mutable list is a lost-update bug.
|
|
- Lifecycle: `async def setup(bot)` in `__init__.py`; `async def cog_load(self)` for I/O-bearing
|
|
startup; `@commands.Cog.listener()` for gateway events.
|
|
- **`cog_unload` is synchronous.** It cannot await, so it cancels held tasks and fires teardown
|
|
off into `self.bot.loop.create_task(self.api.close())`. Start long-lived tasks the same way,
|
|
on the bot's loop.
|
|
- `async def red_delete_data_for_user(self, *, _requester: str, _user_id: int) -> None` is a
|
|
mandatory contract, not optional boilerplate.
|
|
- **The logger name overrides §5**: `logging.getLogger("red.<owner>.<cog>")`, not `__name__`.
|
|
Under `__name__` the cog sits outside the `red.*` tree where Red's own log-level controls
|
|
apply, so the generic rule produces a logger nobody can turn up.
|
|
- Red loads each cog directory independently, so a cross-cog shared module is awkward by design.
|
|
The answer is a per-cog `utils/` package, not a copy-pasted `lib.py` — §8's shared-module rule
|
|
still applies *within* a cog.
|
|
|
|
---
|
|
|
|
## 12. Home Assistant Custom Components
|
|
|
|
- `from __future__ import annotations` at the top of every module.
|
|
- The `async_setup_entry` / `async_unload_entry` / `_async_update_listener` triple, with state on
|
|
**`entry.runtime_data`**, not `hass.data`.
|
|
- **Never create your own `aiohttp.ClientSession`** — take `async_get_clientsession(hass)`, or
|
|
you leak a session per reload and HA logs it.
|
|
- A `DataUpdateCoordinator[T]` subclass whose `_async_update_data` raises `UpdateFailed`. **No
|
|
provider exception may escape it.**
|
|
- Declare dependencies in `manifest.json` `requirements`, not in `pyproject.toml` — the component
|
|
is not a package.
|
|
- Fire events in size-bounded chunks with a run id in the envelope; a single huge event is not
|
|
a thing the bus handles gracefully.
|
|
- PEP 695 syntax is current here: `type MyConfigEntry = ConfigEntry[MyCoordinator]`, and
|
|
`def f[T](…)` for generics.
|