Compare commits

...
20 Commits
Author SHA1 Message Date
BluscreamandClaude Opus 5.5 5e6abadb95 docs: add software-vision and android-backup-restore skills with android scripts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 13:00:44 +01:00
BluscreamandClaude Sonnet 5.5 a16a879b1e Point the AFK status file at tsw-plugins/tasks
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 04:18:26 +02:00
BluscreamandClaude Opus 5.5 44c2d1502c docs(coding): files 1000 lines max, functions 200 lines max, no earlier split point
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 21:52:48 +02:00
BluscreamandClaude Opus 5 e385e01993 fix(gsc): correct the claim that no off-engine compiler exists
gsc-tool compiles GSC outside the engine and reproduces the engine's own
diagnostics, so the skill's "no compiler, linter, or test harness worth the
name" framing was wrong and sent the reader to load a map instead of running a
50ms check. §1 is now the gate: `--mode comp --game <title> --system pc --dry`,
with the warning that `--mode parse` is syntax-only and accepts code the
compiler rejects, and that `--game` takes the engine codename (h2 for MW2CR)
rather than the marketing name.

A clean compile proves the script loads, not that it works, so the
"behaviour is unverified until the map runs" half is kept and sharpened.

Also adds two §8 entries found the hard way: a local declared inside a block is
not visible after it, and there are no file-scope constants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-09 03:04:02 +02:00
BluscreamandClaude Opus 5 6207de1a97 docs(code-languages): add Lua and GSC skills
Lua covers dialect pinning (5.1/LuaJIT/5.2/5.3+/HavokScript), luac -p and
luacheck as the only real gate for an embedded host, locals over globals,
pcall at the boundary, the #t / pairs / pattern traps, and the sandboxed-host
rules including Call of Duty LUI specifics.

GSC covers the absence of any compiler, linter, or test harness and what to
do instead, scope prefixing on level/self, isdefined on every external read,
endon before the first wait, the per-frame instruction budget, load-phase
precaching, and side effects on shared level state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-09 02:28:16 +02:00
Bluscream b8505fc623 docs(cli): add desktop Steam lifecycle guidance via steamcli client 2026-10-07 23:53:04 +02:00
BluscreamandClaude Opus 5 5aef9707d2 feat(php): add a PHP skill, grounded in the synchra-php SDK and the bleichi site
PHP was the one language with real code in two shapes here and no skill: a
Composer library held at PHPStan level max, and a plain PHP-FPM application
behind nginx with no framework. The rules come from reading both rather than
from general PHP advice, so each one names the failure it prevents.

The two halves differ enough to be separate sections. A library requires PSR
interfaces and ships no implementation; an application's rules are about
escaping, keeping the document root unwritable, atomic writes and sessions.
Everything in §1-§8 applies to both.

Three findings are worth the space they take because they were each paid for
once already: `json_encode` with JSON_UNESCAPED_SLASHES into an inline
<script> is an XSS hole, a web user that cannot write its own cache fails
silently on every request, and inside the dev container the host's own `php`
bridge shadows the real interpreter so every vendor/bin tool dies naming
distrobox.

docs(python): cover MCP servers, Red cogs and HA components, and correct three
rules the owner's own code contradicts

The skill described a toolchain and was silent on the four things most of this
Python actually is. Three of its rules are also wrong in context, and the code
is right: getLogger(__name__) puts a cog outside the red.* tree where Red's log
controls apply; TaskGroup's cancel-on-first-error is the wrong default for
fan-out over N providers, where the real pattern is a semaphore plus
gather(return_exceptions=True); and the gate runs locally and through
pre-commit, not through CI, which never runs under this account.

Also: ruff ALL-minus-documented-ignores is the preferred shape rather than the
explicit select list, lockfiles split by artifact type, and bare `python` and
bare `pip` are different interpreters on this host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-06 17:40:23 +02:00
BluscreamandClaude Opus 5 1b688d128c docs(bash): make the pgrep self-match section about misreading, not just killing
The existing warning only covered piping a self-match into kill. The quieter
failure is reading your own shell's PID as "the app is running" and refusing to
touch its config — which is exactly what happened while rotating webhooks in a
closed VRCNext.

Adds: always pass -a so a self-match is visible; prefer -ax (name only, never
scans arguments) for "is it running?"; -A/--ignore-ancestors as the fix when -f
is unavoidable; and why the [b]racket trick is weaker than it looks — the term
can appear elsewhere on the same command line. Points at systemctl is-active
for supervised units.

cli-tooling.md gains a one-line orientation pointer rather than a copy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-02 23:01:57 +02:00
Bluscream 5029ce5e8b docs(afk-sleep-work): add window mouse-click trap warning, task-scaled timeouts, and background tasks 2026-10-02 00:43:49 +02:00
Bluscream 289db75cb6 refactor(afk-sleep-work): compact skill while preserving all directives and tables 2026-10-02 00:41:18 +02:00
Bluscream 87a456cd31 docs(afk-sleep-work): warn against runaway log files in last-breath trap 2026-10-02 00:40:33 +02:00
Bluscream dc666bc56b feat(skills): add afk-sleep-work skill for autonomous unattended execution 2026-10-02 00:39:51 +02:00
Bluscream b08fe50e92 feat(coding-dev): require domain values to be typed, not primitives
A value with rules of its own — a URL, a path, a closed set, an identifier —
gets a type of its own. Passing it as a bare string discards those rules at the
first function boundary and leaves every consumer re-deriving them.

Added as a subsection of Architecture Preferences, with a table mapping the
common cases to the type to reach for in any language, the reasoning, and the
boundary rule: parse once at the edge, keep the type inward, serialize back only
where the wire actually demands a string. Notes the counter-case, so this is not
read as a mandate to wrap every string.

python.md gains the concrete choices: yarl.URL (already present wherever aiohttp
is), pathlib, StrEnum, and NewType for non-interchangeable identifiers.
2026-09-18 18:22:26 +02:00
BluscreamandClaude Opus 5 3171595940 fix(python): PLR0915 counts statements, not lines
Verified with `ruff rule PLR0915`: it is `too-many-statements` (default 50),
not a line-length rule. Ruff has no line-based function-length rule at all.
python.md now says so and points both limits at a pytest case; coding-dev.md
gains the general distinction between a real length lint and a proxy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:56:12 +02:00
BluscreamandClaude Opus 5 32b47b075b fix(coding-dev): prefer a test over the build script for size limits
A shell check only runs where that script runs. A test travels with the repo,
needs no extra tooling, and fails with a message naming the offending files.
Enforcement is now a preference order: native linter rule > test > build
script as fallback, with a worked example and a note on why a per-compile
build hook usually is not worth the incremental-build cost.

Propagated per language: rust (clippy has no file-length lint -> workspace
root test), js-typescript (ESLint max-lines covers both natively, no test
needed), go/python/csharp (linter for functions, test for files), bash (don't
reimplement in the gate what a test can assert).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:52:37 +02:00
BluscreamandClaude Opus 5 5d96e69d2b refactor(skills): inline cross-links, drop Related sections, fix conventions
- all nine files: trailing "Related" sections removed, their unique links
  inlined where the topic actually arises (FFI boundary -> c-cpp, embedded
  frontend -> js-typescript, build scripts -> bash, etc.)
- README conventions: rule 2 mandated a Related Skills section, which now
  contradicts practice — replaced with inline-linking and one-home-per-fact
  rules. The <repo>/<rel path> rule was appended after the wrong section
  last commit; moved into the list where it belongs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:49:30 +02:00
BluscreamandClaude Opus 5 58525056a1 docs: use <repo>/<rel path> links throughout, document the convention
All cross-skill links now spell out the repository: skills-public/foo.md,
skills-private/bar.md. These resolve from the shared parent directory, so one
spelling works from either repo and from code-languages/ subdirectories —
replacing the previous mix of ./, ../ and absolute file:// URLs. Link labels
normalised to match. Convention added to the README conventions list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:44:23 +02:00
BluscreamandClaude Opus 5 72834167b6 feat(cli-tooling): add generic CLI, tooling and script-placement rules
Extracted from the per-agent profile files and the private ai-agents skill,
where they were duplicated and environment-coupled:

- discovery over guessing: walk the help tree, never invent flags, never
  hand-roll a script when a dedicated CLI exists
- non-interactive invocation: explicit targets, the tool's own --wait
- machine-readable output flags (json/compact/csv) vs. never filtering
  diagnostics
- orient before starting: project instructions, git log, prior work records,
  and verifying TODOs against the code
- script placement: dedicated version-controlled dir, symlinked into PATH
- destructive/outward-facing command care; environment drift verification

coding-dev.md and code-languages/bash.md now cross-reference it instead of
carrying their own one-line versions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:40:55 +02:00
BluscreamandClaude Opus 5 9571eb52d2 fix(coding-dev): scoped suppressions, contribution-dependent precedence
- suppressions: prefer the narrowest scope that covers the problem (line,
  function, module) with a reason comment; a project-wide disable silences
  the rule in code nobody has written yet. Always name the specific rule.
  Propagated to rust (#[allow]), csharp (#pragma/SuppressMessage).
- precedence: a project's own rules win only when contributing upstream.
  For local/private work these rules win, adopting only the project's
  load-bearing conventions.
- type checking must pass with zero errors *or warnings*.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:34:51 +02:00
BluscreamandClaude Opus 5 57968904bf feat(skills): add coding-dev.md and per-language rules under code-languages/
Generic coding rules and preferences distilled from existing agent rule
files, project AGENTS.md/.rules, and saved memories:

- strictness maximal; suppressions must be reported to the user
- never filter build/lint/test output through grep/tail/head
- reuse > fork > self-implement; generic code becomes its own package
- 1000-line file / 100-line function limits, split by concern
- regression test every fixed bug; never test against real user data
- events over polling; scope discipline; honest reporting

Per-language: rust, js-typescript, bash, python, go, csharp-dotnet, c-cpp.
Each extends coding-dev.md rather than repeating it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:33:44 +02:00
24 changed files with 4415 additions and 1 deletions

No files matched your search

+51 -1
View File
@@ -6,7 +6,57 @@ A collection of public, reusable agent skills, cheatsheets, and operational work
## 📂 Skills Index
*(Public skills will be added here as generic, open workflows)*
| Skill File | Scope / Topics Covered |
| :--- | :--- |
| **[`coding-dev.md`](skills-public/coding-dev.md)** | Standing rules for all code work: maximum linter/compiler strictness, never filtering build output, reuse-before-reinvent dependency ordering, file/function size limits, test & regression discipline, verification gates, scope control, commit hygiene, and honest reporting. |
| **[`software-vision.md`](skills-public/software-vision.md)** | Starting from a software idea: turn it into a requirements checklist, search the web first (projects, forks, skeletons, libraries, user reviews), then merge in training-data recall checked against current sources, shallow-clone promising candidates to a temp dir to check them, then present use / extend / fork / start-fresh options before building anything. |
| **[`cli-tooling.md`](skills-public/cli-tooling.md)** | Using command-line tools correctly: reading the help tree instead of guessing flags, preferring an existing CLI over an ad-hoc script, non-interactive invocation, machine-readable output formats, orienting on prior work before starting, script placement & symlinking, and care around destructive commands. |
| **[`afk-sleep-work.md`](skills-public/afk-sleep-work.md)** | Operating unattended and autonomously: continuous forward momentum, avoiding polling traps and deadlocks, host preservation and resource budgeting (disk, packages, processes), isolated tool provisioning, avoiding "last breath" traps on token/quota exhaustion, bypassing or driving GUI launchers, and wake-up checkpoints. |
| **[`android-backup-restore.md`](skills-public/android-backup-restore.md)** | Extracting/backing up/restoring Android data by access tier before a wipe or migration: EDL/firehose & bootloader, custom recovery (TWRP/OrangeFox/CWM), root, device-owner, Shizuku, plain ADB, and on-device apps; FBE decryption limits, per-partition dumps (userdata, EFS/modem/IMEI, persist), SELinux/UID-correct restores, the death of `adb backup`, and app-specific data (WhatsApp, Signal, TOTP/Keystore) that does not survive. |
### 🔤 Per-Language Rules ([`code-languages/`](skills-public/code-languages/))
| Language | Skill File | Highlights |
| :--- | :--- | :--- |
| **Rust** | [`rust.md`](skills-public/code-languages/rust.md) | `--workspace` clippy gate, `[workspace.lints]` single source of truth, no `unwrap`/`unsafe`, core-crate split |
| **TypeScript / JavaScript** | [`js-typescript.md`](skills-public/code-languages/js-typescript.md) | TS > JSDoc > plain JS, banned constructs table, React & hooks rules, resource lifecycle, sandbox boundaries |
| **Bash / Shell** | [`bash.md`](skills-public/code-languages/bash.md) | `set -euo pipefail`, shellcheck, no wait loops, destructive-command guards, build-gate structure |
| **Python** | [`python.md`](skills-public/code-languages/python.md) | `mypy --strict`, broad ruff rule set, asyncio task discipline, rate-limit queueing, MCP/cog/HA framework rules |
| **PHP** | [`php.md`](skills-public/code-languages/php.md) | `strict_types` everywhere, PHPStan `level: max`, PER-CS 2.0 risky rules, `final readonly` value objects, PSR interfaces over implementations, escaping & unwritable document root |
| **Go** | [`go.md`](skills-public/code-languages/go.md) | strict `golangci-lint` set, concrete types over `interface{}`, context everywhere, CGO-free static builds |
| **C# / .NET** | [`csharp-dotnet.md`](skills-public/code-languages/csharp-dotnet.md) | `TreatWarningsAsErrors`, nullable enabled, no `.Result`, trimming/single-file publish caveats, SDK shims |
| **C / C++** | [`c-cpp.md`](skills-public/code-languages/c-cpp.md) | full warning set + `-Werror`, sanitizers, RAII/Rule of Zero, target-based CMake, FFI ownership |
| **Lua** | [`lua.md`](skills-public/code-languages/lua.md) | pin the dialect first, `luac -p` + luacheck as the only gate, locals over globals, `pcall` at the boundary, `#t`/pattern traps, embedded & HavokScript hosts |
| **GSC** | [`gsc.md`](skills-public/code-languages/gsc.md) | compile off-engine with `gsc-tool --mode comp`, `isdefined` on everything external, `endon` before the first wait, per-frame budget, precache at load, level-state side effects |
Each language file extends `coding-dev.md` rather than repeating it. A project's own
`AGENTS.md` / `CLAUDE.md` / `.rules` always overrides both.
---
## ✍️ Skill Document Conventions
Skills in this repo follow the same structure as the private counterpart repo, so agents can load either interchangeably:
1. **YAML frontmatter** with `name` (kebab-case, matching the filename) and a `description` starting with *"Use this skill when …"* — the description is what an agent matches against to decide relevance, so it must enumerate concrete triggers.
2. **Numbered `##` sections**, sequential and unique.
3. **Cross-link inline, at the point the other skill becomes relevant** — never in a trailing
"Related Skills" list. A link inside the sentence that raises the topic gets followed; a
list at the bottom of a long document does not, and it drifts out of sync with the body.
4. **One home per fact.** Content that belongs to another skill is *replaced* by a link to it,
not duplicated. Where a skill needs its own specific values for something another skill
owns the rules for (a container's own template fields, say), keep only the values and link
out for the rules.
5. **Link between skills as `<repo>/<relative path>`** — e.g.
`` [`cli-tooling.md`](skills-public/cli-tooling.md) `` or
`` [`network.md`](skills-private/network.md) ``. Links resolve from the shared parent
directory holding both repos, so the same spelling works from either repo and from any
subdirectory. Never `./`, `../`, or absolute `file://` paths. The sole exception is the
per-agent profile files, which state each repository root once in full, since they are
loaded as global instructions with no implied working directory.
6. **CLI discovery over enumeration** — document the *non-obvious* workflows, gotchas, and constraints. For the routine surface, point at `<tool> --help` rather than transcribing flag lists that will go stale.
7. **No secrets inline** — reference the environment file that holds them (e.g. `~/.config/environment.d/…`), never the values themselves.
8. **Verify before documenting** — confirm hostnames, container names, and versions against the live system rather than carrying forward stale claims.
---
+135
View File
@@ -0,0 +1,135 @@
---
name: afk-sleep-work
description: >-
Use this skill when the user is away, AFK, or sleeping, grants broad autonomous execution authority,
asks the agent to work continuously until tokens or objectives are exhausted, or delegates multi-phase
tasks requiring unattended orchestration, safe tool provisioning, avoiding polling traps, respecting
budget margins (the "last breath" rule), and bypassing or driving GUI launchers.
---
# Unattended & Autonomous Execution (AFK / Sleep Work)
When working unattended, the interactive loop is gone. Every decision, recovery, tool provisioning,
and verification step must proceed autonomously without risking host stability or leaving broken state.
---
## 1. Autonomous Execution Contract
1. **Keep Moving Forward:** Do not stop for minor ambiguities. Investigate, formulate hypotheses, test, and pivot independently.
2. **First, Do No Harm:** Broad authority is never permission to fill disks, clobber system packages, reboot, or risk data loss.
3. **Leave a Transparent Audit Trail:** Maintain an orderly checkpoint detailing completed work, failures, state, and exact next steps.
---
## 2. Trap Avoidance: Loops, Timeouts & Interactive Prompts
The primary failure mode of unattended runs is wedging on a blocking command.
### Never write polling wait loops
> [!CAUTION]
> **Strictly forbidden:** Never run `while`, `until`, or sleep polling loops in shell commands
> (see [`code-languages/bash.md` §3](skills-public/code-languages/bash.md)).
> An unattended loop that never satisfies its exit condition hangs the session indefinitely.
- **Tool-native blocking & harness background tasks:** Prefer flags like `--wait`, `--no-detach`, or
dispatch long-running jobs as harness background tasks/daemons (with reactive completion notifications)
so the synchronous execution loop never gets wedged.
- **Enforce task-scaled timeouts:** Never run unbounded commands. Scale timeouts deliberately based on
estimated task duration (e.g. `timeout 30s` for quick checks, `timeout 180s` for tests, `timeout 15m` for
large builds) to prevent infinite hangs on deadlocks or network stalls.
- **Mandatory non-interactive flags:** Supply `-y`, `--yes`, `--batch`, `--headless`, `CI=1`, or
`DEBIAN_FRONTEND=noninteractive`. Never let a tool prompt `[y/N]` (see [`cli-tooling.md` §2](skills-public/cli-tooling.md)).
- **Bounded retries with hypothesis variation:** Never rerun the same failing command unchanged. Cap retries
at 3 attempts; each must test a distinct hypothesis (flags, config, path, or alternative strategy).
If all fail, record diagnostics and pivot.
---
## 3. Host Safety & Resource Budgets ("Do No Harm")
- **Storage & Disk:** Check `df -h <path>` before large fetches or builds. Keep ≥15–20% disk headroom. Clean up
downloaded archives (`.tar.gz`, `.zip`, `.iso`), temp build trees, and intermediate dumps.
- **OS & Package Integrity:** Never run blanket host package upgrades or installs (`sudo dnf`, `sudo apt`,
`pacman -Syu`). Isolate toolchains inside containers (e.g. [`distrobox-buildbox-dev.md`](skills-private/distrobox-buildbox-dev.md),
Docker) or user-space paths (`~/.local/bin/`, venvs).
- **Process Safety:** Never execute `shutdown`, `reboot`, or kill display servers (`wayland`, `xorg`). Never
issue broad `pkill python` or `killall node` that can kill the agent or IDE. Target specific PIDs or match full paths.
- **CPU & Memory:** Cap multi-threaded jobs (`-j$(nproc --ignore=2)`), `nice -n 10` heavy builds, and monitor
`free -m` to prevent kernel OOM panics.
---
## 4. Tool Provisioning: Containerized & Headless First
1. **Verify existence first:** Run `command -v <tool>` before downloading.
2. **Provision in isolation:** Prefer existing repo toolchains > build containers (Distrobox/Docker) >
standalone binaries (`~/.local/bin/`) > user-scoped package managers (`cargo install`, `pip install --user`).
3. **Headless / CLI over GUI:** Never launch heavy graphical interfaces when headless/batch modes exist:
- *Ghidra:* `analyzeHeadless <proj_dir> <proj_name> -import <bin> -postScript <script.py>`
- *Packets:* `tshark` / `tcpdump` instead of GUI Wireshark
- *Debuggers:* `gdb -batch -ex ...` or `lldb --batch -o ...`
---
## 5. Launchers & GUI Automation: Bypass vs. Drive
When targets are shielded by splash launchers, web wrappers, or login dialogs:
| Priority | Strategy | Action |
| :--- | :--- | :--- |
| **1. Direct Bypass (Best)** | Run child binary directly | Inspect `ps aux` or wrapper configs while the launcher runs to extract child binary path and arguments, then execute it directly. |
| **2. CLI / Config Flags** | Suppress launcher UI | Pass switches like `-nosplash`, `-nolauncher`, `-batch`, `--skip-intro`, or set `SkipLauncher=true` in `.ini`/`.json` configs. |
| **3. GUI Automation (Fallback)** | Drive the window | Target window non-interactively (`wmctrl -lp`, `xdotool search --name <name>`, Wayland tree, or desktop MCP), capture screenshot/dump, send keys/clicks, and verify state change with a timeout guard. |
> [!WARNING]
> **Beware mouse-click traps in window utilities:** Never run `xwininfo`, `xprop`, `xdotool selectwindow`,
> or ImageMagick `import` without explicit window IDs or query flags (`-root`, `-name`, `-id`). Without them,
> these tools pause and wait indefinitely for the user to click a window, hanging unattended execution instantly.
---
## 6. Budget Exhaustion, Runaway Logs & The "Last Breath" Rule
In unattended sessions, context window, token budgets, or subscription quotas will eventually exhaust.
> [!CAUTION]
> **The "Last Breath" Trap:** Never initiate an open-ended, non-atomic, or high-risk mutating operation
> when tokens, context, or quota are nearly depleted.
Rushing into a complex task on a thin margin guarantees failure if the harness terminates or pauses
execution mid-flight:
- **Corrupted source trees:** Half-applied multi-file refactors leaving syntax errors and broken builds.
- **Dangling locks:** Incomplete package or git locks (`.git/index.lock`, `/var/lib/dpkg/lock`).
- **Uncommitted state:** Half-finished rebases, stashes, or uncommitted edits.
- **Zombie processes & runaway logs:** Background daemons, watchers, or detached tasks spewing debug output
into unrotated log files, silently filling the disk while the agent is paused or waiting for quota.
### The Controlled Landing protocol
When remaining budget or context drops below a safe margin:
1. **Halt scope expansion:** Do not start new goals or dependencies.
2. **Finish or roll back atomic edits:** Leave the repo in a compiling, testable state (`git checkout -- <file>` if incomplete).
3. **Kill ephemeral processes & log streams:** Terminate background test runners, servers, watchers, and tasks redirecting output to unrotated files.
4. **Run verification gates:** Verify `git status` and run quick build/lint checks.
5. **Commit atomic progress:** Save clean progress with conventional commits.
6. **Write the recovery checkpoint:** Update the status log before the session halts.
---
## 7. Checkpointing & The "Wake-Up" Audit Trail
Maintain a persistent status file (for the tsw-plugins project `tsw-plugins/tasks/{todo,in-progress,done}.md`; otherwise a session artifact) updated at each milestone:
| Section | Contents |
| :--- | :--- |
| **Objective** | Target goal and current operational phase. |
| **Completed Work** | Bulleted list of verified accomplishments (code, passing tests, tools configured). |
| **Current State** | Branch, commit hash, build status, active containers or services. |
| **Findings & Discoveries** | Key technical observations, reversed structs, or API quirks. |
| **Obstacles & Fallbacks** | Failed attempts with concise explanations and how you pivoted. |
| **Budget / Harness State** | Token/quota status; whether stopped by task completion or budget limits. |
| **Immediate Next Steps** | Exact command line or target file for the next session to resume seamlessly. |
Conclude with a single clear summary pointing the returning user directly to this checkpoint.
+562
View File
@@ -0,0 +1,562 @@
---
name: android-backup-restore
description: >-
Use this skill when extracting, backing up, migrating, or restoring data from an Android
device — especially before a wipe, ROM flash, factory reset, or device-to-device move, and
when deciding what is salvageable at a given access level. Covers the full ladder of access
tiers (EDL/firehose & bootloader, custom recovery CWM/TWRP/OrangeFox, root, device-owner,
Shizuku, plain ADB, and on-device apps), file-based-encryption decryption limits, per-partition
dumps (userdata, EFS/modem/IMEI, persist), SELinux/UID-correct restores, the death of
`adb backup`, and app-specific data (WhatsApp, Signal, TOTP/Keystore) that does not survive.
---
# Android Backup & Restore — Extraction by Access Level
How to pull as much as possible off an Android device before wiping, flashing, or migrating —
and how to put it back. Organised as a **ladder of access tiers**: the more privileged the
access, the more you can extract. Work out which tier you have, pull from the highest tier
available, and fall back down the ladder for whatever that tier cannot reach.
This is a *data-recovery and migration* skill for devices you own or are authorised to service.
Shell orchestration here follows [`bash.md`](skills-public/code-languages/bash.md) (strict mode,
hard `timeout`s, no wait-loops); CLI invocation follows
[`cli-tooling.md`](skills-public/cli-tooling.md) (read `<tool> --help` before trusting any flag
below — tool surfaces drift, and the exact partition/block paths are **per-device**).
> [!IMPORTANT]
> **The single biggest variable is encryption.** Every modern Android device uses File-Based
> Encryption (FBE); the `/data` partition is a ciphertext blob until something that holds the
> user credential derives the keys. That "something" is normally the booted OS. A custom
> recovery can only read plaintext `/data` if it was *built to decrypt your exact Android
> version* and you type the lockscreen PIN/password into it. See §3. This is why "boot it
> normally and pull over ADB/root" beats "boot to recovery and dd" whenever the device still
> boots.
---
## 1. The Access Ladder (decision table)
Pick the highest row you can reach. Each tier can extract everything below it plus its own column.
| Tier | Access you need | Reaches raw partitions? | Reaches decrypted `/data/data` app data? | Reaches EFS/IMEI/persist? | Notes |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **EDL / firehose / BROM** (§8) | Vendor edl/mtk loader, device in 9008/BROM | ✅ full flash, offline | ⚠️ only as ciphertext (still FBE-locked) | ✅ | Last resort for dead/bricked devices; dumps are encrypted blobs unless you also have the keys. |
| **Bootloader / fastboot** (§8) | Unlocked bootloader | ⚠️ some (`fastboot fetch` on newer; mostly flash-only) | ❌ | partial | Mostly a *flashing* interface, not a dumping one. |
| **Custom recovery** TWRP/OrangeFox/CWM (§3) | Unlocked bootloader + matching recovery | ✅ every partition via `dd`/built-in backup | ⚠️ **only if recovery decrypts your FBE** + you enter the PIN | ✅ | Classic "nandroid". Decryption is the catch on Android 10+. |
| **Root (booted OS)** (§4) | Magisk/KernelSU/APatch on a booting device | ✅ (block devices readable) | ✅ **fully decrypted** (OS is running) | ✅ | The best tier for a *working* device — plaintext data + raw partitions together. |
| **Device Owner / DPC** (§5) | `dpm set-device-owner` on a fresh/unprovisioned device | ❌ | ❌ (own managed scope only) | ❌ | Niche; grants/denies permissions, reads some providers; not a bulk dumper. |
| **Shizuku / ADB-shell privilege** (§6) | Shizuku (wireless-debug or one-time ADB) | ❌ | ⚠️ APKs, `Android/data`, `Android/obb`, media — **not** `/data/data` of release apps | ❌ | `adb shell` UID (2000) reach, on-device, no PC tether. |
| **Plain ADB** (§6) | USB debugging authorised | ❌ | ⚠️ same as Shizuku + `run-as` for *debuggable* apps only | ❌ | `adb backup` is effectively dead (§7). Pull storage, providers, bugreport. |
| **On-device app, no root** (§6.4) | Just the unlocked phone | ❌ | ❌ (own data only) | ❌ | Vendor cloud/D2D, Seedvault, per-app export. Last-mile for the non-technical case. |
**Rule of thumb for a pre-wipe sweep:** if it boots and you can unlock it → get **root** (§4) if
possible, else **ADB** (§6). If it does *not* boot → **custom recovery** (§3), else **EDL** (§8).
Always grab **EFS/IMEI/persist** (§4.3) before any flash regardless of tier.
---
## 2. First: inventory what is actually worth saving
Before dumping 128 GB of userdata, enumerate. Most of the value is small and specific.
- **Media & documents** — `/sdcard` (a.k.a. `/storage/emulated/0`): DCIM, Pictures, Download,
Documents, Music, ringtones. Reachable at *every* tier including plain ADB. Pull this first; it
is the easy win and the bulk of what users actually miss.
- **App data** — `/data/data/<pkg>` and `/data/user/0/<pkg>` (credential-encrypted, "CE") and
`/data/user_de/0/<pkg>` (device-encrypted, "DE"). Needs root or recovery-decrypt. This is where
game saves, app logins, notes, 2FA seeds live.
- **Scoped external app data** — `Android/data/<pkg>` and `Android/obb/<pkg>` on `/sdcard`.
Blocked from ordinary file managers since Android 11/13 but reachable via ADB/Shizuku/root.
- **Telephony/PIM** — SMS/MMS, call log, contacts, calendar. See §6.2 (providers) and §9.
- **Identity partitions** — EFS/modem (`modemst1`, `modemst2`, `fsg`), `persist`, `efs`. IMEI,
Wi-Fi/BT MAC, sensor calibration. §4.3. **Irreplaceable — back up before any flash.**
- **The things that will NOT survive** no matter the tier — hardware-backed Keystore keys, and
by extension many 2FA/banking apps. §10. Plan migrations for these *before* wiping.
---
## 3. Custom Recovery (TWRP / OrangeFox / CWM) — the offline nandroid
Use when the device will not boot normally, or you want a block-level image independent of the OS.
Requires an **unlocked bootloader** (itself a wipe on most devices — so unlock *before* you have
data worth keeping, or accept the loss). Boot without flashing when you can:
`fastboot boot twrp.img` leaves the installed recovery untouched.
> [!TIP]
> **Try the *installed* recovery before downloading or temp-booting one.** `adb reboot recovery`,
> then `adb shell id` — many custom ROMs ship a current OrangeFox/TWRP whose `adbd` already runs as
> **root** and, crucially, is new enough to **decrypt FBE on the installed Android version** (so
> `/data/data` shows real package names and readable file contents). This beats `fastboot boot`-ing
> an old recovery image: a recovery built from a much older branch often won't boot on a new ROM at
> all (its kernel panics and the device falls back to the OS — you'll land back in Android as
> `uid=2000(shell)`, the tell-tale that it didn't take). Confirm decryption with
> `ls /data/data` (real names, not hex) and by reading one `*/shared_prefs/*.xml`.
### 3.1 The decryption problem (read this before trusting a backup)
TWRP/OrangeFox can only back up **plaintext** `/data` if its decryption code matches your device's
FBE implementation *and* you enter the lockscreen credential in recovery (Keymaster derives the
keys from it). Reality on Android 10+:
- Use a recovery **built from the branch matching your Android version** (e.g. an `android-12.1`+
build for Android 12+). A recovery built from an old branch shows `/data` as garbage folder names
or fails to mount.
- Devices that **shipped on Android ≤11 and were upgraded** to 12+ often cannot be decrypted by any
TWRP build — the keystore/Keymaster path changed. OrangeFox sometimes succeeds where TWRP fails;
try both. If neither decrypts, you can still image the partition but only as ciphertext (useless
without the keys).
- If decryption fails, **do not factory-reset to "fix" it** — that destroys the keys and the data.
Verify decryption worked before relying on it: in the TWRP terminal, `ls -la /data/data` should
show real package names (`com.whatsapp`, …), not hex blobs.
### 3.2 Built-in backup vs. raw `dd`
- **Built-in backup** (TWRP "Backup" / OrangeFox): tick System, Data, Boot, EFS, etc. Produces a
nandroid set under `/sdcard/TWRP/BACKUPS/`. If `/data` decrypted, the Data archive holds plaintext
app data; if not, it backs up the raw (encrypted) partition. Pull the whole folder off afterwards.
- **Raw partition image** when you want a specific partition or the built-in backup won't cover it.
From the recovery `adb shell` or terminal — **block paths are per-device**, find them first:
```bash
# from recovery shell — enumerate by name, DO NOT guess the numbers
ls -l /dev/block/by-name/ # symlinks: userdata, boot_a, modemst1, efs, persist, …
cat /proc/partitions
# image a named partition to portable storage (not back onto /data)
timeout 1800 dd if=/dev/block/by-name/userdata of=/sdcard/userdata.img bs=8M
```
Pull with `adb pull /sdcard/userdata.img` (or write to an SD card / OTG drive). A raw `userdata`
image is only restorable/mountable if you also have the FBE keys — prefer the decrypted built-in
Data backup for app data, and reserve raw images for small partitions (§4.3).
### 3.3 Restore
Restore the built-in nandroid through the recovery UI, or `dd` a raw image back **only onto the
exact same partition of the same device** (never another device's EFS — §4.3). After a decrypted
Data restore across Android versions, expect some apps to misbehave; a clean reinstall + selective
data restore (§4.2) is often more reliable than a whole-partition nandroid across versions.
---
## 4. Root on a booting device — the best tier for a live phone
If the device boots and you can unlock it, root (Magisk / KernelSU / APatch) is the strongest
position: the OS has already derived the FBE keys, so **all app data is plaintext**, and you still
have raw block-device access. (For installing root on this workstation's own test devices, Magisk
APKs/modules may already be on the NAS — see
[check-nas-before-downloading](check-nas-before-downloading.md) before fetching.)
### 4.1 Bulk app-data pull
```bash
# as root, on device or via `adb shell su -c`
# tar preserves owners/modes/symlinks — plain `cp`/`adb pull` loses them (see §4.4)
su -c 'tar -cpf /sdcard/appdata-whatsapp.tar \
/data/data/com.whatsapp /data/user_de/0/com.whatsapp'
# then from the PC:
adb pull /sdcard/appdata-whatsapp.tar
```
Pull `/data/data/<pkg>` (CE) **and** `/data/user_de/0/<pkg>` (DE) for each app — some apps split
state across both. For a whole-device grab, `tar` `/data/data`, `/data/user/0`, `/data/user_de/0`
and `/data/misc` (wifi configs, etc.), but mind disk budget on the pulling host.
> [!TIP]
> **Stream straight off the device with `adb exec-out` — no on-device staging.** When `/data` has
> little free space (you can't write a multi-GB tar back onto the partition you're reading), pipe
> the archive directly to the host, and `dd` images the same way (`exec-out` is binary-safe; plain
> `adb shell` mangles bytes):
> ```bash
> adb exec-out 'cd /data && tar -cp -f - data user_de system misc vendor adb' > data-full.tar
> adb exec-out 'dd if=/dev/block/by-name/boot status=none' > boot.img
> ```
> Toybox/busybox `tar` can't store SELinux contexts and exits non-zero when it skips sockets
> (`unknown file type '140000'` — harmless). Capture contexts separately as a restore sidecar:
> `for d in /data/data/* /data/user_de/0/*; do stat -c '%a %U %G %C %n' "$d"; done`. The per-app
> SELinux *categories* (`:c123,c456`) encode the app sandbox and are **not** regenerable by
> `restorecon` alone — this sidecar is how you put them back.
>
> **Record restore metadata while you have the device.** A `dumpsys package` capture gives, per
> app: `codePath` (→ system/vendor/product/`/data/app`=user), `pkgFlags` (SYSTEM/UPDATED_SYSTEM_APP),
> `installerPackageName` (Play `com.android.vending` vs sideloaded `null`), versionCode, and granted
> runtime permissions; `cmd appops dump` adds special ops (SYSTEM_ALERT_WINDOW,
> MANAGE_EXTERNAL_STORAGE, …). Parse these into a manifest so you can later restore *only* user apps
> and re-grant their exact permissions.
Prefer dedicated tools over hand-rolled tar when restoring to a *different* device or Android
version, because they record ownership/SELinux/permissions that a naive tar restore drops (§4.4):
- **Swift Backup** (root; Shizuku for a weaker subset) — app + data, permissions, SSAID.
- **Neo Backup** (FOSS, OAndBackupX fork; root incl. KernelSU/APatch) — app + data, schedules.
- **OAndBackupX / Migrate** — FOSS nandroid-style app+data sets.
### 4.2 Raw partitions while rooted
Same `dd` from `/dev/block/by-name/` as §3.2, but from the running OS (`su -c dd …`). Good for
grabbing `boot`/`vendor_boot` to patch, or EFS/persist (§4.3) without rebooting to recovery.
### 4.3 EFS / modem / IMEI / persist — back these up before ANY flash
These hold IMEI, baseband calibration, Wi-Fi/BT MAC, and sensor calibration. Corrupting them bricks
the radio permanently and there is no download that restores *your* unique values. **Partition names
and block numbers are per-device — look them up, never copy numbers from a guide.**
```bash
su -c 'ls -l /dev/block/by-name/' | grep -Ei 'efs|modemst|fsg|fsc|persist|nvram|nvdata|protect'
# Qualcomm: commonly modemst1, modemst2, fsg, fsc, persist
# MediaTek: commonly nvram, nvdata, protect_f, protect_s, persist
for p in modemst1 modemst2 fsg persist; do
su -c "dd if=/dev/block/by-name/$p of=/sdcard/efs-$p.img bs=4M"
done
```
Pull the images **off the device and keep copies on a host/NAS** — a backup that lives only on the
partition you are about to wipe is no backup. Restore the same way (`dd … of=/dev/block/by-name/$p`)
**only to the identical partition on the identical device**. Never flash one phone's EFS onto
another — you would overwrite its unique identity with useless values.
### 4.4 Restoring raw `/data/data` tars correctly (UID + SELinux)
A tar of `/data/data/<pkg>` restored by hand will *not* launch unless you fix two things Android
assigns at install time:
1. **Install the app first** (same package, ideally same version) so Android creates the data dir
with the correct per-app UID and SELinux context.
2. Restore the files, then **fix ownership and labels**:
```bash
# find the app's real UID from the freshly-created dir, don't assume 100xx
su -c 'ls -ld /data/data/com.example' # e.g. u0_a142 u0_a142
su -c 'tar -xpf /sdcard/appdata-example.tar -C /'
su -c 'chown -R u0_a142:u0_a142 /data/data/com.example'
su -c 'restorecon -RFD /data/data/com.example' # argument order matters; -v to see work
```
Then reboot before launching. Skipping the UID chown or `restorecon` is the usual reason a
"successful" restore produces an app that crashes on open. Dedicated backup apps (§4.1) do this
bookkeeping for you — hand-tar only when you must.
### 4.5 Getting root headlessly (no GUI taps) when the device boots
If the bootloader is unlocked and you need root to reach §4 but can't tap through the Magisk app,
patch and flash boot entirely from the shell:
1. **Get the *exact* stock boot.img.** The authoritative copy is the live partition — dump it from
a root recovery (§3) with `dd`/`exec-out`. Don't flash a boot.img of unknown origin: check its
cmdline first (`androidboot.hardware=qcom` = Qualcomm vs MediaTek's `bootopt=…` — a wrong-SoC
image bricks the device).
2. **Patch it** with Magisk's own tooling. The Magisk APK is a zip: its `assets/boot_patch.sh` needs
the arm64 libs renamed (`lib/arm64-v8a/libmagiskboot.so`→`magiskboot`, `libmagiskinit.so`→
`magiskinit`, `libmagisk.so`→`magisk`, `libinit-ld.so`→`init-ld`) plus `stub.apk` and
`util_functions.sh` in the same dir, then `sh boot_patch.sh boot.img` → `new-boot.img`.
- Set the flags **explicitly** (standalone `boot_patch.sh` treats unset as false, which can
bootloop): for a modern FBE device with a vbmeta partition and dynamic partitions use
`KEEPVERITY=true KEEPFORCEENCRYPT=true PATCHVBMETAFLAG=false RECOVERYMODE=false LEGACYSAR=false`.
`Failed to patch` on the kernel DTB step is benign (nothing to patch).
- The recovery installer (`META-INF/com/google/android/update-binary`, which auto-detects these
flags via `get_flags()`) is cleaner **but** fails outside a real sideload: it tries to mount
`/system`, which on super/dynamic-partition ROMs is `/dev/mapper/system` and won't mount in a
plain `adb shell`. `boot_patch.sh` with explicit flags sidesteps that.
3. **Flash** `new-boot.img` (verify it unpacks: `magiskboot unpack new-boot.img`) with
`fastboot flash boot` or, from a root recovery, `dd … of=/dev/block/by-name/boot`. Keep the stock
image for instant rollback. Reboot: `magiskd` in `ps -A` + `magisk -V` confirms root took and
survives reboot.
4. **Grant su to the ADB shell** (Magisk denies it by default, silently): Magisk app → Superuser
access = "Apps and ADB" + Automatic response = "Grant", or seed a `policies` row
(`uid=2000 policy=2`) in `/data/adb/magisk.db`. Until then `adb shell su -c …` returns
`Permission denied` even though root is installed and working.
---
## 5. Device Owner (DPC) — narrow, not a bulk extractor
A device-owner app (set with `adb shell dpm set-device-owner <pkg>/<receiver>` on a device with
**no accounts added yet**) gets elevated *policy* powers, not filesystem powers. It is relevant to
extraction only in that it can silently grant its own runtime permissions and read some content
providers that would otherwise prompt — useful for a managed enrolment flow, not for cleaning out a
phone before a wipe. Hard limits:
- `setPermissionGrantState()` for **sensor** permissions (location, camera, mic, body sensors,
activity recognition) is restricted/deprecated for *profile* owners on Android 12+ (silently
returns false); device owners retain more but the surface is shrinking each release.
- It cannot read other apps' `/data/data`, cannot image partitions, and cannot reach EFS.
If you find yourself reaching for device-owner to extract data, the real answer is almost always
root (§4) or ADB providers (§6.2) instead.
---
## 6. ADB & Shizuku — no root, device boots
Plain ADB runs as the `shell` user (UID 2000). Shizuku grants that **same** privilege to on-device
apps (started via wireless debugging or a one-time ADB command), so their reach is identical to ADB
— handy when you have no PC.
### 6.1 External storage & scoped app dirs
```bash
adb pull /sdcard/ ./sdcard-backup/ # DCIM, Download, Documents, etc. — works everywhere
# Android/data and Android/obb are blocked from on-device file managers (11/13+) but
# the adb shell UID can still read them:
adb pull /sdcard/Android/data/ ./android-data/
adb pull /sdcard/Android/obb/ ./android-obb/
```
### 6.2 Content providers (telephony/PIM without root)
`adb shell content query` reaches providers the `shell` UID is allowed to read. What actually works
on a stock device is uneven:
```bash
adb shell content query --uri content://contacts/phones # contacts — usually readable
adb shell content query --uri content://com.android.calendar/events
# SMS/MMS and call_log generally FAIL for the shell UID (need READ_SMS / be default app):
adb shell content query --uri content://sms # typically permission denied
```
For SMS/MMS, call logs, and contacts as restorable files, use an on-device FOSS app rather than
fighting providers over ADB:
- **SMS Import / Export** (`com.github.tmo1.sms_ie`, F-Droid) — SMS+MMS+call log+contacts to
JSON/ZIP, no root, and it *restores* too (set it as default SMS app during restore).
- **Open Android Backup** (mrrfv) — PC-driven, rootless, uses a companion app (not `adb backup`):
restores apps-without-data, internal storage, and contacts (vCard); exports SMS (CSV) and call
log (txt) as **view-only** (no restore, no MMS attachments).
- Contacts also export cleanly from the stock Contacts app ("Export to .vcf") or via Google account
sync — often the least-effort path for the non-technical case.
### 6.3 `run-as` — app data for *debuggable* apps only
```bash
adb shell run-as com.example.debug tar -cpf - ./ > example-data.tar
```
`run-as` only works for apps with `android:debuggable="true"` (i.e. your own dev builds, not Play
Store releases). Shizuku-based file managers use the same `run-as` path and inherit the same limit.
Release-app `/data/data` is **not** reachable at this tier — that needs root (§4) or a
recovery-decrypt (§3).
### 6.4 Bugreport & app lists (cheap, always worth grabbing)
```bash
adb bugreport ./bugreport.zip # logs, settings dump, installed-package metadata
adb shell pm list packages -3 -f # third-party packages + APK paths, to reinstall later
adb shell cmd package list packages --show-versioncode
```
To re-pull the APKs themselves for reinstall:
```bash
adb shell pm path com.example # -> package:/data/app/.../base.apk + splits
adb pull /data/app/~~.../com.example-.../base.apk # repeat for each split; reinstall with
adb install-multiple base.apk split_config.*.apk # install-multiple
```
### 6.5 Vendor D2D and cloud (no-tech last mile)
- **Google** cloud backup + Pixel/Android cable "device-to-device" restore at setup. App data rides
along *only* if the Google **account** is transferred (Block Store is account-bound); untick
nothing under "Accounts". `allowBackup="false"` stops cloud backup but D2D still transfers app data
unless the app sets `dataExtractionRules`/`android:fullBackupOnly` to opt out.
- **Samsung Smart Switch** (PC or cable): backs up most app data, but **encrypted app data (e.g.
WhatsApp), banking logins, and sideloaded APKs do not transfer** — handle those per §9/§10.
- **Seedvault** (CalyxOS/GrapheneOS/LineageOS): the FOSS backup transport. On Android 15 builds its
**device-to-device** mode (Expert settings) is default-on and captures far more app data than the
old cloud-style transport, because it presents as a D2D migration to each app.
---
## 7. The death of `adb backup` (don't build a plan on it)
`adb backup` was deprecated in Android 10 and gutted in Android 12 (API 31): app data is excluded by
default and only included if the app is **debuggable**, so on essentially all release apps across
Android 12–16 `adb backup` produces an archive with no app data. Treat it as unavailable:
- Don't rely on it for a migration. Use root tooling (§4.1), on-device FOSS exporters (§6.2), or
vendor/Seedvault D2D (§6.5) instead.
- `adb restore` likewise cannot restore encrypted backups and is a dead end for modern data.
- The `bmgr`/LocalTransport backup-manager plumbing still exists for *developers testing their own
backup agents* (`adb shell bmgr …`), but it is a per-app developer tool, not a device-wide
extractor — out of scope for a pre-wipe sweep.
---
## 8. EDL / firehose / BROM — dead or bootloader-locked devices
The lowest level, for devices that will not boot to Android *or* recovery. These talk to the SoC's
emergency loader below the OS:
- **Qualcomm EDL (9008)** — [`bkerler/edl`](https://github.com/bkerler/edl) with a firehose
programmer (`.mbn`/`.elf`) matching the SoC. `edl printgpt` lists partitions; `edl rl dumps
--skip=userdata --genxml` dumps all partitions (minus the huge userdata) to a folder with a
`rawprogram` XML; `edl r <partition> <out.img>` grabs one. Newer SoCs need `--loader=<file>`
because loader autodetection no longer works.
- **MediaTek BROM** — [`bkerler/mtkclient`](https://github.com/bkerler/mtkclient). `./mtk printgpt`,
`./mtk rl out` dumps all partitions to `out/`, `./mtk r <partition> <out.img>` for one; `--preloader`
for BROM mode.
The hard limit: **these dumps are ciphertext.** EDL reads the raw flash below the OS, so `userdata`
comes out FBE-encrypted and is not decryptable without the keys (which live in secure hardware keyed
by the lockscreen credential). EDL is therefore excellent for **small identity partitions**
(EFS/persist/nvram — §4.3), full-flash forensic images, and un-bricking, but it is **not** a way to
read a locked phone's app data. Treat downloaded dumps/tools as untrusted input: keep each in its own
directory, pass paths as arguments, and run any helper scripts from outside the dump directory.
---
## 9. App-specific data that needs its own handling
Some high-value data lives behind app-level encryption and ignores the tier ladder:
- **WhatsApp** — local backup at `/sdcard/WhatsApp/Databases/msgstore.db.crypt15` (+ `/Media`). If
the user enabled **end-to-end encrypted backups**, decryption needs the **64-digit key** (or the
`encrypted_backup.key` file from `/data/data/com.whatsapp/files/`, root-only). Tools:
[`wa-crypt-tools`](https://pypi.org/project/wa-crypt-tools/). Easiest migration is WhatsApp's own
built-in transfer or Google-Drive backup with the same phone number — pull the `crypt15` + media
as insurance, not as the primary path.
- **Signal** — does **not** sync to any cloud. Migrate with Signal's in-app device-transfer, or its
encrypted local backup (Settings → Chats → Backups) whose **30-digit passphrase you must record** —
there is no recovery without it.
- **Authenticator / 2FA** — Google Authenticator exports via its QR "export accounts" flow;
**Aegis** (FOSS) exports an encrypted JSON vault. Do this *before* wiping; TOTP seeds not exported
are gone.
---
## 10. What will NOT survive a wipe (plan around it)
No tier recovers these; the only mitigation is to migrate them *before* the wipe:
- **Hardware-backed Keystore keys** are non-exportable by design — key material never leaves the
TEE/StrongBox. Any app that pinned a device-bound key (banking apps, some password managers,
FIDO/passkeys stored in hardware) will treat the restored/new device as untrusted and re-enrol.
- **Play Integrity / SafetyNet attestation** fails on rooted or custom-ROM devices independently of
your data — so a perfectly-restored banking app may still refuse to run. That is an attestation
problem, not a backup problem; don't chase it as data loss.
- **SSAID / `ANDROID_ID`** changes on wipe (per app-signing-key + user), which can log apps out or
reset entitlements even when their `/data/data` is restored byte-perfect.
Before wiping: export 2FA (§9), confirm password-manager vaults sync to their cloud, record Signal
(§9) and WhatsApp keys, and re-enrol passkeys/banking on the new device rather than expecting a
restore to carry them.
---
## 11. Pre-wipe checklist (the one-screen version)
1. **Decide the tier** (§1). Booting + unlockable → root (§4) or ADB (§6). Not booting → recovery
(§3) → EDL (§8).
2. **EFS/modem/persist first** (§4.3) — before any unlock or flash, off-device.
3. **Media & docs** — `adb pull /sdcard` (§6.1).
4. **App data** — root tar + dedicated tool (§4.1), or recovery-decrypted Data backup (§3).
5. **Scoped dirs** — `Android/data`, `Android/obb` (§6.1).
6. **PIM** — SMS/call-log/contacts via SMS Import-Export or vCard (§6.2).
7. **App-specific** — WhatsApp/Signal/2FA via their own flows (§9).
8. **App list + APKs** — `pm list packages -3`, pull splits (§6.4) to rebuild the app set.
9. **System state the app tars miss** (§12.1) — keystore DB, SSAIDs, URI grants, doze whitelist,
app-ops, launcher/shortcuts/QS/controls, Wi-Fi, SD-card UUIDs, root grants, old `packages.list`.
10. **Baseline + app-sweep** the old ROM (`measure.sh`, `test-all-apps.sh`, §12.4) so the new one
has something to be compared against.
11. **Passwords apps keep themselves** — App Manager's KeyStore password, backup-app passphrases,
Signal/WhatsApp keys (§9). Write them down *before* the wipe; none are recoverable after.
12. **Verify the backups off-device** (open the archive, check `ls /data/data` really decrypted)
*before* you type `fastboot -w` or confirm the reset. A backup you have not read back is not a
backup.
---
## 12. Restoring onto a new ROM (root) — what a data tar does not carry
Learned from a full LineageOS-based A16 → Infinity-X A17 switch (Redmi Note 7, Magisk). Restoring
`/data/data` + `user_de` + `Android/data` per §4.4 brings back most apps, but **every app gets a new
UID** on the new ROM, and a lot of state is keyed by UID or lives in system files. The scripts in
[`scripts/android/`](skills-public/scripts/android/) automate the parts below; they take the
device from `ANDROID_SERIAL` like `adb` itself.
### 12.1 Grab these too, before the wipe (all root, all small)
| What | Where | Why it matters on restore |
| :--- | :--- | :--- |
| Old UID map | `/data/system/packages.list` | Remap everything keyed by UID (SSAID, keystore, grants) |
| Keystore2 DB | `/data/misc/keystore/persistent.sqlite` | Apps whose keys vanish show "Keystore error"/log out (§12.3) |
| SSAIDs | `/data/system/users/0/settings_ssaid.xml` | `ANDROID_ID` change logs apps out (GitHub, Bolt, Lovense, …) |
| Persistent SAF grants | `/data/system/urigrants.xml` | Folder pickers (music players, camera, sync apps) lose their folders |
| Doze whitelist | `/data/system/deviceidle.xml` | Notification/automation apps silently stop in background |
| App-ops & special access | `cmd appops dump`, `cmd notification` policy access, `dumpsys package` | `MANAGE_EXTERNAL_STORAGE`, usage stats, DND, `READ_LOGS`, install-packages, notification listeners are **not** runtime permissions |
| Launcher + pinned shortcuts | launcher DB, `/data/system_ce/0/shortcut_service/` | Home screen; pinned shortcuts (e.g. Ice Box) are owned by the system, not the launcher |
| QS tiles, device controls | `settings get secure sysui_qs_tiles`; SystemUI `user_de` `files/controls_favorites.xml` + `shared_prefs/controls_prefs.xml` | Home Assistant controls, dev tiles |
| Wi-Fi | `/data/misc/apex/com.android.wifi/` or `cmd wifi list-networks` + PSKs | Also note which SSIDs must **not** auto-join |
| Root grants + Magisk settings | `magisk --sqlite "SELECT * FROM policies"`, `… FROM settings`, `… FROM denylist`, `/data/adb/modules` (note which are disabled) | UIDs change, grants don't follow; Zygisk off ⇒ DenyList silently does nothing |
| Accounts | `dumpsys account`; `/data/system_ce/0/accounts_ce.db` | Shows which logins (2nd Google account, app-registered accounts) must be re-added by hand — they can't be transplanted |
| SD-card volume IDs | `blkid` of each SD partition | Public volume ID = filesystem UUID; reformat ⇒ new ID ⇒ broken paths and grants |
| Freezer state | `pm list packages -d` + the freezer app's own list | Updates/reinstalls unfreeze |
### 12.2 Restore order and traps
- **Update first, restore data second.** Installing the old APKs and then the store's update is
fine; restoring data *into* an older APK than the data was written by is a downgrade — apps wipe
or crash on it. Never re-run a full restore pass over updated apps; restore data only.
- **Installer of record:** `pm set-installer` refuses even root. Reinstall the same APKs with
`adb install-multiple -r -i <installer>` (keeps data) so Play/Aurora/F-Droid/Obtainium see their
apps as updatable (`fix-installers.py`).
- **New signature, same package** (forks such as Shizuku → Shizuku-Next for a newer Android): tar the
data dirs, uninstall, install, untar, chown/chcon, re-grant, re-freeze — `reinstall-keep-data.sh`.
- **System files are read only at `system_server` start.** Write `urigrants.xml`,
`settings_ssaid.xml`, `shortcut_service/*` etc. (owner `system:system`, mode 600,
`u:object_r:system_data_file:s0`), then `kill -9 $(pidof system_server)` — a soft restart that
does **not** save over them first, which a normal reboot does. It locks the user: unlock with
wake → swipe → `input text <PIN>` → ENTER (the bouncer screenshots black; retry if needed).
- **fs-verity:** files under `shortcut_service` are verity-protected — `rm` then write, never edit.
- **Restored caches can kill an app:** Ice Box OOM-crashed on a restored MMKV package cache. Drop
`cache`, `code_cache`, WebView caches and per-app package-list caches; keep the real state.
- **Re-map UID-keyed state** with the old/new `packages.list` (SSAID `name="<uid>"`, keystore
`namespace`, Magisk `policies.uid`, doze lists are by package and need no remap).
- **SD card:** if the partition had to be recreated, set the old UUID back
(`tune2fs -U <old>` for ext4, `fsck` before and after) rather than fixing every app.
- **Wi-Fi:** restored networks auto-join in their own order; re-add the ones that must stay
manual with `cmd wifi add-network <ssid> wpa2 <psk> -d`.
- **QS tiles** on A17: `settings put secure sysui_qs_tiles` is ignored; use
`cmd statusbar set-tiles "<list>"`. Developer tiles must be enabled first
(`pm enable 'com.android.settings/.development.qstile.DevelopmentTiles$SensorsOff'`).
- **Launcher switch** (Nova had no build for the new Android): converting Nova's `favorites`
table into Lawnchair's Launcher3 schema works (`nova-to-lawnchair.py`) — mark widgets
`restored=1` so Launcher3 binds fresh widget IDs, and re-pin deep shortcuts for the new launcher.
**Banking-app root checks** are not a restore problem, but they show up right after one. Order
that worked (2026-10, Magisk 31, A17): first the ROM's own spoofing settings, if it has any (PIF,
TrickyStore target and keybox, hide developer status). If an app still crashes on purpose, look for
an Xposed module that defuses that app's check. For PayPal 10.18 that was oslokrok on Vector, the
LSPosed successor. Configure Vector headlessly with
`/data/adb/modules/zygisk_vector/cli modules enable <pkg>` and `cli scope set <pkg> <app>/0`.
**Take the app off the Magisk DenyList**: the DenyList strips every Zygisk module from the process,
including the hook.
### 12.3 Keystore: transplanting app keys
Hardware-backed keys never leave the TEE, but keystore2 stores them **wrapped** in its SQLite DB.
If the new ROM runs on the same TEE (compare `keyentry.km_uuid` old vs new — same device, same
firmware ⇒ same), the old wrapped blobs still unwrap. Keys whose `blobmetadata` has tag 0
(`EncryptedBy`) are additionally super-encrypted with the **old lock-screen credential's** key and
cannot be moved. Domain 0 (APP) rows have `namespace = uid` → remap. Stop the apps, back up the live
DB, insert `keyentry`/`blobentry`/`blobmetadata`/`keyparameter`/`keymetadata` with fresh ids, reboot.
Apps that already regenerated keys on the new ROM also need their data re-restored to match.
`scripts/android/keystore-transplant.py OLD_DB [--replace] OLD_UID:NEW_UID …` writes the SQL; dry-run
it on a host copy of the live DB (`pragma integrity_check`) before applying with the device's own
`sqlite3`. Tested: transplanted keys load with no keystore errors after a reboot. That does not
bring back sessions the server ended, or apps that refuse to run because their build is too old.
### 12.4 Verify like a user would
- `test-all-apps.sh OUT` cold-starts every user app (unfreezing and re-freezing), screenshots it,
and greps its log for crashes, ANRs and keystore/crypto errors → `results.tsv`. Look at every
screenshot: logouts, licence checks, ToS/"what's new" dialogs and permission prompts don't crash.
- Android 17 shell launches must match the activity's intent filter:
`am start -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n 'pkg/act'`
(single-quote it — component names contain `$`). `-n` alone gives "Access blocked".
- `measure.sh OUT [pkg…]` snapshots memory, crashes/ANRs/lmk kills since boot and cold-launch
times; run it on the old ROM before the wipe so there is a baseline.
- Keep a running log of every security setting weakened during the restore (adb stay-awake, root
grants, doze exemptions, …) with its revert command; the restore needs many of them and nobody
remembers them afterwards.
+325
View File
@@ -0,0 +1,325 @@
---
name: cli-tooling
description: >-
Use this skill whenever invoking a command-line tool, choosing between a CLI and an ad-hoc
script, writing a standalone utility script, or orienting on unfamiliar work before starting.
Covers discovering a CLI's real interface instead of guessing flags, non-interactive and
machine-readable invocation, checking an API's payload rather than its exit status, keeping
credentials out of argv, eventually-consistent systems that briefly report the old state,
script placement and symlinking, and care around destructive or outward-facing commands —
including additive changes that silently take over a name already serving traffic.
---
# CLI & Tooling — Discovery, Invocation, and Script Placement
How to use command-line tools correctly, and where utility scripts belong.
Companion to [`coding-dev.md`](skills-public/coding-dev.md), which covers writing the code itself.
---
## 1. Discovery Over Guessing
> [!IMPORTANT]
> **Read the tool's own help before using it. Never guess subcommands or flags.**
> ```bash
> <tool> --help
> <tool> <subcommand> --help
> ```
A tool's built-in help is authoritative and current; memory and training data are neither.
Flags get renamed, subcommands get restructured, and a confidently-invented flag either
errors out or — far worse — silently means something else.
**This applies to listing capabilities, too.** When asked what a tool can do, run its help
and report what it actually says. Do not produce a speculative command list; a plausible-
looking invented flag is indistinguishable from a real one to whoever reads it next.
### Walk the whole help tree
Multi-level CLIs hide most of their surface one level down. `<tool> --help` shows the
subcommand list; the flags that matter usually live in `<tool> <subcommand> --help`.
### Don't reinvent a tool that exists
**When a dedicated CLI already provides the functionality, use it** rather than writing an
ad-hoc script, a one-off `curl` pipeline, or a bespoke API client. The CLI already handles
authentication, retries, pagination, error formatting, and the edge cases you have not
thought of yet.
Before writing a script to do something, check whether:
- a dedicated CLI for that system is already installed,
- the tool you are already using has a subcommand for it,
- the project's own `scripts/` directory already contains it.
### Desktop Steam Client Lifecycle (`steamcli`)
> [!TIP]
> **Never spawn Steam via raw shell backgrounding (`nohup steam &` or `steam -silent &`) or kill it with `pkill`.**
> On desktop Linux/Bazzite, raw spawning leaves orphaned bash wrappers, misses custom client flags, or fails to initialize properly.
> Instead, manage the desktop Steam client through **`steamcli client`**:
> ```bash
> # Graceful exit and wait for it to finish flushing configuration:
> steamcli client shutdown
>
> # Launch the desktop client cleanly with configured defaults:
> steamcli client launch
>
> # Launch with custom Steam arguments:
> steamcli client launch -- -silent
> ```
---
## 2. Non-Interactive Invocation
Anything run from a script, a CI job, or an agent must not depend on a human being present.
- **Pass required targets explicitly.** A CLI that manages multiple objects will, when no
target is given, either prompt interactively or fail — both of which break automation.
Always pass `--server <id>`, `--profile <name>`, `--target <x>` rather than relying on a
default or a prompt.
- **Use the tool's own blocking flag** (`--wait`, `--follow`, `--no-detach`) when you need an
operation to finish before the next step. Do not build a polling loop around a command that
already knows how to wait — and never write a shell `until`/`while` wait loop
(see [`code-languages/bash.md`](skills-public/code-languages/bash.md)).
- **Prefer a `--dry-run` rehearsal** for anything that deploys, releases, or deletes, when the
tool offers one.
- Set `--yes` / `--non-interactive` deliberately and only where the action is genuinely safe —
not as a reflex to stop a tool from asking.
---
## 3. Machine-Readable Output
When output feeds another program — or an agent's context — ask the tool for a structured
format rather than parsing its human-facing table.
```bash
tool list -o json # full fidelity
tool list -o compact # minimal tokens, good for agent context
tool list -o csv # structured columns for a spreadsheet or script
```
- **Never scrape a table meant for humans.** Column widths, truncation, colour codes, and
alignment all change without notice; a structured format is a contract.
- **Check the payload, not just the exit status.** Plenty of HTTP APIs answer `200 OK` with
`{"success": false, "errors": [...]}` in the body, and `curl` exits `0` because the
*request* worked. A script that trusts the exit code reports a clean success and writes
nothing. Assert on the field the API uses for this, and print `errors[]` when it trips:
```bash
if [[ "$(jq -r '.success' <<<"$response")" != true ]]; then
jq -r '.errors[]? | " - \(.message)"' <<<"$response" >&2
exit 1
fi
```
- Strip colour explicitly (`--color never`, `NO_COLOR=1`) when capturing output, or escape
sequences end up in your parsed values.
- Keep in mind that a "compact" or summary format is the right default for feeding an agent,
while `json` is right for a script that needs every field.
> [!WARNING]
> **Never pipe build, lint, or test output through a filter to make it shorter.** That is a
> different situation and the rule there is the opposite — see
> [`coding-dev.md` §2](skills-public/coding-dev.md). Structured output flags are for data; `grep` on
> diagnostics hides warnings.
---
## 4. Orient Before Starting
Before beginning work in an unfamiliar project or one you have not touched recently, spend a
moment establishing what has already happened. Cheap, and it prevents the most expensive
failure mode: rebuilding something that already exists.
In rough order of value:
1. **The project's own instructions** — `AGENTS.md`, `CLAUDE.md`, `.rules`, `CONTRIBUTING.md`,
`README.md`. Read them before touching code.
2. **Recent history** — `git log --oneline -20`, and `git status --short` to see what is
already in flight so you do not clobber it.
3. **Prior work records** — session/conversation history tooling, issue trackers, or notes
files, scoped to the project directory.
4. **The code itself** — `grep` for the symbol before trusting any document that says it is
missing.
5. **Whether the thing is already running** — before editing a live app's config or starting a
second copy. Ask the supervisor (`systemctl --user is-active <unit>`) and, for a bare
process, `pgrep -ax <name>` — never bare `pgrep -f <name>`, which matches your own shell and
reads as a false positive. See
[`code-languages/bash.md`](code-languages/bash.md#pgrep--f-matches-its-own-shell).
> [!CAUTION]
> **Task documents are leads, not facts.** A `TODO.md` entry can be stale, duplicated, or
> simply wrong — including when it sounds confident and especially when it sounds familiar.
> Verify against the code before acting on one. A single `grep` is cheaper than rediscovering
> that the work shipped last week.
---
## 5. Where Scripts Live
> [!IMPORTANT]
> **Never write a standalone script directly into a `PATH` directory** (`~/.local/bin`,
> `/usr/local/bin`, or similar). Write it into a **dedicated scripts directory** that is
> version-controlled, then **symlink** it into `PATH`.
```bash
# write the script to its real home
$EDITOR /path/to/scripts/my-tool.sh
chmod +x /path/to/scripts/my-tool.sh
# then expose it
ln -s /path/to/scripts/my-tool.sh ~/.local/bin/my-tool
```
**Why:** one versioned source of truth. A script living directly in `PATH` is invisible to
version control, easy to clobber on a reinstall, impossible to diff, and indistinguishable
from a package-manager-owned binary sitting beside it. The symlink makes the provenance
obvious — `ls -l` shows exactly where the real file is.
The same applies within a project: scripts belong in `scripts/`, referenced by the build
system, not scattered at the repository root.
---
## 6. Destructive and Outward-Facing Commands
- **Look at the target before deleting or overwriting it.** Inspect it first — the cost of a
wrong `rm` or a wrong overwrite is unbounded and the check is one command.
- **Read-only first.** When exploring an unfamiliar tool, use its inspection subcommands
(`status`, `list`, `show`, `get`) before any mutating one.
- **Mutating commands that rewrite shared configuration** — syncing settings across machines,
purging caches or state, patching installed applications — should be confirmed before
running, even when they are reversible in principle.
- **Never use real data as scratch space.** Real accounts, real configuration directories,
live services, and the user's actual files are not disposable test fixtures. Use temporary
directories and disposable targets.
- Anything that sends, publishes, deploys, or purchases is outward-facing: confirm first, and
treat approval as applying to that one action rather than the category.
### Adding something can take something over
> [!CAUTION]
> **A change that deletes nothing can still break a live service.** The dangerous operations
> are not only the ones named `delete`. Adding a routing rule, a proxy entry, a redirect, a
> DNS override or a higher-priority config can silently intercept a name that is already
> serving traffic — while the thing it shadows still looks perfectly healthy in every
> listing, because it was never touched.
>
> That is the worst shape a failure can take: the old config is still there, still correct,
> still pointing where it should, and the service is dead anyway. Nothing in the inventory
> says what happened.
**Before claiming a name, ask what currently answers to it** — and refuse by default when
something does:
```bash
# Only an unrouteable placeholder, or an entry this tool made, is safe to assume is free.
if [[ "$(current_target "$host")" != "$PLACEHOLDER" && "$(owner "$host")" != "$MARKER" ]]; then
[[ "$FORCE" == true ]] || die "$host already serves $(current_target "$host") — pass --force"
fi
```
Two things make that check possible:
- **Mark what you create.** A comment field, a tag, a description prefix — anything that
distinguishes *your* entries from ones a human made. It turns "is this mine to replace?"
into a lookup rather than a guess, and it is what lets the matching `--delete` refuse to
remove something it did not create.
- **Make `--force` the only way past**, and name in the refusal what is there now. A tool that
silently overwrites is one nobody can safely run against production.
> [!TIP]
> Test such a tool against a name you **expect** it to reject, not only against a free one.
> The rejection path is where the damage would have been, and it is the half a happy-path test
> never exercises.
---
## 7. Credentials Belong on stdin, Not in `argv`
> [!WARNING]
> **`argv` is world-readable.** Any user on the box can run `ps` and read the full command
> line of every process. A token passed as `-H "Authorization: Bearer $TOKEN"`, `--password`,
> or `--token` is therefore exposed for as long as the call runs — and lands in shell history
> and process accounting too.
Most CLIs that accept a secret as a flag also accept it somewhere safer. Use that instead:
```bash
# WRONG — the token is visible in `ps` for the life of the request
curl -H "Authorization: Bearer $TOKEN" "$url"
# RIGHT — curl reads its options from stdin; argv holds nothing sensitive
printf 'header = "Authorization: Bearer %s"\n' "$TOKEN" | curl --config - "$url"
```
The equivalent elsewhere: `--password-stdin` (docker login, many package tools), `-u` with a
`.netrc`, `--token-file`, or simply an environment variable the tool reads by name. Reach for
the env var when nothing better exists — it is readable through `/proc/<pid>/environ` but only
by the owner and root, which is a meaningfully smaller exposure than `argv`.
See [`code-languages/bash.md` §8](skills-public/code-languages/bash.md) for the surrounding rules on
where secrets are stored and why they are never echoed.
---
## 8. Eventually-Consistent Systems Lie Briefly
Edge networks, DNS, CDNs, object stores, container registries and clustered schedulers all
acknowledge a write before it is visible everywhere. **A read taken immediately after a
successful write is not evidence of anything.**
The failure mode is believing the tool is broken and "fixing" it:
- A changed rule still returns the **old** value for a while — which reads as "my write was
ignored".
- A brand-new name returns an **error** for a while — a 5xx, an NXDOMAIN, a 404 — which reads
much worse than it is.
So: take the write's own success response as the result, and treat the first few reads as
unreliable. When a verification read matters, **say how long propagation takes** in the tool's
own help or header comment, so the next person does not rediscover it at 2am. State it as a
number — "~30s for a rule, ~40s for a new name" — because "eventually" tells nobody when to
start worrying.
Do not wrap this in a polling loop (see
[`code-languages/bash.md` §3](skills-public/code-languages/bash.md)); take one sample per
invocation and let the caller run it again.
> [!WARNING]
> **Checking too early can make the delay much worse.** A failed lookup is itself cached. Query
> a DNS name before it exists and the resolver stores the `NXDOMAIN` for the SOA's negative TTL
> — commonly 300-1800s — so the name stays invisible to *you* long after it is live for
> everyone else. Polling during propagation is what creates this, which is a second reason not
> to loop.
>
> When a check does come back negative, confirm where the negative lives before concluding
> anything: query the authoritative source directly and compare. The answer is often "the
> record is fine, my resolver is lying", and flushing the local cache will not help if the
> stale entry is on an upstream resolver you do not control.
>
> ```bash
> dig +short @1.1.1.1 "$name" # authoritative-ish: is it actually published?
> dig +short "$name" # what my resolver believes
> curl --resolve "$name:443:<ip>" "https://$name/" # bypass DNS entirely to test the service
> ```
---
## 9. Environment Constraints
- **Entering a container or changing shells resets the working directory.** Chain `cd` inside
the same command string rather than relying on it persisting:
```bash
container-run <name> -- bash -c "cd /path/to/project && make"
```
- **Verify what actually exists rather than assuming from documentation.** Container names,
tool versions, and installed toolchains drift. A one-command check (`<tool> list`,
`<tool> --version`, `command -v <tool>`) costs nothing and prevents a confident invocation
of something that was removed months ago.
- **Check for an optional dependency before the step that needs it**, and fail loudly if it
is missing. A packaging step that silently skips when its tool is absent ships an
incomplete release.
- **Don't saturate the machine.** A full build consuming every core makes the workstation
unusable. When something latency-sensitive is running, reduce the job count and `nice` the
work — and say that you did.
+207
View File
@@ -0,0 +1,207 @@
---
name: bash
description: >-
Use this skill when writing or modifying Bash/shell scripts, build pipelines, release
scripts, or when running shell commands as an agent. Covers strict mode, shellcheck,
quoting, the no-wait-loops rule, build-script structure, and destructive-command safety.
---
# Bash / Shell — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds shell specifics.
Bash is for **orchestration** — sequencing tools, wiring a build pipeline, gluing CLIs.
Once a script grows real data structures, arithmetic, or parsing, it wants to be
[Python](skills-public/code-languages/python.md) or a compiled tool instead.
---
## 1. Strict Mode & Linting
Every script starts with:
```bash
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
```
- `-e` exit on error, `-u` error on unset variable, `-o pipefail` so a failure mid-pipe is
not masked by a successful final command.
- **`shellcheck` must be clean** and belongs in the build gate. Do not silence a finding with
a `# shellcheck disable=` without a comment saying why.
- Format with `shfmt`.
> [!IMPORTANT]
> `set -e` does **not** fire inside `if`, `&&`, `||`, or command substitution used in a
> condition. Do not assume a failing command inside those aborts the script — check
> explicitly where it matters.
---
## 2. Quoting & Expansion
- **Quote every expansion**: `"$var"`, `"${arr[@]}"`, `"$(cmd)"`. Unquoted expansion word-splits
and glob-expands, which is how a path with a space becomes two broken arguments.
- `"${var:-default}"` for defaults; `"${var:?message}"` for required values.
- Use arrays for argument lists — never build a command as a string and `eval` it.
- `[[ ]]` over `[ ]`. `$(…)` over backticks.
- `local` every variable inside a function.
- `printf` over `echo` for anything with escapes or leading dashes.
---
## 3. Never Write Wait Loops
> [!CAUTION]
> **Do not write `until <check>; do sleep 2; done` or `while ! <check>; do sleep …; done`.**
> They hang the terminal, and the only way out is an interrupt.
To wait for something:
- Run the work in the **background** and wait for its completion signal.
- To check external state, **take a single sample per invocation** rather than looping.
- If a tool has a real blocking wait (`--wait`, `wait-for`, `systemctl is-active --wait`),
use that instead of polling around it.
This mirrors the general events-over-polling preference: prefer a notification to a loop.
---
## 4. Never Filter Build Output
Repeating the global rule because shell is where it gets violated:
```bash
# WRONG
cargo build 2>&1 | grep -E "error|warning"
npm run build | tail -20
# RIGHT
cargo build
npm run build
```
Tee to a log if you need a copy — `cmd 2>&1 | tee build.log` preserves the full output — but
never `grep`/`head`/`tail` it away. Narrow the command instead when output is genuinely huge.
---
## 5. Destructive Commands
- **Look at the target before deleting or overwriting it.** `ls` it, `stat` it, read it.
- Never `rm -rf "$VAR"` without first asserting `$VAR` is non-empty and is what you think:
```bash
[[ -n "${BUILD_DIR:-}" && -d "$BUILD_DIR" ]] || { echo "refusing: BUILD_DIR unset" >&2; exit 1; }
```
- **A shared output directory belongs to nobody.** If two scripts write to `dist/`, neither
may `rm -rf` it — each cleans only its own outputs.
- Write scratch files to a `mktemp -d` and trap cleanup:
```bash
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
```
- Prefer a `--dry-run` flag on anything that deploys, releases, or deletes — and implement it
honestly.
### `pgrep -f` matches its own shell
> [!WARNING]
> The command line containing the pattern **is itself a process**, so `pgrep -f foo.conf`
> matches the shell running it. Piping that into `kill` kills the running script — which reads
> exactly like "the thing I was testing never happened".
The quieter failure is **misreading** the result. `pgrep -f VRCNext` prints a PID, you conclude
the app is running, and you refuse to edit its config — when the only match was your own shell.
Agents hit this constantly, because an agent's `bash -c` command line is enormous: it carries a
snapshot path, the whole `eval '...'` of the script, and every argument inside it.
**Always pass `-a`.** Bare `pgrep` prints naked PIDs, which cannot be sanity-checked; `-a`
prints the full command line, so a self-match is obvious on sight:
```bash
pgrep -af myapp # 3201245 /bin/bash -c ... eval 'pgrep -af myapp' ... ← that is you
```
Then pick the invocation that cannot match you in the first place:
```bash
pgrep -ax myapp # match the process NAME only; never scans arguments ← prefer this
pgrep -Af myapp # -A/--ignore-ancestors: excludes your own shell and its parents
pgrep -af 'my[a]pp' # the bracket trick: the regex no longer matches its own literal text
```
- **`-x` is the strongest and the default choice** when you are asking "is this program
running?". It compares against the command name, so the pattern is never tested against any
argument list — including yours. Reach for `-f` only when you must distinguish two instances
of the same binary by their arguments.
- **`-A` is the right fix when `-f` is unavoidable**, and it is robust where the bracket trick
is not. Confirm it exists before relying on it in a script (`pgrep --help | grep ancestors`);
it is present in procps-ng 4.0.6.
- **The bracket trick is weaker than it looks.** `pgrep -f '[V]RCNext'` can still match the
calling shell, because the word `VRCNext` may appear *elsewhere* on that command line — such
as in a `cd ~/.config/VRCNext` earlier in the same script. The trick only defends against
matching the pattern's own literal text, not against the term occurring anywhere else in the
command being run.
> [!TIP]
> For a service under systemd, don't use `pgrep` at all — ask the supervisor, which has no
> self-match problem and distinguishes *stopped* from *failed*:
> `systemctl --user is-active <unit>`.
---
## 6. Build Script Structure
A build script is the project's **quality gate**, and it should be exhaustive enough to catch
mistakes before a human does.
- **Fixed action order regardless of flag order**, with any failure aborting the rest:
```
bump → lint → build → artifacts → stop → deploy → start → commit → push → release
```
- **A skipped test fails the gate.** Never let a skip report green.
- **Print a summary with numbers**, not just a verdict — counts of warnings, suppressions, and
file-size violations. A suppression count that has grown is worth seeing even on a pass.
- **Don't reimplement in the gate what a test can assert.** Checks like file-size limits
belong in the test suite where they travel with the repo and fail with a useful message;
the build script is the fallback for projects with no suite to hang them on. See
[`coding-dev.md` §4](skills-public/coding-dev.md).
- Keep per-step logs in a known directory so a failure can be read after the fact.
- **Never bypass a failed gate** by copying a binary or running the install step by hand. Fix
the failing step; do not weaken the check to make it pass.
- Support `--dry-run` for the full release path.
- **Throttle heavy builds.** Detect when something latency-sensitive is running and drop to
half the cores at `nice -n 10`, announcing that it did so. Provide an override env var.
---
## 7. Paths & Portability
- Resolve the script's own directory rather than assuming the caller's `cwd`:
```bash
script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
```
- Prefer absolute paths in anything non-interactive.
- **Entering a container resets the working directory** — chain `cd` inside the same command
string: `container-run -- bash -c "cd /path && make"`.
- Don't hardcode `/tmp` when the content must survive a cleanup; `/var/tmp` persists longer.
- `command -v tool >/dev/null || { echo "tool required" >&2; exit 1; }` before using an
optional dependency. **A silently skipped step is worse than a failure** — a packaging step
that quietly skips when its tool is missing will ship an incomplete release.
---
## 8. Secrets
- Read secrets from an environment file or secret store; **never** inline them in a script.
- Never `echo` a secret, and never pass one as a command-line argument — argv is world-readable
in the process table. Use an env var or stdin.
- Never commit a file containing real credentials; commit an example with empty values.
---
## 9. Script Placement
Standalone utility scripts belong in a **dedicated, version-controlled scripts directory**,
symlinked into `PATH` rather than copied there. Never write a loose script directly into
`~/.local/bin` or another `PATH` directory — the source of truth should be one versioned
location. See [`cli-tooling.md` §5](skills-public/cli-tooling.md) for the reasoning and the pattern.
+146
View File
@@ -0,0 +1,146 @@
---
name: c-cpp
description: >-
Use this skill when writing, building, or debugging C or C++ — including compiler warning
configuration, sanitizers, modern C++ resource management, build systems (CMake/Meson),
32-bit/cross builds, and FFI boundaries exposed to other languages.
---
# C / C++ — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds C/C++ specifics.
C and C++ are for the cases with no better option: existing codebases, hard real-time paths,
and FFI boundaries. **For new standalone work, prefer [Rust](skills-public/code-languages/rust.md)** — the memory-safety
guarantees are worth more than the familiarity.
---
## 1. Warnings — Everything On, Errors
```cmake
target_compile_options(mytarget PRIVATE
-Wall -Wextra -Wpedantic -Werror
-Wshadow -Wconversion -Wsign-conversion
-Wnon-virtual-dtor -Wold-style-cast -Wcast-align
-Wunused -Woverloaded-virtual -Wdouble-promotion
-Wformat=2 -Wnull-dereference -Wimplicit-fallthrough
)
```
- **`-Werror` in the build gate.** `-Wconversion` and `-Wshadow` are noisy on first contact and
catch real bugs — do not drop them because a legacy file complains; fix the file or scope the
suppression to it.
- Run **both** GCC and Clang where possible; each diagnoses things the other misses.
- `clang-tidy` with `bugprone-*`, `cert-*`, `performance-*`, `modernize-*`.
- `clang-format` with a committed `.clang-format`. No hand-formatting.
---
## 2. Sanitizers — Use Them
```bash
cmake -DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer"
```
- **ASan + UBSan on every debug/test build.** They find the bugs that warnings cannot.
- TSan for anything threaded (separate build — it is incompatible with ASan).
- Valgrind when a sanitizer build is not possible.
- **A sanitizer finding is a bug, not noise.** Fix it.
---
## 3. Resource Management (C++)
- **RAII for everything.** Every raw `new`/`delete`, `malloc`/`free`, `fopen`/`fclose`, and
native handle pair is a leak waiting for an early return or a throw.
- `std::unique_ptr` by default; `std::shared_ptr` only where ownership is genuinely shared.
`std::weak_ptr` to break cycles.
- **No owning raw pointers.** A raw pointer or reference in a signature means "borrowed,
non-owning, outlives the call".
- `std::string_view` / `std::span` for non-owning views — and never return one that outlives
its backing storage.
- **The Rule of Zero**: design so no destructor, copy, or move needs writing. If you must write
one, write all five.
- `constexpr`, `const`, and `noexcept` wherever they hold. `[[nodiscard]]` on anything whose
return value must not be dropped.
---
## 4. C Specifics
- C17 or later unless the target forbids it.
- **Check every allocation and every return value.** In C, unchecked returns are the primary
bug source.
- Fixed-width types from `<stdint.h>` (`uint32_t`), never bare `int` for anything with a
defined width on the wire.
- Always the bounded string functions (`snprintf`, `strncpy` with explicit termination); never
`strcpy`, `sprintf`, `gets`.
- `static` on every function and global not exported from the translation unit.
- Prefer a struct-and-functions module boundary with an opaque pointer over exposing layout.
---
## 5. Build Systems
- **CMake** (3.20+, target-based) or **Meson** for new projects. Never a hand-written Makefile
for anything non-trivial.
- Target-based CMake only: `target_link_libraries`, `target_include_directories`,
`target_compile_options`. **No directory-level `include_directories` or global
`CMAKE_CXX_FLAGS` mutation.**
- `PRIVATE` / `PUBLIC` / `INTERFACE` chosen deliberately — this is what keeps include leaks out
of dependents.
- **Register every new source file with the build system** in the same commit that adds it. A
source file the build never compiles will sit there looking correct while every edit to it
does nothing.
- Vendor dependencies with `FetchContent` or submodules; document which and why.
---
## 6. Cross & Multi-Arch Builds
- Keep 32-bit and 64-bit outputs in **separate build directories**; never reuse one.
- **A component built for one architecture cannot load one built for another.** An
`ELFCLASS64` / `ELFCLASS32` error means a loader path is reaching the wrong build — check the
library search path, not the code.
- When a build must match an already-installed runtime's version, **pin it explicitly** and
note the constraint. A silently mismatched pair fails at load time with a message that
points nowhere useful.
- Record the exact toolchain and container used for a cross build. "It built on my machine" is
not reproducible for native code.
---
## 7. FFI Boundaries
When exposing C/C++ to another language:
- `extern "C"` on the boundary, with a hand-written, stable, documented header.
- **Ownership across the boundary must be explicit** — who allocates, who frees, and with which
function. Provide a matching `free` for every allocation you hand out.
- **Never let an exception cross an `extern "C"` boundary.** Catch everything at the edge and
convert to an error code.
- Bindings generated by a tool (`bindgen`, etc.) are regenerated, never hand-edited.
- On the consuming side, keep all `unsafe`/unmanaged interop in **one thin wrapper module**
that exposes a safe API — never scattered through call sites.
> [!WARNING]
> **Binding crates/packages that reach into a library's struct fields break when that library
> makes the struct opaque.** Pin to a binding version that matches the installed library
> generation, and note the constraint where someone will read it before bumping.
---
## 8. Testing & Debugging
- GoogleTest or Catch2; CTest integration so `ctest` runs the suite.
- **Regression test every fixed bug.**
- Debug builds keep frame pointers (`-fno-omit-frame-pointer`) so stack traces are usable.
- `-g` even in release builds; ship separate debug symbols rather than none.
- **Record dead ends in the project's notes**, especially for native debugging — the
investigation that ruled something out is expensive and will otherwise be repeated. Note
what did *not* work alongside what did.
- When a binary search through a packed/embedded blob finds nothing, that is not evidence the
content is absent — packed asset formats defeat naive string searches. Re-check without
anchoring before concluding.
+150
View File
@@ -0,0 +1,150 @@
---
name: csharp-dotnet
description: >-
Use this skill when writing, building, or releasing C# / .NET code — including nullable
reference types, warnings-as-errors, analyzer configuration, async/await rules, trimming
and single-file publishing, and multi-generation SDK compatibility shims.
---
# C# / .NET — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds .NET specifics.
---
## 1. Project Settings — Strict By Default
```xml
<PropertyGroup>
<Nullable>enable</Nullable>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<EnforceCodeStyleInBuild>true</EnforceCodeStyleInBuild>
<AnalysisLevel>latest-all</AnalysisLevel>
<ImplicitUsings>enable</ImplicitUsings>
<LangVersion>latest</LangVersion>
</PropertyGroup>
```
Put these in a `Directory.Build.props` at the repository root so every project inherits them
and none can quietly drift.
**Every build carries `-warnaserror`:**
```bash
dotnet build MySolution.sln -warnaserror
```
- **Nullable warnings are real bugs.** Never silence one with `!` (null-forgiving) — narrow
the type or fix the contract. `!` is an assertion, and an assertion you cannot prove is a
future `NullReferenceException`.
- Never blanket-disable an analyzer. When a rule is genuinely wrong for a piece of code,
scope the suppression to it — `#pragma warning disable CA1822` around the smallest region
that needs it (with a matching `restore`), or a targeted `[SuppressMessage]` attribute —
and always with a `Justification` / comment saying why. An `.editorconfig` entry turns the
rule off for every file it covers; use it only when the rule is wrong codebase-wide.
---
## 2. Async
- **`async`/`await` throughout; never `.Result` or `.Wait()`** — both deadlock in any context
with a synchronization context, and turn an exception into an `AggregateException`.
- `async Task`, never `async void`, except for genuine event handlers.
- `ConfigureAwait(false)` in library code; not needed in app code on modern hosts.
- Accept a `CancellationToken` on every async public method and actually honor it.
- `IAsyncEnumerable<T>` for streaming rather than materializing a whole collection.
- **Events over polling** — subscribe to the framework's event/observable API; don't run a
timer loop. (Global rule; it applies fully here.)
---
## 3. Errors, Disposal & Logging
- Specific exception types; never `catch (Exception)` without a reason and a log.
- **Never an empty catch block.**
- `using`/`await using` declarations for everything `IDisposable`/`IAsyncDisposable`.
Disposal is not optional for streams, HTTP handlers, DB contexts, or native handles.
- **One static `HttpClient` (or `IHttpClientFactory`)** — a per-request `HttpClient` exhausts
sockets under load.
- Structured logging (Serilog or `ILogger<T>`) with message templates, not string interpolation:
`logger.LogInformation("Loaded {Count} items", count)` — the template is what makes the log
queryable.
- **Never log secrets.**
---
## 4. Style
- File-scoped namespaces, `var` for obvious types, target-typed `new()`.
- `record` for immutable value types; `readonly struct` for small value types.
- Pattern matching and switch expressions over `if`/`else` ladders.
- `IReadOnlyList<T>` / `IReadOnlyDictionary<K,V>` in public signatures returning collections.
- `internal` by default; `public` only for genuine API surface.
- File and function size limits from the general rules apply — put the file limit in a test
rather than an MSBuild target (see [`coding-dev.md` §4](skills-public/coding-dev.md)).
- `dotnet format` before committing.
---
## 5. Publishing — Trimming & Single File
> [!IMPORTANT]
> **The trimmer only runs on a Release/trimmed/single-file publish.** A Debug build cannot
> reproduce the trimmer dropping a type that only reflection reaches. If your release pipeline
> ships trimmed binaries, the publish step is the **only** one that exercises the trimmer —
> so it must run before release, not after.
- Annotate reflection-reached types with `[DynamicallyAccessedMembers]` or a
`TrimmerRootDescriptor`, and **smoke-test the published binary**, not just the build.
- Enable `<IsTrimmable>` per library deliberately; blanket trimming of a plugin host breaks
plugin loading.
- **A release is a fixed, complete set of assets.** Define what that set is, and have the
release step verify it is complete rather than trusting whatever happens to be in the output
directory. A packaging tool that silently skips a step when a dependency is missing will
otherwise ship a release with a missing platform.
---
## 6. Multi-Generation SDK Compatibility
When targeting several generations of a host SDK whose API changed shape (a class becoming an
interface, a type being renamed):
- **All compatibility code lives in dedicated shim files** — e.g. `GlobalUsings.cs` and
`<Host>SdkShims.cs`. Never scatter `#if` directives through feature code.
- Select the target with an MSBuild property so the build is explicit about which generation
it produced:
```bash
dotnet build MyModules.csproj -c Release -p:HostTarget=stable
dotnet build MyModules.csproj -c Release -p:HostTarget=beta
```
- Document the differences between generations in the project's agent/contributor file —
which types changed, and which direction each shim adapts.
- Keep a local package feed for SDK versions not published to the public registry, and script
the packing step rather than doing it by hand.
---
## 7. Testing
- xUnit (or the project's existing framework); `FluentAssertions` for readable assertions.
- **Regression test every fixed bug.**
- Inject abstractions for filesystem, clock, and network so they can be faked — do not reach
for a static.
- **Never test mutations against real user profiles, real config directories, or live
services.** Fixtures and injected backends only.
- Test the **published** artifact for anything involving trimming, single-file packing, or
native dependencies — the assembly that passed unit tests is not the one you shipped.
---
## 8. Toolchain Notes
- When the SDK is not installed on the host (immutable OS, mismatched glibc), build inside the
project's designated container. Entering one resets the working directory — chain `cd` in the
same command string.
- Prefer the project's build script over hand-typed `dotnet` calls; it carries `-warnaserror`,
the correct action ordering, and the asset completeness checks — see
[`bash.md`](skills-public/code-languages/bash.md) for how that script should be structured.
- **Never stop, deploy to, or restart an application while the user has it running**, unless
they have said to.
+153
View File
@@ -0,0 +1,153 @@
---
name: go
description: >-
Use this skill when writing, building, linting, testing, or releasing Go — including
golangci-lint configuration, strict typing over map[string]interface{}, official SDK
preference, CGO-free static builds, context handling, and the build gate.
---
# Go — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Go specifics.
---
## 1. The Gate
```bash
gofmt -l . # must print nothing
go vet ./...
golangci-lint run
go test -race ./...
go build ./...
```
Wire this into a single entry point (`make check`, `scripts/build.sh`) so it cannot be
partially run. **A skipped test fails the gate.** Add `govulncheck` for binary vulnerability
scanning on anything that ships.
---
## 2. Linting — Strict Set
`golangci-lint` with a deliberately broad selection, not the defaults:
```yaml
linters:
enable:
- staticcheck
- errcheck # every returned error is handled
- gosec # security findings
- noctx # no HTTP request without a context
- revive
- misspell
- unconvert
- bodyclose # every response body is closed
- goconst
- ineffassign
- nilerr
```
`errcheck` and `bodyclose` catch the two most common real Go bugs. As always: fix rather than
suppress, and when suppressing use a specific `//nolint:name // reason` with the reason.
---
## 3. Typing — Concrete, Not Dynamic
- **Strongly typed structs everywhere.** Avoid `map[string]interface{}` and reflection when a
struct can be defined — including for JSON, config, and API payloads.
- **Typed constants/enums**, not bare strings:
```go
type SyncStatus string
const (
StatusSynced SyncStatus = "synced"
StatusOutdated SyncStatus = "outdated"
StatusDirty SyncStatus = "dirty"
)
```
- **Concrete error types** with `errors.Is` / `errors.As`, not string comparison.
- Keep types shared with a frontend structurally in sync with that frontend's types; when both
exist, a change to one is incomplete without the other.
---
## 4. Errors
- **Every error is handled.** `errcheck` enforces it; do not `_ =` an error away without a
comment explaining why it cannot matter.
- Wrap with context: `fmt.Errorf("fetching %s: %w", name, err)`. Always `%w`, never `%v`, when
the caller might need to inspect it.
- Sentinel errors as package-level `var ErrNotFound = errors.New(...)`; typed errors when the
caller needs data from them.
- **`panic` only for genuinely unrecoverable programmer error**, never for expected runtime
conditions. A library never panics on input it was designed to receive.
- `defer` the cleanup immediately after acquiring the resource, always.
---
## 5. Context
- **Every function that does I/O takes `ctx context.Context` as its first parameter.**
- Never store a context in a struct field.
- Never pass `nil` — `context.TODO()` marks a genuine gap.
- Honor cancellation: select on `ctx.Done()` in loops, and pass ctx down to every HTTP call
(`noctx` enforces this).
- Set timeouts on outbound requests. A default `http.Client` has no timeout at all.
---
## 6. Dependencies — Official SDKs First
Follow the global reuse-first ordering. In Go, strongly prefer **official/first-party SDKs**
over third-party wrappers — they track API changes and handle auth and pagination correctly.
Choose **pure-Go implementations over CGO ones** whenever the trade is acceptable: a CGO
dependency turns a static single-binary build into a toolchain and glibc problem, and rules
out a `FROM scratch`/Alpine container image.
```bash
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o bin/app ./cmd/app
```
`go mod tidy` before committing, and commit `go.sum`.
---
## 7. Concurrency
- **Share memory by communicating.** Channels for handoff; `sync.Mutex` for guarding a struct's
own fields. Don't mix both for the same state.
- **`go test -race` is not optional** — it belongs in the gate, not in an occasional manual run.
- Every goroutine has a defined exit path. `errgroup.Group` for a set of related goroutines so
errors propagate and cancellation fans out.
- **Events over polling** — subscribe to a watch/notify API rather than looping on a ticker.
Where a ticker is genuinely necessary, make the interval configurable and document why.
- Never leak a goroutine on a channel nobody reads.
---
## 8. Structure
- `cmd/<binary>/` for entry points; `pkg/` or `internal/` for the actual code.
- `internal/` for anything that must not become public API — it is compiler-enforced.
- One package per concern; **package names are lowercase, single-word, no underscores**, and
never `util` or `common`.
- Keep the exported surface small. Unexported by default.
- Embed frontend/static assets with `//go:embed` for single-binary distribution — the frontend
itself follows [`js-typescript.md`](skills-public/code-languages/js-typescript.md).
- File and function size limits from the general rules apply — `funlen` in `golangci-lint`
covers functions; put the file limit in a test (see [`coding-dev.md` §4](skills-public/coding-dev.md)).
---
## 9. Testing
- Table-driven tests are the default idiom.
- `t.Parallel()` where tests are genuinely independent.
- `t.TempDir()` for filesystem work — it cleans itself up.
- `httptest` for HTTP handlers and for faking upstreams; do not hit real services in unit tests.
- **Regression test every fixed bug.**
- **Never run destructive operations against real accounts or user data during validation** —
temporary fixtures only.
+262
View File
@@ -0,0 +1,262 @@
---
name: gsc
description: >-
Use this skill when writing, reviewing, or debugging GSC (Game Script / Call of Duty script)
or CSC — including entity-vs-level scope, threads and notify/wait semantics, endon/waittill
discipline, undefined-value guards, the no-OOP/no-exceptions reality, asset precaching,
per-frame cost, block scoping, and compiling off-engine with gsc-tool.
---
# GSC — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds GSC specifics.
GSC is the Call of Duty engine's own scripting language (`.gsc` server-side, `.csc`
client-side). It is **not Lua, not JavaScript, and not C** despite resembling all three. It has
no type system, no exceptions, no classes, no closures in most dialects, and no test harness.
It can be compiled off-engine (see §1), which catches syntax and scope errors — but nothing
checks a type, a field that was never set, or a builtin that does not exist in the target title.
So the code still has to be written so that it cannot be wrong.
> [!IMPORTANT]
> **Establish the dialect before writing anything.** IW3/IW4 (MW/MW2), IW5 (MW3), T5/T6 (Black
> Ops 1/2), IW6, s1/s2, H1/H2 (MWR / MW2CR), and the various mod loaders (h2-mod, iw4x, plutonium,
> t6 GSC Studio) each differ in available builtins, in how scripts get loaded, and in whether
> `foreach`, `switch` fallthrough, or function pointers exist. A builtin from the wrong title is
> a silent `undefined` or a hard crash.
---
## 1. The Gate — `gsc-tool`, Then The Game
**There is an off-engine GSC compiler: `gsc-tool`.** Reach for it first; it is
easy to assume none exists and verify by loading a map instead.
```bash
gsc-tool --mode comp --game <title> --system pc --dry path/to/script.gsc
```
It exits non-zero and prints the engine's own diagnostic, e.g.
`[ERROR]:compiler:script:97:25: local variable 'previous_count' not found`.
- **Use `--mode comp`, not `--mode parse`.** `parse` is syntax-only and happily
accepts code the engine's compiler rejects, including the block-scope error
above. `comp` with `--dry` writes nothing and does the real thing.
- **`--game` must match the title**, and it is not the marketing name: MW2
Campaign Remastered is `h2`, MWR is `h1`. The choices are `iw5 iw6 iw7 iw8
iw9 s1 s2 s4 h1 h2 t6 t7 t8 t9 jup`. Wrong title, wrong builtin table.
- Prebuilt binaries ship on its GitHub releases; there is no distro package.
- Many mod trees already vendor it (Overlord has it under `deps/gsc-tool`).
What that still leaves:
| Check | How |
| :--- | :--- |
| **Syntax + semantics** | `gsc-tool --mode comp`. Put it in the build gate. |
| **Builtin exists for this title** | `gsc-tool`'s own engine tables, e.g. `src/gsc/engine/h2_meth.cpp`. Grep before using a builtin you remember from another title. |
| **Runtime errors** | In-game console / `developer 1`. Many mod loaders print a script stack trace. |
| **Tests** | None off-host. Behaviour is verified by playing the map. |
So a compile is cheap and mandatory, and **behaviour is still unverified until
the map runs.** Both halves matter:
- **`developer 1` (or the loader's equivalent) is mandatory during development.** Without it the
engine suppresses script warnings and an undefined-value error looks like "the feature did
nothing".
- **Load the script and read the console on every change**, before judging behaviour. "Nothing
happened" is far more often a parse error at load than a logic bug.
- **Instrument with `iprintln` / `iprintlnbold` / `println`** on the success *and* the failure
path. A function that silently returns on a guard is indistinguishable from one that never ran.
- **Say plainly in your report that GSC behaviour is unverified** unless you actually loaded the
map and watched it. A clean `gsc-tool` compile proves the script loads, not that it works.
---
## 2. Scope — `self`, `level`, `game`, and Locals
| Scope | Lifetime | Use for |
| :--- | :--- | :--- |
| local variable | the function call | everything it can hold |
| `self` | the entity | per-entity state (a player's, an actor's) |
| `level` | the current map/level | per-round state, map config, shared handles |
| `game` | across map restarts | persistent state only |
- **Prefix your own fields.** `level.mymod_spawn_queue`, not `level.queue`. `level` and `self`
are one flat namespace shared with the stock game scripts and every other mod; a collision
overwrites engine state and the symptom appears somewhere unrelated.
- **Never write to a stock field you did not create** unless that is the documented integration
point. Read it, copy it, leave it.
- **A function called *on* an entity gets it as `self`**: `player giveweapon("m4")`. Calling the
same function without a prefix leaves `self` undefined and the first `self.x` access errors.
- `self` is not inherited into a `thread`ed call's children automatically in the way callers
expect — `ent thread f()` gives `f` `self == ent`, but a plain `f()` inside it does not.
---
## 3. Undefined Is The Default — Guard Everything External
GSC has no types and no nullability. **Reading a field that was never set yields `undefined`,
and using it in arithmetic or a method call is a runtime error that aborts the thread** — often
silently from the player's point of view.
```gsc
// WRONG: any of these can be undefined
ent = level.target;
ent setorigin(level.player.origin + (0, 0, 64));
// RIGHT
if (!isdefined(level.target) || !isdefined(level.player))
return;
```
- **`isdefined()` on every entity, every `level.*`/`self.*` read you did not write in the same
function, and every return value from a lookup.**
- **Entities die.** A handle held across a `wait` may be a dead entity by the time the thread
resumes. Re-check `isdefined(ent)` *after* every `wait`, not only before.
- `isalive(ent)` is the check for an actor you intend to act on; `isdefined` only proves the
handle exists.
- This is the one place where the general "don't over-guard" rule inverts: in GSC a value from
outside the current function genuinely can be undefined, so the check is not dead code.
- Arrays: `array.size` is safe on an empty array but errors on an undefined one. Guard the array,
then the index.
---
## 4. Threads, `notify`, `waittill`, `endon`
```gsc
monitor_thing()
{
level endon("game_ended");
self endon("death");
for (;;)
{
self waittill("trigger_activated");
do_work();
}
}
```
- **`thread` starts a coroutine and returns immediately.** It does not return the function's
value — a `thread`ed function's `return` value is unreachable. Pass a handle or notify.
- **Every long-lived thread needs an `endon`.** Without one, a loop keeps running after the
entity dies or the level ends, and then errors on a dead `self` — or leaks, one copy per spawn.
`level endon("game_ended")` plus the relevant entity `endon` is the baseline.
- **`endon` must be registered before the first `wait`**, at the top of the function. Registering
it later leaves a window.
- **`waittill` is the event primitive — prefer it to polling.** `self waittill("death")` is
correct; `while (isalive(self)) wait 0.05;` is the same thing, worse, and costs every frame
forever. This is the general events-over-polling rule, and GSC gives you the event.
- **A `notify` with no waiter is lost.** There is no queue. If a thread might not be waiting yet,
set a flag as well as notifying, and have the waiter check the flag first.
- `waittill("x", a, b)` receives the notify's extra parameters. The names are positional.
- **`waittillframeend`** is the tool for "after everything else this frame" — use it instead of
`wait 0.05` when the intent is ordering, not delay.
- `wait 0` is not zero; it is one frame. `wait 0.05` is one server frame at 20Hz — write the
intent, not a magic number, and name the constant.
---
## 5. Per-Frame Cost Is Your Budget
The script VM shares the server frame with everything else, and there is a **hard instruction
budget per frame**. Exceeding it produces `script runtime error: exceeded maximum number of
script variables` or a visible server hitch.
- **One polling loop per concern, never one per entity.** A `for(;;) { wait 0.05 }` loop on each
of 60 actors is 60 wakeups a frame. Keep one manager loop over an array.
- **Where an event exists, the polling loop should not.** See §4.
- **Never loop without a `wait` inside.** A `for(;;)` with no `wait` hangs the server frame and
takes the match down — this is the single most common fatal GSC mistake.
- Keep the body of a per-frame loop trivial. Move anything expensive behind a condition or a
longer interval, and say in a comment why that interval.
- Spawning is expensive and entity counts are capped. Reuse and pool; delete what you spawned.
---
## 6. Assets Must Be Precached
```gsc
init()
{
precachemodel("com_laptop_open");
precacheshader("hud_icon_frag");
precachestring(&"MYMOD_HINT");
}
```
- **Precaching only works during the level's load phase.** A `precachemodel` after the map is
live either errors or silently fails, and the model then renders as a missing-asset
placeholder or nothing at all.
- Put every precache in the init path, unconditionally — not behind the feature flag that decides
whether the asset gets used.
- **A model or weapon name not present in the loaded level's zone will not spawn**, regardless of
precaching. Asset availability is per-map; a name verified on one map proves nothing about
another. State which map you verified on.
- Localized strings use the `&"REFERENCE"` form and must exist in the string table; a missing one
displays the raw reference.
---
## 7. Language Facts That Surprise People
- **No exceptions, no `try`.** An error aborts the current thread and nothing else. There is no
recovery and no handler — which is why §3's guards are the error handling.
- **No classes, no inheritance, no `new`.** "Objects" are entities with fields, or plain arrays
used as structs. `spawnstruct()` is the idiom for a bag of fields with no world presence.
- **Function pointers**: `::funcname` (and `[[ptr]]()` to call) exist in most dialects but not
all. Verify before relying on them.
- **No closures** in most dialects — an inner function cannot capture a local. Pass state
explicitly or hang it on a struct.
- `foreach (x in arr)` exists in IW4+ and T6 but not in the earliest dialects; `for (i = 0;
i < arr.size; i++)` always works.
- **Arrays are associative.** `array[]` with integer keys behaves like a list, but deleting an
element leaves a hole and `.size` counts keys. Rebuild rather than delete-in-place.
- String comparison is `==`; there is no `switch` on anything but constants, and fallthrough
rules differ by dialect.
- Vectors are `(x, y, z)` literals with `+`, `-`, and scalar `*`. Component access is `v[0]`,
`v[1]`, `v[2]`. `anglestoforward` / `vectornormalize` / `distance` are builtins — do not
hand-roll vector maths.
---
## 8. Style & Structure
- **Allman braces, 4 spaces** — the stock scripts' convention; match it, because GSC is almost
always read next to engine scripts.
- `main()` is the level entry point; `init()` is the convention for a module's setup called from
it. Which one the loader calls is dialect- and loader-specific — verify, do not assume.
- `#include maps\_utility;` (dialect-dependent path syntax) at the top; prefer the stock utility
functions over reimplementing them. `maps\_utility` and `common_scripts\utility` already have
flag, wait, and array helpers.
- **`/* */` comments do not nest.** Commenting out a block that contains one breaks the file.
- **A local declared inside a block is not visible after it.** Writing it in one `if` and reading
it in a second does not compile — declare it unconditionally at function scope. Assigning
`undefined` to an entity field unsets it, which is usually the right "restore nothing".
- **No file-scope constants.** `MY_CONST = 0.2;` outside a function is not valid in most
dialects; use a local at the point of use or a `level.` field set during init.
- File and function size limits from the general rules apply, and matter more here than usual:
there is no debugger worth the name, so a 300-line function is unanalysable.
- **Delete dead code.** With no linter to flag it, commented-out GSC accumulates forever.
---
## 9. Side Effects on Level State
GSC runs inside someone else's running game. A change to shared state is not scoped to your
feature and is not undone when your thread ends.
- **Mutating a stock spawner, trigger, or flag changes the mission.** Setting
`spawner.count = 999` to get an extra AI makes that spawner infinite for the rest of the level.
Save the old value and restore it, or clone rather than mutate.
- **Anything you spawn, you delete.** Track handles in your own `level.mymod_*` array and provide
a cleanup path; an orphaned entity survives until map change and counts against the cap.
- **`setdvar` is global and persistent.** A dvar used as a command channel must be cleared
immediately on read, and the clear must happen even on the failure path — otherwise the next
poll re-fires the same command.
- A dvar as an IPC channel is a **poll**, which §5 says to avoid; it is acceptable only because
GSC cannot receive a host event, and the interval and the clear-on-read should both be
explicit and commented.
- Never leave the player in a modified state (god mode, `notarget`, a changed movement speed)
when the feature that set it is turned off. Toggles need an off path that actually restores.
+199
View File
@@ -0,0 +1,199 @@
---
name: js-typescript
description: >-
Use this skill when writing, building, linting, testing, or releasing JavaScript or
TypeScript — including Node, Bun, Deno, React, Next.js, and Astro projects. Covers the
TypeScript-over-JSDoc-over-JS preference, strict compiler settings, banned constructs,
React/hooks rules, resource lifecycle, and package/tooling choices.
---
# TypeScript / JavaScript — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds JS/TS specifics.
---
## 1. Language Choice — A Strict Preference Order
1. **TypeScript** — always the default for new code.
2. **JavaScript with JSDoc type annotations** — acceptable when a project genuinely cannot
adopt a build step, and `checkJs` is enabled so the annotations are actually enforced.
3. **Plain untyped JavaScript** — last resort only.
Migrating an existing JS project to TS is worth proposing; do not do it unasked (scope rule).
In a plain-JS codebase, **match it** — do not introduce a lone `.ts` file and a build step
into a project that has neither.
---
## 2. Strict Compiler Settings
```jsonc
{
"compilerOptions": {
"strict": true,
"noUncheckedIndexedAccess": true,
"noImplicitOverride": true,
"noFallthroughCasesInSwitch": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"exactOptionalPropertyTypes": true,
"isolatedModules": true
}
}
```
**`tsc --noEmit` must pass with zero errors *and* zero warnings**, and it belongs in the build
gate alongside lint and tests — not as a thing someone runs occasionally in their editor.
> [!NOTE]
> Some established codebases deliberately relax `noImplicitAny` or `noUncheckedIndexedAccess`
> because their idioms depend on it. When **contributing upstream**, respect the project's
> `tsconfig.json` as-is. In **our own repos and non-upstreamed forks**, the settings above are
> the target — tightening them is fair game (as its own change, not smuggled into a feature).
---
## 3. Banned Constructs
| Never | Use instead |
| :--- | :--- |
| `any` | `unknown` + narrowing, or a real generic |
| `as unknown as T` | a type guard, or fix the source type |
| `@ts-ignore` | `@ts-expect-error` with a one-line reason, only when unavoidable |
| `!` non-null assertion | narrow with `if` or an early return |
| `enum` / `const enum` | `as const` object + literal union |
| `namespace` | ES modules |
| Boxed primitives (`Number`, `String`, `Object`, `{}`, `Function`) | the primitive types |
| `../../..` relative climbs | the configured path alias (`@/*`) |
| `JSON.parse(JSON.stringify(x))` | `structuredClone` or `lodash.cloneDeep` |
| `await new Promise(r => setTimeout(r, ms))` | a `sleep` helper from your utils |
| `console.log` in shipped code | a real logger with levels |
| Empty `catch {}` | handle it, or let it propagate |
- **`satisfies T` over `as T`** when the value already conforms. `as` is for genuine narrowing
only — it is an assertion that silences the checker, not a conversion.
- **Zero `any` in shared model types.** Frontend types that mirror a backend schema must stay
structurally in sync with it and fully typed.
---
## 4. Use What Exists
**Search the codebase's own `utils/`, `common/`, and shared modules before writing anything.**
Re-implementing an export that already exists is the single most common avoidable diff.
Never hand-roll: debounce/throttle (`lodash`), deep clone, clamp, date formatting
(`date-fns` / `dayjs` / project standard), URL parsing, regex escaping, UUID generation,
retry/backoff, or a semver comparator.
In a UI framework or client mod, **never recreate a component the host already provides** —
a hand-rolled `<button>` loses theming, focus rings, and accessibility.
---
## 5. React
- **Conditional rendering returns `null`**, never `undefined` or `false`.
- **List keys are stable identifiers.** Never `key={index}` on a mutable list.
- **Don't mirror external state into `useState`.** Subscribe to the store with its own hook.
- No `useMemo` / `useCallback` on cheap values that do not feed a dependency array. They are
not free, and they obscure what is actually expensive.
- **Hoist constant dependency arrays** to module scope (`const KEYS = [...] as const`). A fresh
array literal in the render body re-subscribes on every render.
- Prefer `<>…</>` fragments; `React.Fragment` only when the fragment needs a `key`.
- Type props with a named `interface`, not `React.FC`.
- Wrap anything injected into a host application's tree in an error boundary — an
uncaught throw in injected code takes down the host's render, not just yours.
- Components used as dynamic render targets must live at **module scope**. Defining a
component inside another component remounts its whole subtree every render.
---
## 6. Resource Lifecycle — Everything Started Must Be Stopped
Every `setInterval`, long `setTimeout`, `addEventListener`, subscription, `ResizeObserver`,
`IntersectionObserver`, `AbortController`, and `WebSocket` created in a setup path must be
torn down in the matching teardown path — `stop()`, `useEffect` cleanup, or `finally`.
> [!CAUTION]
> **`addEventListener` needs a named function reference.** An inline arrow passed to
> `addEventListener` cannot be matched by `removeEventListener`, so the listener leaks
> silently and accumulates across every mount.
```ts
let intervalId: ReturnType<typeof setInterval> | undefined;
function start() { intervalId = setInterval(tick, 30_000); }
function stop() { if (intervalId !== undefined) { clearInterval(intervalId); intervalId = undefined; } }
```
Attach an `AbortController` to every long-lived `fetch` and abort it in cleanup.
---
## 7. Structure & Naming
- **Components**: PascalCase `.tsx`.
- **Modules / utilities**: kebab-case `.ts`.
- **Imports**: prefer the configured alias (`@/…`) over relative climbs.
- Indentation and quote/semicolon style: **match the file you are editing**. Do not introduce
a new formatter or lint config into a project that already has one.
- **One file until it earns a split** — when it nears the 1000-line limit, or when a second
file is genuinely reusable (`utils.ts`, `types.ts`, `components/`).
The limits from the general rules (1000 lines per file, 200 per function) apply: ESLint's `max-lines` and
`max-lines-per-function` cover both natively, so prefer those over a separate test here.
---
## 8. Async & Data
- `async`/`await` over raw `.then()` chains. No mixing both in one flow.
- **Never leave a promise unawaited** unless you explicitly mean to, and mark it (`void p`)
when you do.
- `Promise.all` for independent work; sequential `await` only when there is a real dependency.
- **Subscribe to events; never poll.** (Global rule — it applies with full force here.)
- **Never store secrets in `localStorage` / `sessionStorage`.** Use the platform's real
storage API, and namespace your keys.
---
## 9. Node / Runtime Boundaries
- In any sandboxed or privileged-split architecture (Electron, browser extensions, plugin
hosts), the untrusted side must **never** import `fs`, `path`, `child_process`, `net`,
`http`, or `electron` directly. Route through the designated IPC/native boundary.
- Every IPC handler argument is **untrusted** — validate type, length, and content.
- Filesystem paths from the other side: `normalize()` then assert `startsWith(<allowed root>)`
before any read or write.
- Subprocesses: `spawn` with an argv **array**. Never `exec` a constructed string, never
`shell: true` with untrusted input.
- Network: allowlist by origin, cap response size, cap redirect depth.
- Errors crossing the boundary must be scrubbed — a raw `Error` leaks absolute paths.
---
## 10. Tooling & Tests
- Respect the project's runtime and package manager (`npm` / `pnpm` / `yarn` / `bun`) and its
lockfile. Do not switch package managers or add a second lockfile. Build and release scripts
around it follow [`bash.md`](skills-public/code-languages/bash.md).
- Test runner: whatever the project uses (`vitest`, `bun test`, `jest`). Tests live beside the
code as `*.test.ts(x)` unless the project says otherwise.
- **Add unit tests for new logic and route tests for new handlers.** Prioritize coverage on
auth, persistence, and anything that talks to a third party.
- Lint with the project's config (`eslint.config.mjs` and friends). **Do not add a new lint or
format config** to a repo that already has one.
- **Know the project's expected test baseline.** Some inherited suites are not all-green on a
clean checkout; judge regressions against the documented baseline rather than a raw run —
and if a baseline file exists, use it.
---
## 11. Framework Versions Move Fast
> [!WARNING]
> Next.js, Astro, React, and the major bundlers ship breaking changes regularly, and their
> current APIs frequently differ from what is in a model's training data. **Read the version's
> own docs in `node_modules/<pkg>/dist/docs/` or the project's pinned documentation before
> writing code against them.** Heed deprecation notices rather than pattern-matching from memory.
+232
View File
@@ -0,0 +1,232 @@
---
name: lua
description: >-
Use this skill when writing, reviewing, or debugging Lua — including Lua 5.1/5.2/5.3+ and
LuaJIT dialect differences, HavokScript (HKS) as used by Call of Duty's LUI, luacheck
configuration, locals over globals, pcall error handling, the integer-vs-float and
table-length pitfalls, module layout, and embedded/sandboxed-host constraints.
---
# Lua — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Lua specifics.
Lua is almost always **embedded**: the host decides the dialect, the available standard
library, the module loader, and whether an uncaught error takes the process down. **Establish
which host and which dialect you are writing for before writing a line** — the same file is
correct in one and a crash in another.
---
## 1. Pin the Dialect First
| Dialect | Notable traits |
| :--- | :--- |
| **5.1** | `loadstring`, `unpack`, `setfenv`/`getfenv`, `module()`, no integer type, no goto |
| **LuaJIT** | 5.1 plus `bit`, `ffi`, `jit`; a few 5.2 extensions |
| **5.2** | `load` replaces `loadstring`, `_ENV` replaces `setfenv`, `goto`, `table.unpack` |
| **5.3** | distinct integer subtype, `//`, bitwise operators, `math.type` |
| **5.4** | `<const>` / `<close>` attributes, generational GC |
| **HavokScript (HKS)** | a 5.1 fork; see §8 |
- **Determine it from the host, not from habit.** `print(_VERSION)` in the target runtime, or
read the host's embedding code. A `goto` in a 5.1 host is a parse error, and a `loadstring`
in 5.4 is a nil call.
- **Write for the host's dialect; do not write portable-across-all-five code** unless the
module genuinely ships to several. Compatibility shims
(`local unpack = table.unpack or unpack`) are worth it only where that is true.
- Lua 5.1 has **no integer type** — every number is a double. `1/2` is `0.5`, and
`tostring(3.0)` is `"3"` in 5.1 but `"3.0"` in 5.3. Never round-trip a number through a
string and expect the same text across dialects.
---
## 2. The Gate
```bash
luacheck . # lint; must be clean
stylua --check . # or lua-format; formatting
luac -p $(git ls-files '*.lua') # parse check against the TARGET dialect's luac
busted # tests, where the code is testable off-host
```
- **`luac -p` is the cheapest real check available** and the one most often skipped. It catches
syntax errors, and when run with the host's own Lua version it catches dialect mismatches
before the host does. For an embedded host whose scripts cannot be unit tested, this plus
`luacheck` *is* the gate — wire it into the build script and never commit a file that fails it.
- `luacheck` needs its globals declared or it reports the host's API as undefined. Declare them
rather than disabling the check:
```lua
-- .luacheckrc
std = "lua51"
read_globals = { "Engine", "LUI", "GenericButtonSettings" }
globals = { "MyModuleState" } -- only what this code deliberately writes
max_line_length = 120
```
- `read_globals` for the host's API, `globals` only for names you intentionally define. If the
list of writable globals is growing, that is the finding — see §3.
---
## 3. Locals, Not Globals
**Every variable is `local` unless it must be visible to the host.** An unqualified assignment
creates a global, silently, with no warning from the language.
```lua
-- WRONG: typo creates a new global; the real one keeps its old value
local function update() cofnig.volume = 1 end
-- RIGHT
local config = require("config")
local function update() config.volume = 1 end
```
- Locals are also **faster** — they are register slots, while a global is a hash lookup in
`_ENV` on every access. Hoist hot library functions into locals at the top of a module:
`local floor, insert = math.floor, table.insert`.
- **Namespace what must be global.** One table (`MyMod.state`), never a scatter of
`myModFoo` / `myModBar`. In a shared host every global is in one namespace with every other
script's.
- `luacheck` flags accidental globals. Keep it clean and the typo class disappears.
- The 200-local-per-function limit is real; hitting it means the function needed splitting long ago.
---
## 4. Errors — `pcall` at the Boundary, Not Everywhere
- **`error()` for real failures; return `nil, message` for expected ones.** A file that may not
exist returns `nil, err`; a caller passing the wrong type gets an `error`.
- **`pcall` / `xpcall` at the boundary only** — a callback the host invokes, a plugin entry
point, a request handler. Wrapping every internal call turns a stack trace into a string and
hides where the fault was.
- **`xpcall(f, debug.traceback)` when you need the trace.** Plain `pcall` discards it, and the
message alone rarely identifies the line in a long callback chain.
- **Never swallow it.** `pcall(f)` with the result ignored is an empty catch block:
```lua
-- WRONG
pcall(risky)
-- RIGHT
local ok, err = pcall(risky)
if not ok then log.warn("risky failed: " .. tostring(err)) end
```
- `error("msg", 2)` blames the caller instead of the raising line — the right level for an
argument-validation error.
- `error()` can be given a **table**, which is how you carry structured error data; the
`tostring` of it is then meaningless, so handlers must check `type(err)`.
---
## 5. Tables — The Traps
- **`#t` is only defined for a sequence** (keys `1..n` with no holes). With a hole it may return
either boundary, and the choice can change between runs. Track the count yourself when a
table can be sparse, and never use `#t` on a map.
- **`table.insert` / `table.remove` shift every later element** — O(n). Appending with
`t[#t+1] = v` is cheaper than `table.insert(t, v)`; building a new table beats removing from
the middle in a loop.
- **Never mutate a table while iterating it with `pairs`.** Assigning to an existing key is
defined; adding a new one is not. Collect keys first, then act.
- **`pairs` order is unspecified** and varies run to run. Anything user-visible or compared
between runs must sort its keys first.
- `ipairs` stops at the first `nil`. Use it for sequences and `pairs` for maps — the choice
documents which the table is.
- `nil` cannot be stored in a table; `t[k] = nil` is a delete, and a sequence containing `nil`
is not a sequence.
- Beware `0` and `""` being **truthy**. Only `nil` and `false` are false, so
`if count then` is true at zero — test `if count ~= 0 then`.
- `1` and `1.0` are the **same key** in 5.3+ (the float is normalized), but a non-integral
float is its own key.
---
## 6. Strings
- **Never build a string in a loop with `..`.** Each concatenation allocates; collect into a
table and `table.concat` once.
- **Lua patterns are not regex.** `%` is the escape (not `\`), there is no alternation and no
`+`/`?` on groups, character classes are `%a %d %s %w %p %c %x` and their uppercase
complements. Reach for `lpeg` when the grammar is real.
- `string.format("%q", s)` produces a Lua-readable quoted literal — correct for serialization,
wrong for display.
- `%s` in `string.format` calls `tostring`, so a nil argument renders `"nil"` rather than
erroring — the opposite of `..`, which errors on nil. That asymmetry is a frequent crash
source in host callbacks: **prefer `string.format` over `..` for any message built from
values that could be nil**, or guard with `tostring()`.
- Indices are 1-based and inclusive on both ends: `s:sub(1, 3)` is the first three bytes.
- Strings are **byte** strings. `#s` and `s:sub` count bytes, not characters — use `utf8` (5.3+)
or a library for anything non-ASCII.
---
## 7. Modules & Structure
```lua
local M = {}
local function helper(x) -- private: a local, not M.helper
return x * 2
end
function M.public(x)
return helper(x)
end
return M
```
- **Return a local table. Never set globals, never use `module()`** (deprecated in 5.1,
removed in 5.2).
- `require` caches by module name — a module body runs once. Do not rely on re-`require` to
reset state; expose an explicit `M.reset()` if a reset is needed.
- `:` for methods that take `self`, `.` for everything else. Be consistent — a `.` call on a
`:` method passes the first argument as `self` and fails confusingly.
- File and function size limits from the general rules apply.
---
## 8. Embedded & Sandboxed Hosts
When the host owns the interpreter (a game engine, nginx, Redis, Neovim, a plugin API), its
constraints outrank every preference above.
- **Verify an API exists before calling it.** The host's surface differs between versions and
builds, and a missing function is a `nil` call that takes down whatever invoked you:
```lua
if type(Engine.SetDvarString) == "function" then
Engine.SetDvarString(name, value)
end
```
Prefer `type(x) == "function"` over a bare `if x then` — it also rejects a truthy non-callable.
- **An uncaught error in a host callback may crash the process, not just the script.** In a C
host the error unwinds through C frames that were not written to expect it. Treat every
callback body as a trust boundary and `pcall` it.
- **Do not monkey-patch host globals** (`print`, `require`, a host API function) outside a
tightly scoped save/restore — and even then, remember the restore does not happen if the body
errors, so do it in the `pcall` result path. Reentrancy makes the saved original a wrapper.
- **Load order is the host's, not yours.** Do not assume a sibling script has run. Read shared
state inside callbacks rather than at load time, and register your own state defensively.
- **Check what the standard library actually is.** Many hosts remove `io`, `os`, and `debug`, or
replace them with their own (`io.readfile` rather than `io.open`). Feature-test; do not assume.
- Logging is whatever the host provides. Use it rather than `print`, and never leave a
high-frequency print in a per-frame callback.
### HavokScript (HKS) — Call of Duty LUI
Call of Duty's UI (LUI) runs **HavokScript**, a proprietary Lua 5.1 fork:
- It is **5.1-based**: `loadstring`, `unpack`, `setfenv` — no `goto`, no integer division, no
bitwise operators, no `table.unpack`.
- There is **no `require` for siblings inside a UI script folder** in h2-mod-style loaders; a
module directory's `__init__.lua` is loaded on its own. Keep each module self-contained in one
file rather than splitting it and discovering the split does not load.
- There is **no off-host test harness**. `luac -p` with a 5.1 `luac`, `luacheck --std lua51`,
and reading the game's console output are the entire verification story — so lint and parse
checks are not optional here, they are the only gate that exists.
- **A dvar read is not type-safe.** `Engine.GetDvarString` on an unregistered dvar returns `nil`,
and `nil .. "text"` is an error that propagates out of the LUI event dispatcher as an access
violation. Register the dvar first, feature-test the accessor, and build messages with
`string.format`.
- **Hot-reload with `dofile` leaves the previous registration in place.** Guard re-entry with a
module flag, and expect duplicate menu buttons when you do not.
+402
View File
@@ -0,0 +1,402 @@
---
name: php
description: >-
Use this skill when writing, linting, analysing, testing, or releasing PHP — libraries,
Composer packages, generated SDKs, and plain PHP-FPM web applications behind nginx. Covers
strict_types, PHPStan at max, PER-CS 2.0 with risky rules, final readonly value objects,
backed enums, PSR interfaces over implementations, output escaping, and PHPUnit strictness.
---
# PHP — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds PHP specifics.
PHP here means one of two things, and they differ: a **library** published through Composer,
which depends on PSR interfaces and ships no implementation, and a **plain PHP-FPM application**
behind nginx with no framework, where the filesystem is the router. Both carry §1–§8; §9 is the
library half and §10 the application half.
---
## 1. The Gate
```bash
composer lint # php-cs-fixer fix --dry-run --diff
composer stan # phpstan analyse
composer test # phpunit
composer check # all three, in that order
```
Define those four as `scripts` in `composer.json` so the gate has one name. `composer check`
must be clean before anything is called done — zero fixer diffs, zero PHPStan errors, zero
test failures, and no warnings, notices or deprecations (see §7).
- Use `@php tools/x.php` rather than `php tools/x.php` in a composer script: it runs the
interpreter Composer itself is running under, which matters on a machine with several.
- `composer test` must never touch the network. Point `defaultTestSuite` at the unit suite so
a bare run is safe to fire blind.
- Name tool configs `.dist` — `phpstan.neon.dist`, `phpunit.xml.dist`, `.php-cs-fixer.dist.php` —
so a developer can drop a local override beside them without dirtying the repo. Add the
caches (`.php-cs-fixer.cache`, `.phpunit.cache`) to `.gitignore`.
- Lockfile by artifact type: an **application commits `composer.lock`**, a **library ignores
it**. Getting this backwards shows up in every diff.
---
## 2. There Is No PHP On The Host
On this workstation PHP is not installed on the host at all. `~/.local/bin/php` is a three-line
bridge that execs into the development container, and **`composer` has no bridge** — see
[`distrobox-buildbox-dev.md`](skills-private/distrobox-buildbox-dev.md) for the container, and
[`coding-dev.md` §13](skills-public/coding-dev.md) for why toolchains never land on an immutable
host root.
```bash
# php works from the host. composer does not — go through the container, and chain the cd,
# because entering one resets the working directory.
distrobox enter <container> -- bash -c "cd /abs/path/to/project && composer check"
```
> [!CAUTION]
> **Inside the container, prepend `PATH=/usr/bin:/bin`.** The host `~/.local/bin` is on PATH
> ahead of `/usr/bin` there, so the host's own `php` bridge shadows the container's real
> interpreter and dies with `/usr/bin/distrobox: No such file or directory`. Every
> `vendor/bin/*` tool is a PHP script, so `vendor/bin/phpstan` fails this way and the error
> names distrobox rather than anything to do with PHP. `php vendor/bin/phpstan` with the fixed
> PATH works.
Edit PHP sources with `Read`/`Edit`/`Write`, not with `sed` or a `php -r` patch script. The
nested quoting inside `distrobox enter … bash -c "…"` mangles `$`, backslashes and quotes in
exactly the places PHP uses them, and a half-applied patch script reports success.
---
## 3. Strictness — `strict_types` In Every File, PHPStan At Max
```php
<?php
declare(strict_types=1);
```
**Every file, with no exceptions**, enforced mechanically by the fixer's `declare_strict_types`
rule rather than by review. Without it a `"7"` from an API quietly becomes `int 7` and a
hydrator whose whole purpose is to fail loudly on drift stops failing at all.
```neon
# phpstan.neon.dist
parameters:
level: max
paths:
- src
- tests
- tools
- examples
treatPhpDocTypesAsCertain: false
```
- **Analyse tests, tools and examples too**, not just `src`. Those are the three places authors
assume are exempt, and they are where a convention actually breaks first.
- `treatPhpDocTypesAsCertain: false` stops PHPStan trusting a PHPDoc that is narrower than the
signature, so genuine runtime `is_string`/`is_array` guards are not reported as always-true.
- **No baseline and no `ignoreErrors`.** Level max is reachable with standard tags and real
guards; a baseline is a list of things nobody will ever go back to.
- A suppression, when genuinely needed, is the narrowest `@phpstan-ignore-next-line` with the
reason on it. Reaching for `@phpstan-*` or `@psalm-*` to describe a *type* is a smell — the
standard tags in §5 carry everything level max needs.
- **Never relax a strictness flag without an adjacent comment saying why.** This is the house
habit across every config file in the repo, and it is what makes a relaxation reviewable
instead of load-bearing folklore.
---
## 4. Formatting — PER-CS 2.0, Risky Rules On
```php
// .php-cs-fixer.dist.php
->setRiskyAllowed(true)
->setRules([
'@PER-CS2.0' => true,
'@PHP82Migration' => true,
'declare_strict_types' => true,
'strict_param' => true,
'strict_comparison' => true,
'global_namespace_import' => ['import_classes' => false, 'import_functions' => false],
'no_unused_imports' => true,
'ordered_imports' => ['sort_algorithm' => 'alpha'],
'php_unit_test_case_static_method_calls' => ['call_type' => 'self'],
])
```
php-cs-fixer and PHPStan, not Pint and not Psalm. The fixer config does more normative work
than the prose does: these rules are where five of the conventions below stop being review
items.
- `strict_comparison` + `strict_param` mean every comparison is `===`/`!==` and every
`in_array()` carries its third argument. Loose `in_array` matches numeric strings.
- `global_namespace_import => false` is why every global function, constant and class is written
with a leading backslash: `\sprintf`, `\JSON_THROW_ON_ERROR`, `\DateTimeImmutable`,
`\RuntimeException`. Inside a namespace an unqualified call makes PHP look up the namespaced
name first and fall back to global — a per-call cost, and a hazard the day something defines
`App\json_decode`.
- Guard a fixer API that newer releases added, rather than calling it unconditionally and
breaking the lint step on the oldest version `composer.json` still allows:
`if (\method_exists($config, 'setUnsupportedPhpVersionAllowed')) { … }`.
---
## 5. Banned Constructs
| Never | Use instead |
| :--- | :--- |
| `==` / `!=` | `===` / `!==` (the fixer enforces it) |
| `in_array($x, $a)` | `\in_array($x, $a, true)` |
| `stdClass` from `json_decode` | `\json_decode($b, true, 512, \JSON_THROW_ON_ERROR)` into `array<string, mixed>`, then a typed model |
| `json_last_error()` checks | `\JSON_THROW_ON_ERROR` on every encode and decode |
| `\DateTime` | `\DateTimeImmutable` — a mutable date in a `readonly` class is a hole in the guarantee |
| `setX()` on a value object | `withX()` returning a new instance |
| `http_build_query()` | a builder that drops nulls, spells booleans `true`/`false`, and repeats a key instead of emitting `k[0]=` |
| `extract()`, `eval()`, `$$var` | named variables |
| `@` error suppression | check the return value |
| `array()` long syntax | `[]` |
| A pure `enum` for a wire value | a backed `enum X: string` (§6) |
| `echo $x` in HTML | `\htmlspecialchars($x, \ENT_QUOTES, 'UTF-8')` (§10) |
| Unqualified `strlen`/`count`/`sprintf` inside a namespace | `\strlen`, `\count`, `\sprintf` |
- **PHPDoc only says what the signature cannot**: array shapes, generics, `class-string`,
`list<T>`, `@template`, `@implements`, `@return never`. A `@param string $x` restating a
declared scalar is a second source of truth that will drift from the first.
- **`@return never` in a docblock over a native `: mixed`** is the deliberate exception: it
gives PHPStan the unreachability while still letting the helper sit in a ternary tail
position (`return \is_string($v) ? $v : $this->fail(…);`), which a native `: never` forbids.
---
## 6. Types — `final readonly` By Default
- **`final readonly class` is the default shape** for anything carrying data. Plain
`final class` only for things that memoise, hold mutable state, or are pure behaviour.
`readonly` on the class is what makes "models are immutable, build a new one" enforced by the
engine instead of promised in a docblock.
- **Constructor property promotion for every property**, with `private readonly` written out
explicitly even inside a `final` class. The one property declared the long way should be the
one the constructor has to run through a normaliser — promotion cannot transform a value, so
that asymmetry is a readable signal rather than an inconsistency.
- **`new` in a parameter default**, not nullable-plus-`??`: `private readonly LoggerInterface
$logger = new NullLogger()`. The signature then states the real type and PHPStan never sees a
nullable to guard. Keep `?X $x = null` only where a *static factory* forwards the argument and
"the caller said nothing" has to stay distinguishable.
- **Named arguments** wherever a call has more than two or three arguments or skips one. A
wide constructor built positionally silently mis-wires the day two same-typed fields swap
order.
- **First-class callable syntax** (`Model::fromArray(...)`) over a `'Model::fromArray'` string —
it is checked at compile time. Mark a closure that needs no binding `static`, so it cannot
capture `$this` into a long-lived callable and keep the owner alive.
- Model properties may keep the **API's own `snake_case` names** rather than being renamed to
camelCase. A rename table has to be maintained in both directions, silently drops whatever
the mapping forgot, and makes a devtools capture stop matching the code.
---
## 7. Enums & Wire Values
- **Backed enums only** (`enum Provider: string`). A backed case is what lets one branch
collapse any enum to its wire form and a generic reader be typed
`class-string<T of \BackedEnum>`.
- **An unrecognised wire value throws**, naming the model, the field, the value, the enum class
and the remedy. `::from()` raises a bare `\ValueError` with no field name; an `Unknown`
fallback case hands the caller a value the API's own type does not have and defers the failure
to someone else's `match`.
- Where a field genuinely carries any JSON value, type it **`mixed` and explain it in prose**. A
PHPDoc union nothing actually validates is a lie PHPStan will then trust.
---
## 8. Errors & Exceptions
- **The root of the hierarchy is an interface**, `interface AppException extends \Throwable {}`,
not a base class. Callers can then `catch (AppException $e)` without catching unrelated
`\RuntimeException`s, while each exception still extends the most apt SPL class — which an
abstract base makes impossible the moment two of them want different parents.
- One **static factory mapping status to class** (`match (true)` over 400/401/403/404/409/422/
429/`>= 500`). Declare the base's constructor `final public function __construct` so `new
$class(…)` in that factory stays safe.
- Carry the **parsed error envelope, the raw body, and the request method and URI** on the
exception, and truncate the raw body — a 2 MB proxy error page does not belong in a log line.
Document that callers branch on an error *code*, never on the message, which is prose.
- **A failure to parse the error body must never mask the real status.** A 502 from a load
balancer returns HTML; if the envelope parser throws, the caller sees a JSON parse error
instead of "the service is down". Return null from the parser on every one of those paths.
- **Retry only idempotent methods** — `GET`, `HEAD`, `OPTIONS`, `PUT`, `DELETE`. POST and PATCH
need an explicit opt-in, because a timeout on a request that actually succeeded is how a
giveaway gets drawn twice. Honour `Retry-After` over the computed backoff, and inject the
sleeper so the retry test does not take 30 seconds of wall clock.
---
## 9. Libraries — PSR Interfaces, Not Implementations
Follow the global ordering (reuse → fork → write, [`coding-dev.md` §3](skills-public/coding-dev.md)).
For a published package specifically:
- **Require PSR interfaces and ship no implementation.** `psr/http-client` (18),
`psr/http-factory` (17), `psr/http-message` (7), `psr/log` (3), plus `php-http/discovery`.
Guzzle and a PSR-7 implementation go in `require-dev` with a `suggest` entry saying any
implementation works. Pinning a client forces a version on every consuming application.
- Widen the constraint on an interface package that has had a major (`"psr/http-message":
"^1.1 || ^2.0"`) so a consumer's choice does not conflict with yours.
- **Wrap the PSR client behind your own one-method transport interface.** That is what lets the
whole unit suite run against a fake and assert on exact bytes, instead of building PSR-7
mocks in every test.
- Convert a discovery miss into your own configuration exception whose message says what to
`composer require`.
- **Secrets enter through an interface**, never a string parameter: a token provider whose
implementation reads the environment, with a null return treated as legitimate because some
endpoints are public. A token passed as a constructor literal lands in every stack trace and
`var_dump`. See [`coding-dev.md` §10](skills-public/coding-dev.md) for the general rule.
- **Partial updates**: distinguish "field absent" from "field is null" per field. Serialise a
fetched model, PATCH it back, and a blanket `array_filter(…, fn($v) => $v !== null)` wipes
every server-side value the caller never mentioned — while the opposite default breaks the
fields for which null *is* the value.
- Accept an `int` where the schema says `number`. JSON has one number type, so a whole number
arrives as an int and strict `is_float()` rejects every such value. This wants a named
regression test.
**Generated code** lives in directories that are wiped and rewritten whole, never overwritten
file by file — otherwise a deleted endpoint lingers as a class that still compiles and 404s.
Every generated file opens with a banner carrying the exact regeneration command, generated and
hand-written namespaces never share a directory, and **nothing is added to a generated
namespace** — a method on a generated enum is deleted by the next run, so it belongs on a
hand-written sibling instead.
---
## 10. Web Applications — Escaping, Writes, Sessions
A plain PHP-FPM app with no framework still gets **one `require __DIR__ . '/../vendor/autoload.php'`
per entry point** and PSR-4 for everything else. Hand-rolled `require` graphs between
application files are what produce double-inclusion and load-order bugs.
- **Entry points are three lines over a class.** Two copies of the same handler are how one of
them loses its CSRF check.
- **Compute first, echo second.** With no template engine the discipline is the separation: all
logic above the `?>`, then markup with `<?= ?>` echoes of values already shaped. Logic mixed
into markup is where an unescaped echo hides.
- **Escape at every echo**, with the explicit form `\htmlspecialchars($t, \ENT_QUOTES, 'UTF-8')`.
There are no trusted values.
> [!CAUTION]
> **`json_encode` into an inline `<script>` is an XSS hole**, and `JSON_UNESCAPED_SLASHES`
> opens it by removing the `<\/script>` escaping that would otherwise have saved you. Any
> `</script>` or `<!--` inside the data closes the element and the rest is parsed as HTML. Use
> `JSON_HEX_TAG | JSON_HEX_AMP | JSON_HEX_APOS | JSON_HEX_QUOT`, or emit a
> `<script type="application/json">` block and `JSON.parse` it.
- **The document root stays unwritable.** A web server that can create files in its own
document root is one bug away from writing a `.php` there. Anything PHP writes goes under
`sys_get_temp_dir()`; a file it must update in place gets permission on *the file*, not on
the directory. If an upload directory must live under the root anyway, sniff content
(`getimagesize()` plus an image-type→extension allowlist, never the uploaded name) **and**
deny `.php` under it in the server config.
- **Check the return value of every write.** Both halves of the uid trap recorded in this
machine's memory were silent: a cache the web user could not create meant every page view
re-scraped three third-party APIs, and an `.env` the web user could not read looked exactly
like "no token configured".
- **Atomic writes, two routes.** Write a temp file beside the target and `rename()` it, copying
the victim's mode *and group* first — a rename-created file lands in PHP's group and locks
everyone else out of editing it. Fall back to `r+` plus `flock(LOCK_EX)` plus `ftruncate`,
never `w`, which truncates before the lock and happily creates a stray file at a typo'd path.
Treat a short write as a failure.
- **Optimistic concurrency by content hash**, not mtime — the mtime of a file on a network
mount is not something to bet a save on. Validate the whole document before writing; a
partial save is never attempted.
- **Validate a filename against a listing**, not by sanitising it: `in_array($name,
array_column($this->backups(), 'name'), true)`. Cleaning the string is how `../../.env` gets
through, and for a path under a base directory only comparing `realpath()` results catches a
symlink.
- **Sessions in one place.** One guarded `session_start()`, an explicit `save_path` you know is
writable (a session that silently fails to persist looks exactly like a login that silently
fails), `use_strict_mode=1`, `use_only_cookies=1`, and an `httponly` + `samesite=Lax` +
`secure` cookie. Decide `secure` by **defaulting to true and excepting loopback**: behind a
terminating proxy `$_SERVER['HTTPS']` is empty on a real HTTPS visit, and reading
`X-Forwarded-Proto` lets a forged header strip the flag. Expire by inactivity, which the client
does not control, and `session_regenerate_id(true)` the moment a visitor gains privilege.
- **Re-check authorisation per request**, never from an `is_admin` flag stamped into the session
at login — otherwise revoking access means waiting for a session to expire. Request the minimum
OAuth scope and discard the access token after use; storing it turns a session leak into an
account compromise.
- **Discard unsafe input rather than cleaning it**: a redirect target must start with `/`, must
not start with `//`, and must have no control characters or backslashes; strip CR/LF from any
header value built from a query string.
- **An authenticated JSON API answers 401 with JSON**, not a 302 to an HTML login page, which is
useless to `fetch()` and surfaces as a parse error. Every admin call is a POST carrying a CSRF
token compared with `hash_equals` — **including the reads**, since a GET is reachable from
another site's page — dispatched through `match` with a `default` that 400s.
- **One response shape**, `{ok: bool, …}` with a snake_case `reason` code, and set the content
type, `X-Content-Type-Options: nosniff` and the cache policy *before* anything can fail so the
error path carries them too. Choose per endpoint whether upstream trouble is a 200 with
`ok: false` (a public endpoint the page polls) or a real status code (a command channel).
- **A multipart request has already been consumed into `$_POST`/`$_FILES`**, so check for the
upload action *before* reading `php://input` — otherwise the generic JSON path reports "not
valid JSON" for a body PHP ate.
- **Every outbound call gets a timeout and a connect timeout**, `http_errors => false` so a 4xx
body can be read for its reason, and redirects pinned to `https`. No timeout means a hung third
party holding an FPM worker.
- **A missing credential is a supported state**: separate "configured" from "not configured",
answer with a machine-readable reason, and render a page saying it is not set up rather than a
500 nobody can diagnose.
---
## 11. Testing
- **PHPUnit with attributes**, never docblock annotations: `#[CoversClass]`, `#[DataProvider]`,
`#[Group]`. Annotations are deprecated in PHPUnit 11 and the config below turns that into a
failure rather than a warning.
- Turn on every escalation in `phpunit.xml.dist`: `failOnWarning`, `failOnNotice`,
`failOnDeprecation`, `failOnRisky`, `failOnEmptyTestSuite`, `beStrictAboutOutputDuringTests`.
`failOnSkipped="false"` is the one justified exception, because a suite that skips itself
without credentials is the intended outcome on a machine without them — say so in an XML
comment next to it.
- Test classes are `final`; assertions are `self::assertSame(…)` (the fixer enforces it), while
expectation setup, which is genuinely instance state, stays on `$this`.
- Test method names are full sentences: `testAWrongTypeIsReportedRatherThanCoerced()`.
- **Mock at the transport boundary with a fake that fails loudly.** Running out of queued
responses throws, naming the method, the URI and how many were already served — a fake
returning null turns "one more request than expected" into a type error three frames away.
Record every send so a test can assert on exact bytes.
- **Fixtures are sanitised real captures**, not hand-written JSON. A hand-written fixture
encodes what the author *thinks* the API returns, so a drift test passes against an imagined
payload. Keep shape, keys and enum values exactly; replace every identifier, handle, display
name, avatar URL and message body with a deterministic placeholder so the diff stays stable.
The raw capture stays out of the repository.
- **A suite needing credentials is a separate testsuite**, tagged, not the default, and skips
per test by re-reading the environment in a helper — so the skip message is attached to the
test that needed it. `\getenv()` returns `false` when unset, so guard with `is_string` before
`trim`. Everything in it is a read, because the only credentials anyone will run it with
belong to something live. **Regression test every fixed bug.**
- **Put the file and function size limits in a test**, since neither PHPStan nor the fixer has a
length rule — see [`coding-dev.md` §4](skills-public/coding-dev.md). Measure function length
by walking `\PhpToken::tokenize()`, not by counting braces in text: a `{` inside a string or a
comment looks exactly like a block opener. Skip generated files, whose length is a property of
the API surface. Carry over-limit files in an `ACCEPTED` map whose value is the written reason,
and fail when a listed file drops back under so the entry gets removed.
---
## 12. CI
A PHP workflow runs `shivammathur/setup-php@v2` with `tools: composer:v2`, then
`composer install --prefer-dist --no-interaction --no-progress`, then the gate in order: fixer
dry-run, PHPStan, unit suite. Set `permissions: contents: read` explicitly. Put a
credential-needing suite behind a `workflow_dispatch` input and pass the secret as `env:`,
never as a command-line argument ([`cli-tooling.md` §7](skills-public/cli-tooling.md)).
> [!IMPORTANT]
> **Under this account, only `workflow_dispatch` may be live.** Every automatic trigger —
> `push`, `pull_request`, `schedule` — stays in the file but commented out, under a comment
> saying why, so a fork restores CI by deleting `#` characters. See the GitHub Actions section
> of the agent profile; a workflow written with `on: push` looks correct and never runs.
A job that reports upstream drift should `::warning::` rather than fail: the API moving is
information, not a defect in this repository.
+314
View File
@@ -0,0 +1,314 @@
---
name: python
description: >-
Use this skill when writing, linting, type-checking, testing, or packaging Python —
including async code, CLI tools, MCP servers, Red-DiscordBot cogs, Home Assistant custom
components, and library packages. Covers strict typing with mypy, ruff configuration,
virtualenv discipline, async patterns, framework conventions, and packaging.
---
# Python — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Python specifics.
For the surrounding build and deploy scripts, see [`bash.md`](skills-public/code-languages/bash.md).
§1–§9 are the general rules. §10–§12 are the three frameworks most of this Python is written
against, each of which overrides a general rule somewhere — and says so.
---
## 1. The Gate
```bash
ruff format --check .
ruff check .
mypy --strict .
pytest
```
All four clean before anything is called done. Configure them in `pyproject.toml` so the
editor, the CLI, and the hooks all read the same settings.
**The gate runs locally and through `pre-commit`, not through CI** — workflows do not run under
this account (see the GitHub Actions section of the agent profile), so a green repository with no
local run means nothing was checked. A `.pre-commit-config.yaml` with `ruff-pre-commit`
(check + format) and `mirrors-mypy` scoped `files: src` is where enforcement actually lives.
Scale the gate to what the code is: **a new owner-authored project gets all four**. A vendored
fork gets lint only — match the surrounding style, keep the patch minimal and reviewable, and
do not open a mass reformat of someone else's package (`coding-dev.md` §8 scope discipline).
---
## 2. Typing — Strict
**Every function gets annotated**, including `-> None`. Type hints are not optional
documentation; they are the checker's input.
```toml
[tool.mypy]
strict = true
warn_unreachable = true
warn_return_any = true
disallow_untyped_defs = true
```
- **Never bare `Any`.** Use a `Protocol`, a `TypeVar`, or `object` + narrowing.
- `X | None` over `Optional[X]`; `list[str]` over `List[str]` (modern syntax, no `typing` import).
- `TypedDict` or a dataclass for structured dicts — not `dict[str, Any]`.
- `Literal` and `Enum` for closed sets of values, not bare strings.
- `@dataclass(frozen=True, slots=True)` for value objects.
- Third-party library missing stubs: install `types-*` if it exists, otherwise a narrowly
scoped `# type: ignore[import-untyped]` with a comment — never a blanket module-level ignore.
For a **framework with no stubs at all**, the per-import ignore does not scale; use a named
`[[tool.mypy.overrides]]` block listing those modules with `ignore_missing_imports = true`
and a comment saying why, so the exemption is one reviewable list rather than scattered.
- Worth adding beyond `strict`: `no_implicit_optional`, `warn_redundant_casts`,
`warn_unused_ignores`, `warn_unused_configs`, `show_error_codes`, `allow_redefinition = false`.
### Value types over primitives
The general rule is in [`coding-dev.md` §7](skills-public/coding-dev.md); Python's concrete
choices are:
- **`yarl.URL` for URLs**, not f-string assembly. It ships with `aiohttp`, so any async HTTP
project already has it. `URL(base) / "api" / "v4" / "users"` joins without double slashes,
and `.with_query({...})` escapes parameters that string formatting would corrupt.
- **`pathlib.Path` for paths**, never `os.path` + `str`. Ruff's `PTH` rules enforce this;
keep `os.fdopen` where a descriptor rather than a name is being adopted.
- `enum.StrEnum` (3.11+) where a closed set must also serialize as a string, so the wire
format stays a string without the code comparing bare literals.
- `typing.NewType` for identifiers that should not be interchangeable — a `UserId` and a
`RepoId` are both `int` and nothing otherwise stops them being swapped.
- Parse at the boundary: take `str` in the public signature if callers supply one, convert
immediately, and keep the real type internal.
---
## 3. Linting
`ruff` is the linter and formatter (it replaces flake8, isort, black, pyupgrade, and most
plugins). The preferred shape is **everything on, minus a documented ignore list**:
```toml
[tool.ruff.lint]
select = ["ALL"]
ignore = [
"COM812", # conflicts with the formatter
"D203", "D213", # mutually exclusive with the docstring style chosen below
"E501", # the formatter owns line length
]
```
The explicit set below is the floor, for a codebase where `ALL` would bury real findings:
```toml
select = ["E", "F", "W", "I", "N", "UP", "B", "A", "C4", "SIM", "RUF", "ASYNC", "S", "PTH"]
```
- `S` (bandit security rules) and `ASYNC` are worth the noise. `PTH` keeps filesystem access on
`pathlib`. Add `PLR0915` and `C901` as the function-size proxies (§8).
- **Comment the ignore list too.** Every entry gets the reason beside it, exactly as a `# noqa`
does — an uncommented ignore list is a set of rules nobody can tell were disabled on purpose.
- As always: fix, don't suppress — and when you must suppress, use a specific code
(`# noqa: S603`) with a reason.
- Do not add `black` alongside ruff. Where both appear it is an inherited upstream toolchain,
not a choice; ruff's formatter is the one to keep in new work.
---
## 4. Environments
- **Always a virtualenv or project-managed environment** (`uv`, `poetry`, `pdm`, `venv`).
Never install into the system interpreter, and never `pip install --user` as a workaround
for an immutable OS — use a container or a venv.
- `uv` is the preferred installer/resolver where free choice exists; it is dramatically faster
and lockfile-native. **Check it exists before relying on it** (`command -v uv`) — a committed
`uv.lock` is not evidence that the tool is installed on this machine.
> [!CAUTION]
> **Bare `python` and bare `pip` can be different interpreters.** On this host `python` is the
> system 3.14 while `pip`/`pip3` on PATH are bridge wrappers into the development container's
> 3.13 — so `pip install x` followed by `python x.py` produces a `ModuleNotFoundError` that
> looks like a bad install. Always go through a project venv, or `python -m pip` with the
> interpreter named explicitly. See [`distrobox-buildbox-dev.md`](skills-private/distrobox-buildbox-dev.md).
- **Lockfile by artifact type**: an application commits its lock, a library does not. A
throwaway script directory legitimately stays on a `requirements.txt` with floor pins and no
lock at all.
- Dev dependencies go in PEP 735 `[dependency-groups] dev = [...]`. `[tool.uv]
dev-dependencies` is the older uv-specific form — fine to leave in a fork, not what new work
uses.
- Pin the Python version in `pyproject.toml` (`requires-python`). Where there is no `[project]`
table at all — a plugin or custom component rather than a package — the pin lives in ruff's
`target-version` and mypy's `python_version` instead, and **those two will drift apart unless
something checks them**; set them in the same edit.
- `pathlib.Path` over `os.path` everywhere (ruff's `PTH` rules enforce this).
---
## 5. Errors & Logging
- **Specific exceptions only.** `except Exception:` needs a reason; bare `except:` is never
acceptable (it catches `KeyboardInterrupt` and `SystemExit`).
- **Never swallow silently.** If you catch and continue, log at `warning` or above.
- `raise ... from err` to preserve the chain.
- Custom exception types deriving from a single package-level base, so callers can catch the
whole family.
- **`logging` module, never `print`,** in anything that isn't a CLI writing to stdout as its
actual output. One `logger = logging.getLogger(__name__)` per module, at module scope — unless
the framework owns the logger tree, in which case its prefix wins (§11).
- **Never log secrets** — tokens, auth headers, message content, personal data.
- A blanket `contextlib.suppress(Exception)` is permitted in exactly one situation: a
development-ergonomics escape hatch whose every suppression carries a written paragraph saying
what staleness or platform quirk it absorbs. The rule is not "never suppress", it is
**suppression is allowed when the comment carries the argument**.
---
## 6. Async
- `asyncio` with `async`/`await`. **Never block the event loop** — no `time.sleep`, no
synchronous `requests`, no blocking file I/O in a coroutine. Use `asyncio.to_thread` for
genuinely blocking calls.
- `httpx` (or the framework's own client) over `requests` in async code.
- **`asyncio.TaskGroup`** (3.11+) over bare `create_task` **where one failure should cancel the
rest** — it propagates exceptions instead of losing them to an unawaited task.
- **For fan-out over N external calls, TaskGroup is the wrong default**: cancel-on-first-error
means one dead provider takes down the other nine. Use a bounded helper instead — an
`asyncio.Semaphore(limit)` around each worker plus `asyncio.gather(..., return_exceptions=True)`
— and keep it in one shared function rather than re-deriving it per caller.
- **Every `create_task` result must be held in a reference**, or the garbage collector can
cancel it mid-flight. Clear it in `finally`.
- Cancellation is cooperative: clean up in `finally`, and never swallow `CancelledError` —
but **distinguish a cancellation you asked for from a framework shutdown**. Re-raise unless a
stop was requested; converting a real shutdown into an application error strands it.
- Where a blocking library runs **its own** background thread, don't wrap it in `to_thread` —
bridge back onto the loop with `loop.call_soon_threadsafe(future.set_result, …)`.
- **Events over polling.** Subscribe to the library's event/callback API; do not loop on a
timer. Where polling is genuinely unavoidable (no event source exists), use a **staggered
backoff** rather than a fixed interval, and say why polling was necessary.
---
## 7. Rate Limiting & External Services
When talking to a rate-limited API (a chat platform, a store, a tracker):
- **Funnel outbound calls through a single queue/worker** rather than calling from arbitrary
call sites. It is the only place a limit can actually be enforced.
- **Backoff with jitter** on retry; never a tight retry loop.
- Respect `Retry-After` and documented limits; assume undocumented ones exist.
- **Be extremely careful scraping real consumer services.** Conservative delays, realistic
headers, plenty of failsafes, and abort fast when a response looks unexpected. Assume you
are one mistake away from a ban.
---
## 8. Structure
- Package layout: `src/<package>/`, tests in `tests/`. A single-binary stdlib-only CLI may stay
flat; a plugin or component follows its framework's layout instead.
- A tool that has to run on an immutable host is worth keeping **dependency-free** where the
stdlib covers it (`argparse`, `subprocess`, `shlex`, `json`, `pathlib`) — nothing to provision
is the strongest form of "it works on that machine".
- Entry points are `[project.scripts]`. `typer` for a CLI with subcommands, stdlib `argparse`
for everything else; do not add a CLI framework to a two-flag script.
- **Shared behavior goes in a real shared module**, imported by each consumer — not
copy-pasted between sibling packages. When three components grow the same queue/retry/
formatting logic, that is the signal to extract it.
- Reusable, generic functionality (a format parser, a protocol client) becomes **its own
installable package** with its own repo and version, consumed as a dependency.
- Keep persistent state in the framework's own config/storage system so it survives restarts;
do not invent a parallel JSON file beside it.
- File and function size limits from the general rules apply. Ruff has **no line-based**
function-length rule — `PLR0915` (`too-many-statements`, default 50) and `C901`
(cyclomatic complexity) are the closest proxies and are worth enabling, but neither
measures lines. Put **both** the function-length and file-length limits in a pytest
case (see [`coding-dev.md` §4](skills-public/coding-dev.md)): file 1000 lines max, function
200 lines max measured by walking the `ast` rather than counting lines of text. **Guard the guard** with a test asserting the glob matched anything at
all — a pattern that matches nothing makes every limit vacuous and still passes.
---
## 9. Testing
- `pytest`, with `pytest-asyncio`. Set `asyncio_mode = "auto"` in
`[tool.pytest.ini_options]` — it removes the per-test marker entirely — plus
`asyncio_default_fixture_loop_scope = "function"`.
- Fixtures over setup/teardown methods; `tmp_path` for anything touching the filesystem.
- **Regression test every fixed bug**, and give them a home: a `test_regressions.py` is where
they stop being scattered and start being a list.
- **Mock at a boundary, not inside your own code.** A test that patches the code under test
proves nothing. The boundary can be a network mock (`respx`, `responses`), a **real local fake
service** the test drives end to end, or **recorded payloads fed to the provider seam** — the
last two are usually better evidence than a mocked HTTP layer, because they exercise the
parsing too.
- **Never exercise destructive operations against real accounts or real data.** Temporary
fixtures only.
- `pytest -p no:randomly` only when order genuinely matters, and note why.
---
## 10. MCP Servers
> [!CAUTION]
> **On the stdio transport, stdout is the protocol.** A stray `print`, a `logging`
> `StreamHandler` pointed at stdout, or a library's progress bar corrupts the frame and the
> client sees a parse error rather than your message. Log to **stderr** only, and keep §5's
> no-`print` rule absolute here.
- Make the transport an explicit option (`--transport stdio|http`) rather than hardcoding one.
- **Tool schemas are derived from the signature and the docstring**, so §2's "every function
annotated" rule has a second payoff: an unannotated parameter produces a useless client-facing
schema. Never hand-write JSON schema.
- Set the behavioural annotations on every tool — `readOnlyHint`, `openWorldHint`, a human
`title`.
- **Errors are raised, not returned.** Raise (FastMCP serialises it) or return
`isError=True` with a `TextContent` on the low-level SDK. A tool that returns the string
`"ERROR: …"` as its normal value is indistinguishable from data to the client and defeats
its retry logic.
- `pydantic-settings` for layered config is good, but note what using a settings value as a
**tool parameter default** means: it is frozen at import time *and* published in the advertised
schema. Fine for a hostname, a secret leak for anything else.
---
## 11. Red-DiscordBot Cogs
- `self.config = Config.get_conf(self, identifier=<int>, force_registration=True)`, then
`register_global`/`register_guild` for every key. **Never a side-car JSON file** beside the cog.
- Read-modify-write goes through the context manager — `async with
self.config.guild(ctx.guild).channels() as channels:` — because a plain get/set around a
mutable list is a lost-update bug.
- Lifecycle: `async def setup(bot)` in `__init__.py`; `async def cog_load(self)` for I/O-bearing
startup; `@commands.Cog.listener()` for gateway events.
- **`cog_unload` is synchronous.** It cannot await, so it cancels held tasks and fires teardown
off into `self.bot.loop.create_task(self.api.close())`. Start long-lived tasks the same way,
on the bot's loop.
- `async def red_delete_data_for_user(self, *, _requester: str, _user_id: int) -> None` is a
mandatory contract, not optional boilerplate.
- **The logger name overrides §5**: `logging.getLogger("red.<owner>.<cog>")`, not `__name__`.
Under `__name__` the cog sits outside the `red.*` tree where Red's own log-level controls
apply, so the generic rule produces a logger nobody can turn up.
- Red loads each cog directory independently, so a cross-cog shared module is awkward by design.
The answer is a per-cog `utils/` package, not a copy-pasted `lib.py` — §8's shared-module rule
still applies *within* a cog.
---
## 12. Home Assistant Custom Components
- `from __future__ import annotations` at the top of every module.
- The `async_setup_entry` / `async_unload_entry` / `_async_update_listener` triple, with state on
**`entry.runtime_data`**, not `hass.data`.
- **Never create your own `aiohttp.ClientSession`** — take `async_get_clientsession(hass)`, or
you leak a session per reload and HA logs it.
- A `DataUpdateCoordinator[T]` subclass whose `_async_update_data` raises `UpdateFailed`. **No
provider exception may escape it.**
- Declare dependencies in `manifest.json` `requirements`, not in `pyproject.toml` — the component
is not a package.
- Fire events in size-bounded chunks with a run id in the envelope; a single huge event is not
a thing the bus handles gracefully.
- PEP 695 syntax is current here: `type MyConfigEntry = ConfigEntry[MyCoordinator]`, and
`def f[T](…)` for generics.
+199
View File
@@ -0,0 +1,199 @@
---
name: rust
description: >-
Use this skill when writing, building, linting, testing, or releasing Rust code — including
workspace lint configuration, clippy/fmt gates, error handling with Result/thiserror,
unsafe policy, async patterns, and cargo workspace/submodule layout.
---
# Rust — Rules & Preferences
Extends [`coding-dev.md`](skills-public/coding-dev.md). Everything there applies; this adds Rust specifics.
---
## 1. The Gate
Nothing is "done" until all of these are clean:
```bash
cargo fmt --all --check
cargo clippy --workspace --all-targets --all-features
cargo test --workspace
cargo doc --workspace --no-deps
cargo build --release
```
> [!IMPORTANT]
> **`--workspace` is not optional.** Without it only the root package is checked and every
> member crate's warnings stay invisible. This has hidden real warnings for weeks at a time.
> `--all-targets` additionally covers tests, benches, and examples — which are also code.
- `cargo fmt` before every commit. **No hand-formatting**, ever.
- `cargo doc` matters: a broken intra-doc link is a reference to something that no longer
exists, which is worth knowing about.
- Run the project's build script (`./scripts/build.sh`) if it has one — it wires these
together plus size checks and a smoke run.
---
## 2. Lints — Pedantic On, Configured In One Place
```toml
# Cargo.toml (workspace root)
[workspace.lints.clippy]
pedantic = { level = "warn", priority = -1 }
too_many_lines = "warn"
unwrap_used = "warn"
[workspace.lints.rust]
unsafe_code = "forbid"
```
```toml
# every member crate
[lints]
workspace = true
```
> [!CAUTION]
> **Lint config lives in `[workspace.lints]` only.** A package that repeats the configuration
> carries a second copy that drifts — which is how a binary once kept reading its own stale
> allows while the workspace's were being actively edited.
- **Never `#[allow]` on a whim** — but when a lint is genuinely wrong for a piece of code,
**allow it at the narrowest scope that covers it** (the expression, the function, the
module) with a comment saying why:
```rust
// The FFI callback signature is fixed by the C API; we cannot take fewer arguments.
#[allow(clippy::too_many_arguments)]
extern "C" fn on_event(/* … */) { }
```
- A **workspace-wide** allow in `[workspace.lints]` is for the rare case where the lint is
wrong for the *entire* codebase — not a shortcut for one stubborn call site. It silences
the rule in code nobody has written yet.
- **Re-test every `#[allow]` periodically.** If removing one produces no warning, delete it.
- `too_many_lines` is the mechanical enforcement of the 200-line function limit. Clippy's
default threshold is 100, so set it in `clippy.toml`:
```toml
too-many-lines-threshold = 200
```
A function that trips it wants extracting, not an allow.
- Clippy has **no file-length lint**, so the 1000-line file limit belongs in a test — see
[`coding-dev.md` §4](skills-public/coding-dev.md) for the pattern. Put it in the workspace
root's `tests/` so one test covers every member crate, walking `src/` plus each
`crates/*/src`. `cargo test --workspace` then enforces it for free.
---
## 3. Errors & Panics
- **`Result` and `Option` everywhere.** `thiserror` for library error enums;
`anyhow` is acceptable at the application/binary boundary only.
- **No `unwrap()` or `expect()` outside tests and `main`.** Anything that can be absent at
runtime — a device, a file, a network peer, an env var — must be handled. Hardware and
network resources appear and disappear while the program runs.
- No `panic!` as flow control. A library never panics on input it was designed to receive.
- Never `?`-swallow context — attach it (`.context(...)` / a typed variant) so the error says
where it came from.
---
## 4. `unsafe`
**Forbidden by default**, enforced in `Cargo.toml` with `unsafe_code = "forbid"`.
If an FFI binding genuinely requires it, it goes behind a **safe wrapper in exactly one
module** (see [`c-cpp.md`](skills-public/code-languages/c-cpp.md) for the other side of that boundary), with a `// SAFETY:` comment on every block stating the invariant being upheld.
`unsafe` scattered through call sites is not acceptable.
---
## 5. Dependencies & Workspace Layout
Follow the global ordering (reuse → fork → write). In Rust specifically:
- Search crates.io and check maintenance status before adding. Prefer pure-Rust crates over
ones requiring a C toolchain when the difference is not material — a C dependency turns a
one-command build into a container requirement.
- **Pin precisely when it matters.** If you need an API that only exists on a git revision,
pin the revision and **write down why** — a future reader will otherwise "helpfully"
bump it to the crates.io release that lacks the API.
- **Reusable logic goes in its own crate**, published separately, with the application as a
consumer. The workspace layout that follows from this:
```
crates/<name>-core/ # all reusable logic, no UI, no framework deps
crates/<name>-proto/ # wire types & protocol parsing, dependency-light
src/ # or crates/<name>-ui — presentation only
```
- **Submodule hygiene**: commit and push inside the submodule crate first, then commit the
updated SHA pointer in the parent. Match each repo's own commit-trailer and identity
conventions — a public library repo and a private app repo often differ deliberately.
---
## 6. Async
- Tokio runtime with `futures` streams; `async_trait` where trait methods must be async.
- **Model new flows as streams or async methods**, matching what the codebase already does.
- **Never block in async context.** Use `spawn_blocking` for genuinely blocking work.
- Subscribe to event streams; never poll on a timer (see the global events-over-polling rule).
- Don't short-circuit a hand-written `Stream` impl that manually pumps backends — it usually
encodes ordering guarantees that are not obvious from the signature.
---
## 7. Feature Flags & `cfg`
- Feature flags declared in the root `Cargo.toml`.
- **Gate at module level, not per function** — per-function `cfg` leaves a trail of empty
stubs and makes the real platform differences hard to see.
- Adding a platform backend: declare the feature, create the gated module, and **log which
backend was selected at startup**. Silent backend selection is very hard to debug remotely.
- Use `build.rs`-exported configs for conditions that aren't plain features.
---
## 8. Style Details
- `unsafe`-free, `unwrap`-free, and formatted — see above.
- **No `pub` on anything that doesn't need to cross a module boundary.** Default to private
and widen deliberately.
- Max ~5 parameters; past that, pass a struct.
- Prefer `f32` for UI/geometry when the framework's native unit is `f32` — mixing in `f64`
produces a stream of casts and rounding inconsistencies.
- Derive rather than hand-implement (`Debug`, `Clone`, `Default`, `PartialEq`) unless the
hand implementation is doing something a derive cannot.
- `#[non_exhaustive]` on public enums/structs that are likely to grow.
---
## 9. Testing
- Unit tests inline in `#[cfg(test)] mod tests`; integration tests in `tests/`.
- `cargo test --workspace` — plus `--release` when timing or optimization matters to the
behavior under test.
- **Regression test every fixed bug.**
- A dummy/mock backend that exercises the full pipeline without real hardware is worth
building early — it is the only way most device code gets tested at all.
- Never point test output at real hardware. Route test signal to a virtual/null endpoint and
**verify the route resolved** — most tools silently fall back to the system default when
the requested target does not exist, which sends your test straight at the user.
---
## 10. Building in Containers
When the crate needs system headers the host does not have (audio, graphics, webkit),
build inside the project's designated container. Chain `cd` inside the same command string —
entering a container resets the working directory:
```bash
distrobox enter <container> -- bash -c "cd /path/to/project && cargo build --release"
```
Prefer the project's `scripts/build.sh` over a hand-typed invocation; it also handles job
throttling so a release build does not saturate the machine.
**Split so that as little as possible needs the container**: a `-core` crate with no GUI
dependencies builds and tests directly on the host, which is seconds instead of minutes.
+392
View File
@@ -0,0 +1,392 @@
---
name: coding-dev
description: >-
Use this skill whenever writing, reviewing, refactoring, building, testing, or releasing code
in any language. Defines standing preferences on strictness, dependency choice, file and function
size limits, test discipline, build output handling, error/warning policy, and commit hygiene.
Links to per-language rules under code-languages/ for Rust, TypeScript/JavaScript, Bash, Python,
PHP, Go, C#/.NET, and C/C++.
---
# Coding & Development — Standing Rules and Preferences
These are the defaults for **all** code work, in any language. They are preferences with
reasons attached, not style trivia — most exist because ignoring them caused a real problem.
> [!IMPORTANT]
> **Whose rules win depends on where the code is going.** Always read a repository's
> `AGENTS.md`, `CLAUDE.md`, `AGENT.md`, `.rules`, or `CONTRIBUTING.md` first — but what you do
> on a conflict differs:
>
> | Situation | Precedence |
> | :--- | :--- |
> | **Contributing upstream** — a PR, a patch, or anything landing in someone else's repo | **The project's rules win.** Match their conventions exactly, even where they contradict this document. A PR that fights the house style gets rejected on style instead of reviewed on substance. |
> | **Local / private work** — our own repos, forks we don't upstream, private branches, local patches | **These rules win.** A project's conventions are a data point, not an obligation. Adopt the ones that are load-bearing (their build gate, their architecture contracts, anything that breaks if ignored) and apply our own preferences for the rest. |
>
> When in doubt about which mode you are in, **ask** — it changes the answer on strictness,
> file size, typing, and test policy all at once.
> [!IMPORTANT]
> **A new software idea starts with research, not code.** When the user describes a tool, app,
> service, or other project they want built, follow
> [`software-vision.md`](skills-public/software-vision.md) first: find existing projects to use,
> extend, or fork (web searches and user reviews first, then training-data recall, then
> throwaway clones), and present the options before scaffolding anything — unless the user
> explicitly opted out of research.
Per-language specifics live in [`code-languages/`](skills-public/code-languages/):
| Language | Rules |
| :--- | :--- |
| Rust | [`code-languages/rust.md`](skills-public/code-languages/rust.md) |
| TypeScript / JavaScript | [`code-languages/js-typescript.md`](skills-public/code-languages/js-typescript.md) |
| Bash / Shell | [`code-languages/bash.md`](skills-public/code-languages/bash.md) |
| Python | [`code-languages/python.md`](skills-public/code-languages/python.md) |
| PHP | [`code-languages/php.md`](skills-public/code-languages/php.md) |
| Go | [`code-languages/go.md`](skills-public/code-languages/go.md) |
| C# / .NET | [`code-languages/csharp-dotnet.md`](skills-public/code-languages/csharp-dotnet.md) |
| C / C++ | [`code-languages/c-cpp.md`](skills-public/code-languages/c-cpp.md) |
| Lua | [`code-languages/lua.md`](skills-public/code-languages/lua.md) |
| GSC (Call of Duty script) | [`code-languages/gsc.md`](skills-public/code-languages/gsc.md) |
---
## 1. Strictness — Maximum, Always
**Configure every linter, compiler, and type checker as strictly as the toolchain allows.**
Strict mode is the starting point, not an aspiration to migrate toward later.
- Warnings are errors in CI and in the build gate (`-warnaserror`, `-D warnings`, `--strict`).
- Enable the pedantic/extra lint tiers, not just the defaults.
- Type checking must pass with **zero errors or warnings** — `tsc --noEmit`, `mypy`,
`cargo check` alike.
- Never weaken a check to make a build pass. Fix the failing step. If a check is genuinely
wrong for a specific piece of code, **suppress it at the narrowest scope that covers the
problem — the line, the function, or the module — with a comment explaining why.** A
project-wide disable is a bigger hammer than it looks: it silences the rule everywhere,
including in code written later by someone who never saw the reason. Reach for it only when
the rule is genuinely wrong for the *whole* codebase, never as a shortcut for a stubborn
call site.
### Suppressions are a last resort, and they expire
- Try hard to actually fix an error or warning before suppressing it.
- When you do suppress one, **say so explicitly in your report to the user** — name the
warning, where it is, and why it could not be fixed. A silent suppression is the failure
mode this rule exists to prevent.
- **Every suppression carries a comment saying why.** A bare `#[allow]` / `// eslint-disable`
/ `# noqa` with no reason is indistinguishable from someone giving up, and nobody will ever
dare remove it.
- **Name the specific rule**, never a blanket suppression: `# noqa: S603`, not `# noqa`;
`//nolint:errcheck`, not `//nolint`.
- **Re-test suppressions periodically.** Removing one that no longer suppresses anything
should produce no warning — if it doesn't, delete it. Stale suppressions accumulate and
hide real regressions.
---
## 2. Never Hide Build, Test, or Lint Output
**Do not pipe build, compile, lint, format, or test output through `grep`, `tail`, `head`,
`sed`, or any other filter.** Run the command so its complete output is visible.
**Why:** filtering hides warnings, notes, and diagnostics that did not match the pattern.
Grepping for `^warning|^error` drops multi-line notes, help text, deprecation notices, and
anything a tool emits in an unexpected format.
```bash
# WRONG — silently drops notes, helps, and deprecations
cargo clippy 2>&1 | grep -E "^(warning|error)"
# RIGHT
cargo clippy --workspace --all-targets --all-features
```
If output is genuinely too large, **narrow the command** (one test, one package, one target)
rather than filtering its output. The same applies to logging: do not swallow stderr, do not
redirect diagnostics to `/dev/null`, and do not set a log level that hides warnings during a
verification run.
---
## 3. Dependencies — Reuse Before You Build
A strict ordering for obtaining any non-trivial functionality:
1. **Use an existing library / crate / package / official SDK** that already does it.
2. **If one is close but not exactly right, fork it** (as a git submodule or vendored fork)
and modify it.
3. **Write your own implementation only when nothing else fits** — and say why, explicitly.
**Why:** the ecosystem has already solved correctness, edge cases, and platform quirks that
a fresh implementation will rediscover slowly and incompletely.
Before hand-rolling any subsystem — a protocol client, a parser, a date formatter, a
debounce, a retry/backoff, a semver comparator — **search first and state the finding.**
Prefer official/first-party SDKs over third-party wrappers. The same ordering applies to a
whole project idea, not just a subsystem — see
[`software-vision.md`](skills-public/software-vision.md).
### The corollary: new generic code becomes its own package
**When you implement something generic that no existing package provided — a parser or
reader for a file format, a protocol implementation, a client for an undocumented API —
it becomes its own package/crate/module in its own repository**, which the application then
depends on like any other dependency.
**Why:** it keeps the app's tree focused on the app, it forces a clean public API boundary,
and it gives back the library that was missing in the first place. Structure the split so
the reusable core has no UI, no framework, and no app-specific types — only the app shell
depends on those.
> [!TIP]
> Design for this from the start: put **all reusable logic in a core library** and keep the
> UI/CLI layer as presentation only. A core without GUI dependencies also builds and tests
> far faster, often without the app's heavyweight toolchain.
---
## 4. Size Limits — Files and Functions
| Unit | Limit |
| :--- | :--- |
| **Source file** | **1000 lines** max |
| **Function** | **200 lines** max |
| **Function parameters** | ~5, then pass a struct/object |
| **Nesting depth** | 3, then extract |
These are the limits, not targets to stay well under: there is no earlier "split at" point.
A file of 700 lines with one coherent responsibility is fine and is not split for its size.
### Enforce both mechanically, never by eye
A limit nobody checks is a suggestion. Pick the **highest** applicable option:
1. **A linter rule**, where one exists — the function limit is usually covered natively
(Clippy's `too_many_lines` with `too-many-lines-threshold = 200`, ESLint's
`max-lines-per-function`, `golangci-lint`'s `funlen`), configured to 200 lines.
A function that trips it wants extracting, not a suppression. Note that some linters only
offer *proxies* for length — Ruff's `PLR0915` counts statements, not lines — in which case
enable the proxy and put the real limit in a test.
2. **A test**, for anything the linter does not cover — file length almost always falls here.
A test is the best home for it: it runs with the suite everyone already runs, it needs no
extra tooling, it fails with a message naming the offending files, and it travels with the
repository rather than living in one machine's shell script.
3. **The build script**, only as a fallback when the project has no test suite to hang it on.
Whichever you use, it should **fail over the limit** and name the offending files.
```rust
#[test]
fn no_source_file_exceeds_limits() {
const LIMIT: usize = 1000;
let mut over = Vec::new();
for path in source_files() {
let lines = std::fs::read_to_string(&path).unwrap().lines().count();
if lines > LIMIT {
over.push(format!("{} — {lines} lines (limit {LIMIT})", path.display()));
}
}
assert!(over.is_empty(), "files over the size limit:\n{}", over.join("\n"));
}
```
> [!NOTE]
> A test runs with the suite, not on every compile. If you genuinely need the check on every
> build, most toolchains have a pre-build hook for it (Cargo's `build.rs`, an MSBuild target,
> a bundler plugin) — but that cost is paid on every incremental build forever, and the suite
> is usually soon enough. Reach for the hook only if oversized files keep landing anyway.
- **Split by concern, never by line count.** A large `strip.rs` becomes
`strip/fader.rs`, `strip/knobs.rs`, `strip/routing.rs` — never `strip1.rs` / `strip2.rs`.
- One coherent responsibility per module. A module that draws widgets should not also own
application state.
---
## 5. Tests
- **Write tests for every feature where testing is possible.** State plainly when something
is untestable (GUI, hardware, live third-party service) rather than quietly skipping it.
- **Every bug that was fixed gets a regression test.** A bug that broke once will break again;
the test is what makes the second time cheap. This is the highest-value test you can write.
- **A skipped test fails the gate.** Silently skipped tests are worse than missing ones —
they report green.
- Unit tests for logic; integration tests for protocol/boundary behavior.
- **Never test destructive operations against real data.** Use fixtures, temporary
directories, injected backends, and disposable test accounts. The user's real config,
data, devices, accounts, and files must never be used as scratch space.
- **Prove the test exercises the path under test.** A tool that silently falls back to a
default target when the requested one does not resolve will produce a passing measurement
of the wrong thing. Verify the wiring before trusting the result.
- **Test the value, not the presence of the key.** A check that counts entries which exist
will pass against a cleared or null value — and reads exactly like a working fix.
---
## 6. Verification — Before Claiming Anything Works
Run the **whole** gate, not the convenient part of it. A typical full gate:
> format check → lint (strict/pedantic) → type check → build → tests (with race/sanitizer
> where available) → docs build → size limits → smoke run
- **Prefer the project's build script** (`./scripts/build.sh`, `make check`, `./build.sh --lint`)
over hand-typed commands. It encodes steps you will otherwise forget, and it stays correct
when the toolchain moves.
- **Read the numbers, not just the verdict.** A suppression count that has grown is worth
noticing even when everything passes.
- **Confirm the binary you are testing contains the change.** A file existing proves a file
exists, not that it is current. Compare its mtime against the commit, or wait for the build
to actually report success — testing a stale binary reads exactly like "the fix didn't work".
- **A change to visual/UI code is not done until it has been looked at.** Build, run,
screenshot, compare.
- **State plainly what is stubbed.** A component that draws its frame but has no behavior is a
scaffold, not a finished component — say so rather than implying completion.
---
## 7. Architecture Preferences
- **Events over polling — always.** When there is an event, callback, channel, or watch API to
subscribe to, use it. Do **not** settle for a polling loop because a benchmark shows it
currently costs nothing; a measurement of 0% CPU does not make polling the right design. It
adds latency and can miss or coalesce events. If existing code polls where an event exists,
treat that as a finding worth fixing. Only poll when no event source exists — and say so.
- **Don't over-guard.** Check only conditions that can actually occur. A null check on a
non-optional value, a `try/catch` around code that cannot throw, or an optional-chain on a
type-guaranteed value is dead defensive code, not safety.
- **No abstraction for one or two call sites.** A factory used once is just a function call.
No class wrapping a single function, no state machine for a boolean, no
`Manager`/`Handler`/`Strategy`/`Provider` suffix unless multiple implementations exist.
- **Less code with the same behavior wins.** Between equivalent solutions, the shorter ships.
Three similar lines beat a premature helper.
- **Delete dead code, don't comment it out.** That is what version control is for.
- **Don't cache what a store already owns.** Read from the source of truth on demand; cache
only what has no source of truth.
- **A search-and-replace that matches nothing is a silent failure.** Assert the target string
exists before replacing, and verify the result afterward.
### Model domain values as types, not primitives
**A value with rules of its own gets a type of its own.** Passing it around as a bare string,
int, or dict throws those rules away at the first function boundary and makes every consumer
re-derive them.
| Value | Use | Not |
| :--- | :--- | :--- |
| URL | the platform's URL type (`yarl.URL`, `Uri`, `url::Url`, `URL`) | `str` + f-string assembly |
| Filesystem path | `Path` / `PathBuf` / `std::filesystem::path` | `str` + manual separators |
| Closed set of values | enum / union of literals | bare strings compared with `==` |
| Structured record | struct, dataclass, or typed record | `dict[str, Any]` / `map[string]any` |
| Quantity with a unit | a wrapper carrying the unit | a bare number and a naming convention |
| Identifier of a specific kind | a distinct newtype | a `str` that any other id also fits |
**Why:** the type is where the rules live. A URL type knows how to join a path without
producing a double slash, escapes query parameters, and cannot be concatenated into something
malformed. A path type knows the platform separator. An enum cannot hold a typo. Each one
turns a class of runtime bug into something the compiler or checker rejects — and makes the
signature say what it accepts instead of `str`.
- **Convert at the boundary.** Parse incoming strings into the real type once, at the edge,
and keep the type inward. Serialize back to a string only where the wire or the filesystem
actually demands one.
- **Don't reach for a wrapper where the primitive genuinely is the value** — a person's name
is a string. The test is whether the value has invariants, a format, or operations that
callers keep re-implementing.
- Retrofit opportunistically: convert what you touch rather than rewriting a stringly-typed
codebase in one pass.
---
## 8. Scope Discipline
- **Solve the problem at hand.** No refactors unless asked. No "while I was in there" cleanups.
- **Smallest diff that correctly solves it.**
- **Don't refactor code you weren't asked to touch**, and don't mix unrelated concerns in one
change — backend work and UI work land separately.
- When you notice adjacent work worth doing, **describe it** rather than absorbing it.
- **Check `git status --short` before editing** and preserve unrelated in-progress changes.
- **Verify a TODO before acting on it.** Task documents are leads, not facts — grep for the
symbol, read the function, check the history. Work has been redone from scratch because a
stale TODO claimed something was missing that had shipped hours earlier.
---
## 9. Comments & Naming
- **Comment the *why*, never the *what*.** "A gap here, matching the reference layout" is
useful; `// draw the button` is not.
- **Don't narrate changes in comments** ("changed this to fix X") — that is git's job.
- Keep existing comments; they carry context you do not have.
- **Natural human text in user-facing strings** — errors, descriptions, toasts. Sentence case,
ending in a period, no robotic phrasing.
- **Never hardcode** what a constant, config file, or generated table already defines —
endpoints, colors, class names, translated strings, magic identifiers.
- Names checked mechanically stay correct; names checked by memory drift. If a project has a
naming rule that matters (a one-letter difference between the app and its library, say),
**put it in the build gate**.
---
## 10. Errors, Logging & Security
- Use the language's structured error type; do not stringify errors as the primary mechanism.
- **Never swallow an error silently.** No empty catch blocks.
- Use a real logger with levels — `debug` for traces, `info` for lifecycle, `warn` for
recoverable, `error` for unrecoverable. Not bare `print`/`console.log`.
- **Never log secrets** — tokens, auth headers, passwords, message content, personal data.
- **Never commit secrets.** Reference an environment file or secret store; commit only an
`.env.example` with empty values.
- **Treat every input from outside the trust boundary as hostile** — validate type, length,
and content. Normalize and bounds-check any path before touching the filesystem with it.
Never pass an externally-derived string to a shell.
- **Be extremely careful automating against real third-party services.** Use conservative
rate limits, generous delays, retries with backoff, and abort fast when a response looks
wrong. Assume you are one mistake away from a ban or a real-world side effect.
---
## 11. Git & Commits
- **Conventional Commits**: `feat(scope): …`, `fix(translator): …`, `test: …`, `docs: …`.
- Short, imperative subject lines.
- **Commit incrementally as work lands.** Never leave a large batch of uncommitted work at
the end of a task.
- **Submodules**: commit and push the submodule first, then the updated pointer in the parent.
Never leave a submodule commit unpushed — the parent would reference a commit nobody can fetch.
- Match the repository's existing trailer and attribution conventions; some repositories
deliberately carry none.
---
## 12. Reporting Work Honestly
- If tests fail, **say so and show the output**. If a step was skipped, say that.
- If a warning was suppressed rather than fixed, **name it**.
- Distinguish verified from unverified: GUI behavior, packaging, hardware, and live
third-party interactions are usually unverified unless you actually exercised them.
- Do not claim a visual match without having compared the two images.
- **Prefer the boring explanation.** A typo, a stale binary, or a self-matching pattern is
far more likely than a toolchain or sandbox bug — and an environment explanation is more
flattering and arrives faster, which is exactly why it is the one to distrust.
---
## 13. Environment & Tooling
Full version in [`cli-tooling.md`](skills-public/cli-tooling.md); the essentials:
- **Read `<tool> --help` before using a CLI.** Do not guess flags, and do not write an ad-hoc
script when an existing tool already does the job.
- **Standalone scripts live in a dedicated, version-controlled scripts directory**, symlinked
into `PATH` — never written directly into `~/.local/bin` or another `PATH` directory.
- **Build inside the project's designated environment** (container, venv, toolchain manager)
rather than the host, when the project specifies one. Note that entering a container
typically resets the working directory — chain `cd` inside the same command string.
- **Don't saturate the machine.** A full release build that consumes every core makes the
workstation unusable. When something latency-sensitive is running, halve the job count and
`nice` the build.
- **Prefer long-running work in the background** over blocking wait loops. Never write
`until <check>; do sleep 2; done` — it hangs the terminal and has to be interrupted by hand.
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env python3
"""Reinstall each restored app with its original installer (pm set-installer refuses root).
Same APKs, `install-multiple -r -i <installer>` keeps data. The device comes from ANDROID_SERIAL, as for adb. Usage: fix-installers.py BACKUP_DIR WORK_DIR
"""
import re
import subprocess
import sys
import time
from pathlib import Path
backup, work = Path(sys.argv[1]), Path(sys.argv[2])
log_file = (work / "restore.log").open("a")
def log(msg: str) -> None:
line = f"{time.strftime('%H:%M:%S')} {msg}"
print(line, flush=True)
log_file.write(line + "\n")
log_file.flush()
def adb(*args: str, timeout: int = 900) -> str:
r = subprocess.run(["adb", *args], capture_output=True, text=True, timeout=timeout)
return (r.stdout + r.stderr).strip()
def installers(text: str) -> dict:
# `pm list packages -i` prints "package:PKG", with -f "package:PATH=PKG".
return dict(re.findall(r"[:=]([\w.]+)\s+installer=(\S+)", text))
want = installers((backup / "rooted-fresh/packages-3.txt").read_text())
now = installers(adb("shell", "pm list packages -3 -i"))
disabled = set(re.findall(r"package:([\w.]+)", adb("shell", "pm list packages -d")))
log("=== installer fix-up start")
for pkg, inst in sorted(want.items()):
if inst == "null" or pkg not in now or now[pkg] == inst or inst not in now | {"com.android.vending": 1}:
continue
apks = sorted(str(p) for p in (backup / "apks" / pkg).glob("*.apk"))
if not apks:
continue
# Reinstalling re-enables a frozen app; disable it again afterwards.
out = adb("install-multiple", "-r", "-d", "-i", inst, "--bypass-low-target-sdk-block", *apks)
if pkg in disabled:
adb("shell", f"su -c 'pm disable-user --user 0 {pkg}'")
log(f"{pkg}: installer {now[pkg]} -> {inst}: {'OK' if 'Success' in out else 'FAILED ' + out[-200:]}")
log("=== installer fix-up done")
+74
View File
@@ -0,0 +1,74 @@
#!/usr/bin/env python3
"""Emit SQL that copies apps' keystore2 keys from an old ROM's DB into the live one, under the new UIDs.
Only works when both ROMs use the same TEE (compare keyentry.km_uuid). Keys super-encrypted with the
old lock-screen credential (blobmetadata tag 0, EncryptedBy) cannot be moved and are skipped.
Usage: keystore-transplant.py OLD_DB [--replace] OLD_UID:NEW_UID [...] > transplant.sql
--replace first delete the app's keys that already exist on the new ROM (needed when it regenerated them)
Apply with the apps force-stopped: `su -c 'sqlite3 /data/misc/keystore/persistent.sqlite < transplant.sql'`, reboot.
"""
import secrets
import sqlite3
import sys
DOMAIN_APP = 0
TAG_ENCRYPTED_BY = 0
def lit(v: object) -> str:
if v is None:
return "NULL"
if isinstance(v, bytes):
return f"X'{v.hex()}'"
if isinstance(v, str):
return "'" + v.replace("'", "''") + "'"
return str(v)
def delete_keys(where: str) -> list[str]:
ids = f"(SELECT id FROM keyentry WHERE {where})"
return [f"DELETE FROM blobmetadata WHERE blobentryid IN (SELECT id FROM blobentry WHERE keyentryid IN {ids});",
f"DELETE FROM blobentry WHERE keyentryid IN {ids};",
f"DELETE FROM keyparameter WHERE keyentryid IN {ids};",
f"DELETE FROM keymetadata WHERE keyentryid IN {ids};",
f"DELETE FROM grant WHERE keyentryid IN {ids};",
f"DELETE FROM keyentry WHERE {where};"]
args = sys.argv[1:]
old = sqlite3.connect(f"file:{args.pop(0)}?mode=ro", uri=True)
replace = "--replace" in args
pairs = [tuple(map(int, a.split(":"))) for a in args if a != "--replace"]
# Older keystore2 schemas have no blobentry.state; every blob there is current (0).
blob_state = "state" if any(c[1] == "state" for c in old.execute("PRAGMA table_info(blobentry)")) else "0"
out = ["BEGIN;"]
for old_ns, new_ns in pairs:
if replace:
out += delete_keys(f"domain={DOMAIN_APP} AND namespace={new_ns}")
keys = old.execute("SELECT id, key_type, alias, state, km_uuid FROM keyentry WHERE domain=? AND namespace=?",
(DOMAIN_APP, old_ns)).fetchall()
for kid, key_type, alias, state, km_uuid in keys:
blobs = old.execute(f"SELECT id, subcomponent_type, blob, {blob_state} FROM blobentry WHERE keyentryid=?", (kid,)).fetchall()
if any(old.execute("SELECT 1 FROM blobmetadata WHERE blobentryid=? AND tag=?", (b[0], TAG_ENCRYPTED_BY)).fetchone()
for b in blobs):
print(f"-- skip {old_ns} {alias!r}: super-encrypted with the old lock-screen credential", file=sys.stderr)
continue
new_id = secrets.randbits(63) * (1 if secrets.randbits(1) else -1)
# The old key wins over one the app generated under the same alias on the new ROM.
out += delete_keys(f"domain={DOMAIN_APP} AND namespace={new_ns} AND alias={lit(alias)}")
out.append(f"INSERT INTO keyentry (id,key_type,domain,namespace,alias,state,km_uuid) VALUES "
f"({new_id},{key_type},{DOMAIN_APP},{new_ns},{lit(alias)},{state},{lit(km_uuid)});")
for bid, sub, blob, bstate in blobs:
out.append(f"INSERT INTO blobentry (subcomponent_type,keyentryid,blob,state) VALUES ({sub},{new_id},{lit(blob)},{bstate});")
for tag, data in old.execute("SELECT tag, data FROM blobmetadata WHERE blobentryid=?", (bid,)):
out.append(f"INSERT INTO blobmetadata (blobentryid,tag,data) VALUES ((SELECT max(id) FROM blobentry WHERE keyentryid={new_id}),{tag},{lit(data)});")
for tag, data, level in old.execute("SELECT tag, data, security_level FROM keyparameter WHERE keyentryid=?", (kid,)):
out.append(f"INSERT INTO keyparameter (keyentryid,tag,data,security_level) VALUES ({new_id},{tag},{lit(data)},{level});")
for tag, data in old.execute("SELECT tag, data FROM keymetadata WHERE keyentryid=?", (kid,)):
out.append(f"INSERT INTO keymetadata (keyentryid,tag,data) VALUES ({new_id},{tag},{lit(data)});")
print(f"-- {old_ns}->{new_ns} {alias!r}: {len(blobs)} blobs", file=sys.stderr)
out.append("COMMIT;")
print("\n".join(out))
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
# Cold-start an app's launcher activity the way Android 17 accepts from the shell. Usage: launch.sh PKG [WAIT] [SHOT.png]
p=$1 w=${2:-6} shot=${3:-}
act=$(adb shell "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" </dev/null | tr -d '\r' | tail -1)
adb shell "am start -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" </dev/null >/dev/null
sleep "$w"
[ -n "$shot" ] && adb exec-out screencap -p > "$shot" </dev/null
true
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# Snapshot phone health: memory, load, crashes/ANRs since boot, low-memory kills, log errors, cold launches.
# Usage: measure.sh OUT_FILE [launch-package ...]
set -uo pipefail
out=$1; shift
a() { adb shell "$@" </dev/null; }
{
echo "# $(date '+%F %T') $(a getprop ro.build.display.id) / Android $(a getprop ro.build.version.release)"
echo "## system"
a 'uptime; cat /proc/loadavg'
a 'grep -E "MemTotal|MemAvailable|Cached:|SwapTotal|SwapFree" /proc/meminfo'
a 'su -c "cat /proc/sys/vm/swappiness; cat /sys/block/zram0/mm_stat 2>/dev/null"'
echo "procs: $(a 'ls -d /proc/[0-9]* | wc -l')"
echo "## top cpu"
a 'top -b -n 1 -m 12 -o PID,USER,%CPU,%MEM,NAME' | tail -13
echo "## dropbox since boot"
boot=$(a 'date -d @$(( $(date +%s) - $(cut -d. -f1 /proc/uptime) )) "+%Y-%m-%d %H:%M:%S"')
a "dumpsys dropbox" | awk -v b="$boot" '$1" "$2 >= b && $3 ~ /anr|crash|watchdog|tombstone|wtf|lowmem|native/ {print $3}' | sort | uniq -c
echo "## crash buffer (FATAL / ANR per process)"
a 'logcat -d -b crash' | grep -oE "Process: [^,]+|FATAL EXCEPTION|>>> [^ ]+ <<<" | sort | uniq -c | sort -rn | head -20
a 'logcat -d -b events' | grep -oE "am_anr: \[[^]]*" | awk -F, '{print "ANR " $3}' | sort | uniq -c
echo "## lmkd / am_kill"
a 'logcat -d -b events' | grep -cE "am_kill|lmk_kill" | sed 's/^/kills: /'
a 'logcat -d -b events' | grep -oE "am_kill: \[[^]]*" | awk -F, '{print $5}' | sort | uniq -c | sort -rn | head -10
echo "## logcat error tags (top 25)"
a 'logcat -d *:E' | awk '{print $6}' | sort | uniq -c | sort -rn | head -25
echo "## thermal / battery"
a 'dumpsys battery | grep -E "level|temperature|status"; for z in /sys/class/thermal/thermal_zone*; do echo "$(cat $z/type) $(cat $z/temp)"; done 2>/dev/null | sort -k2 -n | tail -5'
if [ $# -gt 0 ]; then
echo "## cold launches (ms, am start -W -S, 3 runs)"
for p in "$@"; do
t=""
for _ in 1 2 3; do
act=$(a "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" | tr -d '\r' | tail -1)
v=$(a "am start -W -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" | awk '/TotalTime/{print $2}')
t="$t ${v:-fail}"; a 'input keyevent KEYCODE_HOME'; sleep 2
done
echo "$p:$t"
done
fi
} 2>&1 | tee "$out"
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Rebuild the Nova home screen in Lawnchair, plus the Ice Box shortcuts Nova had pinned.
Usage: nova-to-lawnchair.py NOVA_DB LAWNCHAIR_DB OLD_ICEBOX_SHORTCUT_XML OUT_DIR
Writes OUT_DIR/launcher_5_5_5.db (Lawnchair schema, Nova's items), OUT_DIR/icebox.xml
(shortcut_service package file with only the pinned shortcuts) and OUT_DIR/lawnchair-pins.xml
(shortcut_service launcher file pinning them for Lawnchair).
"""
import re
import shutil
import sqlite3
import sys
from pathlib import Path
LAUNCHER = "app.lawnchair.nightly"
ICEBOX = "com.catchingnow.icebox"
# From /data/system/packages.xml on the new ROM: versionCode, lastUpdateTime, SHA-256 of the cert.
LAUNCHER_INFO = ("1600020100", "1792700106881", "dHw2RbNXJYsuI+hR5TyWdH/grdAH5bos2X6MhVcuTcU=")
ICEBOX_INFO = ("154821395", "1792699567277", "gvW+cC4DIBdGDKg3b22r1yWp5S3SSSDdXl17WF+/4bY=")
# Nova-only containers (gesture/dock-swipe slots); Lawnchair has nothing to put them in.
SKIP_CONTAINERS = {-207, -208}
ITEM_APP, ITEM_FOLDER, ITEM_WIDGET, ITEM_DEEP_SHORTCUT = 0, 2, 4, 6
FLAG_ID_NOT_VALID = 1 # Launcher3 restore flag: allocate and bind a fresh widget id on load
MISSING_PACKAGES = {"com.dot.gallery"} # not installed on the new ROM
# Google's smartspace widget on row 0 of the first page: Lawnchair reserves that row for its own.
SKIP_IDS = {56}
nova_db, lc_db, icebox_xml, out = map(Path, sys.argv[1:5])
out.mkdir(parents=True, exist_ok=True)
def package_info(version: str, updated: str, sig: str) -> str:
return (f'<package-info version="{version}" last_udpate_time="{updated}" shadow="0" allow-backup="1" '
f'allow-backup-initialized="1" bk_src_version="-1" bk_src_backup-allowed="0">\n'
f'<signature hash="{sig}" />\n</package-info>\n')
nova = sqlite3.connect(nova_db)
nova.row_factory = sqlite3.Row
rows = [r for r in nova.execute("select * from favorites where _id >= 0 order by _id")
if r["container"] not in SKIP_CONTAINERS and r["_id"] not in SKIP_IDS]
folders = {r["_id"] for r in rows if r["itemType"] == ITEM_FOLDER}
kept = []
for r in rows:
pkg = re.search(r"component=([\w.]+)/", r["intent"] or "")
if pkg and pkg.group(1) in MISSING_PACKAGES:
print(f"skip {r['title']}: {pkg.group(1)} not installed")
continue
if r["container"] >= 0 and r["container"] not in folders:
continue
kept.append(r)
db = out / "launcher_5_5_5.db"
shutil.copy(lc_db, db)
lc = sqlite3.connect(db)
lc.execute("delete from favorites")
pins = []
for r in kept:
widget = r["itemType"] == ITEM_WIDGET
hotseat = r["container"] == -101
in_folder = r["container"] >= 0
x, y = int(max(r["cellX"] or 0, 0)), int(max(r["cellY"] or 0, 0))
lc.execute(
"insert into favorites (_id,title,intent,container,screen,cellX,cellY,spanX,spanY,itemType,"
"appWidgetId,appWidgetProvider,restored,profileId,rank,options) values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
(r["_id"], r["title"], r["intent"], r["container"],
x if hotseat else (0 if in_folder else r["screen"]),
x, 0 if hotseat else y, int(r["spanX"]), int(r["spanY"]), r["itemType"],
r["appWidgetId"] if widget else -1, r["appWidgetProvider"],
FLAG_ID_NOT_VALID if widget else 0, 0, r["rank"], 0))
if r["itemType"] == ITEM_DEEP_SHORTCUT:
pins.append(re.search(r"S\.shortcut_id=([^;]+)", r["intent"]).group(1))
lc.commit()
print(f"{len(kept)} items written, pins: {pins}")
old = Path(icebox_xml).read_text()
blocks = [m.group(0) for m in re.finditer(r'<shortcut id="([^"]+)".*?</shortcut>\n', old, re.S) if m.group(1) in pins]
assert len(blocks) == len(pins), f"found {len(blocks)} of {len(pins)} shortcuts"
head = re.search(r"<package name=[^>]*>\n", old).group(0)
(out / "icebox.xml").write_text("<?xml version='1.0' encoding='utf-8' standalone='yes' ?>\n" + head
+ package_info(*ICEBOX_INFO) + "".join(blocks) + "</package>\n")
(out / "lawnchair-pins.xml").write_text(
"<?xml version='1.0' encoding='utf-8' standalone='yes' ?>\n"
f'<launcher-pins package-name="{LAUNCHER}" launcher-user="0">\n' + package_info(*LAUNCHER_INFO)
+ f'<package package-name="{ICEBOX}" package-user="0">\n'
+ "".join(f'<pin value="{p}" />\n' for p in pins) + "</package>\n</launcher-pins>\n")
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Replace an app with a differently signed build while keeping its data, permissions,
# installer and frozen state. Data is also saved to BACKUP_DIR before the uninstall.
# Usage: reinstall-keep-data.sh PACKAGE BACKUP_DIR APK [APK ...]
set -euo pipefail
pkg=$1 bak=$2; shift 2
a() { adb shell "$@" </dev/null; }
su_() { a "su -c '$1'"; }
mkdir -p "$bak"
installer=$(a "pm list packages -3 -i $pkg" | tr -d '\r' | grep "package:$pkg " | sed 's/.*installer=//')
disabled=$(a "pm list packages -d $pkg" | tr -d '\r' | grep -cx "package:$pkg" || true)
perms=$(a "dumpsys package $pkg" | sed -n '/runtime permissions:/,/^ *[a-z]* .*:$/p' | grep -oE '[a-zA-Z0-9_.]+: granted=true' | cut -d: -f1 | sort -u | tr '\n' ' ')
echo "$pkg: installer=$installer disabled=$disabled perms=[$perms]"
dirs="/data/data/$pkg /data/user_de/0/$pkg"
[ "$(su_ "[ -d /data/media/0/Android/data/$pkg ] && echo y")" = y ] && dirs="$dirs /data/media/0/Android/data/$pkg"
su_ "am force-stop $pkg"
# The phone writes the tar to stdout; exec-out keeps it binary-safe.
adb exec-out "su -c 'tar -cf - --exclude=$pkg/cache --exclude=$pkg/code_cache $dirs 2>/dev/null'" > "$bak/$pkg.tar"
echo "saved $(stat -c %s "$bak/$pkg.tar") bytes to $bak/$pkg.tar"
[ "$(stat -c %s "$bak/$pkg.tar")" -gt 10240 ] || { echo "backup suspiciously small, aborting"; exit 1; }
a "pm uninstall $pkg"
iopt=(); [ -n "$installer" ] && [ "$installer" != null ] && iopt=(-i "$installer")
adb install-multiple -r "${iopt[@]}" --bypass-low-target-sdk-block "$@"
uid=$(su_ "stat -c %u /data/data/$pkg")
cache_gid=$((uid - 10000 + 20000))
adb exec-in "su -c 'tar -xf - -C /'" < "$bak/$pkg.tar"
su_ "while pgrep -x tar >/dev/null; do sleep 0.3; done"
for d in /data/data/$pkg /data/user_de/0/$pkg; do
su_ "chown -R $uid:$uid $d; for c in cache code_cache; do mkdir -p $d/\$c; chown $uid:$cache_gid $d/\$c; chmod 2771 $d/\$c; done; chcon -hR \"\$(stat -c %C $d)\" $d"
done
case " $dirs " in *Android/data*)
su_ "d=/data/media/0/Android/data/$pkg; chown -R $uid:1078 \$d; find \$d -type d -exec chmod 2770 {} +; find \$d -type f -exec chmod 660 {} +; chcon -R u:object_r:media_rw_data_file:s0 \$d" ;;
esac
for p in $perms; do a "pm grant $pkg $p" 2>/dev/null || true; done
[ "$disabled" = 1 ] && su_ "pm disable-user --user 0 $pkg"
echo "$pkg: reinstalled as uid $uid, $(a "dumpsys package $pkg" | grep -m1 versionName | xargs)"
+144
View File
@@ -0,0 +1,144 @@
#!/usr/bin/env python3
"""Restore user apps, app data and permissions onto a freshly flashed, rooted phone.
Sources (from the backup folder):
apks/<pkg>/*.apk APK + splits
rooted-fresh/dumpsys-package.txt installer, granted runtime permissions
rooted-fresh/packages-3.txt which packages are user apps
rooted-fresh/packages-disabled.txt frozen/disabled state
rooted/apps-manifest.json special app-ops (overlay, all-files, ...)
<work>/data/<pkg>, <work>/user_de/0/<pkg> unpacked data-fresh.tar
The device comes from ANDROID_SERIAL, as for adb. Usage: restore.py BACKUP_DIR WORK_DIR [--data-only] [pkg ...] (no pkgs = all user apps)
--data-only only put the data back (apps already installed and updated): keeps the current APK,
installer, permissions and frozen state. Never re-run a full restore over updated apps.
Every action and its result is appended to WORK_DIR/restore.log.
"""
import json
import re
import shlex
import subprocess
import sys
import time
from pathlib import Path
SKIP = {"com.topjohnwu.magisk"} # installed fresh, its data must not be reused
# Dropped on restore: regenerated by the app, and the bulk of most backups.
VOLATILE = ["cache", "code_cache", "app_webview/Default/Cache", "app_webview/Default/Code Cache"]
backup, work = Path(sys.argv[1]), Path(sys.argv[2])
data_only = "--data-only" in sys.argv[3:]
only = set(sys.argv[3:]) - {"--data-only"}
log_file = (work / "restore.log").open("a")
def log(msg: str) -> None:
line = f"{time.strftime('%H:%M:%S')} {msg}"
print(line, flush=True)
log_file.write(line + "\n")
log_file.flush()
def adb(*args: str, timeout: int = 300, stdin=None) -> subprocess.CompletedProcess:
return subprocess.run(["adb", *args], capture_output=True, text=stdin is None,
timeout=timeout, stdin=stdin)
def su(cmd: str, timeout: int = 300) -> str:
r = adb("shell", f"su -c {shlex.quote(cmd)}", timeout=timeout)
return (r.stdout + r.stderr).strip()
def parse_dumpsys(text: str) -> dict:
"""installer + granted runtime permissions per package."""
out = {}
for block in re.split(r"\n Package \[", text)[1:]:
name = block.split("]", 1)[0]
installer = re.search(r"\n installerPackageName=(\S+)", block)
runtime = block.split("runtime permissions:", 1)
perms = re.findall(r"\n ([\w.]+): granted=true", runtime[1]) if len(runtime) > 1 else []
out[name] = {"installer": installer.group(1) if installer else None, "perms": sorted(set(perms))}
return out
fresh = backup / "rooted-fresh"
meta = parse_dumpsys((fresh / "dumpsys-package.txt").read_text(errors="replace"))
manifest = json.loads((backup / "rooted/apps-manifest.json").read_text())["apps"]
user_pkgs = sorted(set(re.findall(r"=([\w.]+)\s+installer=", (fresh / "packages-3.txt").read_text())) - SKIP)
disabled = set(re.findall(r"package:([\w.]+)", (fresh / "packages-disabled.txt").read_text()))
targets = [p for p in user_pkgs if not only or p in only]
log(f"=== restore start: {len(targets)} packages")
# Pass 1: install every APK set, so installers like Obtainium exist before pass 2 names them.
for pkg in [] if data_only else targets:
apks = sorted(str(p) for p in (backup / "apks" / pkg).glob("*.apk"))
if not apks:
log(f"{pkg}: NO APK, skipped")
continue
# --bypass-low-target-sdk-block: Android 14+ refuses targetSdk < 24 (Automagic workarounds).
r = subprocess.run(["adb", "install-multiple", "-r", "-d", "-g",
"--bypass-low-target-sdk-block", *apks],
capture_output=True, text=True, timeout=900)
log(f"{pkg}: install {'OK' if 'Success' in r.stdout else 'FAILED ' + (r.stdout + r.stderr).strip()[-200:]}")
installed = set(re.findall(r"package:([\w.]+)", adb("shell", "pm list packages").stdout))
# Pass 2: data, ownership, SELinux labels, installer, permissions, disabled state.
for pkg in targets:
if pkg not in installed:
continue
info = meta.get(pkg, {})
su(f"am force-stop {pkg}")
uid = su(f"stat -c %u /data/data/{pkg}")
if not uid.isdigit():
log(f"{pkg}: no data dir after install ({uid}), data skipped")
continue
cache_gid = int(uid) - 10000 + 20000 # u0_aNNN_cache
members = [m for m in (f"data/{pkg}", f"user_de/0/{pkg}") if (work / m).is_dir()]
if members:
for m in members:
su(f"find /data/{m} -mindepth 1 -maxdepth 1 ! -name lib -exec rm -rf {{}} +")
excludes = [f"--exclude={m}/{v}" for m in members for v in VOLATILE]
tar = subprocess.Popen(["tar", "-cf", "-", "-C", str(work), *excludes, *members],
stdout=subprocess.PIPE)
r = adb("exec-in", "su -c 'tar -xf - -C /data'", stdin=tar.stdout, timeout=1800)
tar.wait()
# exec-in returns once our side is sent; the phone's tar may still be writing.
su("while pgrep -x tar >/dev/null; do sleep 0.3; done", timeout=600)
for m in members:
su(f"chown -R {uid}:{uid} /data/{m}; "
f"for d in cache code_cache; do mkdir -p /data/{m}/$d; chown {uid}:{cache_gid} /data/{m}/$d; "
# restorecon gives only app_data_file:s0; the per-app MLS categories that
# installd put on the top directory must be copied onto its contents.
f"chmod 2771 /data/{m}/$d; done; chcon -hR \"$(stat -c %C /data/{m})\" /data/{m}",
timeout=900)
wrong = su(f"find {' '.join('/data/' + m for m in members)} ! -user {uid} | wc -l")
log(f"{pkg}: data {'+'.join(members)} rc={r.returncode} uid={uid} wrong_owner={wrong}")
else:
log(f"{pkg}: no data in backup")
if data_only:
continue
inst = info.get("installer")
if inst and inst != "null" and inst in installed | {"com.android.vending"}:
log(f"{pkg}: installer {inst} -> {su(f'pm set-installer {pkg} {inst}') or 'OK'}")
# -g granted everything at install; revoke what was not granted before.
wanted = set(info.get("perms", []))
now = re.findall(r"\n\s+([\w.]+): granted=true", su(f"dumpsys package {pkg} | sed -n '/runtime permissions:/,$p'"))
revoked = [p for p in set(now) - wanted]
for p in revoked:
su(f"pm revoke {pkg} {p}")
for p in wanted - set(now):
su(f"pm grant {pkg} {p}")
ops = manifest.get(pkg, {}).get("specialAppOps", {})
for op, mode in ops.items():
su(f"appops set {pkg} {op} {mode}")
log(f"{pkg}: perms granted={len(wanted)} revoked={len(revoked)} appops={','.join(ops) or '-'}")
if pkg in disabled:
log(f"{pkg}: disable-user -> {su(f'pm disable-user --user 0 {pkg}')}")
log("=== restore done")
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
# Launch every user app with a launcher activity, one by one: unfreeze if frozen, cold start,
# wait, screenshot, collect its crash/ANR/keystore log lines, check it is alive and in front,
# then stop it and re-freeze. Writes OUT_DIR/results.tsv, OUT_DIR/<pkg>.png, OUT_DIR/<pkg>.log.
# Usage: test-all-apps.sh OUT_DIR [pkg ...]
set -uo pipefail
out=$1; shift
mkdir -p "$out"
a() { adb shell "$@" </dev/null; }
WAIT=${WAIT:-8}
if [ $# -gt 0 ]; then pkgs=("$@"); else
mapfile -t pkgs < <(a 'pm list packages -3' | tr -d '\r' | sed 's/package://' | sort)
fi
frozen=$(a 'pm list packages -d' | tr -d '\r' | sed 's/package://')
printf 'pkg\twas_frozen\tlaunch_ms\talive\tforeground\tcrash\tanr\tkeystore\tnote\n' > "$out/results.tsv"
for p in "${pkgs[@]}"; do
wf=0; grep -qx "$p" <<<"$frozen" && wf=1
[ $wf = 1 ] && a "su -c 'pm enable --user 0 $p'" >/dev/null
act=$(a "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" | tr -d '\r' | tail -1)
if [[ "$act" != */* ]]; then
[ $wf = 1 ] && a "su -c 'pm disable-user --user 0 $p'" >/dev/null
printf '%s\t%s\t-\t-\t-\t-\t-\t-\tno launcher activity\n' "$p" "$wf" >> "$out/results.tsv"; continue
fi
a 'logcat -c'
ms=$(a "am start -W -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" | awk '/TotalTime/{print $2}')
sleep "$WAIT"
adb exec-out screencap -p > "$out/$p.png" </dev/null
pid=$(a "pidof $p" | tr -d '\r')
fg=$(a 'dumpsys activity activities' | grep -m1 -E 'topResumedActivity|mResumedActivity' | grep -c "$p/")
a 'logcat -d' > "$out/$p.log"
crash=$(grep -cE "FATAL EXCEPTION|Process: $p|Fatal signal" "$out/$p.log")
anr=$(grep -cE "ANR in $p" "$out/$p.log")
ks=$(grep -ciE "AEADBadTag|KeyStoreException|VERIFICATION_FAILED|KeyPermanentlyInvalidated|UnrecoverableKey|BadPaddingException|file is not a database|SQLiteDatabaseCorrupt" "$out/$p.log")
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t\n' "$p" "$wf" "${ms:-?}" "${pid:+yes}" "$fg" "$crash" "$anr" "$ks" >> "$out/results.tsv"
echo "$p frozen=$wf ms=${ms:-?} alive=${pid:+yes} fg=$fg crash=$crash anr=$anr ks=$ks"
a "am force-stop $p; input keyevent KEYCODE_HOME"
[ $wf = 1 ] && a "su -c 'pm disable-user --user 0 $p'" >/dev/null
done
+201
View File
@@ -0,0 +1,201 @@
---
name: software-vision
description: >-
Use this skill whenever the user describes an idea for a new piece of software, a tool, an app,
a service, a bot, a plugin, a mod, or an automation — "I want something that …", "build me a …",
"could we make a tool for …", "I need a service that …" — before choosing a language, scaffolding
a repository, or writing any code. Requires prior-art research first: web searches for existing
projects, forks, skeletons, libraries and what users say about them (reviews, complaints,
comparisons), only then merged with training-data recall as a secondary source, a
throwaway shallow clone of promising candidates to check them against the requirements, and a
set of options (fork, depend on, extend, start fresh) for the user to choose from.
---
# Software Vision — Find What Exists Before Building
When the user has an idea for a piece of software, the first job is **not** to design it. It is
to find out what already exists that does all or most of it. Building a full stack from nothing
when a project covering 80% of the requirements is one `git clone` away wastes days, and the
result is usually worse than the thing that already had years of bug reports behind it.
This is the project-level form of the dependency ordering in
[`coding-dev.md` §3](skills-public/coding-dev.md): use what exists, fork what is close, write
your own only when nothing fits. That section governs individual subsystems; this one governs
the whole idea.
> [!IMPORTANT]
> **Never assume nothing exists.** However niche the idea sounds, someone may have built it,
> half-built it, abandoned it, drafted a skeleton, written a blog post about the protocol, or
> published research on the hard part. "I couldn't think of anything" is not a finding — a list
> of the searches you ran and what they turned up is.
---
## 1. When This Applies — and the Only Opt-Out
Run this whole process whenever the user describes software they want to exist: a new app,
CLI, service, bot, plugin, mod, integration, library, or automation — whether it is a fresh
repository or a large new component inside an existing one.
**The only way to skip it is an explicit opt-out from the user**, such as "no online research",
"don't look at other projects", or "I want to write this from scratch myself". Then:
- Skip the web searches and the clones (§3–§4).
- Still use the libraries rule in [`coding-dev.md` §3](skills-public/coding-dev.md) for
individual subsystems unless the user excluded that too.
- Do not argue for research afterwards; the user has already decided.
Urgency, a small-sounding scope, or "it's just a quick script" are **not** opt-outs. A few
searches take a minute; reinventing a maintained tool takes far longer.
---
## 2. Pin Down the Requirements First
Before searching, turn the user's description into a short checklist, because that list is what
every candidate gets measured against:
- **Must-haves** — what the software has to do to be useful at all.
- **Nice-to-haves** — features the user mentioned that are not deal-breakers.
- **Constraints** — platform (Linux, Windows, Wine, Android, a game engine), runtime, language
preferences, self-hosted vs cloud, license needs, offline use, performance.
Show the checklist to the user with the results (§6) so they can correct it. If a requirement
is genuinely ambiguous *and* would change which candidates qualify, ask — otherwise state the
assumption and carry on.
---
## 3. Search First, Then Merge in Recall
The searches happen anyway, so they come **first**, and the decision rests on what they show.
Training data can be months or years out of date: projects get abandoned, renamed, relicensed,
or overtaken, and newer ones may never have been seen at all. Starting from recall anchors the
whole evaluation on a stale picture, so recall comes second and only fills gaps.
### 3.1 Web searches first
**Run at least three to five searches**, varied in phrasing and source, before settling on any
candidate. Cover:
| Where | What it finds |
| :--- | :--- |
| General web search with the problem phrased several ways ("self-hosted X", "open source alternative to Y", "X for Linux") | Products, blog posts, comparisons |
| GitHub / GitLab / Codeberg search, topics, and `awesome-*` lists | Projects, forks, skeletons, abandoned prototypes |
| The package registry for the likely language (crates.io, npm, PyPI, NuGet, pkg.go.dev, …) | Libraries that do the hard part |
| Forums and communities (Reddit, Hacker News, the domain's own Discord/forum, issue trackers of adjacent projects) | "Does anything do X?" threads, half-finished attempts, known dead ends |
| Papers, specs, reverse-engineering notes for niche domains | The research behind the hard part, even if nobody shipped an app |
Search for **parts** as well as the whole: an idea with no complete match often splits into
pieces that each have a mature library or tool.
For each promising hit, note what can be checked without cloning: last commit or release date,
license, language, open issue count and how maintainers respond, stars or downloads as a weak
signal, and whether its README claims the must-haves.
### 3.2 What people say about it
A project's own README describes what it *claims* to do. What users say shows how it actually
holds up, so search for opinions on every serious candidate too: reviews, "X vs Y" comparisons,
Reddit and Hacker News threads, forum posts, and the complaints in its issue tracker. Look for:
- Recurring pain points (crashes, data loss, missing features people keep requesting)
- Whether people are moving to it or away from it, and to what
- How the maintainers respond to bugs and pull requests
- Workarounds people rely on, which often point at the gaps a fork would need to fill
Weight recent opinions over old ones, and many independent voices over a single loud one. Put
this in the comparison (§6) next to the technical facts.
### 3.3 Then merge in recall
Only after searching, add what you already know from training data: projects, products,
libraries, and standards that the searches missed. Every recalled item must be checked with a
quick search before it goes into the comparison (does it still exist, is it maintained, has the
license changed?). Where recall and current sources disagree, **current sources win.**
---
## 4. Clone the Promising Candidates — Then Delete Them
When a candidate looks close but it is unclear whether it ticks every must-have, **look inside
it** rather than guessing from the README.
1. **Shallow-clone into a throwaway directory** — the session's scratchpad or a fresh `mktemp -d`,
never inside a project tree:
```bash
dir=$(mktemp -d) && git clone --depth 1 <url> "$dir/<name>"
```
2. **Read, don't run.** A clone is untrusted code. Inspect the README, directory layout, entry
points, dependency manifest, build files, tests, and open issues. Do not run its install
scripts, build, or test suite as part of evaluation — that is a decision for after the user
picks it.
3. **Map it against the checklist from §2**: which must-haves it covers, which are partial, which
are missing, and roughly how much work each gap is. Note the architecture (does it have a
plugin/extension point where the gaps could go?), code quality, and the license.
4. **Delete the clone when done.** Nothing from evaluation stays on disk or in a project. If the
user later chooses to fork it, that is a fresh, deliberate clone or fork in the right place.
Clone only the few candidates that are genuinely in contention, not every search hit.
---
## 5. Pick Likely Libraries
Whether the plan ends up as a fork or a fresh start, identify the libraries the work would lean
on, for the language that fits best (the existing candidate's language if forking, otherwise the
best fit for the constraints and the user's preferences). For each major subsystem — protocol,
file format, UI toolkit, storage, networking, parsing — name one or two maintained candidates
and why. The per-language rules under
[`code-languages/`](skills-public/code-languages/) apply to whatever is chosen.
---
## 6. Present Options — the User Chooses
Report back **before** scaffolding or writing code, with:
1. **The requirements checklist** (§2), so the user can correct it.
2. **What was searched** — the queries and sources, briefly, so an empty result is credible.
3. **A comparison of the real candidates**:
| Candidate | Covers | Missing / partial | License | Health | User opinion | Fit |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
4. **Options**, typically some of:
- **Use it as is** — it already does the job; maybe configure or script around it.
- **Extend it** — via its plugin/extension API, with no fork.
- **Fork and modify** — close, but the gaps need changes to its core.
- **Contribute upstream** — the gaps are useful to everyone; send them as PRs (then the
project's own rules win, per the precedence table in
[`coding-dev.md`](skills-public/coding-dev.md)).
- **Combine** — several existing tools or libraries glued together.
- **Start fresh** — on top of the libraries from §5, with a short, explicit reason why none of
the candidates were a usable base.
5. **A recommendation**, with the reason — but the decision is the user's. Do not quietly start
on the recommended option. When the user has said to work autonomously and decide without
them, take the recommendation and record the choice and the alternatives where they will see
it.
If nothing usable exists, say so plainly, list what was checked, and still mention near misses
worth borrowing ideas, formats, or code from (license permitting).
**Keep hosting out of the proposal.** The options are about *what software* to run, not how to
deploy it. Containers, proxies, tunnels, extra helper services, and the database layout belong to
the environment's own infrastructure skill, which gets read once an option is chosen and before
anything is deployed. Note a hard requirement that affects the choice (it needs Postgres, it
needs a GPU) and stop there.
---
## 7. Checklist
- [ ] Requirements turned into must-haves / nice-to-haves / constraints
- [ ] At least three to five varied web searches first, covering the whole idea and its parts
- [ ] User reviews, comparisons, and complaints gathered for each serious candidate
- [ ] Training-data recall merged in afterwards, each item checked against current sources
- [ ] Promising candidates shallow-cloned to a temp directory, read without running, mapped
against the checklist, then deleted
- [ ] Libraries identified for the likely language
- [ ] Options and a recommendation presented; nothing built before the user chose (or before
an autonomous decision was recorded)