docs: add software-vision and android-backup-restore skills with android scripts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
BluscreamandClaude Opus 5.5 committed
1 parent
a16a879b1e
commit
5e6abadb95
12 files changed
+1261
-1
No files matched your search
@@ -9,8 +9,10 @@ A collection of public, reusable agent skills, cheatsheets, and operational work
|
|||||||
| Skill File | Scope / Topics Covered |
|
| Skill File | Scope / Topics Covered |
|
||||||
| :--- | :--- |
|
| :--- | :--- |
|
||||||
| **[`coding-dev.md`](skills-public/coding-dev.md)** | Standing rules for all code work: maximum linter/compiler strictness, never filtering build output, reuse-before-reinvent dependency ordering, file/function size limits, test & regression discipline, verification gates, scope control, commit hygiene, and honest reporting. |
|
| **[`coding-dev.md`](skills-public/coding-dev.md)** | Standing rules for all code work: maximum linter/compiler strictness, never filtering build output, reuse-before-reinvent dependency ordering, file/function size limits, test & regression discipline, verification gates, scope control, commit hygiene, and honest reporting. |
|
||||||
|
| **[`software-vision.md`](skills-public/software-vision.md)** | Starting from a software idea: turn it into a requirements checklist, search the web first (projects, forks, skeletons, libraries, user reviews), then merge in training-data recall checked against current sources, shallow-clone promising candidates to a temp dir to check them, then present use / extend / fork / start-fresh options before building anything. |
|
||||||
| **[`cli-tooling.md`](skills-public/cli-tooling.md)** | Using command-line tools correctly: reading the help tree instead of guessing flags, preferring an existing CLI over an ad-hoc script, non-interactive invocation, machine-readable output formats, orienting on prior work before starting, script placement & symlinking, and care around destructive commands. |
|
| **[`cli-tooling.md`](skills-public/cli-tooling.md)** | Using command-line tools correctly: reading the help tree instead of guessing flags, preferring an existing CLI over an ad-hoc script, non-interactive invocation, machine-readable output formats, orienting on prior work before starting, script placement & symlinking, and care around destructive commands. |
|
||||||
| **[`afk-sleep-work.md`](skills-public/afk-sleep-work.md)** | Operating unattended and autonomously: continuous forward momentum, avoiding polling traps and deadlocks, host preservation and resource budgeting (disk, packages, processes), isolated tool provisioning, avoiding "last breath" traps on token/quota exhaustion, bypassing or driving GUI launchers, and wake-up checkpoints. |
|
| **[`afk-sleep-work.md`](skills-public/afk-sleep-work.md)** | Operating unattended and autonomously: continuous forward momentum, avoiding polling traps and deadlocks, host preservation and resource budgeting (disk, packages, processes), isolated tool provisioning, avoiding "last breath" traps on token/quota exhaustion, bypassing or driving GUI launchers, and wake-up checkpoints. |
|
||||||
|
| **[`android-backup-restore.md`](skills-public/android-backup-restore.md)** | Extracting/backing up/restoring Android data by access tier before a wipe or migration: EDL/firehose & bootloader, custom recovery (TWRP/OrangeFox/CWM), root, device-owner, Shizuku, plain ADB, and on-device apps; FBE decryption limits, per-partition dumps (userdata, EFS/modem/IMEI, persist), SELinux/UID-correct restores, the death of `adb backup`, and app-specific data (WhatsApp, Signal, TOTP/Keystore) that does not survive. |
|
||||||
|
|
||||||
### 🔤 Per-Language Rules ([`code-languages/`](skills-public/code-languages/))
|
### 🔤 Per-Language Rules ([`code-languages/`](skills-public/code-languages/))
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,562 @@
|
|||||||
|
---
|
||||||
|
name: android-backup-restore
|
||||||
|
description: >-
|
||||||
|
Use this skill when extracting, backing up, migrating, or restoring data from an Android
|
||||||
|
device — especially before a wipe, ROM flash, factory reset, or device-to-device move, and
|
||||||
|
when deciding what is salvageable at a given access level. Covers the full ladder of access
|
||||||
|
tiers (EDL/firehose & bootloader, custom recovery CWM/TWRP/OrangeFox, root, device-owner,
|
||||||
|
Shizuku, plain ADB, and on-device apps), file-based-encryption decryption limits, per-partition
|
||||||
|
dumps (userdata, EFS/modem/IMEI, persist), SELinux/UID-correct restores, the death of
|
||||||
|
`adb backup`, and app-specific data (WhatsApp, Signal, TOTP/Keystore) that does not survive.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Android Backup & Restore — Extraction by Access Level
|
||||||
|
|
||||||
|
How to pull as much as possible off an Android device before wiping, flashing, or migrating —
|
||||||
|
and how to put it back. Organised as a **ladder of access tiers**: the more privileged the
|
||||||
|
access, the more you can extract. Work out which tier you have, pull from the highest tier
|
||||||
|
available, and fall back down the ladder for whatever that tier cannot reach.
|
||||||
|
|
||||||
|
This is a *data-recovery and migration* skill for devices you own or are authorised to service.
|
||||||
|
Shell orchestration here follows [`bash.md`](skills-public/code-languages/bash.md) (strict mode,
|
||||||
|
hard `timeout`s, no wait-loops); CLI invocation follows
|
||||||
|
[`cli-tooling.md`](skills-public/cli-tooling.md) (read `<tool> --help` before trusting any flag
|
||||||
|
below — tool surfaces drift, and the exact partition/block paths are **per-device**).
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> **The single biggest variable is encryption.** Every modern Android device uses File-Based
|
||||||
|
> Encryption (FBE); the `/data` partition is a ciphertext blob until something that holds the
|
||||||
|
> user credential derives the keys. That "something" is normally the booted OS. A custom
|
||||||
|
> recovery can only read plaintext `/data` if it was *built to decrypt your exact Android
|
||||||
|
> version* and you type the lockscreen PIN/password into it. See §3. This is why "boot it
|
||||||
|
> normally and pull over ADB/root" beats "boot to recovery and dd" whenever the device still
|
||||||
|
> boots.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. The Access Ladder (decision table)
|
||||||
|
|
||||||
|
Pick the highest row you can reach. Each tier can extract everything below it plus its own column.
|
||||||
|
|
||||||
|
| Tier | Access you need | Reaches raw partitions? | Reaches decrypted `/data/data` app data? | Reaches EFS/IMEI/persist? | Notes |
|
||||||
|
| :--- | :--- | :--- | :--- | :--- | :--- |
|
||||||
|
| **EDL / firehose / BROM** (§8) | Vendor edl/mtk loader, device in 9008/BROM | ✅ full flash, offline | ⚠️ only as ciphertext (still FBE-locked) | ✅ | Last resort for dead/bricked devices; dumps are encrypted blobs unless you also have the keys. |
|
||||||
|
| **Bootloader / fastboot** (§8) | Unlocked bootloader | ⚠️ some (`fastboot fetch` on newer; mostly flash-only) | ❌ | partial | Mostly a *flashing* interface, not a dumping one. |
|
||||||
|
| **Custom recovery** TWRP/OrangeFox/CWM (§3) | Unlocked bootloader + matching recovery | ✅ every partition via `dd`/built-in backup | ⚠️ **only if recovery decrypts your FBE** + you enter the PIN | ✅ | Classic "nandroid". Decryption is the catch on Android 10+. |
|
||||||
|
| **Root (booted OS)** (§4) | Magisk/KernelSU/APatch on a booting device | ✅ (block devices readable) | ✅ **fully decrypted** (OS is running) | ✅ | The best tier for a *working* device — plaintext data + raw partitions together. |
|
||||||
|
| **Device Owner / DPC** (§5) | `dpm set-device-owner` on a fresh/unprovisioned device | ❌ | ❌ (own managed scope only) | ❌ | Niche; grants/denies permissions, reads some providers; not a bulk dumper. |
|
||||||
|
| **Shizuku / ADB-shell privilege** (§6) | Shizuku (wireless-debug or one-time ADB) | ❌ | ⚠️ APKs, `Android/data`, `Android/obb`, media — **not** `/data/data` of release apps | ❌ | `adb shell` UID (2000) reach, on-device, no PC tether. |
|
||||||
|
| **Plain ADB** (§6) | USB debugging authorised | ❌ | ⚠️ same as Shizuku + `run-as` for *debuggable* apps only | ❌ | `adb backup` is effectively dead (§7). Pull storage, providers, bugreport. |
|
||||||
|
| **On-device app, no root** (§6.4) | Just the unlocked phone | ❌ | ❌ (own data only) | ❌ | Vendor cloud/D2D, Seedvault, per-app export. Last-mile for the non-technical case. |
|
||||||
|
|
||||||
|
**Rule of thumb for a pre-wipe sweep:** if it boots and you can unlock it → get **root** (§4) if
|
||||||
|
possible, else **ADB** (§6). If it does *not* boot → **custom recovery** (§3), else **EDL** (§8).
|
||||||
|
Always grab **EFS/IMEI/persist** (§4.3) before any flash regardless of tier.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. First: inventory what is actually worth saving
|
||||||
|
|
||||||
|
Before dumping 128 GB of userdata, enumerate. Most of the value is small and specific.
|
||||||
|
|
||||||
|
- **Media & documents** — `/sdcard` (a.k.a. `/storage/emulated/0`): DCIM, Pictures, Download,
|
||||||
|
Documents, Music, ringtones. Reachable at *every* tier including plain ADB. Pull this first; it
|
||||||
|
is the easy win and the bulk of what users actually miss.
|
||||||
|
- **App data** — `/data/data/<pkg>` and `/data/user/0/<pkg>` (credential-encrypted, "CE") and
|
||||||
|
`/data/user_de/0/<pkg>` (device-encrypted, "DE"). Needs root or recovery-decrypt. This is where
|
||||||
|
game saves, app logins, notes, 2FA seeds live.
|
||||||
|
- **Scoped external app data** — `Android/data/<pkg>` and `Android/obb/<pkg>` on `/sdcard`.
|
||||||
|
Blocked from ordinary file managers since Android 11/13 but reachable via ADB/Shizuku/root.
|
||||||
|
- **Telephony/PIM** — SMS/MMS, call log, contacts, calendar. See §6.2 (providers) and §9.
|
||||||
|
- **Identity partitions** — EFS/modem (`modemst1`, `modemst2`, `fsg`), `persist`, `efs`. IMEI,
|
||||||
|
Wi-Fi/BT MAC, sensor calibration. §4.3. **Irreplaceable — back up before any flash.**
|
||||||
|
- **The things that will NOT survive** no matter the tier — hardware-backed Keystore keys, and
|
||||||
|
by extension many 2FA/banking apps. §10. Plan migrations for these *before* wiping.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Custom Recovery (TWRP / OrangeFox / CWM) — the offline nandroid
|
||||||
|
|
||||||
|
Use when the device will not boot normally, or you want a block-level image independent of the OS.
|
||||||
|
Requires an **unlocked bootloader** (itself a wipe on most devices — so unlock *before* you have
|
||||||
|
data worth keeping, or accept the loss). Boot without flashing when you can:
|
||||||
|
`fastboot boot twrp.img` leaves the installed recovery untouched.
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> **Try the *installed* recovery before downloading or temp-booting one.** `adb reboot recovery`,
|
||||||
|
> then `adb shell id` — many custom ROMs ship a current OrangeFox/TWRP whose `adbd` already runs as
|
||||||
|
> **root** and, crucially, is new enough to **decrypt FBE on the installed Android version** (so
|
||||||
|
> `/data/data` shows real package names and readable file contents). This beats `fastboot boot`-ing
|
||||||
|
> an old recovery image: a recovery built from a much older branch often won't boot on a new ROM at
|
||||||
|
> all (its kernel panics and the device falls back to the OS — you'll land back in Android as
|
||||||
|
> `uid=2000(shell)`, the tell-tale that it didn't take). Confirm decryption with
|
||||||
|
> `ls /data/data` (real names, not hex) and by reading one `*/shared_prefs/*.xml`.
|
||||||
|
|
||||||
|
### 3.1 The decryption problem (read this before trusting a backup)
|
||||||
|
|
||||||
|
TWRP/OrangeFox can only back up **plaintext** `/data` if its decryption code matches your device's
|
||||||
|
FBE implementation *and* you enter the lockscreen credential in recovery (Keymaster derives the
|
||||||
|
keys from it). Reality on Android 10+:
|
||||||
|
|
||||||
|
- Use a recovery **built from the branch matching your Android version** (e.g. an `android-12.1`+
|
||||||
|
build for Android 12+). A recovery built from an old branch shows `/data` as garbage folder names
|
||||||
|
or fails to mount.
|
||||||
|
- Devices that **shipped on Android ≤11 and were upgraded** to 12+ often cannot be decrypted by any
|
||||||
|
TWRP build — the keystore/Keymaster path changed. OrangeFox sometimes succeeds where TWRP fails;
|
||||||
|
try both. If neither decrypts, you can still image the partition but only as ciphertext (useless
|
||||||
|
without the keys).
|
||||||
|
- If decryption fails, **do not factory-reset to "fix" it** — that destroys the keys and the data.
|
||||||
|
|
||||||
|
Verify decryption worked before relying on it: in the TWRP terminal, `ls -la /data/data` should
|
||||||
|
show real package names (`com.whatsapp`, …), not hex blobs.
|
||||||
|
|
||||||
|
### 3.2 Built-in backup vs. raw `dd`
|
||||||
|
|
||||||
|
- **Built-in backup** (TWRP "Backup" / OrangeFox): tick System, Data, Boot, EFS, etc. Produces a
|
||||||
|
nandroid set under `/sdcard/TWRP/BACKUPS/`. If `/data` decrypted, the Data archive holds plaintext
|
||||||
|
app data; if not, it backs up the raw (encrypted) partition. Pull the whole folder off afterwards.
|
||||||
|
- **Raw partition image** when you want a specific partition or the built-in backup won't cover it.
|
||||||
|
From the recovery `adb shell` or terminal — **block paths are per-device**, find them first:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# from recovery shell — enumerate by name, DO NOT guess the numbers
|
||||||
|
ls -l /dev/block/by-name/ # symlinks: userdata, boot_a, modemst1, efs, persist, …
|
||||||
|
cat /proc/partitions
|
||||||
|
|
||||||
|
# image a named partition to portable storage (not back onto /data)
|
||||||
|
timeout 1800 dd if=/dev/block/by-name/userdata of=/sdcard/userdata.img bs=8M
|
||||||
|
```
|
||||||
|
|
||||||
|
Pull with `adb pull /sdcard/userdata.img` (or write to an SD card / OTG drive). A raw `userdata`
|
||||||
|
image is only restorable/mountable if you also have the FBE keys — prefer the decrypted built-in
|
||||||
|
Data backup for app data, and reserve raw images for small partitions (§4.3).
|
||||||
|
|
||||||
|
### 3.3 Restore
|
||||||
|
|
||||||
|
Restore the built-in nandroid through the recovery UI, or `dd` a raw image back **only onto the
|
||||||
|
exact same partition of the same device** (never another device's EFS — §4.3). After a decrypted
|
||||||
|
Data restore across Android versions, expect some apps to misbehave; a clean reinstall + selective
|
||||||
|
data restore (§4.2) is often more reliable than a whole-partition nandroid across versions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Root on a booting device — the best tier for a live phone
|
||||||
|
|
||||||
|
If the device boots and you can unlock it, root (Magisk / KernelSU / APatch) is the strongest
|
||||||
|
position: the OS has already derived the FBE keys, so **all app data is plaintext**, and you still
|
||||||
|
have raw block-device access. (For installing root on this workstation's own test devices, Magisk
|
||||||
|
APKs/modules may already be on the NAS — see
|
||||||
|
[check-nas-before-downloading](check-nas-before-downloading.md) before fetching.)
|
||||||
|
|
||||||
|
### 4.1 Bulk app-data pull
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# as root, on device or via `adb shell su -c`
|
||||||
|
# tar preserves owners/modes/symlinks — plain `cp`/`adb pull` loses them (see §4.4)
|
||||||
|
su -c 'tar -cpf /sdcard/appdata-whatsapp.tar \
|
||||||
|
/data/data/com.whatsapp /data/user_de/0/com.whatsapp'
|
||||||
|
# then from the PC:
|
||||||
|
adb pull /sdcard/appdata-whatsapp.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
Pull `/data/data/<pkg>` (CE) **and** `/data/user_de/0/<pkg>` (DE) for each app — some apps split
|
||||||
|
state across both. For a whole-device grab, `tar` `/data/data`, `/data/user/0`, `/data/user_de/0`
|
||||||
|
and `/data/misc` (wifi configs, etc.), but mind disk budget on the pulling host.
|
||||||
|
|
||||||
|
> [!TIP]
|
||||||
|
> **Stream straight off the device with `adb exec-out` — no on-device staging.** When `/data` has
|
||||||
|
> little free space (you can't write a multi-GB tar back onto the partition you're reading), pipe
|
||||||
|
> the archive directly to the host, and `dd` images the same way (`exec-out` is binary-safe; plain
|
||||||
|
> `adb shell` mangles bytes):
|
||||||
|
> ```bash
|
||||||
|
> adb exec-out 'cd /data && tar -cp -f - data user_de system misc vendor adb' > data-full.tar
|
||||||
|
> adb exec-out 'dd if=/dev/block/by-name/boot status=none' > boot.img
|
||||||
|
> ```
|
||||||
|
> Toybox/busybox `tar` can't store SELinux contexts and exits non-zero when it skips sockets
|
||||||
|
> (`unknown file type '140000'` — harmless). Capture contexts separately as a restore sidecar:
|
||||||
|
> `for d in /data/data/* /data/user_de/0/*; do stat -c '%a %U %G %C %n' "$d"; done`. The per-app
|
||||||
|
> SELinux *categories* (`:c123,c456`) encode the app sandbox and are **not** regenerable by
|
||||||
|
> `restorecon` alone — this sidecar is how you put them back.
|
||||||
|
>
|
||||||
|
> **Record restore metadata while you have the device.** A `dumpsys package` capture gives, per
|
||||||
|
> app: `codePath` (→ system/vendor/product/`/data/app`=user), `pkgFlags` (SYSTEM/UPDATED_SYSTEM_APP),
|
||||||
|
> `installerPackageName` (Play `com.android.vending` vs sideloaded `null`), versionCode, and granted
|
||||||
|
> runtime permissions; `cmd appops dump` adds special ops (SYSTEM_ALERT_WINDOW,
|
||||||
|
> MANAGE_EXTERNAL_STORAGE, …). Parse these into a manifest so you can later restore *only* user apps
|
||||||
|
> and re-grant their exact permissions.
|
||||||
|
|
||||||
|
Prefer dedicated tools over hand-rolled tar when restoring to a *different* device or Android
|
||||||
|
version, because they record ownership/SELinux/permissions that a naive tar restore drops (§4.4):
|
||||||
|
|
||||||
|
- **Swift Backup** (root; Shizuku for a weaker subset) — app + data, permissions, SSAID.
|
||||||
|
- **Neo Backup** (FOSS, OAndBackupX fork; root incl. KernelSU/APatch) — app + data, schedules.
|
||||||
|
- **OAndBackupX / Migrate** — FOSS nandroid-style app+data sets.
|
||||||
|
|
||||||
|
### 4.2 Raw partitions while rooted
|
||||||
|
|
||||||
|
Same `dd` from `/dev/block/by-name/` as §3.2, but from the running OS (`su -c dd …`). Good for
|
||||||
|
grabbing `boot`/`vendor_boot` to patch, or EFS/persist (§4.3) without rebooting to recovery.
|
||||||
|
|
||||||
|
### 4.3 EFS / modem / IMEI / persist — back these up before ANY flash
|
||||||
|
|
||||||
|
These hold IMEI, baseband calibration, Wi-Fi/BT MAC, and sensor calibration. Corrupting them bricks
|
||||||
|
the radio permanently and there is no download that restores *your* unique values. **Partition names
|
||||||
|
and block numbers are per-device — look them up, never copy numbers from a guide.**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
su -c 'ls -l /dev/block/by-name/' | grep -Ei 'efs|modemst|fsg|fsc|persist|nvram|nvdata|protect'
|
||||||
|
# Qualcomm: commonly modemst1, modemst2, fsg, fsc, persist
|
||||||
|
# MediaTek: commonly nvram, nvdata, protect_f, protect_s, persist
|
||||||
|
for p in modemst1 modemst2 fsg persist; do
|
||||||
|
su -c "dd if=/dev/block/by-name/$p of=/sdcard/efs-$p.img bs=4M"
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
Pull the images **off the device and keep copies on a host/NAS** — a backup that lives only on the
|
||||||
|
partition you are about to wipe is no backup. Restore the same way (`dd … of=/dev/block/by-name/$p`)
|
||||||
|
**only to the identical partition on the identical device**. Never flash one phone's EFS onto
|
||||||
|
another — you would overwrite its unique identity with useless values.
|
||||||
|
|
||||||
|
### 4.4 Restoring raw `/data/data` tars correctly (UID + SELinux)
|
||||||
|
|
||||||
|
A tar of `/data/data/<pkg>` restored by hand will *not* launch unless you fix two things Android
|
||||||
|
assigns at install time:
|
||||||
|
|
||||||
|
1. **Install the app first** (same package, ideally same version) so Android creates the data dir
|
||||||
|
with the correct per-app UID and SELinux context.
|
||||||
|
2. Restore the files, then **fix ownership and labels**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# find the app's real UID from the freshly-created dir, don't assume 100xx
|
||||||
|
su -c 'ls -ld /data/data/com.example' # e.g. u0_a142 u0_a142
|
||||||
|
su -c 'tar -xpf /sdcard/appdata-example.tar -C /'
|
||||||
|
su -c 'chown -R u0_a142:u0_a142 /data/data/com.example'
|
||||||
|
su -c 'restorecon -RFD /data/data/com.example' # argument order matters; -v to see work
|
||||||
|
```
|
||||||
|
|
||||||
|
Then reboot before launching. Skipping the UID chown or `restorecon` is the usual reason a
|
||||||
|
"successful" restore produces an app that crashes on open. Dedicated backup apps (§4.1) do this
|
||||||
|
bookkeeping for you — hand-tar only when you must.
|
||||||
|
|
||||||
|
### 4.5 Getting root headlessly (no GUI taps) when the device boots
|
||||||
|
|
||||||
|
If the bootloader is unlocked and you need root to reach §4 but can't tap through the Magisk app,
|
||||||
|
patch and flash boot entirely from the shell:
|
||||||
|
|
||||||
|
1. **Get the *exact* stock boot.img.** The authoritative copy is the live partition — dump it from
|
||||||
|
a root recovery (§3) with `dd`/`exec-out`. Don't flash a boot.img of unknown origin: check its
|
||||||
|
cmdline first (`androidboot.hardware=qcom` = Qualcomm vs MediaTek's `bootopt=…` — a wrong-SoC
|
||||||
|
image bricks the device).
|
||||||
|
2. **Patch it** with Magisk's own tooling. The Magisk APK is a zip: its `assets/boot_patch.sh` needs
|
||||||
|
the arm64 libs renamed (`lib/arm64-v8a/libmagiskboot.so`→`magiskboot`, `libmagiskinit.so`→
|
||||||
|
`magiskinit`, `libmagisk.so`→`magisk`, `libinit-ld.so`→`init-ld`) plus `stub.apk` and
|
||||||
|
`util_functions.sh` in the same dir, then `sh boot_patch.sh boot.img` → `new-boot.img`.
|
||||||
|
- Set the flags **explicitly** (standalone `boot_patch.sh` treats unset as false, which can
|
||||||
|
bootloop): for a modern FBE device with a vbmeta partition and dynamic partitions use
|
||||||
|
`KEEPVERITY=true KEEPFORCEENCRYPT=true PATCHVBMETAFLAG=false RECOVERYMODE=false LEGACYSAR=false`.
|
||||||
|
`Failed to patch` on the kernel DTB step is benign (nothing to patch).
|
||||||
|
- The recovery installer (`META-INF/com/google/android/update-binary`, which auto-detects these
|
||||||
|
flags via `get_flags()`) is cleaner **but** fails outside a real sideload: it tries to mount
|
||||||
|
`/system`, which on super/dynamic-partition ROMs is `/dev/mapper/system` and won't mount in a
|
||||||
|
plain `adb shell`. `boot_patch.sh` with explicit flags sidesteps that.
|
||||||
|
3. **Flash** `new-boot.img` (verify it unpacks: `magiskboot unpack new-boot.img`) with
|
||||||
|
`fastboot flash boot` or, from a root recovery, `dd … of=/dev/block/by-name/boot`. Keep the stock
|
||||||
|
image for instant rollback. Reboot: `magiskd` in `ps -A` + `magisk -V` confirms root took and
|
||||||
|
survives reboot.
|
||||||
|
4. **Grant su to the ADB shell** (Magisk denies it by default, silently): Magisk app → Superuser
|
||||||
|
access = "Apps and ADB" + Automatic response = "Grant", or seed a `policies` row
|
||||||
|
(`uid=2000 policy=2`) in `/data/adb/magisk.db`. Until then `adb shell su -c …` returns
|
||||||
|
`Permission denied` even though root is installed and working.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Device Owner (DPC) — narrow, not a bulk extractor
|
||||||
|
|
||||||
|
A device-owner app (set with `adb shell dpm set-device-owner <pkg>/<receiver>` on a device with
|
||||||
|
**no accounts added yet**) gets elevated *policy* powers, not filesystem powers. It is relevant to
|
||||||
|
extraction only in that it can silently grant its own runtime permissions and read some content
|
||||||
|
providers that would otherwise prompt — useful for a managed enrolment flow, not for cleaning out a
|
||||||
|
phone before a wipe. Hard limits:
|
||||||
|
|
||||||
|
- `setPermissionGrantState()` for **sensor** permissions (location, camera, mic, body sensors,
|
||||||
|
activity recognition) is restricted/deprecated for *profile* owners on Android 12+ (silently
|
||||||
|
returns false); device owners retain more but the surface is shrinking each release.
|
||||||
|
- It cannot read other apps' `/data/data`, cannot image partitions, and cannot reach EFS.
|
||||||
|
|
||||||
|
If you find yourself reaching for device-owner to extract data, the real answer is almost always
|
||||||
|
root (§4) or ADB providers (§6.2) instead.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. ADB & Shizuku — no root, device boots
|
||||||
|
|
||||||
|
Plain ADB runs as the `shell` user (UID 2000). Shizuku grants that **same** privilege to on-device
|
||||||
|
apps (started via wireless debugging or a one-time ADB command), so their reach is identical to ADB
|
||||||
|
— handy when you have no PC.
|
||||||
|
|
||||||
|
### 6.1 External storage & scoped app dirs
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb pull /sdcard/ ./sdcard-backup/ # DCIM, Download, Documents, etc. — works everywhere
|
||||||
|
# Android/data and Android/obb are blocked from on-device file managers (11/13+) but
|
||||||
|
# the adb shell UID can still read them:
|
||||||
|
adb pull /sdcard/Android/data/ ./android-data/
|
||||||
|
adb pull /sdcard/Android/obb/ ./android-obb/
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.2 Content providers (telephony/PIM without root)
|
||||||
|
|
||||||
|
`adb shell content query` reaches providers the `shell` UID is allowed to read. What actually works
|
||||||
|
on a stock device is uneven:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb shell content query --uri content://contacts/phones # contacts — usually readable
|
||||||
|
adb shell content query --uri content://com.android.calendar/events
|
||||||
|
# SMS/MMS and call_log generally FAIL for the shell UID (need READ_SMS / be default app):
|
||||||
|
adb shell content query --uri content://sms # typically permission denied
|
||||||
|
```
|
||||||
|
|
||||||
|
For SMS/MMS, call logs, and contacts as restorable files, use an on-device FOSS app rather than
|
||||||
|
fighting providers over ADB:
|
||||||
|
|
||||||
|
- **SMS Import / Export** (`com.github.tmo1.sms_ie`, F-Droid) — SMS+MMS+call log+contacts to
|
||||||
|
JSON/ZIP, no root, and it *restores* too (set it as default SMS app during restore).
|
||||||
|
- **Open Android Backup** (mrrfv) — PC-driven, rootless, uses a companion app (not `adb backup`):
|
||||||
|
restores apps-without-data, internal storage, and contacts (vCard); exports SMS (CSV) and call
|
||||||
|
log (txt) as **view-only** (no restore, no MMS attachments).
|
||||||
|
- Contacts also export cleanly from the stock Contacts app ("Export to .vcf") or via Google account
|
||||||
|
sync — often the least-effort path for the non-technical case.
|
||||||
|
|
||||||
|
### 6.3 `run-as` — app data for *debuggable* apps only
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb shell run-as com.example.debug tar -cpf - ./ > example-data.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
`run-as` only works for apps with `android:debuggable="true"` (i.e. your own dev builds, not Play
|
||||||
|
Store releases). Shizuku-based file managers use the same `run-as` path and inherit the same limit.
|
||||||
|
Release-app `/data/data` is **not** reachable at this tier — that needs root (§4) or a
|
||||||
|
recovery-decrypt (§3).
|
||||||
|
|
||||||
|
### 6.4 Bugreport & app lists (cheap, always worth grabbing)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb bugreport ./bugreport.zip # logs, settings dump, installed-package metadata
|
||||||
|
adb shell pm list packages -3 -f # third-party packages + APK paths, to reinstall later
|
||||||
|
adb shell cmd package list packages --show-versioncode
|
||||||
|
```
|
||||||
|
|
||||||
|
To re-pull the APKs themselves for reinstall:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb shell pm path com.example # -> package:/data/app/.../base.apk + splits
|
||||||
|
adb pull /data/app/~~.../com.example-.../base.apk # repeat for each split; reinstall with
|
||||||
|
adb install-multiple base.apk split_config.*.apk # install-multiple
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.5 Vendor D2D and cloud (no-tech last mile)
|
||||||
|
|
||||||
|
- **Google** cloud backup + Pixel/Android cable "device-to-device" restore at setup. App data rides
|
||||||
|
along *only* if the Google **account** is transferred (Block Store is account-bound); untick
|
||||||
|
nothing under "Accounts". `allowBackup="false"` stops cloud backup but D2D still transfers app data
|
||||||
|
unless the app sets `dataExtractionRules`/`android:fullBackupOnly` to opt out.
|
||||||
|
- **Samsung Smart Switch** (PC or cable): backs up most app data, but **encrypted app data (e.g.
|
||||||
|
WhatsApp), banking logins, and sideloaded APKs do not transfer** — handle those per §9/§10.
|
||||||
|
- **Seedvault** (CalyxOS/GrapheneOS/LineageOS): the FOSS backup transport. On Android 15 builds its
|
||||||
|
**device-to-device** mode (Expert settings) is default-on and captures far more app data than the
|
||||||
|
old cloud-style transport, because it presents as a D2D migration to each app.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. The death of `adb backup` (don't build a plan on it)
|
||||||
|
|
||||||
|
`adb backup` was deprecated in Android 10 and gutted in Android 12 (API 31): app data is excluded by
|
||||||
|
default and only included if the app is **debuggable**, so on essentially all release apps across
|
||||||
|
Android 12–16 `adb backup` produces an archive with no app data. Treat it as unavailable:
|
||||||
|
|
||||||
|
- Don't rely on it for a migration. Use root tooling (§4.1), on-device FOSS exporters (§6.2), or
|
||||||
|
vendor/Seedvault D2D (§6.5) instead.
|
||||||
|
- `adb restore` likewise cannot restore encrypted backups and is a dead end for modern data.
|
||||||
|
- The `bmgr`/LocalTransport backup-manager plumbing still exists for *developers testing their own
|
||||||
|
backup agents* (`adb shell bmgr …`), but it is a per-app developer tool, not a device-wide
|
||||||
|
extractor — out of scope for a pre-wipe sweep.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. EDL / firehose / BROM — dead or bootloader-locked devices
|
||||||
|
|
||||||
|
The lowest level, for devices that will not boot to Android *or* recovery. These talk to the SoC's
|
||||||
|
emergency loader below the OS:
|
||||||
|
|
||||||
|
- **Qualcomm EDL (9008)** — [`bkerler/edl`](https://github.com/bkerler/edl) with a firehose
|
||||||
|
programmer (`.mbn`/`.elf`) matching the SoC. `edl printgpt` lists partitions; `edl rl dumps
|
||||||
|
--skip=userdata --genxml` dumps all partitions (minus the huge userdata) to a folder with a
|
||||||
|
`rawprogram` XML; `edl r <partition> <out.img>` grabs one. Newer SoCs need `--loader=<file>`
|
||||||
|
because loader autodetection no longer works.
|
||||||
|
- **MediaTek BROM** — [`bkerler/mtkclient`](https://github.com/bkerler/mtkclient). `./mtk printgpt`,
|
||||||
|
`./mtk rl out` dumps all partitions to `out/`, `./mtk r <partition> <out.img>` for one; `--preloader`
|
||||||
|
for BROM mode.
|
||||||
|
|
||||||
|
The hard limit: **these dumps are ciphertext.** EDL reads the raw flash below the OS, so `userdata`
|
||||||
|
comes out FBE-encrypted and is not decryptable without the keys (which live in secure hardware keyed
|
||||||
|
by the lockscreen credential). EDL is therefore excellent for **small identity partitions**
|
||||||
|
(EFS/persist/nvram — §4.3), full-flash forensic images, and un-bricking, but it is **not** a way to
|
||||||
|
read a locked phone's app data. Treat downloaded dumps/tools as untrusted input: keep each in its own
|
||||||
|
directory, pass paths as arguments, and run any helper scripts from outside the dump directory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. App-specific data that needs its own handling
|
||||||
|
|
||||||
|
Some high-value data lives behind app-level encryption and ignores the tier ladder:
|
||||||
|
|
||||||
|
- **WhatsApp** — local backup at `/sdcard/WhatsApp/Databases/msgstore.db.crypt15` (+ `/Media`). If
|
||||||
|
the user enabled **end-to-end encrypted backups**, decryption needs the **64-digit key** (or the
|
||||||
|
`encrypted_backup.key` file from `/data/data/com.whatsapp/files/`, root-only). Tools:
|
||||||
|
[`wa-crypt-tools`](https://pypi.org/project/wa-crypt-tools/). Easiest migration is WhatsApp's own
|
||||||
|
built-in transfer or Google-Drive backup with the same phone number — pull the `crypt15` + media
|
||||||
|
as insurance, not as the primary path.
|
||||||
|
- **Signal** — does **not** sync to any cloud. Migrate with Signal's in-app device-transfer, or its
|
||||||
|
encrypted local backup (Settings → Chats → Backups) whose **30-digit passphrase you must record** —
|
||||||
|
there is no recovery without it.
|
||||||
|
- **Authenticator / 2FA** — Google Authenticator exports via its QR "export accounts" flow;
|
||||||
|
**Aegis** (FOSS) exports an encrypted JSON vault. Do this *before* wiping; TOTP seeds not exported
|
||||||
|
are gone.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. What will NOT survive a wipe (plan around it)
|
||||||
|
|
||||||
|
No tier recovers these; the only mitigation is to migrate them *before* the wipe:
|
||||||
|
|
||||||
|
- **Hardware-backed Keystore keys** are non-exportable by design — key material never leaves the
|
||||||
|
TEE/StrongBox. Any app that pinned a device-bound key (banking apps, some password managers,
|
||||||
|
FIDO/passkeys stored in hardware) will treat the restored/new device as untrusted and re-enrol.
|
||||||
|
- **Play Integrity / SafetyNet attestation** fails on rooted or custom-ROM devices independently of
|
||||||
|
your data — so a perfectly-restored banking app may still refuse to run. That is an attestation
|
||||||
|
problem, not a backup problem; don't chase it as data loss.
|
||||||
|
- **SSAID / `ANDROID_ID`** changes on wipe (per app-signing-key + user), which can log apps out or
|
||||||
|
reset entitlements even when their `/data/data` is restored byte-perfect.
|
||||||
|
|
||||||
|
Before wiping: export 2FA (§9), confirm password-manager vaults sync to their cloud, record Signal
|
||||||
|
(§9) and WhatsApp keys, and re-enrol passkeys/banking on the new device rather than expecting a
|
||||||
|
restore to carry them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Pre-wipe checklist (the one-screen version)
|
||||||
|
|
||||||
|
1. **Decide the tier** (§1). Booting + unlockable → root (§4) or ADB (§6). Not booting → recovery
|
||||||
|
(§3) → EDL (§8).
|
||||||
|
2. **EFS/modem/persist first** (§4.3) — before any unlock or flash, off-device.
|
||||||
|
3. **Media & docs** — `adb pull /sdcard` (§6.1).
|
||||||
|
4. **App data** — root tar + dedicated tool (§4.1), or recovery-decrypted Data backup (§3).
|
||||||
|
5. **Scoped dirs** — `Android/data`, `Android/obb` (§6.1).
|
||||||
|
6. **PIM** — SMS/call-log/contacts via SMS Import-Export or vCard (§6.2).
|
||||||
|
7. **App-specific** — WhatsApp/Signal/2FA via their own flows (§9).
|
||||||
|
8. **App list + APKs** — `pm list packages -3`, pull splits (§6.4) to rebuild the app set.
|
||||||
|
9. **System state the app tars miss** (§12.1) — keystore DB, SSAIDs, URI grants, doze whitelist,
|
||||||
|
app-ops, launcher/shortcuts/QS/controls, Wi-Fi, SD-card UUIDs, root grants, old `packages.list`.
|
||||||
|
10. **Baseline + app-sweep** the old ROM (`measure.sh`, `test-all-apps.sh`, §12.4) so the new one
|
||||||
|
has something to be compared against.
|
||||||
|
11. **Passwords apps keep themselves** — App Manager's KeyStore password, backup-app passphrases,
|
||||||
|
Signal/WhatsApp keys (§9). Write them down *before* the wipe; none are recoverable after.
|
||||||
|
12. **Verify the backups off-device** (open the archive, check `ls /data/data` really decrypted)
|
||||||
|
*before* you type `fastboot -w` or confirm the reset. A backup you have not read back is not a
|
||||||
|
backup.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Restoring onto a new ROM (root) — what a data tar does not carry
|
||||||
|
|
||||||
|
Learned from a full LineageOS-based A16 → Infinity-X A17 switch (Redmi Note 7, Magisk). Restoring
|
||||||
|
`/data/data` + `user_de` + `Android/data` per §4.4 brings back most apps, but **every app gets a new
|
||||||
|
UID** on the new ROM, and a lot of state is keyed by UID or lives in system files. The scripts in
|
||||||
|
[`scripts/android/`](skills-public/scripts/android/) automate the parts below; they take the
|
||||||
|
device from `ANDROID_SERIAL` like `adb` itself.
|
||||||
|
|
||||||
|
### 12.1 Grab these too, before the wipe (all root, all small)
|
||||||
|
|
||||||
|
| What | Where | Why it matters on restore |
|
||||||
|
| :--- | :--- | :--- |
|
||||||
|
| Old UID map | `/data/system/packages.list` | Remap everything keyed by UID (SSAID, keystore, grants) |
|
||||||
|
| Keystore2 DB | `/data/misc/keystore/persistent.sqlite` | Apps whose keys vanish show "Keystore error"/log out (§12.3) |
|
||||||
|
| SSAIDs | `/data/system/users/0/settings_ssaid.xml` | `ANDROID_ID` change logs apps out (GitHub, Bolt, Lovense, …) |
|
||||||
|
| Persistent SAF grants | `/data/system/urigrants.xml` | Folder pickers (music players, camera, sync apps) lose their folders |
|
||||||
|
| Doze whitelist | `/data/system/deviceidle.xml` | Notification/automation apps silently stop in background |
|
||||||
|
| App-ops & special access | `cmd appops dump`, `cmd notification` policy access, `dumpsys package` | `MANAGE_EXTERNAL_STORAGE`, usage stats, DND, `READ_LOGS`, install-packages, notification listeners are **not** runtime permissions |
|
||||||
|
| Launcher + pinned shortcuts | launcher DB, `/data/system_ce/0/shortcut_service/` | Home screen; pinned shortcuts (e.g. Ice Box) are owned by the system, not the launcher |
|
||||||
|
| QS tiles, device controls | `settings get secure sysui_qs_tiles`; SystemUI `user_de` `files/controls_favorites.xml` + `shared_prefs/controls_prefs.xml` | Home Assistant controls, dev tiles |
|
||||||
|
| Wi-Fi | `/data/misc/apex/com.android.wifi/` or `cmd wifi list-networks` + PSKs | Also note which SSIDs must **not** auto-join |
|
||||||
|
| Root grants + Magisk settings | `magisk --sqlite "SELECT * FROM policies"`, `… FROM settings`, `… FROM denylist`, `/data/adb/modules` (note which are disabled) | UIDs change, grants don't follow; Zygisk off ⇒ DenyList silently does nothing |
|
||||||
|
| Accounts | `dumpsys account`; `/data/system_ce/0/accounts_ce.db` | Shows which logins (2nd Google account, app-registered accounts) must be re-added by hand — they can't be transplanted |
|
||||||
|
| SD-card volume IDs | `blkid` of each SD partition | Public volume ID = filesystem UUID; reformat ⇒ new ID ⇒ broken paths and grants |
|
||||||
|
| Freezer state | `pm list packages -d` + the freezer app's own list | Updates/reinstalls unfreeze |
|
||||||
|
|
||||||
|
### 12.2 Restore order and traps
|
||||||
|
|
||||||
|
- **Update first, restore data second.** Installing the old APKs and then the store's update is
|
||||||
|
fine; restoring data *into* an older APK than the data was written by is a downgrade — apps wipe
|
||||||
|
or crash on it. Never re-run a full restore pass over updated apps; restore data only.
|
||||||
|
- **Installer of record:** `pm set-installer` refuses even root. Reinstall the same APKs with
|
||||||
|
`adb install-multiple -r -i <installer>` (keeps data) so Play/Aurora/F-Droid/Obtainium see their
|
||||||
|
apps as updatable (`fix-installers.py`).
|
||||||
|
- **New signature, same package** (forks such as Shizuku → Shizuku-Next for a newer Android): tar the
|
||||||
|
data dirs, uninstall, install, untar, chown/chcon, re-grant, re-freeze — `reinstall-keep-data.sh`.
|
||||||
|
- **System files are read only at `system_server` start.** Write `urigrants.xml`,
|
||||||
|
`settings_ssaid.xml`, `shortcut_service/*` etc. (owner `system:system`, mode 600,
|
||||||
|
`u:object_r:system_data_file:s0`), then `kill -9 $(pidof system_server)` — a soft restart that
|
||||||
|
does **not** save over them first, which a normal reboot does. It locks the user: unlock with
|
||||||
|
wake → swipe → `input text <PIN>` → ENTER (the bouncer screenshots black; retry if needed).
|
||||||
|
- **fs-verity:** files under `shortcut_service` are verity-protected — `rm` then write, never edit.
|
||||||
|
- **Restored caches can kill an app:** Ice Box OOM-crashed on a restored MMKV package cache. Drop
|
||||||
|
`cache`, `code_cache`, WebView caches and per-app package-list caches; keep the real state.
|
||||||
|
- **Re-map UID-keyed state** with the old/new `packages.list` (SSAID `name="<uid>"`, keystore
|
||||||
|
`namespace`, Magisk `policies.uid`, doze lists are by package and need no remap).
|
||||||
|
- **SD card:** if the partition had to be recreated, set the old UUID back
|
||||||
|
(`tune2fs -U <old>` for ext4, `fsck` before and after) rather than fixing every app.
|
||||||
|
- **Wi-Fi:** restored networks auto-join in their own order; re-add the ones that must stay
|
||||||
|
manual with `cmd wifi add-network <ssid> wpa2 <psk> -d`.
|
||||||
|
- **QS tiles** on A17: `settings put secure sysui_qs_tiles` is ignored; use
|
||||||
|
`cmd statusbar set-tiles "<list>"`. Developer tiles must be enabled first
|
||||||
|
(`pm enable 'com.android.settings/.development.qstile.DevelopmentTiles$SensorsOff'`).
|
||||||
|
- **Launcher switch** (Nova had no build for the new Android): converting Nova's `favorites`
|
||||||
|
table into Lawnchair's Launcher3 schema works (`nova-to-lawnchair.py`) — mark widgets
|
||||||
|
`restored=1` so Launcher3 binds fresh widget IDs, and re-pin deep shortcuts for the new launcher.
|
||||||
|
|
||||||
|
**Banking-app root checks** are not a restore problem, but they show up right after one. Order
|
||||||
|
that worked (2026-10, Magisk 31, A17): first the ROM's own spoofing settings, if it has any (PIF,
|
||||||
|
TrickyStore target and keybox, hide developer status). If an app still crashes on purpose, look for
|
||||||
|
an Xposed module that defuses that app's check. For PayPal 10.18 that was oslokrok on Vector, the
|
||||||
|
LSPosed successor. Configure Vector headlessly with
|
||||||
|
`/data/adb/modules/zygisk_vector/cli modules enable <pkg>` and `cli scope set <pkg> <app>/0`.
|
||||||
|
**Take the app off the Magisk DenyList**: the DenyList strips every Zygisk module from the process,
|
||||||
|
including the hook.
|
||||||
|
|
||||||
|
### 12.3 Keystore: transplanting app keys
|
||||||
|
|
||||||
|
Hardware-backed keys never leave the TEE, but keystore2 stores them **wrapped** in its SQLite DB.
|
||||||
|
If the new ROM runs on the same TEE (compare `keyentry.km_uuid` old vs new — same device, same
|
||||||
|
firmware ⇒ same), the old wrapped blobs still unwrap. Keys whose `blobmetadata` has tag 0
|
||||||
|
(`EncryptedBy`) are additionally super-encrypted with the **old lock-screen credential's** key and
|
||||||
|
cannot be moved. Domain 0 (APP) rows have `namespace = uid` → remap. Stop the apps, back up the live
|
||||||
|
DB, insert `keyentry`/`blobentry`/`blobmetadata`/`keyparameter`/`keymetadata` with fresh ids, reboot.
|
||||||
|
Apps that already regenerated keys on the new ROM also need their data re-restored to match.
|
||||||
|
`scripts/android/keystore-transplant.py OLD_DB [--replace] OLD_UID:NEW_UID …` writes the SQL; dry-run
|
||||||
|
it on a host copy of the live DB (`pragma integrity_check`) before applying with the device's own
|
||||||
|
`sqlite3`. Tested: transplanted keys load with no keystore errors after a reboot. That does not
|
||||||
|
bring back sessions the server ended, or apps that refuse to run because their build is too old.
|
||||||
|
|
||||||
|
### 12.4 Verify like a user would
|
||||||
|
|
||||||
|
- `test-all-apps.sh OUT` cold-starts every user app (unfreezing and re-freezing), screenshots it,
|
||||||
|
and greps its log for crashes, ANRs and keystore/crypto errors → `results.tsv`. Look at every
|
||||||
|
screenshot: logouts, licence checks, ToS/"what's new" dialogs and permission prompts don't crash.
|
||||||
|
- Android 17 shell launches must match the activity's intent filter:
|
||||||
|
`am start -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n 'pkg/act'`
|
||||||
|
(single-quote it — component names contain `$`). `-n` alone gives "Access blocked".
|
||||||
|
- `measure.sh OUT [pkg…]` snapshots memory, crashes/ANRs/lmk kills since boot and cold-launch
|
||||||
|
times; run it on the old ROM before the wipe so there is a baseline.
|
||||||
|
- Keep a running log of every security setting weakened during the restore (adb stay-awake, root
|
||||||
|
grants, doze exemptions, …) with its revert command; the restore needs many of them and nobody
|
||||||
|
remembers them afterwards.
|
||||||
+11
-1
@@ -26,6 +26,14 @@ reasons attached, not style trivia — most exist because ignoring them caused a
|
|||||||
> When in doubt about which mode you are in, **ask** — it changes the answer on strictness,
|
> When in doubt about which mode you are in, **ask** — it changes the answer on strictness,
|
||||||
> file size, typing, and test policy all at once.
|
> file size, typing, and test policy all at once.
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> **A new software idea starts with research, not code.** When the user describes a tool, app,
|
||||||
|
> service, or other project they want built, follow
|
||||||
|
> [`software-vision.md`](skills-public/software-vision.md) first: find existing projects to use,
|
||||||
|
> extend, or fork (web searches and user reviews first, then training-data recall, then
|
||||||
|
> throwaway clones), and present the options before scaffolding anything — unless the user
|
||||||
|
> explicitly opted out of research.
|
||||||
|
|
||||||
Per-language specifics live in [`code-languages/`](skills-public/code-languages/):
|
Per-language specifics live in [`code-languages/`](skills-public/code-languages/):
|
||||||
|
|
||||||
| Language | Rules |
|
| Language | Rules |
|
||||||
@@ -115,7 +123,9 @@ a fresh implementation will rediscover slowly and incompletely.
|
|||||||
|
|
||||||
Before hand-rolling any subsystem — a protocol client, a parser, a date formatter, a
|
Before hand-rolling any subsystem — a protocol client, a parser, a date formatter, a
|
||||||
debounce, a retry/backoff, a semver comparator — **search first and state the finding.**
|
debounce, a retry/backoff, a semver comparator — **search first and state the finding.**
|
||||||
Prefer official/first-party SDKs over third-party wrappers.
|
Prefer official/first-party SDKs over third-party wrappers. The same ordering applies to a
|
||||||
|
whole project idea, not just a subsystem — see
|
||||||
|
[`software-vision.md`](skills-public/software-vision.md).
|
||||||
|
|
||||||
### The corollary: new generic code becomes its own package
|
### The corollary: new generic code becomes its own package
|
||||||
|
|
||||||
|
|||||||
Executable
+49
@@ -0,0 +1,49 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Reinstall each restored app with its original installer (pm set-installer refuses root).
|
||||||
|
|
||||||
|
Same APKs, `install-multiple -r -i <installer>` keeps data. The device comes from ANDROID_SERIAL, as for adb. Usage: fix-installers.py BACKUP_DIR WORK_DIR
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
backup, work = Path(sys.argv[1]), Path(sys.argv[2])
|
||||||
|
log_file = (work / "restore.log").open("a")
|
||||||
|
|
||||||
|
|
||||||
|
def log(msg: str) -> None:
|
||||||
|
line = f"{time.strftime('%H:%M:%S')} {msg}"
|
||||||
|
print(line, flush=True)
|
||||||
|
log_file.write(line + "\n")
|
||||||
|
log_file.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def adb(*args: str, timeout: int = 900) -> str:
|
||||||
|
r = subprocess.run(["adb", *args], capture_output=True, text=True, timeout=timeout)
|
||||||
|
return (r.stdout + r.stderr).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def installers(text: str) -> dict:
|
||||||
|
# `pm list packages -i` prints "package:PKG", with -f "package:PATH=PKG".
|
||||||
|
return dict(re.findall(r"[:=]([\w.]+)\s+installer=(\S+)", text))
|
||||||
|
|
||||||
|
|
||||||
|
want = installers((backup / "rooted-fresh/packages-3.txt").read_text())
|
||||||
|
now = installers(adb("shell", "pm list packages -3 -i"))
|
||||||
|
disabled = set(re.findall(r"package:([\w.]+)", adb("shell", "pm list packages -d")))
|
||||||
|
log("=== installer fix-up start")
|
||||||
|
for pkg, inst in sorted(want.items()):
|
||||||
|
if inst == "null" or pkg not in now or now[pkg] == inst or inst not in now | {"com.android.vending": 1}:
|
||||||
|
continue
|
||||||
|
apks = sorted(str(p) for p in (backup / "apks" / pkg).glob("*.apk"))
|
||||||
|
if not apks:
|
||||||
|
continue
|
||||||
|
# Reinstalling re-enables a frozen app; disable it again afterwards.
|
||||||
|
out = adb("install-multiple", "-r", "-d", "-i", inst, "--bypass-low-target-sdk-block", *apks)
|
||||||
|
if pkg in disabled:
|
||||||
|
adb("shell", f"su -c 'pm disable-user --user 0 {pkg}'")
|
||||||
|
log(f"{pkg}: installer {now[pkg]} -> {inst}: {'OK' if 'Success' in out else 'FAILED ' + out[-200:]}")
|
||||||
|
log("=== installer fix-up done")
|
||||||
Executable
+74
@@ -0,0 +1,74 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Emit SQL that copies apps' keystore2 keys from an old ROM's DB into the live one, under the new UIDs.
|
||||||
|
|
||||||
|
Only works when both ROMs use the same TEE (compare keyentry.km_uuid). Keys super-encrypted with the
|
||||||
|
old lock-screen credential (blobmetadata tag 0, EncryptedBy) cannot be moved and are skipped.
|
||||||
|
|
||||||
|
Usage: keystore-transplant.py OLD_DB [--replace] OLD_UID:NEW_UID [...] > transplant.sql
|
||||||
|
--replace first delete the app's keys that already exist on the new ROM (needed when it regenerated them)
|
||||||
|
Apply with the apps force-stopped: `su -c 'sqlite3 /data/misc/keystore/persistent.sqlite < transplant.sql'`, reboot.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import secrets
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
|
||||||
|
DOMAIN_APP = 0
|
||||||
|
TAG_ENCRYPTED_BY = 0
|
||||||
|
|
||||||
|
|
||||||
|
def lit(v: object) -> str:
|
||||||
|
if v is None:
|
||||||
|
return "NULL"
|
||||||
|
if isinstance(v, bytes):
|
||||||
|
return f"X'{v.hex()}'"
|
||||||
|
if isinstance(v, str):
|
||||||
|
return "'" + v.replace("'", "''") + "'"
|
||||||
|
return str(v)
|
||||||
|
|
||||||
|
|
||||||
|
def delete_keys(where: str) -> list[str]:
|
||||||
|
ids = f"(SELECT id FROM keyentry WHERE {where})"
|
||||||
|
return [f"DELETE FROM blobmetadata WHERE blobentryid IN (SELECT id FROM blobentry WHERE keyentryid IN {ids});",
|
||||||
|
f"DELETE FROM blobentry WHERE keyentryid IN {ids};",
|
||||||
|
f"DELETE FROM keyparameter WHERE keyentryid IN {ids};",
|
||||||
|
f"DELETE FROM keymetadata WHERE keyentryid IN {ids};",
|
||||||
|
f"DELETE FROM grant WHERE keyentryid IN {ids};",
|
||||||
|
f"DELETE FROM keyentry WHERE {where};"]
|
||||||
|
|
||||||
|
|
||||||
|
args = sys.argv[1:]
|
||||||
|
old = sqlite3.connect(f"file:{args.pop(0)}?mode=ro", uri=True)
|
||||||
|
replace = "--replace" in args
|
||||||
|
pairs = [tuple(map(int, a.split(":"))) for a in args if a != "--replace"]
|
||||||
|
# Older keystore2 schemas have no blobentry.state; every blob there is current (0).
|
||||||
|
blob_state = "state" if any(c[1] == "state" for c in old.execute("PRAGMA table_info(blobentry)")) else "0"
|
||||||
|
|
||||||
|
out = ["BEGIN;"]
|
||||||
|
for old_ns, new_ns in pairs:
|
||||||
|
if replace:
|
||||||
|
out += delete_keys(f"domain={DOMAIN_APP} AND namespace={new_ns}")
|
||||||
|
keys = old.execute("SELECT id, key_type, alias, state, km_uuid FROM keyentry WHERE domain=? AND namespace=?",
|
||||||
|
(DOMAIN_APP, old_ns)).fetchall()
|
||||||
|
for kid, key_type, alias, state, km_uuid in keys:
|
||||||
|
blobs = old.execute(f"SELECT id, subcomponent_type, blob, {blob_state} FROM blobentry WHERE keyentryid=?", (kid,)).fetchall()
|
||||||
|
if any(old.execute("SELECT 1 FROM blobmetadata WHERE blobentryid=? AND tag=?", (b[0], TAG_ENCRYPTED_BY)).fetchone()
|
||||||
|
for b in blobs):
|
||||||
|
print(f"-- skip {old_ns} {alias!r}: super-encrypted with the old lock-screen credential", file=sys.stderr)
|
||||||
|
continue
|
||||||
|
new_id = secrets.randbits(63) * (1 if secrets.randbits(1) else -1)
|
||||||
|
# The old key wins over one the app generated under the same alias on the new ROM.
|
||||||
|
out += delete_keys(f"domain={DOMAIN_APP} AND namespace={new_ns} AND alias={lit(alias)}")
|
||||||
|
out.append(f"INSERT INTO keyentry (id,key_type,domain,namespace,alias,state,km_uuid) VALUES "
|
||||||
|
f"({new_id},{key_type},{DOMAIN_APP},{new_ns},{lit(alias)},{state},{lit(km_uuid)});")
|
||||||
|
for bid, sub, blob, bstate in blobs:
|
||||||
|
out.append(f"INSERT INTO blobentry (subcomponent_type,keyentryid,blob,state) VALUES ({sub},{new_id},{lit(blob)},{bstate});")
|
||||||
|
for tag, data in old.execute("SELECT tag, data FROM blobmetadata WHERE blobentryid=?", (bid,)):
|
||||||
|
out.append(f"INSERT INTO blobmetadata (blobentryid,tag,data) VALUES ((SELECT max(id) FROM blobentry WHERE keyentryid={new_id}),{tag},{lit(data)});")
|
||||||
|
for tag, data, level in old.execute("SELECT tag, data, security_level FROM keyparameter WHERE keyentryid=?", (kid,)):
|
||||||
|
out.append(f"INSERT INTO keyparameter (keyentryid,tag,data,security_level) VALUES ({new_id},{tag},{lit(data)},{level});")
|
||||||
|
for tag, data in old.execute("SELECT tag, data FROM keymetadata WHERE keyentryid=?", (kid,)):
|
||||||
|
out.append(f"INSERT INTO keymetadata (keyentryid,tag,data) VALUES ({new_id},{tag},{lit(data)});")
|
||||||
|
print(f"-- {old_ns}->{new_ns} {alias!r}: {len(blobs)} blobs", file=sys.stderr)
|
||||||
|
out.append("COMMIT;")
|
||||||
|
print("\n".join(out))
|
||||||
Executable
+8
@@ -0,0 +1,8 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Cold-start an app's launcher activity the way Android 17 accepts from the shell. Usage: launch.sh PKG [WAIT] [SHOT.png]
|
||||||
|
p=$1 w=${2:-6} shot=${3:-}
|
||||||
|
act=$(adb shell "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" </dev/null | tr -d '\r' | tail -1)
|
||||||
|
adb shell "am start -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" </dev/null >/dev/null
|
||||||
|
sleep "$w"
|
||||||
|
[ -n "$shot" ] && adb exec-out screencap -p > "$shot" </dev/null
|
||||||
|
true
|
||||||
Executable
+42
@@ -0,0 +1,42 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Snapshot phone health: memory, load, crashes/ANRs since boot, low-memory kills, log errors, cold launches.
|
||||||
|
# Usage: measure.sh OUT_FILE [launch-package ...]
|
||||||
|
set -uo pipefail
|
||||||
|
out=$1; shift
|
||||||
|
a() { adb shell "$@" </dev/null; }
|
||||||
|
|
||||||
|
{
|
||||||
|
echo "# $(date '+%F %T') $(a getprop ro.build.display.id) / Android $(a getprop ro.build.version.release)"
|
||||||
|
echo "## system"
|
||||||
|
a 'uptime; cat /proc/loadavg'
|
||||||
|
a 'grep -E "MemTotal|MemAvailable|Cached:|SwapTotal|SwapFree" /proc/meminfo'
|
||||||
|
a 'su -c "cat /proc/sys/vm/swappiness; cat /sys/block/zram0/mm_stat 2>/dev/null"'
|
||||||
|
echo "procs: $(a 'ls -d /proc/[0-9]* | wc -l')"
|
||||||
|
echo "## top cpu"
|
||||||
|
a 'top -b -n 1 -m 12 -o PID,USER,%CPU,%MEM,NAME' | tail -13
|
||||||
|
echo "## dropbox since boot"
|
||||||
|
boot=$(a 'date -d @$(( $(date +%s) - $(cut -d. -f1 /proc/uptime) )) "+%Y-%m-%d %H:%M:%S"')
|
||||||
|
a "dumpsys dropbox" | awk -v b="$boot" '$1" "$2 >= b && $3 ~ /anr|crash|watchdog|tombstone|wtf|lowmem|native/ {print $3}' | sort | uniq -c
|
||||||
|
echo "## crash buffer (FATAL / ANR per process)"
|
||||||
|
a 'logcat -d -b crash' | grep -oE "Process: [^,]+|FATAL EXCEPTION|>>> [^ ]+ <<<" | sort | uniq -c | sort -rn | head -20
|
||||||
|
a 'logcat -d -b events' | grep -oE "am_anr: \[[^]]*" | awk -F, '{print "ANR " $3}' | sort | uniq -c
|
||||||
|
echo "## lmkd / am_kill"
|
||||||
|
a 'logcat -d -b events' | grep -cE "am_kill|lmk_kill" | sed 's/^/kills: /'
|
||||||
|
a 'logcat -d -b events' | grep -oE "am_kill: \[[^]]*" | awk -F, '{print $5}' | sort | uniq -c | sort -rn | head -10
|
||||||
|
echo "## logcat error tags (top 25)"
|
||||||
|
a 'logcat -d *:E' | awk '{print $6}' | sort | uniq -c | sort -rn | head -25
|
||||||
|
echo "## thermal / battery"
|
||||||
|
a 'dumpsys battery | grep -E "level|temperature|status"; for z in /sys/class/thermal/thermal_zone*; do echo "$(cat $z/type) $(cat $z/temp)"; done 2>/dev/null | sort -k2 -n | tail -5'
|
||||||
|
if [ $# -gt 0 ]; then
|
||||||
|
echo "## cold launches (ms, am start -W -S, 3 runs)"
|
||||||
|
for p in "$@"; do
|
||||||
|
t=""
|
||||||
|
for _ in 1 2 3; do
|
||||||
|
act=$(a "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" | tr -d '\r' | tail -1)
|
||||||
|
v=$(a "am start -W -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" | awk '/TotalTime/{print $2}')
|
||||||
|
t="$t ${v:-fail}"; a 'input keyevent KEYCODE_HOME'; sleep 2
|
||||||
|
done
|
||||||
|
echo "$p:$t"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
} 2>&1 | tee "$out"
|
||||||
Executable
+88
@@ -0,0 +1,88 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Rebuild the Nova home screen in Lawnchair, plus the Ice Box shortcuts Nova had pinned.
|
||||||
|
|
||||||
|
Usage: nova-to-lawnchair.py NOVA_DB LAWNCHAIR_DB OLD_ICEBOX_SHORTCUT_XML OUT_DIR
|
||||||
|
|
||||||
|
Writes OUT_DIR/launcher_5_5_5.db (Lawnchair schema, Nova's items), OUT_DIR/icebox.xml
|
||||||
|
(shortcut_service package file with only the pinned shortcuts) and OUT_DIR/lawnchair-pins.xml
|
||||||
|
(shortcut_service launcher file pinning them for Lawnchair).
|
||||||
|
"""
|
||||||
|
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import sqlite3
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
LAUNCHER = "app.lawnchair.nightly"
|
||||||
|
ICEBOX = "com.catchingnow.icebox"
|
||||||
|
# From /data/system/packages.xml on the new ROM: versionCode, lastUpdateTime, SHA-256 of the cert.
|
||||||
|
LAUNCHER_INFO = ("1600020100", "1792700106881", "dHw2RbNXJYsuI+hR5TyWdH/grdAH5bos2X6MhVcuTcU=")
|
||||||
|
ICEBOX_INFO = ("154821395", "1792699567277", "gvW+cC4DIBdGDKg3b22r1yWp5S3SSSDdXl17WF+/4bY=")
|
||||||
|
# Nova-only containers (gesture/dock-swipe slots); Lawnchair has nothing to put them in.
|
||||||
|
SKIP_CONTAINERS = {-207, -208}
|
||||||
|
ITEM_APP, ITEM_FOLDER, ITEM_WIDGET, ITEM_DEEP_SHORTCUT = 0, 2, 4, 6
|
||||||
|
FLAG_ID_NOT_VALID = 1 # Launcher3 restore flag: allocate and bind a fresh widget id on load
|
||||||
|
MISSING_PACKAGES = {"com.dot.gallery"} # not installed on the new ROM
|
||||||
|
# Google's smartspace widget on row 0 of the first page: Lawnchair reserves that row for its own.
|
||||||
|
SKIP_IDS = {56}
|
||||||
|
|
||||||
|
nova_db, lc_db, icebox_xml, out = map(Path, sys.argv[1:5])
|
||||||
|
out.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def package_info(version: str, updated: str, sig: str) -> str:
|
||||||
|
return (f'<package-info version="{version}" last_udpate_time="{updated}" shadow="0" allow-backup="1" '
|
||||||
|
f'allow-backup-initialized="1" bk_src_version="-1" bk_src_backup-allowed="0">\n'
|
||||||
|
f'<signature hash="{sig}" />\n</package-info>\n')
|
||||||
|
|
||||||
|
|
||||||
|
nova = sqlite3.connect(nova_db)
|
||||||
|
nova.row_factory = sqlite3.Row
|
||||||
|
rows = [r for r in nova.execute("select * from favorites where _id >= 0 order by _id")
|
||||||
|
if r["container"] not in SKIP_CONTAINERS and r["_id"] not in SKIP_IDS]
|
||||||
|
folders = {r["_id"] for r in rows if r["itemType"] == ITEM_FOLDER}
|
||||||
|
kept = []
|
||||||
|
for r in rows:
|
||||||
|
pkg = re.search(r"component=([\w.]+)/", r["intent"] or "")
|
||||||
|
if pkg and pkg.group(1) in MISSING_PACKAGES:
|
||||||
|
print(f"skip {r['title']}: {pkg.group(1)} not installed")
|
||||||
|
continue
|
||||||
|
if r["container"] >= 0 and r["container"] not in folders:
|
||||||
|
continue
|
||||||
|
kept.append(r)
|
||||||
|
|
||||||
|
db = out / "launcher_5_5_5.db"
|
||||||
|
shutil.copy(lc_db, db)
|
||||||
|
lc = sqlite3.connect(db)
|
||||||
|
lc.execute("delete from favorites")
|
||||||
|
pins = []
|
||||||
|
for r in kept:
|
||||||
|
widget = r["itemType"] == ITEM_WIDGET
|
||||||
|
hotseat = r["container"] == -101
|
||||||
|
in_folder = r["container"] >= 0
|
||||||
|
x, y = int(max(r["cellX"] or 0, 0)), int(max(r["cellY"] or 0, 0))
|
||||||
|
lc.execute(
|
||||||
|
"insert into favorites (_id,title,intent,container,screen,cellX,cellY,spanX,spanY,itemType,"
|
||||||
|
"appWidgetId,appWidgetProvider,restored,profileId,rank,options) values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)",
|
||||||
|
(r["_id"], r["title"], r["intent"], r["container"],
|
||||||
|
x if hotseat else (0 if in_folder else r["screen"]),
|
||||||
|
x, 0 if hotseat else y, int(r["spanX"]), int(r["spanY"]), r["itemType"],
|
||||||
|
r["appWidgetId"] if widget else -1, r["appWidgetProvider"],
|
||||||
|
FLAG_ID_NOT_VALID if widget else 0, 0, r["rank"], 0))
|
||||||
|
if r["itemType"] == ITEM_DEEP_SHORTCUT:
|
||||||
|
pins.append(re.search(r"S\.shortcut_id=([^;]+)", r["intent"]).group(1))
|
||||||
|
lc.commit()
|
||||||
|
print(f"{len(kept)} items written, pins: {pins}")
|
||||||
|
|
||||||
|
old = Path(icebox_xml).read_text()
|
||||||
|
blocks = [m.group(0) for m in re.finditer(r'<shortcut id="([^"]+)".*?</shortcut>\n', old, re.S) if m.group(1) in pins]
|
||||||
|
assert len(blocks) == len(pins), f"found {len(blocks)} of {len(pins)} shortcuts"
|
||||||
|
head = re.search(r"<package name=[^>]*>\n", old).group(0)
|
||||||
|
(out / "icebox.xml").write_text("<?xml version='1.0' encoding='utf-8' standalone='yes' ?>\n" + head
|
||||||
|
+ package_info(*ICEBOX_INFO) + "".join(blocks) + "</package>\n")
|
||||||
|
(out / "lawnchair-pins.xml").write_text(
|
||||||
|
"<?xml version='1.0' encoding='utf-8' standalone='yes' ?>\n"
|
||||||
|
f'<launcher-pins package-name="{LAUNCHER}" launcher-user="0">\n' + package_info(*LAUNCHER_INFO)
|
||||||
|
+ f'<package package-name="{ICEBOX}" package-user="0">\n'
|
||||||
|
+ "".join(f'<pin value="{p}" />\n' for p in pins) + "</package>\n</launcher-pins>\n")
|
||||||
Executable
+40
@@ -0,0 +1,40 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Replace an app with a differently signed build while keeping its data, permissions,
|
||||||
|
# installer and frozen state. Data is also saved to BACKUP_DIR before the uninstall.
|
||||||
|
# Usage: reinstall-keep-data.sh PACKAGE BACKUP_DIR APK [APK ...]
|
||||||
|
set -euo pipefail
|
||||||
|
pkg=$1 bak=$2; shift 2
|
||||||
|
a() { adb shell "$@" </dev/null; }
|
||||||
|
su_() { a "su -c '$1'"; }
|
||||||
|
|
||||||
|
mkdir -p "$bak"
|
||||||
|
installer=$(a "pm list packages -3 -i $pkg" | tr -d '\r' | grep "package:$pkg " | sed 's/.*installer=//')
|
||||||
|
disabled=$(a "pm list packages -d $pkg" | tr -d '\r' | grep -cx "package:$pkg" || true)
|
||||||
|
perms=$(a "dumpsys package $pkg" | sed -n '/runtime permissions:/,/^ *[a-z]* .*:$/p' | grep -oE '[a-zA-Z0-9_.]+: granted=true' | cut -d: -f1 | sort -u | tr '\n' ' ')
|
||||||
|
echo "$pkg: installer=$installer disabled=$disabled perms=[$perms]"
|
||||||
|
|
||||||
|
dirs="/data/data/$pkg /data/user_de/0/$pkg"
|
||||||
|
[ "$(su_ "[ -d /data/media/0/Android/data/$pkg ] && echo y")" = y ] && dirs="$dirs /data/media/0/Android/data/$pkg"
|
||||||
|
su_ "am force-stop $pkg"
|
||||||
|
# The phone writes the tar to stdout; exec-out keeps it binary-safe.
|
||||||
|
adb exec-out "su -c 'tar -cf - --exclude=$pkg/cache --exclude=$pkg/code_cache $dirs 2>/dev/null'" > "$bak/$pkg.tar"
|
||||||
|
echo "saved $(stat -c %s "$bak/$pkg.tar") bytes to $bak/$pkg.tar"
|
||||||
|
[ "$(stat -c %s "$bak/$pkg.tar")" -gt 10240 ] || { echo "backup suspiciously small, aborting"; exit 1; }
|
||||||
|
|
||||||
|
a "pm uninstall $pkg"
|
||||||
|
iopt=(); [ -n "$installer" ] && [ "$installer" != null ] && iopt=(-i "$installer")
|
||||||
|
adb install-multiple -r "${iopt[@]}" --bypass-low-target-sdk-block "$@"
|
||||||
|
|
||||||
|
uid=$(su_ "stat -c %u /data/data/$pkg")
|
||||||
|
cache_gid=$((uid - 10000 + 20000))
|
||||||
|
adb exec-in "su -c 'tar -xf - -C /'" < "$bak/$pkg.tar"
|
||||||
|
su_ "while pgrep -x tar >/dev/null; do sleep 0.3; done"
|
||||||
|
for d in /data/data/$pkg /data/user_de/0/$pkg; do
|
||||||
|
su_ "chown -R $uid:$uid $d; for c in cache code_cache; do mkdir -p $d/\$c; chown $uid:$cache_gid $d/\$c; chmod 2771 $d/\$c; done; chcon -hR \"\$(stat -c %C $d)\" $d"
|
||||||
|
done
|
||||||
|
case " $dirs " in *Android/data*)
|
||||||
|
su_ "d=/data/media/0/Android/data/$pkg; chown -R $uid:1078 \$d; find \$d -type d -exec chmod 2770 {} +; find \$d -type f -exec chmod 660 {} +; chcon -R u:object_r:media_rw_data_file:s0 \$d" ;;
|
||||||
|
esac
|
||||||
|
for p in $perms; do a "pm grant $pkg $p" 2>/dev/null || true; done
|
||||||
|
[ "$disabled" = 1 ] && su_ "pm disable-user --user 0 $pkg"
|
||||||
|
echo "$pkg: reinstalled as uid $uid, $(a "dumpsys package $pkg" | grep -m1 versionName | xargs)"
|
||||||
Executable
+144
@@ -0,0 +1,144 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Restore user apps, app data and permissions onto a freshly flashed, rooted phone.
|
||||||
|
|
||||||
|
Sources (from the backup folder):
|
||||||
|
apks/<pkg>/*.apk APK + splits
|
||||||
|
rooted-fresh/dumpsys-package.txt installer, granted runtime permissions
|
||||||
|
rooted-fresh/packages-3.txt which packages are user apps
|
||||||
|
rooted-fresh/packages-disabled.txt frozen/disabled state
|
||||||
|
rooted/apps-manifest.json special app-ops (overlay, all-files, ...)
|
||||||
|
<work>/data/<pkg>, <work>/user_de/0/<pkg> unpacked data-fresh.tar
|
||||||
|
|
||||||
|
The device comes from ANDROID_SERIAL, as for adb. Usage: restore.py BACKUP_DIR WORK_DIR [--data-only] [pkg ...] (no pkgs = all user apps)
|
||||||
|
--data-only only put the data back (apps already installed and updated): keeps the current APK,
|
||||||
|
installer, permissions and frozen state. Never re-run a full restore over updated apps.
|
||||||
|
Every action and its result is appended to WORK_DIR/restore.log.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import shlex
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SKIP = {"com.topjohnwu.magisk"} # installed fresh, its data must not be reused
|
||||||
|
# Dropped on restore: regenerated by the app, and the bulk of most backups.
|
||||||
|
VOLATILE = ["cache", "code_cache", "app_webview/Default/Cache", "app_webview/Default/Code Cache"]
|
||||||
|
|
||||||
|
backup, work = Path(sys.argv[1]), Path(sys.argv[2])
|
||||||
|
data_only = "--data-only" in sys.argv[3:]
|
||||||
|
only = set(sys.argv[3:]) - {"--data-only"}
|
||||||
|
log_file = (work / "restore.log").open("a")
|
||||||
|
|
||||||
|
|
||||||
|
def log(msg: str) -> None:
|
||||||
|
line = f"{time.strftime('%H:%M:%S')} {msg}"
|
||||||
|
print(line, flush=True)
|
||||||
|
log_file.write(line + "\n")
|
||||||
|
log_file.flush()
|
||||||
|
|
||||||
|
|
||||||
|
def adb(*args: str, timeout: int = 300, stdin=None) -> subprocess.CompletedProcess:
|
||||||
|
return subprocess.run(["adb", *args], capture_output=True, text=stdin is None,
|
||||||
|
timeout=timeout, stdin=stdin)
|
||||||
|
|
||||||
|
|
||||||
|
def su(cmd: str, timeout: int = 300) -> str:
|
||||||
|
r = adb("shell", f"su -c {shlex.quote(cmd)}", timeout=timeout)
|
||||||
|
return (r.stdout + r.stderr).strip()
|
||||||
|
|
||||||
|
|
||||||
|
def parse_dumpsys(text: str) -> dict:
|
||||||
|
"""installer + granted runtime permissions per package."""
|
||||||
|
out = {}
|
||||||
|
for block in re.split(r"\n Package \[", text)[1:]:
|
||||||
|
name = block.split("]", 1)[0]
|
||||||
|
installer = re.search(r"\n installerPackageName=(\S+)", block)
|
||||||
|
runtime = block.split("runtime permissions:", 1)
|
||||||
|
perms = re.findall(r"\n ([\w.]+): granted=true", runtime[1]) if len(runtime) > 1 else []
|
||||||
|
out[name] = {"installer": installer.group(1) if installer else None, "perms": sorted(set(perms))}
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
fresh = backup / "rooted-fresh"
|
||||||
|
meta = parse_dumpsys((fresh / "dumpsys-package.txt").read_text(errors="replace"))
|
||||||
|
manifest = json.loads((backup / "rooted/apps-manifest.json").read_text())["apps"]
|
||||||
|
user_pkgs = sorted(set(re.findall(r"=([\w.]+)\s+installer=", (fresh / "packages-3.txt").read_text())) - SKIP)
|
||||||
|
disabled = set(re.findall(r"package:([\w.]+)", (fresh / "packages-disabled.txt").read_text()))
|
||||||
|
targets = [p for p in user_pkgs if not only or p in only]
|
||||||
|
log(f"=== restore start: {len(targets)} packages")
|
||||||
|
|
||||||
|
# Pass 1: install every APK set, so installers like Obtainium exist before pass 2 names them.
|
||||||
|
for pkg in [] if data_only else targets:
|
||||||
|
apks = sorted(str(p) for p in (backup / "apks" / pkg).glob("*.apk"))
|
||||||
|
if not apks:
|
||||||
|
log(f"{pkg}: NO APK, skipped")
|
||||||
|
continue
|
||||||
|
# --bypass-low-target-sdk-block: Android 14+ refuses targetSdk < 24 (Automagic workarounds).
|
||||||
|
r = subprocess.run(["adb", "install-multiple", "-r", "-d", "-g",
|
||||||
|
"--bypass-low-target-sdk-block", *apks],
|
||||||
|
capture_output=True, text=True, timeout=900)
|
||||||
|
log(f"{pkg}: install {'OK' if 'Success' in r.stdout else 'FAILED ' + (r.stdout + r.stderr).strip()[-200:]}")
|
||||||
|
|
||||||
|
installed = set(re.findall(r"package:([\w.]+)", adb("shell", "pm list packages").stdout))
|
||||||
|
|
||||||
|
# Pass 2: data, ownership, SELinux labels, installer, permissions, disabled state.
|
||||||
|
for pkg in targets:
|
||||||
|
if pkg not in installed:
|
||||||
|
continue
|
||||||
|
info = meta.get(pkg, {})
|
||||||
|
su(f"am force-stop {pkg}")
|
||||||
|
uid = su(f"stat -c %u /data/data/{pkg}")
|
||||||
|
if not uid.isdigit():
|
||||||
|
log(f"{pkg}: no data dir after install ({uid}), data skipped")
|
||||||
|
continue
|
||||||
|
cache_gid = int(uid) - 10000 + 20000 # u0_aNNN_cache
|
||||||
|
|
||||||
|
members = [m for m in (f"data/{pkg}", f"user_de/0/{pkg}") if (work / m).is_dir()]
|
||||||
|
if members:
|
||||||
|
for m in members:
|
||||||
|
su(f"find /data/{m} -mindepth 1 -maxdepth 1 ! -name lib -exec rm -rf {{}} +")
|
||||||
|
excludes = [f"--exclude={m}/{v}" for m in members for v in VOLATILE]
|
||||||
|
tar = subprocess.Popen(["tar", "-cf", "-", "-C", str(work), *excludes, *members],
|
||||||
|
stdout=subprocess.PIPE)
|
||||||
|
r = adb("exec-in", "su -c 'tar -xf - -C /data'", stdin=tar.stdout, timeout=1800)
|
||||||
|
tar.wait()
|
||||||
|
# exec-in returns once our side is sent; the phone's tar may still be writing.
|
||||||
|
su("while pgrep -x tar >/dev/null; do sleep 0.3; done", timeout=600)
|
||||||
|
for m in members:
|
||||||
|
su(f"chown -R {uid}:{uid} /data/{m}; "
|
||||||
|
f"for d in cache code_cache; do mkdir -p /data/{m}/$d; chown {uid}:{cache_gid} /data/{m}/$d; "
|
||||||
|
# restorecon gives only app_data_file:s0; the per-app MLS categories that
|
||||||
|
# installd put on the top directory must be copied onto its contents.
|
||||||
|
f"chmod 2771 /data/{m}/$d; done; chcon -hR \"$(stat -c %C /data/{m})\" /data/{m}",
|
||||||
|
timeout=900)
|
||||||
|
wrong = su(f"find {' '.join('/data/' + m for m in members)} ! -user {uid} | wc -l")
|
||||||
|
log(f"{pkg}: data {'+'.join(members)} rc={r.returncode} uid={uid} wrong_owner={wrong}")
|
||||||
|
else:
|
||||||
|
log(f"{pkg}: no data in backup")
|
||||||
|
if data_only:
|
||||||
|
continue
|
||||||
|
|
||||||
|
inst = info.get("installer")
|
||||||
|
if inst and inst != "null" and inst in installed | {"com.android.vending"}:
|
||||||
|
log(f"{pkg}: installer {inst} -> {su(f'pm set-installer {pkg} {inst}') or 'OK'}")
|
||||||
|
|
||||||
|
# -g granted everything at install; revoke what was not granted before.
|
||||||
|
wanted = set(info.get("perms", []))
|
||||||
|
now = re.findall(r"\n\s+([\w.]+): granted=true", su(f"dumpsys package {pkg} | sed -n '/runtime permissions:/,$p'"))
|
||||||
|
revoked = [p for p in set(now) - wanted]
|
||||||
|
for p in revoked:
|
||||||
|
su(f"pm revoke {pkg} {p}")
|
||||||
|
for p in wanted - set(now):
|
||||||
|
su(f"pm grant {pkg} {p}")
|
||||||
|
ops = manifest.get(pkg, {}).get("specialAppOps", {})
|
||||||
|
for op, mode in ops.items():
|
||||||
|
su(f"appops set {pkg} {op} {mode}")
|
||||||
|
log(f"{pkg}: perms granted={len(wanted)} revoked={len(revoked)} appops={','.join(ops) or '-'}")
|
||||||
|
|
||||||
|
if pkg in disabled:
|
||||||
|
log(f"{pkg}: disable-user -> {su(f'pm disable-user --user 0 {pkg}')}")
|
||||||
|
|
||||||
|
log("=== restore done")
|
||||||
Executable
+40
@@ -0,0 +1,40 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Launch every user app with a launcher activity, one by one: unfreeze if frozen, cold start,
|
||||||
|
# wait, screenshot, collect its crash/ANR/keystore log lines, check it is alive and in front,
|
||||||
|
# then stop it and re-freeze. Writes OUT_DIR/results.tsv, OUT_DIR/<pkg>.png, OUT_DIR/<pkg>.log.
|
||||||
|
# Usage: test-all-apps.sh OUT_DIR [pkg ...]
|
||||||
|
set -uo pipefail
|
||||||
|
out=$1; shift
|
||||||
|
mkdir -p "$out"
|
||||||
|
a() { adb shell "$@" </dev/null; }
|
||||||
|
WAIT=${WAIT:-8}
|
||||||
|
|
||||||
|
if [ $# -gt 0 ]; then pkgs=("$@"); else
|
||||||
|
mapfile -t pkgs < <(a 'pm list packages -3' | tr -d '\r' | sed 's/package://' | sort)
|
||||||
|
fi
|
||||||
|
frozen=$(a 'pm list packages -d' | tr -d '\r' | sed 's/package://')
|
||||||
|
printf 'pkg\twas_frozen\tlaunch_ms\talive\tforeground\tcrash\tanr\tkeystore\tnote\n' > "$out/results.tsv"
|
||||||
|
|
||||||
|
for p in "${pkgs[@]}"; do
|
||||||
|
wf=0; grep -qx "$p" <<<"$frozen" && wf=1
|
||||||
|
[ $wf = 1 ] && a "su -c 'pm enable --user 0 $p'" >/dev/null
|
||||||
|
act=$(a "cmd package resolve-activity --brief -c android.intent.category.LAUNCHER $p" | tr -d '\r' | tail -1)
|
||||||
|
if [[ "$act" != */* ]]; then
|
||||||
|
[ $wf = 1 ] && a "su -c 'pm disable-user --user 0 $p'" >/dev/null
|
||||||
|
printf '%s\t%s\t-\t-\t-\t-\t-\t-\tno launcher activity\n' "$p" "$wf" >> "$out/results.tsv"; continue
|
||||||
|
fi
|
||||||
|
a 'logcat -c'
|
||||||
|
ms=$(a "am start -W -S -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -n '$act'" | awk '/TotalTime/{print $2}')
|
||||||
|
sleep "$WAIT"
|
||||||
|
adb exec-out screencap -p > "$out/$p.png" </dev/null
|
||||||
|
pid=$(a "pidof $p" | tr -d '\r')
|
||||||
|
fg=$(a 'dumpsys activity activities' | grep -m1 -E 'topResumedActivity|mResumedActivity' | grep -c "$p/")
|
||||||
|
a 'logcat -d' > "$out/$p.log"
|
||||||
|
crash=$(grep -cE "FATAL EXCEPTION|Process: $p|Fatal signal" "$out/$p.log")
|
||||||
|
anr=$(grep -cE "ANR in $p" "$out/$p.log")
|
||||||
|
ks=$(grep -ciE "AEADBadTag|KeyStoreException|VERIFICATION_FAILED|KeyPermanentlyInvalidated|UnrecoverableKey|BadPaddingException|file is not a database|SQLiteDatabaseCorrupt" "$out/$p.log")
|
||||||
|
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t\n' "$p" "$wf" "${ms:-?}" "${pid:+yes}" "$fg" "$crash" "$anr" "$ks" >> "$out/results.tsv"
|
||||||
|
echo "$p frozen=$wf ms=${ms:-?} alive=${pid:+yes} fg=$fg crash=$crash anr=$anr ks=$ks"
|
||||||
|
a "am force-stop $p; input keyevent KEYCODE_HOME"
|
||||||
|
[ $wf = 1 ] && a "su -c 'pm disable-user --user 0 $p'" >/dev/null
|
||||||
|
done
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
---
|
||||||
|
name: software-vision
|
||||||
|
description: >-
|
||||||
|
Use this skill whenever the user describes an idea for a new piece of software, a tool, an app,
|
||||||
|
a service, a bot, a plugin, a mod, or an automation — "I want something that …", "build me a …",
|
||||||
|
"could we make a tool for …", "I need a service that …" — before choosing a language, scaffolding
|
||||||
|
a repository, or writing any code. Requires prior-art research first: web searches for existing
|
||||||
|
projects, forks, skeletons, libraries and what users say about them (reviews, complaints,
|
||||||
|
comparisons), only then merged with training-data recall as a secondary source, a
|
||||||
|
throwaway shallow clone of promising candidates to check them against the requirements, and a
|
||||||
|
set of options (fork, depend on, extend, start fresh) for the user to choose from.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Software Vision — Find What Exists Before Building
|
||||||
|
|
||||||
|
When the user has an idea for a piece of software, the first job is **not** to design it. It is
|
||||||
|
to find out what already exists that does all or most of it. Building a full stack from nothing
|
||||||
|
when a project covering 80% of the requirements is one `git clone` away wastes days, and the
|
||||||
|
result is usually worse than the thing that already had years of bug reports behind it.
|
||||||
|
|
||||||
|
This is the project-level form of the dependency ordering in
|
||||||
|
[`coding-dev.md` §3](skills-public/coding-dev.md): use what exists, fork what is close, write
|
||||||
|
your own only when nothing fits. That section governs individual subsystems; this one governs
|
||||||
|
the whole idea.
|
||||||
|
|
||||||
|
> [!IMPORTANT]
|
||||||
|
> **Never assume nothing exists.** However niche the idea sounds, someone may have built it,
|
||||||
|
> half-built it, abandoned it, drafted a skeleton, written a blog post about the protocol, or
|
||||||
|
> published research on the hard part. "I couldn't think of anything" is not a finding — a list
|
||||||
|
> of the searches you ran and what they turned up is.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. When This Applies — and the Only Opt-Out
|
||||||
|
|
||||||
|
Run this whole process whenever the user describes software they want to exist: a new app,
|
||||||
|
CLI, service, bot, plugin, mod, integration, library, or automation — whether it is a fresh
|
||||||
|
repository or a large new component inside an existing one.
|
||||||
|
|
||||||
|
**The only way to skip it is an explicit opt-out from the user**, such as "no online research",
|
||||||
|
"don't look at other projects", or "I want to write this from scratch myself". Then:
|
||||||
|
|
||||||
|
- Skip the web searches and the clones (§3–§4).
|
||||||
|
- Still use the libraries rule in [`coding-dev.md` §3](skills-public/coding-dev.md) for
|
||||||
|
individual subsystems unless the user excluded that too.
|
||||||
|
- Do not argue for research afterwards; the user has already decided.
|
||||||
|
|
||||||
|
Urgency, a small-sounding scope, or "it's just a quick script" are **not** opt-outs. A few
|
||||||
|
searches take a minute; reinventing a maintained tool takes far longer.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Pin Down the Requirements First
|
||||||
|
|
||||||
|
Before searching, turn the user's description into a short checklist, because that list is what
|
||||||
|
every candidate gets measured against:
|
||||||
|
|
||||||
|
- **Must-haves** — what the software has to do to be useful at all.
|
||||||
|
- **Nice-to-haves** — features the user mentioned that are not deal-breakers.
|
||||||
|
- **Constraints** — platform (Linux, Windows, Wine, Android, a game engine), runtime, language
|
||||||
|
preferences, self-hosted vs cloud, license needs, offline use, performance.
|
||||||
|
|
||||||
|
Show the checklist to the user with the results (§6) so they can correct it. If a requirement
|
||||||
|
is genuinely ambiguous *and* would change which candidates qualify, ask — otherwise state the
|
||||||
|
assumption and carry on.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Search First, Then Merge in Recall
|
||||||
|
|
||||||
|
The searches happen anyway, so they come **first**, and the decision rests on what they show.
|
||||||
|
Training data can be months or years out of date: projects get abandoned, renamed, relicensed,
|
||||||
|
or overtaken, and newer ones may never have been seen at all. Starting from recall anchors the
|
||||||
|
whole evaluation on a stale picture, so recall comes second and only fills gaps.
|
||||||
|
|
||||||
|
### 3.1 Web searches first
|
||||||
|
|
||||||
|
**Run at least three to five searches**, varied in phrasing and source, before settling on any
|
||||||
|
candidate. Cover:
|
||||||
|
|
||||||
|
| Where | What it finds |
|
||||||
|
| :--- | :--- |
|
||||||
|
| General web search with the problem phrased several ways ("self-hosted X", "open source alternative to Y", "X for Linux") | Products, blog posts, comparisons |
|
||||||
|
| GitHub / GitLab / Codeberg search, topics, and `awesome-*` lists | Projects, forks, skeletons, abandoned prototypes |
|
||||||
|
| The package registry for the likely language (crates.io, npm, PyPI, NuGet, pkg.go.dev, …) | Libraries that do the hard part |
|
||||||
|
| Forums and communities (Reddit, Hacker News, the domain's own Discord/forum, issue trackers of adjacent projects) | "Does anything do X?" threads, half-finished attempts, known dead ends |
|
||||||
|
| Papers, specs, reverse-engineering notes for niche domains | The research behind the hard part, even if nobody shipped an app |
|
||||||
|
|
||||||
|
Search for **parts** as well as the whole: an idea with no complete match often splits into
|
||||||
|
pieces that each have a mature library or tool.
|
||||||
|
|
||||||
|
For each promising hit, note what can be checked without cloning: last commit or release date,
|
||||||
|
license, language, open issue count and how maintainers respond, stars or downloads as a weak
|
||||||
|
signal, and whether its README claims the must-haves.
|
||||||
|
|
||||||
|
### 3.2 What people say about it
|
||||||
|
|
||||||
|
A project's own README describes what it *claims* to do. What users say shows how it actually
|
||||||
|
holds up, so search for opinions on every serious candidate too: reviews, "X vs Y" comparisons,
|
||||||
|
Reddit and Hacker News threads, forum posts, and the complaints in its issue tracker. Look for:
|
||||||
|
|
||||||
|
- Recurring pain points (crashes, data loss, missing features people keep requesting)
|
||||||
|
- Whether people are moving to it or away from it, and to what
|
||||||
|
- How the maintainers respond to bugs and pull requests
|
||||||
|
- Workarounds people rely on, which often point at the gaps a fork would need to fill
|
||||||
|
|
||||||
|
Weight recent opinions over old ones, and many independent voices over a single loud one. Put
|
||||||
|
this in the comparison (§6) next to the technical facts.
|
||||||
|
|
||||||
|
### 3.3 Then merge in recall
|
||||||
|
|
||||||
|
Only after searching, add what you already know from training data: projects, products,
|
||||||
|
libraries, and standards that the searches missed. Every recalled item must be checked with a
|
||||||
|
quick search before it goes into the comparison (does it still exist, is it maintained, has the
|
||||||
|
license changed?). Where recall and current sources disagree, **current sources win.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Clone the Promising Candidates — Then Delete Them
|
||||||
|
|
||||||
|
When a candidate looks close but it is unclear whether it ticks every must-have, **look inside
|
||||||
|
it** rather than guessing from the README.
|
||||||
|
|
||||||
|
1. **Shallow-clone into a throwaway directory** — the session's scratchpad or a fresh `mktemp -d`,
|
||||||
|
never inside a project tree:
|
||||||
|
```bash
|
||||||
|
dir=$(mktemp -d) && git clone --depth 1 <url> "$dir/<name>"
|
||||||
|
```
|
||||||
|
2. **Read, don't run.** A clone is untrusted code. Inspect the README, directory layout, entry
|
||||||
|
points, dependency manifest, build files, tests, and open issues. Do not run its install
|
||||||
|
scripts, build, or test suite as part of evaluation — that is a decision for after the user
|
||||||
|
picks it.
|
||||||
|
3. **Map it against the checklist from §2**: which must-haves it covers, which are partial, which
|
||||||
|
are missing, and roughly how much work each gap is. Note the architecture (does it have a
|
||||||
|
plugin/extension point where the gaps could go?), code quality, and the license.
|
||||||
|
4. **Delete the clone when done.** Nothing from evaluation stays on disk or in a project. If the
|
||||||
|
user later chooses to fork it, that is a fresh, deliberate clone or fork in the right place.
|
||||||
|
|
||||||
|
Clone only the few candidates that are genuinely in contention, not every search hit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Pick Likely Libraries
|
||||||
|
|
||||||
|
Whether the plan ends up as a fork or a fresh start, identify the libraries the work would lean
|
||||||
|
on, for the language that fits best (the existing candidate's language if forking, otherwise the
|
||||||
|
best fit for the constraints and the user's preferences). For each major subsystem — protocol,
|
||||||
|
file format, UI toolkit, storage, networking, parsing — name one or two maintained candidates
|
||||||
|
and why. The per-language rules under
|
||||||
|
[`code-languages/`](skills-public/code-languages/) apply to whatever is chosen.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Present Options — the User Chooses
|
||||||
|
|
||||||
|
Report back **before** scaffolding or writing code, with:
|
||||||
|
|
||||||
|
1. **The requirements checklist** (§2), so the user can correct it.
|
||||||
|
2. **What was searched** — the queries and sources, briefly, so an empty result is credible.
|
||||||
|
3. **A comparison of the real candidates**:
|
||||||
|
|
||||||
|
| Candidate | Covers | Missing / partial | License | Health | User opinion | Fit |
|
||||||
|
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
||||||
|
|
||||||
|
4. **Options**, typically some of:
|
||||||
|
- **Use it as is** — it already does the job; maybe configure or script around it.
|
||||||
|
- **Extend it** — via its plugin/extension API, with no fork.
|
||||||
|
- **Fork and modify** — close, but the gaps need changes to its core.
|
||||||
|
- **Contribute upstream** — the gaps are useful to everyone; send them as PRs (then the
|
||||||
|
project's own rules win, per the precedence table in
|
||||||
|
[`coding-dev.md`](skills-public/coding-dev.md)).
|
||||||
|
- **Combine** — several existing tools or libraries glued together.
|
||||||
|
- **Start fresh** — on top of the libraries from §5, with a short, explicit reason why none of
|
||||||
|
the candidates were a usable base.
|
||||||
|
5. **A recommendation**, with the reason — but the decision is the user's. Do not quietly start
|
||||||
|
on the recommended option. When the user has said to work autonomously and decide without
|
||||||
|
them, take the recommendation and record the choice and the alternatives where they will see
|
||||||
|
it.
|
||||||
|
|
||||||
|
If nothing usable exists, say so plainly, list what was checked, and still mention near misses
|
||||||
|
worth borrowing ideas, formats, or code from (license permitting).
|
||||||
|
|
||||||
|
**Keep hosting out of the proposal.** The options are about *what software* to run, not how to
|
||||||
|
deploy it. Containers, proxies, tunnels, extra helper services, and the database layout belong to
|
||||||
|
the environment's own infrastructure skill, which gets read once an option is chosen and before
|
||||||
|
anything is deployed. Note a hard requirement that affects the choice (it needs Postgres, it
|
||||||
|
needs a GPU) and stop there.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Checklist
|
||||||
|
|
||||||
|
- [ ] Requirements turned into must-haves / nice-to-haves / constraints
|
||||||
|
- [ ] At least three to five varied web searches first, covering the whole idea and its parts
|
||||||
|
- [ ] User reviews, comparisons, and complaints gathered for each serious candidate
|
||||||
|
- [ ] Training-data recall merged in afterwards, each item checked against current sources
|
||||||
|
- [ ] Promising candidates shallow-cloned to a temp directory, read without running, mapped
|
||||||
|
against the checklist, then deleted
|
||||||
|
- [ ] Libraries identified for the likely language
|
||||||
|
- [ ] Options and a recommendation presented; nothing built before the user chose (or before
|
||||||
|
an autonomous decision was recorded)
|
||||||
Reference in new issue
Block a user