1024x1024 master, favicon
- kopi-docka-800x200.png (72 KB): wide README header banner — used by
the redesigned title in the preceding commit
- kopi-docka-1280x640.png (177 KB): GitHub Open Graph aspect — upload
via Repo Settings → Social preview to control the link card on
X/HN/Reddit/Discord (manual UI step, not scriptable via gh CLI)
- kopi-docka-logo.png (1.2 MB): 1024x1024 master for downstream
derivations (presentations, DR-bundle templates)
- kopi-docka-favicon.png (168 KB): 420x420 for any future doc site
Total +1.58 MB to the repo. All four sit alongside the existing
demo-*.svg files under docs/media/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7 individual SVG embeds back-to-back made the README quite scroll-heavy.
This combines all 7 casts into a single 113-second animation that plays
the scenes sequentially with brief banner cards between them
("1 / 7 · doctor · system health check", etc.), then loops.
The individual SVG files are preserved (other docs may reference them
directly) and exposed as a collapsible "Individual scenes" table below
the combined animation, so the user-flow narrative is still visible at
a glance without expanding anything.
Combined SVG is 743 KB — larger than any single scene but acceptable
inline; GitHub renders animated SVGs without lazy-loading anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real user hit this on Windows: PowerShell 5.1's `>` operator re-encodes
stdout as UTF-16, doubling the byte count and corrupting the AES ZIP
beyond recognition. Even 7-Zip rejects the file ("not an archive").
Confirmed: locally a clean stream is 7 KB; the PS-corrupted version
arrives at 14 KB. cmd.exe's `>` works correctly.
Three touch-points so this is hard to miss:
1. `--stream` help text in disaster_recovery_commands.py now mentions
the PowerShell pitfall and points at docs/DISASTER_RECOVERY.md.
2. README.md gains an inline Windows note next to the stream example
plus a stronger 7-Zip hint at extract time.
3. docs/DISASTER_RECOVERY.md "SSH Stream Mode" section grows two
subsections: "Windows clients" (scp + cmd.exe patterns, verification
command) and "Extracting on Windows" (Explorer's lack of AES-256
support, recommended tools).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bare 'disaster-recovery export ... --yes' demo was too thin — one
panel, just file path and decrypt commands. The legacy
'disaster-recovery' (no subcommand) path triggers a more substantive
flow that's actually nicer for HN/r/selfhosted visitors:
1. Deprecation-warning panel → demonstrates the tool actively
guides users toward the new ZIP format (a trust signal — it
doesn't just silently break)
2. "Disaster Recovery Bundle Creation" intro panel
3. Progress log (rclone.conf added, password sidecar warning)
4. "Bundle Created" panel with all three file paths and the
openssl-decrypt command inline
Source: real transcript provided by the user against the testlab,
faithfully reconstructed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Completes the narrative arc on the README from "setup the tool" to
"recover from total loss":
Doctor → Dry-run → Backup → History → Snapshot list → Restore → DR export
- history.svg (44 KB, ~6s): Backup History table with timestamps,
duration, status, scope, volumes, snapshot IDs. Operations / "did the
backups actually run?" sight.
- snapshot-list.svg (58 KB, ~8s): kopi-docka advanced snapshot list
--snapshots — shows the tag-based organisation (every snapshot tagged
with backup_id/type/unit so the wizard can group recipe + networks +
volumes from one backup_id as a unit). Direct illustration of the
Kopia-tag differentiator vs. archive-name-based tools.
- dr-export.svg (39 KB, ~8s): disaster-recovery export — shows the
encrypted-ZIP bundle creation panel, including the "passphrase NOT
stored in file" reminder.
All three captured from real testlab runs against the rclone+GDrive
backend (32 existing snapshots in the repo).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The --yes (CI/CD) flow worked but missed the actual story: the wizard
is where the Kopia-tag-based session grouping becomes visible (each
"backup session" in the list is one backup_id tying recipe + networks
+ volumes), and where the safety-backup-before-overwrite step earns
user trust.
Recording reconstructed from a real interactive session captured by
the user on the testlab. Compressed from ~2min real time to ~34s while
keeping every decision point visible: session selection, "Recreate
network?" prompt, "Restore volume NOW?" confirmation, target-directory
prompt, completion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follow-up to the doctor/dry-run demos — adds the two missing pieces of
the natural user journey:
- demo-backup.svg (23 KB, ~7s): real cold backup via SFTP. Shows
per-stack container-stop → snapshot → restart cycle for both
test-stack-redis and test-stack-nginx. Real exit 0 capture from the
E2E SFTP test environment.
- demo-restore.svg (93 KB, ~10s): non-interactive 'restore --yes' run
showing the restore-wizard panel, snapshot listing, file restore,
network recreation, and container restart.
Order on the README is now: doctor → dry-run → backup → restore, matching
the actual user workflow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
HN/r/selfhosted-prep: visitors land on the README and want to see
"what does this actually do" in ten seconds, before reading any prose.
Two short animated SVGs (~7-9s each) embedded inline:
- demo-dry-run.svg: discovers the test-lab stacks (redis + nginx),
prints the dry-run plan with estimates.
- demo-doctor.svg: dependency check, repository status, backend sanity,
DR-readiness — the "this tool checks itself" angle.
SVGs are 56 KB / 88 KB, text-selectable when zoomed, rendered by GitHub
inline as raw.githubusercontent.com URLs (works on both GitHub and PyPI
README rendering — the URLs are absolute).
Generated with termtosvg from manually-constructed asciicast files
(the testlab session was too long to script with expect; output captured
once and timed-out for typing animation, ~8s each).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the 8 helper modules previously absent from ARCHITECTURE.md
(backend_helper, sudo_helper, metadata_reader, docker_run_builder,
file_operations, process_lock, system_utils, logging, ui_utils,
constants) — each with a one-paragraph purpose + signature table where
the API is non-trivial.
Adds a new "Repair pattern (rebuild_kopia_params)" section documenting
how advanced config repair-kopia-params dispatches generically through
BackendBase + MissingCredentialsError, with the extension point for
future per-backend repair logic.
Also: final Plan-XXX sweep — README's tested-backends table now uses
version anchors (v7.4.0 + v7.6.1) instead of slug refs.
Plan 0039 — Block D.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Internal planning artifacts ("Plan 0022", "Plan 0028") have no meaning to
end users reading the docs — rewrite to point at the release that
delivered the change (e.g. "since v7.3.0") instead. 8 occurrences across
FEATURES, CONFIGURATION, ARCHITECTURE, TROUBLESHOOTING.
Plan slugs are kept in CHANGELOG section headers where they serve as
release-notes shorthand (e.g. "v7.3.0 — Plan 0028 + post-release"), and
in the plan/ directory itself.
Plan 0039 — Block B.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New "What's NOT in the bundle (and why)" section in
docs/DISASTER_RECOVERY.md spells out:
* Which credentials live OUTSIDE the bundle per backend type (SSH key,
AWS keys, B2 keys, Azure key, GCP service-account JSON).
* The defense-in-depth / NIST SP 800-57 key-separation rationale and a
brief note that restic, Borg, Duplicity, rclone crypt all default to
the same posture.
* Suggested storage for the SSH key (password-manager attachment,
separate USB stick, GPG-symmetric, paper printout).
* The --include-ssh-key opt-in flag with its trade-off explicit.
Plan 0030 documentation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CONFIGURATION.md and TROUBLESHOOTING.md previously instructed
v7.0–v7.3.13 Tailscale users to copy/paste a sed line emitted by
doctor. v7.5.0 ships a real command for this; the docs now show that
instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- CONFIGURATION.md: Tailscale backend section now shows the correct
per-flag kopia_params shape (--path / --host / --username / --keyfile
/ --known-hosts) and links to the migration writeup. Also documents
the Unraid 6.12+ symlink detection.
- TROUBLESHOOTING.md: new "Tailscale-SFTP kopia_params migration"
section walks v7.0.0 – v7.3.13 users through `doctor` → sed →
`advanced repo init`. Adjacent note explains the Unraid persistent-
path handling so users don't worry when the wizard skips the mirror.
- FEATURES.md: Tailscale "Technical Details" lists the v7.4.0
correctness invariants (real FQDN, separate flags, inode-aware
Unraid mirror).
Plan 0029 documentation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported multi-minute waits on a real rclone+GDrive system
(296 s for one `kopia policy set --global`, 160 s for a couple-of-KB
recipe snapshot). Searched the Kopia community — turns out this is
well-documented upstream:
- Kopia CLI docs mark rclone backend as "Not maintained".
- Forum (kopia.discourse.group/t/.../2597): `repository status` 28 s
rclone-GDrive vs 0.8 s SFTP; 40+ min maintenance on a 100 GB rclone
repo vs seconds locally.
- Root cause is Kopia spawning fresh `rclone serve` per call + GDrive's
small-file write overhead — not something kopi-docka can paper over.
This release surfaces the issue everywhere a user would look:
- docs/TROUBLESHOOTING.md: new "Backups against rclone+Google Drive
feel very slow" section with measured numbers, upstream links, and
comparison table (Tailscale → SFTP, Backblaze B2, native gdrive
with its 30 GB service-account-quota caveat).
- docs/CONFIGURATION.md Storage Backends: performance reality check
callout pointing at the troubleshooting entry.
- doctor section 4 (Backend Dependencies): yellow performance warning
when backend is rclone, with extra paragraph if kopia_params look
like Google Drive. Also added to the bottom warnings panel.
Hot-path code unchanged; this is a documentation + observability fix.
1132 unit tests passing. Doctor warning live-verified on the rclone
testlab — yellow block appears under section 4 and the bottom panel
gets a 1-line summary.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Typer CLI itself has been registering both names for a while —
`advanced` as the documented top-level group and `admin` as a
hidden=True alias — but help texts, docstrings, error hints, the
disaster-recovery shell snippets and the bundled systemd templates
mixed the two. `kopi-docka --help` showed `advanced`, but `kopi-docka
advanced policy --help` then listed `prune` with a doc string saying
"use `kopi-docka admin config edit`", and the shipped
`kopi-docka.service` template ran `kopi-docka admin service daemon`.
134 references across 23 files unified to `advanced`. The `admin` name
stays registered as `hidden=True` on the same Typer app, so every
existing user script / cron / systemd unit that still says `admin`
keeps working — this is a documentation / display fix, not a CLI
break.
Notable churn:
- All `kopi_docka/commands/advanced/*.py` module docstrings + help text
- Backend recovery hints in `kopi_docka/backends/rclone.py`
- DR-bundle shell snippets in `cores/disaster_recovery_manager.py`
- Systemd templates (`kopi-docka.service.template`,
`kopi-docka.timer.template`, `kopi-docka-backup.service.template`,
`templates/systemd/README.md`) now use `advanced service daemon` /
`advanced service write-units` for fresh installs
- Top-level docs (CLAUDE.md, CHANGELOG.md, docs/USAGE.md,
docs/FEATURES.md)
Tests: 1124 passing. `kopi-docka --help`, `kopi-docka advanced --help`,
and the hidden `kopi-docka admin snapshot --help` all confirmed working
locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round two of UX cleanup on scripts/migrate-config.sh after a real
production config (full notification setup, legacy hooks,
retention.yearly from an old release) exposed several false positives
and a missing template field.
Fixed
- Stop flagging `template=null` vs scalar as a type mismatch. A null
value in the template marks a "not configured yet" slot —
kopia.password_file, notifications.service, notifications.url and
similar. The live system produced pages of false-positive `!` lines.
Now suppressed in either direction; only genuine clashes surface.
- Trailing empty `! ` entry in the type-mismatch block removed. The
block was built with `+= "$line"$'\n'` plus a terminating newline,
printing one blank `! ` at the end. Switched to a proper bash array
and `printf '%s\n' "${arr[@]}"`.
- Template now ships backup.database_backup. The Pydantic schema has
had it (default "true") since 5.x but config_template.json was
missing it — every install that still had the key got it flagged
as "unknown" on first migrate.
Documentation
- Known legacy renames table in docs/CONFIGURATION.md. The script
doesn't auto-rename (no opinion on your data), but the doc tells you
which "Unknown" lines (retention.yearly → annual, pre/post_backup_hook
→ backup.hooks.*) are safe to --prune-unknown once you've copied the
value to the new key.
Tests: 1124 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UX hardening after the first live-system run of v7.3.2 surfaced
"tried: <empty>" with no actionable hint.
Fixed
- Template auto-locate now has a four-step strategy chain instead of
a single python3 import:
1. --template flag (explicit override)
2. default `python3 -c 'import kopi_docka'`
3. python from `which kopi-docka`'s shebang (handles pipx / venv
installs where /usr/bin/python3 can't import the package)
4. GitHub raw fallback (no install required)
Every failed step logs why; the final "could not locate" error
lists all four strategies with how to fix each one.
- Cleanup trap unified: previous version had two `trap … EXIT`
clauses that clobbered each other, so the GitHub-fallback tempfile
could leak.
- Docs chain `chmod +x` on the same line as `curl -o /usr/local/bin/…`
so first-time copy-paste users don't get a "Permission denied"
after the download.
Added
- `--config` is now optional. When omitted, the script probes the
same defaults kopi-docka itself uses
($HOME/.config/kopi-docka/config.json first, then
/etc/kopi-docka.json), honoring $SUDO_USER so `sudo` finds the
invoking user's per-user config instead of root's.
- Version banner: every run prints `kopi-docka installed: X.Y.Z
(/path/to/bin)` or `<not found on PATH — will use the GitHub-hosted
template>`. Makes which release the migration is checking against
visible up front.
E2E verified on the testlab:
- explicit --config: dry-run + real run + doctor green + rollback
sha256-identical to the original
- no --config: probes user/etc defaults, found the testlab profile
- stubbed PATH without python3 and kopi-docka: banner reports
"not found", GitHub fallback fetches the template, diff produces
the same missing/unknown buckets as the local run
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three usage styles now documented in both CONFIGURATION.md and
TROUBLESHOOTING.md:
1. One-shot from GitHub (curl | sudo bash -s --): no clone, no install,
single line that downloads and runs in one go. Most useful for the
"I just upgraded, fix my config" case.
2. Download once to /usr/local/bin/kopi-docka-migrate-config — for
users who'd rather inspect the script before running it.
3. From a source checkout — for contributors and people who already
have the repo cloned.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Added
- scripts/migrate-config.sh: bash + jq helper that diffs an existing
kopi-docka.json against the template shipped with the installed
kopi-docka and fills in missing keys with template defaults.
Nothing hard-coded — the script reads `paths(...)` off both files
and reports missing / unknown / type-mismatch buckets. Default is
"add missing only" + timestamped backup; --prune-unknown opts into
deleting deprecated keys.
Docs: CONFIGURATION.md "Migrating an older config" section, plus a
one-paragraph pointer from TROUBLESHOOTING.md.
Fixed
- Config template was missing `kopia.profile`. The Pydantic schema
has had it (default "kopi-docka") since 5.x, but config_template.json
didn't ship the field. `kopi-docka advanced config new` generated
configs without a visible profile entry, and the new migration
helper would otherwise flag every multi-profile install's
kopia.profile as "unknown" on first run.
Tests: 1124 passing. Manual verification on the rclone+GDrive testlab:
- dry-run shows 4 missing + 2 deprecated keys (parallel_workers,
task_timeout) as expected
- real run + doctor → Health Check Passed
- rollback via the .backup-YYYYMMDD-HHMMSS file restores the
original byte-for-byte (sha256 verified)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three loose ends after Plan 0028 finalize:
1. Docs
- docs/CONFIGURATION.md: replace the old "Policy State Cache" /
"smart-skip" section with a "Global Retention Policy (since v7.3.0)"
explanation. Trim the historical v5.3.0 path-mismatch story to one
paragraph (Plan 0028 makes the per-path-vs-virtual-path question
moot anyway).
- docs/ARCHITECTURE.md: update BackupManager step list, method
table, and the backup sequence diagram to show
_collect_backup_sources() + sequential repo.create_snapshots(),
instead of the parallel ThreadPool + _ensure_policies path.
- docs/diagrams/04_sequenceDiagram.mmd: regenerated to match.
- docs/architecture_components.json: KopiaPolicyManager method list
updated (list_policies / delete_* in, set_*_for_target out);
KopiaRepository gains create_snapshots.
2. Migration
- New _maybe_cleanup_legacy_state_files() in KopiaRepository removes
the obsolete ~/.config/kopi-docka/policy_state.json (smart-skip
cache from v7.2.0). Idempotent, runs on first _run() call after
upgrade; OSError is downgraded to debug — housekeeping, not a
hard requirement.
- Live-verified on the rclone+GDrive testlab: created an empty
policy_state.json, ran doctor → file removed with the expected
INFO log line, doctor reports clean state.
3. Config templates / dead fields
- kopi_docka/templates/config_template.json: drop parallel_workers
and task_timeout (no longer consumed under Plan 0028).
- kopi_docka/helpers/config.py: keep both Pydantic fields but mark
them "Deprecated since v7.3.0 — ignored" in the description so
existing kopi-docka.json files still validate.
- BackupManager.__init__: remove the dead self.max_workers
assignment that survived Phase 3.
- DryRunReport: drop the "Parallel Workers" line from system info
and config review sections.
Tests: 1121 unit tests pass. New TestLegacyStateCleanup (3 tests) pins
the file-removal contract. test_dry_run_manager.py updated to assert
"Parallel Workers" is absent from the report.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- KopiaCommandError: replaces bare RuntimeError in _run(), carries returncode + stderr_tail (UTF-8 safe)
- is_connected(force_refresh=True): bypasses 60s cache for pre-flight check; negative result also cached
- BackendUnreachableError: new exception in backends/base.py (subclass of ConnectionError)
- Pre-flight backend check in backup_unit() before container teardown; containers NOT stopped on failure
- BackupErrorDetail dataclass in types.py; BackupMetadata.error_details persisted in JSON (backward-compat)
- Verbose failure notifications: phase, exit code, stderr tail in fenced code block (Markdown-injection-safe)
- Markdown body_format for all Apprise sends; services without Markdown degrade gracefully
- send_connectivity_alert() and send_missed_backup_alert() added to NotificationManager
- MissedBackupChecker: time-based overdue detection using MetadataReader; per-unit threshold overrides
- Post-run missed-backup check after every backup_unit(); alert suppression via missed_state.json
- Doctor section 8 "Backup Freshness": age per unit, OVERDUE units highlighted
- Config: notifications.verbose, notifications.preflight_check, alerting.missed_backup.*
- Config templates and examples updated; migration guide in INSTALLATION.md
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
New `kopi-docka history` command to browse past backups from stored
metadata JSONs. Includes table view, detail panels, filters (--unit,
--failed, --last, --since), statistics (--stats), and JSON output
(--json). No root privileges required.
Adds MetadataReader helper and BackupMetadata.from_dict() for reusable
metadata deserialization. 43 new tests, 95%/99% coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- #71: Remove invalid `show=True` argument from cmd_config() calls
- #70: Implement missing get_backend_class() in backends/__init__.py
- #69: Add rsync as SOFT dependency in dependency_manager and docs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New docs/DISASTER_RECOVERY.md with full DR documentation:
ZIP export usage, SSH streaming, recovery walkthrough,
CLI reference, passphrase security, format comparison,
storage best practices, troubleshooting
- README.md: updated DR section, CLI examples, documentation links
- FEATURES.md: added v6.2.0 ZIP export info, updated CLI listing
- USAGE.md: added export subcommand, updated CLI structure and
emergency recovery steps for both ZIP and legacy format
- Add DockerRunBuilder helper to parse inspect.json and reconstruct docker run commands
- Integrate automatic container reconstruction in restore workflow
- Interactive prompts to start containers immediately after restore
- Support for all common Docker parameters (ports, volumes, env, networks, capabilities, etc.)
- Filter Docker-injected environment variables
- Check for existing containers before attempting start
- Add 30+ comprehensive unit tests
- Update documentation (CHANGELOG, USAGE, FEATURES)
- Bump version to 6.1.0
Closes#59
Major refactoring implementing "Think Simple" philosophy:
Kopi-Docka now expects a prepared system instead of automatic installation.
BREAKING CHANGES:
- Removed automatic dependency installation
- Removed distro detection logic
- Removed install-deps command
- Removed distro library dependency
Added:
- Hard/Soft Gate dependency system
- DependencyHelper utility class
- Server-Baukasten integration
- Enhanced doctor command with categories
- Comprehensive test suite (97 new/updated tests)
- Backend dependency enforcement
Changed:
- DependencyManager simplified (40% code reduction)
- All backends refactored with REQUIRED_TOOLS
- Commands now check dependencies at entry point
- Documentation completely rewritten
Migration:
- Users must install Docker + Kopia before upgrading
- Use Server-Baukasten for automated system setup
- See docs/INSTALLATION.md for details
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enhanced email setup wizard to include display name for better sender identification.
## Fixed
- Email notifications now show custom display name in sender field
- Example: 'Kopi-Docka Backup <user@gmail.com>' instead of 'user@gmail.com'
- Improves email readability and professionalism
## Changed
- notification_commands.py:
- Added 'Display Name' prompt in _setup_email()
- Automatically builds 'from' parameter with display name
- Default: 'Kopi-Docka'
- Format: DisplayName<email@example.com>
- Documentation updates:
- docs/NOTIFICATIONS.md: Added display name to setup steps
- docs/CONFIGURATION.md: Updated email example with from parameter
- CHANGELOG.md: Added v5.4.1 release notes
- Version bump:
- pyproject.toml: 5.4.0 → 5.4.1
- kopi_docka/helpers/constants.py: VERSION = '5.4.1'
- kopi_docka/__init__.py: @version 5.4.1
## Technical Details
The from parameter is now automatically constructed as:
```
from_email = f"{display_name}<{username}>"
url = f"mailto://{username}@{smtp_server}:{smtp_port}?to={recipient}&from={from_email}"
```
This follows the standard email format and is properly handled by Apprise.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
- Dynamic version display in --help header
- Rename 'admin' to 'advanced' command group
- Hide wrapper commands (keep functional for backward compatibility)
- Update version to 5.3.1
- Update documentation (CHANGELOG, USAGE, README)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
TZERO78