From 730f215e23c83fd8d721de27bab4770e846307da Mon Sep 17 00:00:00 2001 From: serversdown Date: Wed, 24 Jun 2026 23:02:47 +0000 Subject: [PATCH] docs: mirror dual-send design spec Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/mirror-dual-send-design.md | 51 +++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 docs/mirror-dual-send-design.md diff --git a/docs/mirror-dual-send-design.md b/docs/mirror-dual-send-design.md new file mode 100644 index 0000000..430a071 --- /dev/null +++ b/docs/mirror-dual-send-design.md @@ -0,0 +1,51 @@ +# Watcher Mirror (Dual-Send) — Design + +**Date:** 2026-06-24 · **Branch:** `feat/mirror-dual-send` (both `thor-watcher` + `series3-watcher`) · **Status:** approved, implementing + +## Goal +Let each watcher post every heartbeat and event to a **second ("mirror") server** in addition to the primary, so the office NAS stays a continuous replica of prod during the migration. The eventual cutover becomes a non-event. **Default off** → existing installs unchanged. + +## Pattern +Classic dual-write migration. The primary path stays **exactly as reliable as today**; the mirror is pure best-effort bonus that can **never delay or fail** the primary. + +## The two watchers (parallel changes) +| | thor-watcher (series4) | series3-watcher (s3) | +|---|---|---| +| Config | `config.json` (JSON) | `config.ini` (`[agent]`, INI) | +| Heartbeat fn | `send_api_payload(payload, api_url, timeout)` | `send_api_payload(payload, api_url)` | +| Event fn | `event_forwarder.forward_pending(root, sfm_url, state, …)` | same (`event_forwarder` is a sibling port) | +| Loop | `series4_ingest.run_watcher` | `series3_watcher.run_watcher` | +| Settings GUI | `thor_settings_dialog.py` | `settings_dialog.py` | + +## Design (Approach A — reliable event mirror, best-effort heartbeat) + +### Config additions (default empty = off) +- thor (`config.json`): `mirror_api_url`, `mirror_sfm_url`, `mirror_sfm_state_file` (blank → `/thor_forwarded_mirror.json`). +- s3 (`config.ini`): `MIRROR_API_URL`, `MIRROR_SFM_URL`, `MIRROR_SFM_STATE_FILE` (blank → `sfm_forwarded_mirror.json`). + +The mirror **rides along** with the primary: heartbeat-mirror active iff primary heartbeat on **and** `mirror_api_url` set; event-mirror active iff primary forwarding on **and** `mirror_sfm_url` set. No separate enable flag. + +### Heartbeat mirror (best-effort) +After the primary `send_api_payload`, if `mirror_api_url` is set, fire one more `send_api_payload(payload, mirror_api_url, …)`. Wrapped so it can never raise into the loop; result ignored except a debug log. Bounded by the existing short API timeout (~5 s). + +### Event mirror (reliable, isolated) +- Init a **second** `ForwardState(mirror_state_path)` alongside the primary state. +- Each forward tick, **after** the primary `forward_pending`, run a second `forward_pending(root, mirror_sfm_url, mirror_state, …)` in its **own** try/except. +- **Reachability guard:** before the mirror forward, a quick TCP/HTTP probe of `mirror_sfm_url` (~3 s). If unreachable → skip this pass, log, retry next tick. Prevents a down NAS from stalling the loop on per-event timeouts. The mirror state file means skipped events stay pending → delivered when the NAS returns. **No data loss.** +- Reliable + idempotent via its own sha256 state, fully independent of the primary's state. + +### Isolation invariant (the one rule) +Nothing on the mirror path can delay or fail the primary: separate state file, its own try/except, the reachability guard + bounded timeouts, all exceptions swallowed-and-logged. + +### Settings dialog +Add `Mirror API URL` + `Mirror SFM URL` fields (blank = off) to each watcher's settings dialog. + +## Seeding (operational, for this migration) +The mirror state file should reflect what the NAS already has (everything ≤ the migration snapshot, ~16:23 on 2026-06-24). Recipe: copy the primary state file → mirror state file, then drop entries whose `forwarded_at` is after the snapshot, so the mirror re-delivers exactly the gap. SFM-side dedup covers any overlap. (If a watcher had no events in the gap, a straight copy suffices.) + +## Testing +- thor `test_event_forwarder.py`: mirror uses a separate state file (idempotent, independent); a mirror failure leaves the primary's result + state untouched; reachability guard skips cleanly when the mirror is down. +- s3: add a minimal test for the same isolation/idempotency. + +## Out of scope / handoff +Rebuilding the Windows installers + redeploying (s3 on the Win7 box, thor on the Win10 box) is the operator's job — this change is **code + tests only**.