feat: best-effort mirror (dual-send) of heartbeats + events

Optional second destination so each heartbeat and event is posted to a
mirror server (the office NAS) alongside the primary — dual-write to
de-risk the migration. Default off (blank mirror URLs); existing installs
unchanged.

- event_forwarder: mirror_reachable() fast-fail probe + mirror_forward_pass()
  (reliable event mirror with its OWN state file, total exception isolation,
  never raises into the primary path).
- series4_ingest: mirror_api_url/mirror_sfm_url/mirror_sfm_state_file config;
  best-effort heartbeat mirror; isolated event-mirror pass after the primary.
- settings dialog: new 'Mirror' tab.
- 9 new tests incl. the isolation invariant (down mirror = fast no-op that
  never touches primary state). 42 passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-24 23:10:50 +00:00
parent dae67afeb9
commit 7d22442bad
5 changed files with 328 additions and 1 deletions
+68
View File
@@ -692,6 +692,74 @@ def forward_pending(
return counts
# ── Mirror (dual-send) ────────────────────────────────────────────────────────
def mirror_reachable(base_url: str, timeout: float = 3.0) -> bool:
"""Quick liveness probe of a mirror SFM base URL.
Run before a mirror forward pass so an unreachable mirror can't stall
the watcher loop on a per-event HTTP timeout for every pending file.
Probes ``<base>/health`` (the SFM server exposes it). ANY HTTP
response — even 404/500 — means the server is up, so proceed; only a
connection-level failure (refused / timed out / DNS) counts as down.
Best-effort: never raises.
"""
if not base_url:
return False
url = base_url.rstrip("/") + "/health"
try:
with urllib.request.urlopen(
urllib.request.Request(url, method="GET"), timeout=timeout
):
return True
except urllib.error.HTTPError:
return True # got a status back → server is alive
except Exception:
return False # refused / timeout / DNS / socket → down
def mirror_forward_pass(
watch_dir: str,
mirror_url: str,
mirror_state: ForwardState,
*,
reachable_fn=mirror_reachable,
reachable_timeout: float = 3.0,
**forward_kwargs: Any,
) -> Optional[Dict[str, int]]:
"""One best-effort forwarding pass against a *mirror* SFM server.
The dual-send entry point. Wraps :func:`forward_pending` with two
properties that keep the mirror from ever harming the primary path:
1. **Fast-fail reachability guard** — probe ``mirror_url`` first; if
it's down, return ``None`` immediately rather than let
:func:`forward_pending` block on a per-event timeout for every
pending file.
2. **Total exception isolation** — any error (probe, forward, state
I/O) is swallowed and reported as ``None``. This function NEVER
raises, so the caller's already-completed primary forward is
untouched.
The mirror keeps its OWN ``ForwardState`` file, so its idempotency
and retry are independent of the primary's. Nothing is lost while
the mirror is down — skipped events stay pending and are delivered
on a later pass once it returns.
Returns the :func:`forward_pending` counts dict on a completed pass,
or ``None`` if the mirror was unreachable or the pass raised.
"""
try:
if not mirror_url:
return None
if not reachable_fn(mirror_url, reachable_timeout):
return None
return forward_pending(watch_dir, mirror_url, mirror_state, **forward_kwargs)
except Exception:
return None
# ── Seed-state mode (skip historical backfill on first deploy) ────────────────