fix: bw_report overlay onto event before DB, prevents data lossdocs: three-tier architecture model + strategic roadmap #27
Assignees
serversdown
Clear assignees
serversdown
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: serversdown/seismo-relay#27
Reference in New Issue
Block a user
CLAUDE.md gains an Architecture section near the top describing the canonical three-tier mental model: - SFM: device-side, live connections, /device/* endpoints - SDM: data-side, DB + waveform store + /db/* endpoints (currently living under sfm/ for historical reasons; rename deferred) - Codec library: pure data-interpretation, used by both tiers Future code should be placed and named according to this model even though the directory layout doesn't fully reflect it yet. Decision rule for where new code goes is documented inline. README.md's Roadmap section gains two strategic-direction subsections: - "Strategic direction" — frames the suite-of-components vision and notes that BW ACH + Thor IDF call-home remain the data movers; seismo-relay's value is on the receiving and processing side. - "Terra-View ↔ SFM device control" — the long-term vision where Terra-View can launch into SFM device-control surfaces (operator notices missing unit → clicks "Connect to Device" → live view in browser). Includes concrete implementation checklist (auth, embedded live-monitor view, action history, series IV live support). The existing tactical roadmap items remain unchanged below. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>Mirror what the ingest path does: BW's reported peaks (and sample_rate / record_time) take precedence over codec output where present. Without this, --force backfill silently overwrites bw_report-overlaid DB columns with codec-derived peaks. Wrong for events where the codec doesn't fully decode (waveform walker edge cases on SP0/SS0/SV0-style events, histogram byte[5]!=0 sub-format that isn't yet RE'd), producing PVS=0 on real high-amplitude events. Bit on prod 2026-05-22 with three top-10 waveform events ending up at PVS=0 (rolled back same day, this fix is the proper resolution). New helper minimateplus.event_file_io.apply_bw_report_dict_to_event operates on the projected sidecar dict shape (the structure _bw_report_to_dict produces, which is what gets preserved in the sidecar). Mirrors apply_report_to_event's semantics: only writes fields where bw_report has a non-None value, no-ops cleanly on empty / None input. Dev validation against prod snapshot: pre : 1839.7315 pvs_sum 356 events with DB PVS ≠ sidecar bw_report post : 2016.4902 pvs_sum 2 events still mismatched (both have NULL timestamp + duplicate rows, edge case) Both edge-case events DO get the correct value written by the new backfill — their stale rows from prior backfills remain because UNIQUE(serial, timestamp) doesn't fire on NULL. Separate dedup cleanup needed for those 2 events (0.014% of corpus); not blocking. Backfill remains idempotent + bw_report preservation still passes (0 WIPED, 0 CHANGED on the 3rd consecutive run). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>