CHANGELOG.md

@@ -5,6 +5,56 @@ All notable changes to Terra-View will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.10.0] - 2026-05-14
+
+This release brings terra-view onto the SFM (Seismograph Field Module) event pipeline. Triggered events forwarded by series3-watcher now land in SFM, and terra-view reads from that store as the authoritative source for vibration data. The watcher heartbeat is preserved as a transparent fallback signal.
+
+### Added
+- **SFM Integration**: New fleet-wide events page at `/sfm` listing every event ingested by SFM, with filters for serial, date range, false-trigger flag, and limit. Unit detail pages and project-location pages show their own attributed subsets of the same event stream.
+- **Event Detail Modal**: Shared across `/sfm`, unit detail, and project-location pages — clicking any event opens a rich modal showing peaks per channel (PVS color-coded by magnitude), microphone dB(L) + ZC frequency + time of peak, sensor self-check table with pass/fail per channel, device/recording metadata (firmware, battery, calibration date, geo range), and download buttons for the original Blastware binary and the sidecar JSON. Includes an inline pretty-printed JSON viewer with copy-to-clipboard.
+- **Events Attribution Engine** (`backend/services/sfm_events.py`): Per-event attribution against `UnitAssignment` time windows. Events outside any assignment window surface in an "Unattributed" bucket with the nearest-assignment diagnostic (which location, signed delta in days).
+- **Metadata Backfill Tool** (`/tools` → Backfill from event metadata): Scans operator-typed `project` and `sensor_location` strings in event sidecars, fuzzy-clusters them via `rapidfuzz.WRatio`, and proposes retroactive `UnitAssignment` records to attribute orphan events. Tracks operator decisions per cluster across re-scans.
+- **Project Tidy Tool** (`/tools` → Project Tidy): Fuzzy-detect duplicate projects and bulk-merge them with a single click. Source projects soft-deleted with full audit trail.
+- **Vibration Summary on Project Pages**: New roll-up card on vibration project detail pages showing per-location event counts, the project's "Overall Peak" PVS (false triggers excluded), last event timestamp, and a Top Locations by Activity list.
+- **SFM-Primary Seismograph Status**: `emit_status_snapshot()` now consults SFM's `/db/units` (cached 15s) before falling back to `Emitter.last_seen` for each seismograph. The fresher signal wins; the choice is recorded in a new per-unit `last_seen_source` field. A small `SFM` (orange) or `HB` (gray) badge on each unit's active-table row shows which path is currently driving the status.
+- **Dashboard Rework**: Top row reordered to Recent Alerts → Recent Call-Ins (double-wide) → Fleet Summary. Today's Schedule moved to a horizontal collapsible card below the Fleet Map, auto-expanding only when pending actions exist. Recent Call-Ins now sources from a new `/api/recent-event-callins` endpoint backed by SFM event forwards instead of the watcher-heartbeat endpoint.
+- **Sortable Events Tables**: `/sfm` and unit-detail SFM Events tables now have clickable column headers with ↕/↓/↑ indicators. Default sort is Timestamp DESC. Click same column to toggle direction; click different column to switch and reset to DESC. Pure client-side over cached rows — no re-fetches.
+- **Developer → SFM Admin** (`/admin/sfm`): Health banner with reachability indicator, terra-view↔SFM connection panel, 4 KPI tiles (known units, total events, stale `monitor_log` rows, stale `ach_sessions` rows), per-unit roll-up table, recent-events table with color-coded forwarding latency (so stale watcher forwards stand out), and a raw API tester for any `/api/sfm/*` path.
+- **Developer → SLMM Admin** (`/admin/slmm`): Stripped-down companion page — health, connection info, raw API tester.
+- **Tools Workflow Hub** (`/tools`): New top-level sidebar entry consolidating Pair Devices, Project Tidy, Metadata Backfill, Reports (info card), and Swap Detection (placeholder).
+- **Sidebar Reorganization**: Devices → Projects → Events → Tools → Job Planner → Settings. Devices is now a single entry with internal tabs (All Devices / Seismographs / Sound Level Meters / Modems / Pair Devices) replacing five separate sidebar items.
+- **Synology Deployment Doc** (`docs/SYNOLOGY_DEPLOYMENT.md`): End-to-end playbook for migrating the stack to an always-on office NAS — phased rollout (pre-stage, data rsync, watcher repoint, external access, decommission), Tailscale vs reverse-proxy options, rollback plan, and gotchas.
+
+### Changed
+- **Overall Peak excludes false triggers**: The project-level "Overall Peak" KPI tile (and the underlying `_compute_stats()` function in `sfm_events.py`) now skip events flagged as false triggers when computing the highest PVS, so operators see the highest real event rather than the biggest sensor glitch. `false_trigger_count` still includes flagged events so operators can see how many were filtered out.
+- **`RosterUnit.note` Editing**: Inline edit on seismograph cards is more forgiving and now auto-saves on blur.
+- **Sidebar Nav Renamed**: Old "Fleet" sidebar entry → "Devices" (renamed because it always meant the device list, not the broader fleet view).
+
+### Fixed
+- **Status drift between watcher heartbeat and actual event arrivals**: Seismographs are now reported with whichever signal is more recent — eliminates the case where a unit had recent SFM events but a stale heartbeat (or vice-versa) showed the wrong status.
+- **Event modal: Record Type always showed "Waveform"**: Workaround client-side — Record Type now derived from the Blastware filename's last-char code (`H`=Histogram, `W`=Waveform, `M`=Manual, `E`=Event, `C`=Combo). The proper fix lives in SFM's sidecar parser; tracked separately.
+- **Event modal: Mic PSI tile removed**: Operators only care about dB(L); the redundant PSI tile was dropped.
+
+### Migration Notes
+Run on each database before deploying. Every migration is idempotent.
+
+```bash
+# Cleanest: re-run all migrations in chronological order.
+# Already-applied migrations no-op safely.
+for f in backend/migrate_*.py; do
+  docker exec terra-view-terra-view-1 python3 "/app/backend/$(basename $f)"
+done
+```
+
+Migrations new in this release:
+- `migrate_add_metadata_backfill.py` — adds `unit_assignments.source` column and `metadata_backfill_decisions` table for the Metadata Backfill tool
+
+### Deployment Notes
+- **`SFM_BASE_URL`**: Confirm prod's `docker-compose.yml` sets this for the terra-view service (typically `http://sfm:8200` for the in-stack SFM container, or an external URL if SFM lives elsewhere).
+- **Watcher repoint**: series3-watcher's `sfm_forward_url` should point at `https://<your-terra-view-host>/api/sfm` (proxy-based — no second port forward needed). Watcher composes the full path `/db/import/blastware_file` itself.
+
+---
+
 ## [0.9.4] - 2026-04-06
 
 ### Added

README.md

@@ -1,4 +1,4 @@
-# Terra-View v0.9.4
+# Terra-View v0.10.0
 Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard.
 
 ## Features
@@ -496,6 +496,19 @@ docker compose down -v
 
 ## Release Highlights
 
+### v0.10.0 — 2026-05-14
+- **SFM Integration**: terra-view now consumes events from the SFM (Seismograph Field Module) backend in real time, with a fleet-wide events page at `/sfm`, per-unit attribution against project assignment windows, and a project-level vibration roll-up that uses SFM data as the single source of truth.
+- **SFM-Primary Seismograph Status**: Deployed seismograph status (OK/Pending/Missing) now flows from SFM event forwards first; the watcher heartbeat stays as a transparent backup. Each unit's active table row shows a small `SFM` or `HB` badge so operators can see at a glance which signal is currently driving the status.
+- **Dashboard Rework**: Top row reordered to Recent Alerts → Recent Call-Ins (double-wide) → Fleet Summary. Today's Schedule moves to a horizontal collapsible card below the Fleet Map, auto-expanding only when there's a pending action. Recent Call-Ins now sources from SFM event forwards instead of the legacy watcher-heartbeat endpoint.
+- **Event Detail Modal**: Click any event anywhere in the app to open a rich detail modal showing peak particle velocity per channel, microphone dB(L), sensor self-check results, device/recording metadata, and download buttons for the original Blastware binary and sidecar JSON. Includes an inline JSON viewer with one-click copy.
+- **Sortable Events Tables**: Every events table (project events, unit-detail events, fleet-wide /sfm) now supports clickable column-header sorting with directional indicators. Defaults to newest-first.
+- **Events Attribution & Backfill**: Each SFM event is automatically attributed to a project/location based on `UnitAssignment` time windows. Unattributed events get a diagnostic showing the nearest assignment and a delta-days gap. The metadata-backfill tool in `/tools` scans operator-typed project/sensor-location strings in event sidecars and clusters them via fuzzy matching to propose new assignment retroactives.
+- **Projects Tools**: New `/tools` workflow hub consolidates Pair Devices, Project Tidy (fuzzy-detect + merge duplicate projects), Metadata Backfill, Reports, and Swap Detection (placeholder).
+- **Sidebar Reorganization**: Devices → Projects → Events → Tools → Job Planner → Settings. Devices is now a single entry with internal tabs (All Devices / Seismographs / Sound Level Meters / Modems / Pair Devices).
+- **Developer → SFM Admin**: New `/admin/sfm` page surfacing SFM health, per-unit roll-up from `/db/units`, recent-events table with forwarding latency (so operators can spot stale watcher forwards), stale-table counts, and a raw API tester. Companion `/admin/slmm` page covers SLMM health + raw API.
+- **"Overall Peak" excludes False Triggers**: The project-level Overall Peak KPI tile now excludes events flagged as false triggers — operators see the highest real event, not the biggest sensor glitch.
+- **Synology Deployment Doc**: New `docs/SYNOLOGY_DEPLOYMENT.md` covers migrating the stack to an always-on office NAS, including phased rollout, data rsync, watcher repoint, external-access (Tailscale or reverse-proxy), and rollback plan.
+
 ### v0.8.0 — 2026-03-18
 - **Watcher Manager**: Admin page for monitoring field watcher agents with live status cards, log tails, and one-click update triggering
 - **Watcher Status Fix**: Agent status now reflects heartbeat connectivity (missing if not heard from in >60 min) rather than unit-level data staleness
@@ -599,9 +612,21 @@ MIT
 
 ## Version
 
-**Current: 0.8.0** — Watcher Manager admin page, live agent status refresh, watcher connectivity-based status (2026-03-18)
+**Current: 0.10.0** — SFM integration, SFM-primary seismograph status, dashboard rework, sortable events tables, event detail modal, /admin/sfm + /admin/slmm diagnostic pages, Tools workflow hub (2026-05-14)
 
-Previous: 0.7.1 — Out-for-calibration status, reservation modal, migration fixes (2026-03-12)
+Previous: 0.9.4 — Modular project types, deleted project management, swap modal search, roster auto-refresh fix (2026-04-06)
+
+0.9.3 — Monitoring session detail page, configurable period windows, vibration project redesign, modem assignment on locations (2026-03-28)
+
+0.9.2 — Deployment records, allocated status, quick-info unit modal, inline seismograph editing (2026-03-27)
+
+0.9.1 — Fix location slots not persisting on save/reload (2026-03-20)
+
+0.9.0 — Job Planner redesign, monitoring locations, estimated units, smart color picker, calendar bar tooltips, toast notifications (2026-03-19)
+
+0.8.0 — Watcher Manager admin page, live agent status refresh, watcher connectivity-based status (2026-03-18)
+
+0.7.1 — Out-for-calibration status, reservation modal, migration fixes (2026-03-12)
 
 0.7.0 — Project status management, manual SD card upload, combined report wizard, NL32 support, MonitoringSession rename (2026-03-07)
 

backend/main.py

@@ -30,7 +30,7 @@ Base.metadata.create_all(bind=engine)
 ENVIRONMENT = os.getenv("ENVIRONMENT", "production")
 
 # Initialize FastAPI app
-VERSION = "0.9.4"
+VERSION = "0.10.0"
 if ENVIRONMENT == "development":
     _build = os.getenv("BUILD_NUMBER", "0")
     if _build and _build != "0":