serversdown/terra-view
1
0
Fork 0
Code Issues 16 Pull Requests Actions Packages Projects Releases 4 Wiki Activity

4 Releases 8 Tags

RSS Feed
  • v0.11.0 f4fd1c943d
    Compare

    v0.11.0 - ACH ingest and Project Expansion. Stable

    serversdown released this 2026-05-15 19:18:31 -04:00 | 30 commits to main since this release

    [0.11.0] - 2026-05-15

    Operator-facing polish release. All work builds on the v0.10.0 SFM integration foundation — this release is about making the day-to-day workflows (managing locations, cleaning up bad attributions, browsing deployments) faster and less error-prone.

    Added

    • Soft-remove monitoring locations (POST /api/projects/{p}/locations/{l}/remove + /restore): mark a location as no longer actively monitored without destroying historical events. Cascade-closes active unit assignments and cancels pending scheduled actions at the location. Restored locations rejoin the active list (assignments are NOT auto-reopened — operator creates new ones if resuming). Project page splits locations into Active and Removed sections; removed cards are greyed out, badged with the removal date + reason, and offer a Restore button.
    • Per-unit deployment Gantt chart above the existing Deployment Timeline list on every seismograph unit detail page. Plain-SVG rendering, color per location, today marker (orange dashed line), reduced-opacity bars for closed assignments, blue outlines on metadata-backfilled assignments, dashed blue underlines marking mergeable groups. Click a bar to scroll the matching list row into view with a flash highlight.
    • Merge consecutive same-location assignments (POST /api/projects/{p}/assignments/merge): operators often end up with several rows representing one continuous deployment (after remove/restore, or metadata-backfill adjacent to a manual record). Now auto-detected and surfaceable in the timeline header — one click combines them into a single record. Preserves the earliest record's notes + ingest source, writes an assignment_merged audit entry, deletes the others.
    • Delete assignment for mis-clicks (DELETE /api/projects/{p}/assignments/{a}): hard-deletes a bogus assignment row that was never a real deployment. Trash icon in each row of the location's Deployment History panel. Refuses the delete if any MonitoringSession exists in the assignment's window — those should go through Unassign instead, which preserves audit history. Writes an assignment_deleted UnitHistory row.
    • Drag-to-reorder location cards: each active card has a six-dot drag handle on the left. Drag/drop reorders the DOM and persists via POST /api/projects/{p}/locations/reorder. Implementation uses native HTML5 drag-and-drop (no library). New locations land at the end (sort_order = max + 1); removed locations stay sorted by removal date.
    • Three-dot kebab menu on location cards: replaces the four inline pill buttons (Unassign / Edit / Remove / Delete) with a single ⋮ menu. Click ⋮ to open; click outside or Escape to close; only one menu open at a time.
    • Event count on vibration location cards: vibration cards now show "{N} events" sourced from SFM via concurrent fan-out, instead of "Sessions: 0" (sessions don't exist under the watcher-forward pipeline). Sound locations still show session counts.
    • Project overview location map: right column of every project's overview replaces the lightly-used Upcoming Actions panel with a Leaflet map. One pin per active monitoring location (parsed from the coordinates field). Click pin → scrolls + flashes the matching card. Tooltip on hover. Locations without coordinates surface as an inline hint below the map. If the project has pending scheduled actions, a small "{N} upcoming actions →" link appears in the card header that switches to the Schedules tab.

    Changed

    • Backfill location fuzzy matcher is now stricter: rapidfuzz.WRatio was over-confident on location names because their shared boilerplate vocabulary ("Area", "Loc", numbers) inflated scores. Example false positive that prompted the change: "Area 2 - Brookville Dam - Loc 2 East" vs "Area 1 - Loc 1 - 87 Jenks" scored 86% via WRatio. Now uses token_set_ratio as the base scorer plus a 0.30 penalty when the two strings have disjoint multi-digit numeric tokens. Catches the "same project, different address number" case ("68 Jenks" vs "87 Jenks") that pure token-set scoring still rated above 0.90. Project matching keeps WRatio (where its leniency is desirable for typos like 1-80 vs I-80).

    Fixed

    • Three separate JSON.stringify quote-collision bugs: any inline onclick="...({...} | tojson)" or onclick="...${JSON.stringify(x)}..." where x contained any character that JSON quotes (essentially every real-world string) broke the HTML attribute and silently un-bound the click handler. Surfaced in three places this release; all fixed by switching to data-* attributes plus a trampoline function reading from this.dataset:
      • Location Remove button on the project page
      • Metadata-backfill typeahead dropdown (existing project + location pickers)
      • Project-merge typeahead dropdown (in the per-project header)
    • Project-merge modal too short to show typeahead options without scrolling: modal body's flex-1 overflow-y-auto collapsed tight; added min-height: 480px to the modal container + min-h-[320px] to the body so the dropdown always has room.
    • Project location map covered modals: Leaflet's internal panes carry z-indexes 200–800 by default and the map container didn't establish a stacking context, so those z-indexes leaked into the root and outranked modals' z-50. Fixed by adding isolation: isolate to the map container.
    • delete_assignment crashed with AttributeError: the safety check queried MonitoringSession.start_time but the actual column is started_at. Every DELETE call to /assignments/{id} failed with 500 before doing anything.

    Migration Notes

    Run on each database before deploying. Both migrations are idempotent and non-destructive.

    docker exec terra-view-terra-view-1 python3 /app/backend/migrate_add_location_removed.py
docker exec terra-view-terra-view-1 python3 /app/backend/migrate_add_location_sort_order.py

    Or sweep all migrations at once (safe — already-applied ones no-op):

    for f in backend/migrate_*.py; do
  docker exec terra-view-terra-view-1 python3 "/app/backend/$(basename $f)"
done

    New columns added this release:

    • monitoring_locations.removed_at (DATETIME, nullable) — NULL means active
    • monitoring_locations.removal_reason (TEXT, nullable)
    • monitoring_locations.sort_order (INTEGER, default 0) — seeded to alphabetical-index per project on first migration

    Deploy order matters: migrations must run BEFORE the new code is up, otherwise the running app will throw 500s on the unrecognized columns. Idempotent migrations make this recoverable but it's better avoided — the v0.11.0 deploy on prod hit this exact window after the v0.10.0 release.

    [0.10.0] - 2026-05-14

    This release brings terra-view onto the SFM (Seismograph Field Module) event pipeline. Triggered events forwarded by series3-watcher now land in SFM, and terra-view reads from that store as the authoritative source for vibration data. The watcher heartbeat is preserved as a transparent fallback signal.

    Added

    • SFM Integration: New fleet-wide events page at /sfm listing every event ingested by SFM, with filters for serial, date range, false-trigger flag, and limit. Unit detail pages and project-location pages show their own attributed subsets of the same event stream.
    • Event Detail Modal: Shared across /sfm, unit detail, and project-location pages — clicking any event opens a rich modal showing peaks per channel (PVS color-coded by magnitude), microphone dB(L) + ZC frequency + time of peak, sensor self-check table with pass/fail per channel, device/recording metadata (firmware, battery, calibration date, geo range), and download buttons for the original Blastware binary and the sidecar JSON. Includes an inline pretty-printed JSON viewer with copy-to-clipboard.
    • Events Attribution Engine (backend/services/sfm_events.py): Per-event attribution against UnitAssignment time windows. Events outside any assignment window surface in an "Unattributed" bucket with the nearest-assignment diagnostic (which location, signed delta in days).
    • Metadata Backfill Tool (/tools → Backfill from event metadata): Scans operator-typed project and sensor_location strings in event sidecars, fuzzy-clusters them via rapidfuzz.WRatio, and proposes retroactive UnitAssignment records to attribute orphan events. Tracks operator decisions per cluster across re-scans.
    • Project Tidy Tool (/tools → Project Tidy): Fuzzy-detect duplicate projects and bulk-merge them with a single click. Source projects soft-deleted with full audit trail.
    • Vibration Summary on Project Pages: New roll-up card on vibration project detail pages showing per-location event counts, the project's "Overall Peak" PVS (false triggers excluded), last event timestamp, and a Top Locations by Activity list.
    • SFM-Primary Seismograph Status: emit_status_snapshot() now consults SFM's /db/units (cached 15s) before falling back to Emitter.last_seen for each seismograph. The fresher signal wins; the choice is recorded in a new per-unit last_seen_source field. A small SFM (orange) or HB (gray) badge on each unit's active-table row shows which path is currently driving the status.
    • Dashboard Rework: Top row reordered to Recent Alerts → Recent Call-Ins (double-wide) → Fleet Summary. Today's Schedule moved to a horizontal collapsible card below the Fleet Map, auto-expanding only when pending actions exist. Recent Call-Ins now sources from a new /api/recent-event-callins endpoint backed by SFM event forwards instead of the watcher-heartbeat endpoint.
    • Sortable Events Tables: /sfm and unit-detail SFM Events tables now have clickable column headers with ↕/↓/↑ indicators. Default sort is Timestamp DESC. Click same column to toggle direction; click different column to switch and reset to DESC. Pure client-side over cached rows — no re-fetches.
    • Developer → SFM Admin (/admin/sfm): Health banner with reachability indicator, terra-view↔SFM connection panel, 4 KPI tiles (known units, total events, stale monitor_log rows, stale ach_sessions rows), per-unit roll-up table, recent-events table with color-coded forwarding latency (so stale watcher forwards stand out), and a raw API tester for any /api/sfm/* path.
    • Developer → SLMM Admin (/admin/slmm): Stripped-down companion page — health, connection info, raw API tester.
    • Tools Workflow Hub (/tools): New top-level sidebar entry consolidating Pair Devices, Project Tidy, Metadata Backfill, Reports (info card), and Swap Detection (placeholder).
    • Sidebar Reorganization: Devices → Projects → Events → Tools → Job Planner → Settings. Devices is now a single entry with internal tabs (All Devices / Seismographs / Sound Level Meters / Modems / Pair Devices) replacing five separate sidebar items.
    • Synology Deployment Doc (docs/SYNOLOGY_DEPLOYMENT.md): End-to-end playbook for migrating the stack to an always-on office NAS — phased rollout (pre-stage, data rsync, watcher repoint, external access, decommission), Tailscale vs reverse-proxy options, rollback plan, and gotchas.

    Changed

    • Overall Peak excludes false triggers: The project-level "Overall Peak" KPI tile (and the underlying _compute_stats() function in sfm_events.py) now skip events flagged as false triggers when computing the highest PVS, so operators see the highest real event rather than the biggest sensor glitch. false_trigger_count still includes flagged events so operators can see how many were filtered out.
    • RosterUnit.note Editing: Inline edit on seismograph cards is more forgiving and now auto-saves on blur.
    • Sidebar Nav Renamed: Old "Fleet" sidebar entry → "Devices" (renamed because it always meant the device list, not the broader fleet view).

    Fixed

    • Status drift between watcher heartbeat and actual event arrivals: Seismographs are now reported with whichever signal is more recent — eliminates the case where a unit had recent SFM events but a stale heartbeat (or vice-versa) showed the wrong status.
    • Event modal: Record Type always showed "Waveform": Workaround client-side — Record Type now derived from the Blastware filename's last-char code (H=Histogram, W=Waveform, M=Manual, E=Event, C=Combo). The proper fix lives in SFM's sidecar parser; tracked separately.
    • Event modal: Mic PSI tile removed: Operators only care about dB(L); the redundant PSI tile was dropped.

    Migration Notes

    Run on each database before deploying. Every migration is idempotent.

    # Cleanest: re-run all migrations in chronological order.
# Already-applied migrations no-op safely.
for f in backend/migrate_*.py; do
  docker exec terra-view-terra-view-1 python3 "/app/backend/$(basename $f)"
done

    Migrations new in this release:

    • migrate_add_metadata_backfill.py — adds unit_assignments.source column and metadata_backfill_decisions table for the Metadata Backfill tool

    Deployment Notes

    • SFM_BASE_URL: Confirm prod's docker-compose.yml sets this for the terra-view service (typically http://sfm:8200 for the in-stack SFM container, or an external URL if SFM lives elsewhere).
    • Watcher repoint: series3-watcher's sfm_forward_url should point at https://<your-terra-view-host>/api/sfm (proxy-based — no second port forward needed). Watcher composes the full path /db/import/blastware_file itself.
    Downloads