diff --git a/CHANGELOG.md b/CHANGELOG.md index 727b9ce..5aa9db9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,9 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -SLM live monitoring — fan-out feed + cache-first reads. Targets **0.14.0**. The throughline: the NL-43 allows exactly **one** TCP connection at a time, so every page that opened its own device stream (or sent its own `Measure?`/DOD on load) was competing for that single connection — a second viewer saw nothing, and dashboard loads stole polling resolution from the live feed. This release moves Terra-View entirely onto SLMM's shared, cached monitoring: one DOD poll loop per device, fanned out to all viewers; dashboards read SLMM's cache (a DB read on SLMM's side) instead of touching the device; and the live panels populate instantly from cache on open, upgrading to the live WS only on demand. Paired with the SLMM-side work (adaptive poll rate, unreachable backoff, device-offline alert) on SLMM branch `dev`. +## [0.14.0] - 2026-06-17 -### Added +Rounds out **sound monitoring** and adds a **client-facing portal**, consolidating four threads since 0.13.x: SLM live monitoring (now on SLMM's shared, cached feed), an automated **FTP night-report pipeline**, a read-only **client portal**, and **per-project password auth** for it. Depends on the matching **SLMM `dev`** build — see Upgrade Notes at the end of each section. + +### SLM live monitoring — fan-out feed + cache-first + +SLM live monitoring — fan-out feed + cache-first reads. The throughline: the NL-43 allows exactly **one** TCP connection at a time, so every page that opened its own device stream (or sent its own `Measure?`/DOD on load) was competing for that single connection — a second viewer saw nothing, and dashboard loads stole polling resolution from the live feed. This release moves Terra-View entirely onto SLMM's shared, cached monitoring: one DOD poll loop per device, fanned out to all viewers; dashboards read SLMM's cache (a DB read on SLMM's side) instead of touching the device; and the live panels populate instantly from cache on open, upgrading to the live WS only on demand. Paired with the SLMM-side work (adaptive poll rate, unreachable backoff, device-offline alert) on SLMM branch `dev`. + +#### Added - **Fan-out `/monitor` feed consumption.** The unit live view (`partials/slm_live_view.html`) and the dashboard live tile (`sound_level_meters.html`) now subscribe to SLMM's shared per-device monitor over `WS /api/slmm/{unit}/monitor` instead of each opening its own device stream. Any number of clients attach without each consuming the NL-43's single connection — the "second viewer sees nothing" contention is gone. A WS proxy handler for `/monitor` was added to `backend/routers/slmm.py`. - **L1/L10 percentile lines + cards.** Both the per-unit live chart and the dashboard card chart now plot L1 (purple) and L10 (orange) alongside Lp/Leq, and the KPI cards show L1/L10. Sourced from the DOD feed's `ln1`/`ln2` (DRD streaming can't carry percentiles, DOD can). Missing/`-.-` values leave a gap rather than dropping the line to 0. @@ -18,20 +24,20 @@ SLM live monitoring — fan-out feed + cache-first reads. Targets **0.14.0**. - **Refresh buttons** — one per device-list row, one in the panel header. On-demand, user-initiated single device read via `GET /api/slmm/{unit}/live` (which also refreshes SLMM's cache), with a spinner + success/error toast, then reloads the device list. - **Per-unit live-monitoring (keepalive) toggle on `/admin/slmm`** — turns a device's server-side keepalive feed on/off (`POST /monitor/start|stop`), so alerting can keep a device's feed running with no browser attached. -### Changed +#### Changed - **Dashboard device list + command center read SLMM's cache, not the device.** `slm_dashboard.py`'s `get_slm_units` pulls each unit's cached status from SLMM's `/roster` (one call, a SLMM DB read) for the badge + freshness; the command-center `get_live_view` reads cached `/status` instead of sending `Measure?` + a fresh DOD on every load. This stops dashboard loads from stealing the device's single connection from the live monitor. The elapsed-measurement timer still works because `measurement_start_time` is now included in the cached `/status` response. - **Device-list freshness reflects real monitoring.** The "Last check" line now uses SLMM's cached `last_seen` (which the monitor advances on every successful poll) via `unit.cache_last_seen`, instead of the `slm_last_check` roster field the monitor never updates. The status badge also treats `Measure` as Measuring, matching the panel and SLMM's cache. - **Status badge relocated** to the card's bottom meta row (next to "Last check"), off the top-right corner where it collided with the chart/gear/refresh action icons. -### Fixed +#### Fixed - **Deploy/bench threw `can't access property "dispatchEvent", e is null`.** `toggleSLMDeployed()` and the save-config path called `htmx.trigger('#slm-list', 'load')` guarded only by `typeof htmx !== 'undefined'`; no page has a `#slm-list`, so htmx resolved null and called `null.dispatchEvent(...)`. The deploy POST had already succeeded, so the operator saw both the green success **and** a red error. Both call sites now guard on the element existing (`slm_settings_modal.html`). - **Monitor WS proxy leaked `CancelledError` / "task exception never retrieved"** on stream stop — the cleanup awaited pending tasks but only caught `Exception`, missing `CancelledError` (a `BaseException`). - **"No recent check-in" shown even on an actively-monitored device** — the row read the stale `slm_last_check` roster field instead of SLMM's live cache (see Changed). - **L1/L10 KPI cards populated but the chart drew no L1/L10 lines** — the card chart only had Lp + Leq datasets. -### Upgrade Notes +#### Upgrade Notes Requires the **matching SLMM build (branch `dev`)** — Terra-View now depends on SLMM's fan-out `/monitor` feed, `/history` trail, `/status` carrying `ln1`/`ln2` + `measurement_start_time`, cached `/roster` status, and the `monitor_enabled` keepalive flag. @@ -49,6 +55,54 @@ The two builds must ship **together**. Note the `docker-compose.yml` container --- +### FTP night-report pipeline *(new)* + +Automated daily morning report of last night's noise (7 PM–7 AM) vs a baseline, +per location, for 24/7 remote sound jobs. The meter records to its SD card +regardless of TCP state, so the report pulls the meter's own stored 15-minute +Leq intervals over FTP (via the SLMM proxy) — accurate, and resilient to a +control-path wedge. **Field-tested on a real NL-43.** + +#### Added + +- **Report engine.** Per-location LAmax / LA01 / LA10 / LA90 / LAeq over Evening + (7–10 PM) + Nighttime (10 PM–7 AM); Leq energy-averaged, percentiles/Lmax + arithmetic; the LN→percentile map is read from the device's own `.rnh`. Two + baseline modes: *captured* (weekly average) and *reference* (typed per-location + limits). +- **Renderers.** HTML email body + Excel attachment (per-NRL interval table + + line chart + Last/Base/Δ summary). +- **Capture cycle.** The daily scheduled "24/7 Continuous" cycle stops → + downloads → ingests → re-indexes → restarts the meter, verifies it resumed + measuring via a fresh DOD read, and retries the restart once before alerting. +- **Standardized ingest.** Manual SD upload, manual FTP "Download & Save", and + the scheduled cycle all funnel through one ingest core: keeps the `.rnh` + + 15-minute Leq, drops the 1-second `_Lp_` files, parses the header, dedupes, and + derives the session's real recording window from the Leq rows. +- **UI.** Night Report button/modal (view / run-and-email / recent reports) and a + per-project Settings panel (enable, time, baseline, recipients, test-email); the + per-NRL Data Files tab now matches the project-wide tab. +- **Config-driven SMTP** sender (`REPORT_SMTP_*`), dry-run when unconfigured. + +#### Fixed + +- **NL-43 sessions stamped `now` / zero-duration.** The NL-43 `.rnh` carries no + measurement timestamps, so the session window is now derived from the Leq rows. + Also fixes NL-43 dedupe (it had keyed on an always-empty start time). +- **"Browse Files" did nothing on the NRL Data Files tab** — the FTP-browser + script's global functions collided with the SLM live-view's (both loaded on that + page); it's now namespaced behind `window.FtpBrowser`. + +#### Upgrade Notes + +- **No DB migration** — the `sound_report_configs` table auto-creates on startup. +- Set `REPORT_SMTP_HOST/PORT/SECURITY/USER/PASSWORD/FROM/RECIPIENTS` to send email + (reports build to `data/reports/…` in dry-run until then). +- To automate a job: a **"24/7 Continuous"** recurring schedule (~7:15 AM) + enable + the report (~8:00 AM) + set a baseline. + +--- + ### Client portal *(new — read-only client-facing view)* A scoped, read-only portal at **`/portal/*`** where a client sees only *their* diff --git a/README.md b/README.md index df2cbd5..f06d19d 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ -# Terra-View v0.13.3 -Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard. +# Terra-View v0.14.0 +Backend API and HTMX-powered web interface for managing a mixed fleet of seismographs, sound level meters, and field modems. Track deployments, monitor health in real time, merge roster intent with incoming telemetry, and control your fleet through a unified database and dashboard. ## Features @@ -18,6 +18,9 @@ Backend API and HTMX-powered web interface for managing a mixed fleet of seismog - **Settings & Safeguards**: `/settings` page exposes roster stats, exports, replace-all imports, and danger-zone reset tools - **Device & Modem Metadata**: Capture calibration windows, modem pairings, phone/IP details, and addresses per unit - **Status Management**: Automatically mark deployed units as OK, Pending (>12h), or Missing (>24h) based on recent telemetry +- **Sound Level Meter Monitoring**: Live per-device monitoring through SLMM's shared, cached feed — multiple viewers without contending for the NL-43's single connection — with L1/L10 percentile lines, a measuring/freshness indicator, and on-demand refresh +- **Automated Night Reports**: Daily per-location noise report (last night vs a baseline) for 24/7 remote sound jobs — pulls the meter's 15-minute Leq over FTP and emails an HTML summary + Excel; the meter is auto-cycled (stop → download → ingest → restart, with restart verification) each morning +- **Client Portal** (`/portal/*`): scoped, read-only, client-facing live view of *their* locations only, gated by a per-project link + shared password (argon2-hashed) - **SFM Event DB Manager** (`/admin/events`): cross-unit event browser with bulk false-trigger flagging and admin-only hard-delete (cleans on-disk binaries + sidecars too) for purging bogus events from misbehaving units - **Deployment-History Calendar + Gantt** (`/tools/deployment-history`): fleet-wide 12-month calendar with side-panel day drill-down, plus "Gantt by Project" / "Gantt by Unit" tabs - **Photo Management**: Upload and view photos for each unit diff --git a/backend/main.py b/backend/main.py index 92f2470..e6b5a17 100644 --- a/backend/main.py +++ b/backend/main.py @@ -30,7 +30,7 @@ Base.metadata.create_all(bind=engine) ENVIRONMENT = os.getenv("ENVIRONMENT", "production") # Initialize FastAPI app -VERSION = "0.13.3" +VERSION = "0.14.0" if ENVIRONMENT == "development": _build = os.getenv("BUILD_NUMBER", "0") if _build and _build != "0":