Cut [0.14.0] consolidating SLM live monitoring, the FTP night-report
pipeline (was missing from the changelog entirely), the client portal,
and portal auth Phase 1 under one entry. Bump VERSION + README to 0.14.0
and add the sound-monitoring / night-report / client-portal features to
the README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
If the post-restart DOD check shows the meter isn't measuring, retry once with start_recording (a plain start that does NOT re-index, unlike start_cycle) and re-verify before raising the schedule-failed alert. Retry fires only on a confident not-measuring reading — never on a failed/inconclusive DOD read — so a flaky read can't disrupt an already-running measurement or split the night across two store folders. Turns a transient restart hiccup into a self-heal instead of a meter left stopped overnight.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- delete dead magic-link helpers (resolve_token, ensure_project_client,
mint_link_token, provision_preview_session) + now-unused datetime import
- key brute-force lockout on link_token alone (IP term only enabled a
source-IP-rotation bypass; behind the proxy all clients share one IP)
- drop unused PORTAL_BASE_URL from the retired CLI
- add WebSocket ownership tests (unauth + cross-project both close 1008)
Modules own raw device data; Terra-View owns fleet/project/session/report context. Documents the SFM (read-through) vs SLMM (Terra-View-stored) asymmetry, the rule new modules must follow, and grandfathers SLMM as a deliberate-future-realignment exception. Establishes the docs/adr/ convention.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Caught by adversarial review of the scope test: portal_client_for_project minted a
dedicated client but never set project.client_id, so the client-scoped routes found
no projects — every location 404'd, including the client's own (empty portal). Now
links the project + adds a positive-case test.
Proper fix superseding the fb* prefix band-aid (1801d4e): wrap ftp_browser.html's script in an IIFE and expose only window.FtpBrowser. Its ~11 helpers no longer leak to global scope, so the partial is safe to co-load with other FTP-browsing partials (e.g. slm_live_view's Command Center) without name collisions in either direction. Inline onclick handlers call FtpBrowser.*; showFTPSettings stays global (it's from the included settings modal). Behaviour unchanged — verified full Jinja render + balanced delimiters.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ftp_browser.html and slm_live_view.html are both loaded on the per-NRL detail page (Data Files + Command Center tabs) and each defined loadFTPFiles / downloadToServer / downloadFTPFile / enableFTP / formatFileSize as globals — last to load won. 'Browse Files' then called slm_live_view's loadFTPFiles, which renders into the hidden Command Center's #ftp-files-list, so the FTP request fired but nothing appeared. Prefix ftp_browser's five colliding functions with fb* so each partial keeps its own. (Element IDs don't collide: per-unit vs fixed.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-NRL Data Files tab now reuses the same FTP browser + unified-files partials as the project-wide tab, scoped to the one NRL: ftp-browser and files-unified take an optional location_id. nrl_detail.html drops the flat file_list view for 'Download Files from SLMs' (Browse Files -> Download & Save) plus the grouped 'Project Files' view (edit times / download-all / delete), keeping the NRL upload and adding a refresh button.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ftp-download-folder-to-server and ftp-download-to-server now route NRL data through the shared ingest (ingest_nrl_zip / _ingest_file_entries) instead of hand-rolling DataFile rows on a now/zero-duration session. Folder save requires the unit be assigned to a location; non-NRL single files keep the generic save path. The FTP browser popup now reports how long the measurement ran.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The NL-43 .rnh carries no measurement timestamps, so _ingest_file_entries was stamping every session with utcnow() and no duration. Derive started_at/stopped_at/duration from the Leq .rnd 'Start Time' column when the header lacks them (interval from the .rnh, else inferred from row spacing). Adds an optional unit_id so callers that know the recording unit attribute the session at creation, and returns duration_seconds.
Side effect: NL-43 dedupe now works (it keyed on a previously-empty start_time_str). Affects all ingest paths: manual upload, FTP cycle, stop, download, and manual FTP download.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_execute_stop and _execute_download no longer hand-roll ZIP extraction; all three actions now call a shared _ingest_and_link helper (ingest via ingest_nrl_zip, link the unit, drop the empty placeholder session). Every capture path produces the same clean, .rnh-parsed, percentile-aware, deduped, Leq-only session. _execute_download previously created no session at all (TODO); it now ingests like the others.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the read-only client portal under [Unreleased] alongside the SLM
live-monitoring work: per-client scoping + interim auth, live location view with
the auto-closing WS stream, locations overview map + rollup, the alerts
config→surface→24/7 track, operator sharing tools, the field-instrument design +
light/dark toggle, the security posture, and upgrade notes (migration, SECRET_KEY,
SLMM alert-engine pairing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidates the deferred items (reverse proxy exposing only /portal/*, TLS,
SECRET_KEY, PORTAL_OPEN_LINKS off, M4 auth incl. the operator app + currently-
unauthenticated operator endpoints, and the smaller code-review items) into an
actionable checklist so the hardening session starts from a list, not a recall.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- #1 XSS: escape user-controlled values (location name, baseline values, recent-
report fields, SMTP status message) in the modals via the existing _mergeEsc
helper — they were concatenated raw into innerHTML (stored XSS via location name).
- #2 SMTP: an unrecognized REPORT_SMTP_SECURITY no longer silently downgrades to a
plaintext connection while still calling login() — it falls back to starttls and
warns; warn on intentional security=none + auth.
- #3 scheduler: run the (blocking smtplib + Excel) nightly report in a worker thread
(asyncio.to_thread + its own DB session) so it can't stall the loop that drives
time-sensitive device cycles. New _run_one_report helper.
- #4 cycle ingest: set unit_id on the ingested data session (ingest_nrl_zip leaves
it None) before dropping the empty placeholder, preserving the unit<->session link;
repoint old_session_id at the real row.
- #7 robustness: wrap send_report_email in the orchestrator and run_nightly_report in
/view + /run so a render/SMTP error returns a clean error instead of a raw 500
after artifacts are written.
Verified: SMTP paths (typo->starttls, none, starttls, ssl), off-thread tick stamps
last_run_date + writes the file, /view 200, escaping wired, app imports.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- PORTAL_OPEN_LINKS now defaults OFF — /portal/open/* is an unauthenticated,
proxy-reachable session-minting path (and a linked project's open link grants
the whole client's scope), so it must be explicitly enabled in dev.
- Session cookie: enforce server-side expiry (check iat vs COOKIE_MAX_AGE — was
browser-only) and guard a non-dict signed body (was an uncaught AttributeError →
500, reachable if SECRET_KEY is the insecure default).
- Escape operator-set strings (location/rule/event names) before innerHTML +
Leaflet tooltips — they're client-facing, so a name with markup was stored XSS
in the client's browser. Global esc() helper applied at every injection point.
- WS _scrub_frame drops a non-JSON frame instead of forwarding it raw; /history
rows now whitelisted like the other scoped endpoints.
- Preview-client slug uses the full project id (an 8-char prefix could collide
two projects onto one client).
Verified: cookie reader (fresh/expired/non-dict/missing-iat) + open-links default
off; templates parse; scoped scrubbing intact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New scoped GET /portal/api/location/{id}/thresholds returns the enabled alert
rules (scrubbed: name/metric/comparison/threshold/duration/schedule — no cooldown
or hysteresis internals). Location page renders an "Alert limits" panel above the
history, e.g. "Night noise · Leq above 65 dB for 60s · 22:00–07:00", hidden when
no limits are set. Gives the breach history context.
Verified: portal.py compiles; location script balances; template parses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
render_excel(report): one worksheet per location — interval table, a line chart,
and a Last/Base/Δ summary per window. Metric-driven, so it tracks whatever metric
set is configured.
- orchestrator: render report.xlsx alongside report.html, attach it to the email
(dry-run until SMTP set), expose xlsx_path. Never lets a spreadsheet error sink
the report.
- reports router: /list includes xlsx_url when present; new
GET /archive/{date}/xlsx serves the saved spreadsheet.
- UI: Recent-reports rows get an "Excel" download link.
Verified: real Feb data -> valid .xlsx (sheet per NRL, interval table + chart +
summary with real values), attachment path runs, both archive routes registered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend _execute_cycle (daily stop/download/increment/restart) so the nightly
report's data lands automatically:
- Step 4b: after the device download, fetch the just-finished Auto_#### folder
from SLMM and ingest via ingest_nrl_zip (clean session + DataFiles, Lp filtered,
dedup). Drops the empty "recording" placeholder session once the real data
session exists. New helper _ingest_cycle_folder.
- Step 6b: after restart, verify the meter resumed measuring via a fresh DOD
(measurement_state) — advisory: alerts loudly on failure but doesn't fail the
cycle (keepalive polling re-confirms within ~10s).
Both wrapped defensively so they never break the cycle. Ingest-hook logic verified
with a mocked SLMM (real Feb folder -> session + 2 DataFiles, dedup, empty/HTTP
guards). Device-control paths (restart-verify, live download) are field-untested
— no meter available in dev.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reverts the light-mode ground to a cool light (#eef2f9) with cool navy ink,
borders, and shadow — keeping the solid (opaque, defined) cards from the
un-ghosting pass so it's clean rather than dull. theme-color meta updated to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Light is now the default for new visitors/clients (was dark); the toggle still
flips to dark and the choice persists. Also fixed the mobile theme-color meta to
update the actual <meta> tag (was setting a no-op attribute on <html>) and use the
warm paper shade.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Light mode was washed out. Switch the background to warm paper (#f7f5ef), make
panels solid white (no longer translucent/ghostly) with a warm hairline border
and a grounded two-layer shadow, and warm the text ink. Light-specific hover
shadow (the dark one is invisible on paper). Also fix two dark-only reds — the
alarm banner and active-event text now use var(--lvl-bad) so they read on both
themes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
serversdown