Commit Graph

307 Commits

Author SHA1 Message Date
serversdown 7fcd1261b4 Merge pull request 'feat(reports): FTP night-report pipeline foundation' (#62) from feat/ftp-report-pipeline into dev
Reviewed-on: #62
2026-06-11 23:27:34 -04:00
serversdown 576e4f89ca doc: changelog entry for reports 2026-06-12 03:26:30 +00:00
serversdown 5f02a0bc21 Merge client portal into dev
Reviewed-on: #61
2026-06-11 23:21:52 -04:00
serversdown 684a487203 docs: changelog [Unreleased] — add the client portal feature
Documents the read-only client portal under [Unreleased] alongside the SLM
live-monitoring work: per-client scoping + interim auth, live location view with
the auto-closing WS stream, locations overview map + rollup, the alerts
config→surface→24/7 track, operator sharing tools, the field-instrument design +
light/dark toggle, the security posture, and upgrade notes (migration, SECRET_KEY,
SLMM alert-engine pairing).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 03:20:16 +00:00
serversdown 04cd6b9f24 docs(portal): security hardening backlog for the dedicated pass
Consolidates the deferred items (reverse proxy exposing only /portal/*, TLS,
SECRET_KEY, PORTAL_OPEN_LINKS off, M4 auth incl. the operator app + currently-
unauthenticated operator endpoints, and the smaller code-review items) into an
actionable checklist so the hardening session starts from a list, not a recall.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 02:39:33 +00:00
serversdown fdd0426884 fix(reports): code-review findings — XSS, SMTP, blocking, unit link, email guard
- #1 XSS: escape user-controlled values (location name, baseline values, recent-
  report fields, SMTP status message) in the modals via the existing _mergeEsc
  helper — they were concatenated raw into innerHTML (stored XSS via location name).
- #2 SMTP: an unrecognized REPORT_SMTP_SECURITY no longer silently downgrades to a
  plaintext connection while still calling login() — it falls back to starttls and
  warns; warn on intentional security=none + auth.
- #3 scheduler: run the (blocking smtplib + Excel) nightly report in a worker thread
  (asyncio.to_thread + its own DB session) so it can't stall the loop that drives
  time-sensitive device cycles. New _run_one_report helper.
- #4 cycle ingest: set unit_id on the ingested data session (ingest_nrl_zip leaves
  it None) before dropping the empty placeholder, preserving the unit<->session link;
  repoint old_session_id at the real row.
- #7 robustness: wrap send_report_email in the orchestrator and run_nightly_report in
  /view + /run so a render/SMTP error returns a clean error instead of a raw 500
  after artifacts are written.

Verified: SMTP paths (typo->starttls, none, starttls, ssl), off-thread tick stamps
last_run_date + writes the file, /view 200, escaping wired, app imports.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 02:37:28 +00:00
serversdown fe7cf91488 fix(portal): pre-merge security hardening from code review
- PORTAL_OPEN_LINKS now defaults OFF — /portal/open/* is an unauthenticated,
  proxy-reachable session-minting path (and a linked project's open link grants
  the whole client's scope), so it must be explicitly enabled in dev.
- Session cookie: enforce server-side expiry (check iat vs COOKIE_MAX_AGE — was
  browser-only) and guard a non-dict signed body (was an uncaught AttributeError →
  500, reachable if SECRET_KEY is the insecure default).
- Escape operator-set strings (location/rule/event names) before innerHTML +
  Leaflet tooltips — they're client-facing, so a name with markup was stored XSS
  in the client's browser. Global esc() helper applied at every injection point.
- WS _scrub_frame drops a non-JSON frame instead of forwarding it raw; /history
  rows now whitelisted like the other scoped endpoints.
- Preview-client slug uses the full project id (an 8-char prefix could collide
  two projects onto one client).

Verified: cookie reader (fresh/expired/non-dict/missing-iat) + open-links default
off; templates parse; scoped scrubbing intact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 23:40:52 +00:00
serversdown c1bc391ba2 feat(portal): show the client their active alert limits
New scoped GET /portal/api/location/{id}/thresholds returns the enabled alert
rules (scrubbed: name/metric/comparison/threshold/duration/schedule — no cooldown
or hysteresis internals). Location page renders an "Alert limits" panel above the
history, e.g. "Night noise · Leq above 65 dB for 60s · 22:00–07:00", hidden when
no limits are set. Gives the breach history context.

Verified: portal.py compiles; location script balances; template parses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 23:28:57 +00:00
serversdown ccb70698ba feat(reports): Excel renderer + attachment + archive download
render_excel(report): one worksheet per location — interval table, a line chart,
and a Last/Base/Δ summary per window. Metric-driven, so it tracks whatever metric
set is configured.

- orchestrator: render report.xlsx alongside report.html, attach it to the email
  (dry-run until SMTP set), expose xlsx_path. Never lets a spreadsheet error sink
  the report.
- reports router: /list includes xlsx_url when present; new
  GET /archive/{date}/xlsx serves the saved spreadsheet.
- UI: Recent-reports rows get an "Excel" download link.

Verified: real Feb data -> valid .xlsx (sheet per NRL, interval table + chart +
summary with real values), attachment path runs, both archive routes registered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 22:49:32 +00:00
serversdown 88887a92d8 feat(reports): #2 capture hook — cycle auto-ingests + verifies restart
Extend _execute_cycle (daily stop/download/increment/restart) so the nightly
report's data lands automatically:
- Step 4b: after the device download, fetch the just-finished Auto_#### folder
  from SLMM and ingest via ingest_nrl_zip (clean session + DataFiles, Lp filtered,
  dedup). Drops the empty "recording" placeholder session once the real data
  session exists. New helper _ingest_cycle_folder.
- Step 6b: after restart, verify the meter resumed measuring via a fresh DOD
  (measurement_state) — advisory: alerts loudly on failure but doesn't fail the
  cycle (keepalive polling re-confirms within ~10s).

Both wrapped defensively so they never break the cycle. Ingest-hook logic verified
with a mocked SLMM (real Feb folder -> session + 2 DataFiles, dedup, empty/HTTP
guards). Device-control paths (restart-verify, live download) are field-untested
— no meter available in dev.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 20:55:44 +00:00
serversdown a81764d4bc style(portal): cool light background, keep the solid cards
Reverts the light-mode ground to a cool light (#eef2f9) with cool navy ink,
borders, and shadow — keeping the solid (opaque, defined) cards from the
un-ghosting pass so it's clean rather than dull. theme-color meta updated to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 20:46:04 +00:00
serversdown a555cb74dd style(portal): default to light theme
Light is now the default for new visitors/clients (was dark); the toggle still
flips to dark and the choice persists. Also fixed the mobile theme-color meta to
update the actual <meta> tag (was setting a no-op attribute on <html>) and use the
warm paper shade.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 20:42:55 +00:00
serversdown 505c2e3ca5 style(portal): warm, solid light mode — paper bg + defined cards
Light mode was washed out. Switch the background to warm paper (#f7f5ef), make
panels solid white (no longer translucent/ghostly) with a warm hairline border
and a grounded two-layer shadow, and warm the text ink. Light-specific hover
shadow (the dark one is invisible on paper). Also fix two dark-only reds — the
alarm banner and active-event text now use var(--lvl-bad) so they read on both
themes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 20:34:31 +00:00
serversdown 1d49b54bd1 feat(reports): baseline-source editor in the settings modal
Gear → Settings now has a "Baseline source" toggle:
- Captured nights → the date-range fields (existing).
- Fixed values → a per-NRL grid (metrics × Evening/Nighttime) to type spec
  limits or prior-report averages, with a "Copy first NRL → all" helper.

Loads from GET /reports/baseline, saves mode via PUT /config and the per-NRL
values via PUT /reports/baseline. Verified the template renders + gates to sound.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 20:29:20 +00:00
serversdown c1b5efae56 feat(reports): reference-baseline mode (typed limits / prior averages)
Baseline can now come from fixed values typed per location, not just captured
data — for a spec limit ("L10 = 85") or a prior report's averages when the raw
data isn't available.

- SoundReportConfig.baseline_mode ("captured" | "reference").
- report_pipeline: _location_reference_baseline() reads per-location values from
  location_metadata; build_*_night_report honor baseline_mode (reference cells
  use the typed value; unset metrics compare against nothing).
- reports router: GET/PUT /reports/baseline (mode on config + per-location values
  in location_metadata); config carries baseline_mode; manual view/run fall back
  to the saved config's baseline when no explicit dates are given.
- orchestrator + scheduler tick thread baseline_mode through.

Verified end-to-end: PUT/GET /baseline, reference deltas (L10 66.6 vs 85 -> -18.4),
unset metrics compare against nothing, captured-mode regression intact.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 20:26:23 +00:00
serversdown f760e81309 style(portal): live-console location page, polished access splash, light/dark toggle
- Location page rebuilt as a monitoring console: Leq hero readout (mono, level-
  colored, auto-flips with theme), instrument strip for Lp/Lmax/L1/L10, refined
  dark Chart.js (mono ticks, thin lines), panel-styled alert history, polished
  pause overlay. All live-stream/chart/alert JS hooks preserved.
- Access page → centered branded splash.
- Light/Dark toggle: CSS-variable theme system (structure + level/metric accent
  colors flip), header sun/moon button, localStorage + no-flash boot script,
  smooth body transition. On toggle, a 'portal-theme' event re-skins the Chart.js
  trace and swaps Leaflet tiles (CARTO dark <-> light) + recolors map dots.

All JS hook IDs intact (verified); both themes validated to parse + balance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 20:10:29 +00:00
serversdown 4839d14a22 style(portal): field-instrument redesign — shell + overview
A refined dark "field instrument" aesthetic for the client-facing portal:
- Type: Hanken Grotesk UI + IBM Plex Mono for readings (dB values feel like real
  instrumentation). Tabular numerals.
- Atmosphere: deep navy-black base with a navy/burgundy aurora and a faint fixed
  instrument grid; sticky blurred header with an animated signal-bars mark.
- Panel system (.panel/.panel-hover): translucent, hairline-lit, depth + hover
  lift. Pulsing live dot; staggered load reveal.
- Overview: mono Leq hero on each tile (colored by level when live), pill badges
  with the pulsing dot, rollup pills, dark CARTO map tiles, level-colored dots.

All live-data JS hook IDs preserved (verified). No backend change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:41:48 +00:00
serversdown fa7dc39e5e feat(portal): M2b-3 note — enabled alerts keep the device monitored 24/7
UI note on the SLM alerts card reflecting the SLMM keepalive coupling.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:36:16 +00:00
serversdown 0914cf0a75 feat(portal): M2b-2 — surface alert state + breach history (internal + portal)
Internal (SLM detail page): live alarm-state badge in the Alerts header
(● N active / ✓ all clear), a History list of fired events (onset → clear, peak
dB, ack status) with an Ack button, refreshed every 20s. Reads the existing SLMM
/alerts/events + /ack via the proxy.

Portal (client, read-only, scoped): new GET /portal/api/location/{id}/events —
ownership-gated, returns a scrubbed projection (rule_name/metric/threshold/onset/
peak/clear/status only; no internal ids or ack-by) plus an `active` count. The
location page shows a red "Currently above threshold" banner when active and a
read-only breach history, polled every 20s. No ack on the client side.

Verified: portal.py compiles; both scripts balance; both templates parse.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 19:02:56 +00:00
serversdown 29b974a1f7 feat(portal): M2b-1 — alert rule config UI on the SLM detail page
Adds an "Alerts" card to /slm/{id}: lists rules and a create/edit/delete form
(simple-first — "Alert when [Leq] is [above] [65] dB for [N] s", optional
time-of-day window + day picker, advanced hysteresis/cooldown collapsed). Talks
to the existing SLMM alert CRUD via the proxy (/api/slmm/{unit}/alerts/rules);
no SLMM changes. Rule changes invalidate the evaluator's cache server-side.

Verified: alerts script JS balances, slm_detail.html parses, and the TV proxy
forwards method + JSON body + query params for POST/PUT/DELETE.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 18:01:27 +00:00
serversdown bececafe78 feat(portal): plain no-token "open" links for dev feedback (PORTAL_OPEN_LINKS)
Adds a frictionless shareable link so anyone can open a project's client portal
during dev without minting/copying a magic token. GET /portal/open/{project_id}
(gated by PORTAL_OPEN_LINKS) provisions the client session and lands on /portal;
lives under /portal so it works through a proxy exposing only /portal/*.

The project page's "Copy client link" modal now leads with this Quick share link
(amber, host taken from window.location.origin so it always matches the host you
copied it from — no more LAN-vs-public foot-gun). The token-based generate/list/
revoke stays below for the eventual secure path.

PORTAL_OPEN_LINKS defaults ON for the prototype (whole app is open anyway) and logs
a warning; set =false before real clients. The get_current_client seam is
untouched, so M4 auth still layers in front of the same routes regardless.

Verified: compiles, share script balances, detail.html parses, flag default
on / =false off.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:26:37 +00:00
serversdown 7fb4ba0343 feat(reports): wire run-now, archive, test-email, last-run status into the UI
Backend (reports router):
- POST /reports/test-email — send a test email (body/config recipients; dry-run
  if SMTP unset) to verify the relay.
- GET  /reports/list — list generated report artifacts on disk (newest first).
- GET  /reports/archive/{date} — serve a saved report.html (traversal-guarded).

Frontend (sound project header modals):
- Night Report modal: "Run & Email" button (POST /run) + a "Recent reports" list
  (GET /list → opens the archived report.html in a new tab).
- Settings modal: schedule + last-run status line, and a "Send test email" button.

Verified: endpoints (run→list→archive, traversal blocked, test-email recipient
fallback) and the template renders with all four wired + gated to sound projects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 17:19:30 +00:00
serversdown 2da9493cb5 feat(portal): "Copy client link" — generate/copy/revoke shareable links from the project page
No-CLI way to get a real shareable magic link (/portal/enter/<token>) for a
project's client. Project page gets a "Copy client link" button next to the
preview; opens a modal that lists active links (with revoke), generates a fresh
one, and copies it to the clipboard.

Backend (operator, internal /projects/*):
- POST /projects/{id}/portal-link  -> mint a fresh token, return the full URL
  (built from request.base_url so it uses the operator's host).
- GET  /projects/{id}/portal-links -> list active links (label/created/last-used).
- POST /projects/{id}/portal-link/{tid}/revoke -> revoke one (scoped to the
  project's client).

Refactor: split ensure_project_client() + mint_link_token() out of
provision_preview_session() so minting a shareable link and the preview cookie
share one provisioning path.

Verified: ensure/mint persistence across commits + sessions, minted link resolves,
token stored hashed, second mint = distinct active link (4/4); compiles; share
script balances; detail.html parses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:11:34 +00:00
serversdown b2c54caebd feat(reports): per-project report config + automatic morning run
Add SoundReportConfig (one row per project) + the scheduler tick that runs the
nightly report on its own:
- model SoundReportConfig (enabled, report_time, metric_keys, baseline range,
  recipients, last_run_date) — new table, auto-created by create_all (no migration).
- GET/PUT /api/projects/{id}/reports/config with validation.
- SchedulerService.run_due_reports(): each loop, for every enabled config past
  its report_time, run last night's report once (dedup via last_run_date),
  writing the file + emailing (dry-run until SMTP is set).
- UI: gear button beside "Night Report" opens a settings modal (enable, time,
  baseline range, metrics, recipients) that GET/PUTs the config.

Verified: table registers + auto-creates, config CRUD + validation, tick
runs/dedups, templates render and gate to sound projects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 17:08:58 +00:00
serversdown b908f394ed feat(portal): M2a — live status map + status rollup on the overview
Reuses the existing per-location /live fetch (no backend change):
- Map dots recolor live by current level (green/amber/red bands, grey when
  not measuring/offline) and the tooltip shows the live Leq. Bands are
  placeholders until M2 alert thresholds drive the color.
- Status rollup header: total locations, # live vs offline, and a "Loudest now"
  Leq callout. Aggregated each 15s refresh.

Refactored the refresh into refreshAll() (Promise.all over loadTile -> updateRollup);
loadTile now also feeds liveState + recolors the matching map dot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 06:05:09 +00:00
serversdown 5455d3a931 style(portal): overview map uses dot markers, matching the internal project map
Swap Leaflet's default teardrop pins for L.circleMarker (radius 8, seismo-orange
fill, white border) + a name tooltip, same as partials/projects/location_map.html.
Also disables scroll-wheel zoom to match.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 03:40:17 +00:00
serversdown b971d19068 feat(portal): tile headline is Leq, not Lp; note live-map for M2
Lp (instantaneous) twitches every reading and makes a poor at-a-glance headline;
Leq (energy-average) is the stable, standard sound-monitoring/compliance metric.
Overview tiles now lead with Leq. Design doc: live project map (status-colored
pins + current-reading popups) recorded as an M2 item; headline-metric rationale
noted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 03:33:42 +00:00
serversdown 0103917870 feat(portal): live ~1Hz WS stream with auto-close (visibility + idle cap)
The portal location view is now genuinely live, not a 15s poll. Scoped WS endpoint
/portal/api/location/{id}/stream: authenticates via the session cookie, enforces
ownership (resolve_client_location), then bridges the unit's shared SLMM /monitor
fan-out feed to the browser — a viewer is just one more subscriber, no extra
device connection. Frames are scrubbed to the portal whitelist (drops unit_id,
raw_payload, counter, lmin) before reaching the client.

location.html: cache prefill for instant first paint, then upgrades to the live
socket (cards tick ~1Hz, chart scrolls). Auto-close so an abandoned tab can't pin
the device at 1Hz polling (~8x cellular data):
- closes when the tab is hidden, reopens when visible (Page Visibility) — the main
  guard;
- hard 15-min cap -> "Live paused — click to resume" overlay.

Refactor: client_from_cookie() extracted from get_current_client so the WS handler
(no Request-based Depends) can auth the same way.

Verified: scrub drops internal fields / keeps metrics + heartbeat (7/7), auth
refactor (3/3), portal compiles, location.html JS balances + parses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 03:16:32 +00:00
serversdown 3fc20e104a feat(portal): one-click "View client portal" preview from the project page
Adds a "View client portal" button on the project detail page that opens the
client portal scoped to that project — no CLI. GET /projects/{id}/portal-preview
auto-provisions a client + access token for the project (provision_preview_session)
and seals a portal session cookie, then redirects to /portal.

- Reuses the project's linked client if it has one; otherwise creates/reuses a
  per-project 'preview-<id>' client. Only sets project.client_id when unset, so it
  never clobbers a real client link. Idempotent — repeat clicks reuse the same
  client/token.
- Lives under /projects (not /portal), so a future public proxy exposing only
  /portal/* won't expose this operator shortcut.

Verified: provisioning (unlinked creates+links, idempotent, linked-no-clobber) 7/7.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 02:18:06 +00:00
serversdown c5b5045603 fix: remove stray conflict markers from .gitignore
Leftover <<<<<<</=======/>>>>>>> from an earlier dev merge that got committed
unresolved; keep the real ignore rules (*.db, /data/, /data-dev/, .aider*).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 02:02:08 +00:00
serversdown 2031681d0f docs(portal): add "Going to prod" checklist (migration, SECRET_KEY, exposure)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 02:01:58 +00:00
serversdown dd77f27cf6 Merge branch 'dev' into feat/ftp-report-pipeline
pulled in the live slm stuff
2026-06-11 01:54:21 +00:00
serversdown 1cf80ea7ea fix(portal): portal_admin.py runnable as a script, not just -m
`python3 backend/portal_admin.py` set sys.path[0] to backend/, hiding the
`backend` package and breaking `from backend.database import ...`. Insert the
project root on sys.path so the documented script invocation works.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 01:16:30 +00:00
serversdown 08d3d53702 feat(reports): add "Night Report" button to sound project header
Sound projects only: a Night Report button next to "Generate Combined Report"
opens a small modal (pick night + optional baseline range) that opens the
rendered report (/reports/nightly/view) in a new tab. Defaults the night to
last night; baseline is optional.

Verified the header partial renders and the button is gated to sound_monitoring
(hidden on vibration-only projects); modal + JS wired.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 00:59:50 +00:00
claude 786a9821a3 chore: rename terra-view container for clairity (pt 2) 2026-06-11 00:47:23 +00:00
serversdown a82bf59fb6 feat(reports): manual run/view endpoints for the night report
Add backend/routers/reports.py (registered in main.py):
- GET  /api/projects/{id}/reports/nightly/view — render the night report
  HTML inline (preview; no write, no email)
- POST /api/projects/{id}/reports/nightly/run  — build -> write
  report.html/report.json to disk -> dry-run email -> JSON result + view_url

Same entry point the scheduled morning tick will reuse. Query params:
night_date (default last night, local tz), baseline_start/end, metrics, send.
Orchestrator now also returns the rendered html for inline display.

Verified via FastAPI TestClient on real meter data (200 HTML with the computed
numbers, files written to disk, 400/404 validation paths).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 23:43:17 +00:00
serversdown 26b4b1e7e4 feat(portal): M1 admin CLI — create client, link projects, mint/revoke links
backend/portal_admin.py (run in-container): create-client, link-project (by id/
number/name -> sets Project.client_id), mint-link (prints the full magic URL once,
stores only the hash), list, revoke. PORTAL_BASE_URL controls the printed link base.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:43:28 +00:00
serversdown d3e221b6b1 feat(portal): M1 pages — locations overview + read-only live location view
/portal overview: client's active sound locations as live tiles (current Lp +
Live/Stopped badge + "updated Xm ago", polled from the scoped cache every 15s)
plus a Leaflet map of locations with coordinates. /portal/location/{id}: 404-gated
read-only live panel — Lp/Leq/Lmax/L1/L10 cards + a 4-line Chart.js trace
(backfilled from /history) + measuring/freshness badge. Cache-only, 15s poll, no
device controls, no refresh-from-device. _client_locations() feeds the overview.

Verified: portal.py compiles; both inline scripts balance; all four portal
templates parse in Jinja2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:41:13 +00:00
serversdown 9f40210057 feat(portal): M1 scoping gate + scoped cache endpoints
resolve_client_location() enforces ownership (sound location in one of the
client's active projects) and 404s everything else — same response for missing
and not-yours, so location existence never leaks. active_unit_for_location()
resolves the currently-assigned SLM.

Scoped GET /portal/api/location/{id}/live and /history: gate -> resolve unit ->
read SLMM cache (never the device). /live returns a SCRUBBED projection (sound
metrics + run state only; no battery/SD/raw_payload). Both degrade gracefully
when there's no device or SLMM is down.

Verified: ownership gate (owns / other-client / vibration / deleted-project /
removed / nonexistent) + active-vs-completed unit resolution — 8/8 on a temp DB.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:38:16 +00:00
serversdown 6c048a9c30 feat(portal): M1 auth gate — signed magic-URL session + get_current_client
backend/portal_auth.py: stdlib HMAC-signed session cookie carrying the access-
token id (re-validated against the DB each request, so revoke kills live
sessions), hash_token, resolve_token, and the get_current_client dependency
(raises PortalAuthError). SECRET_KEY env (insecure dev default + warning).

routers/portal.py: /portal/enter/{token} mints the cookie -> /portal; /logout;
/access; /portal home stub. main.py registers the router + a PortalAuthError
handler (HTML access page for pages, 401 JSON for /portal/api/*).

Portal shell templates (base, access_required, overview stub), branded dark.

Verified: cookie round-trip + tamper/garbage rejection, token resolution
(valid/bad), get_current_client (valid/no-cookie/revoked) — 8/8 against a temp DB.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:36:09 +00:00
serversdown 80a8470b55 feat(portal): M1 data model — Client, ClientAccessToken, Project.client_id
Client (customer org), ClientAccessToken (interim hashed magic-URL gate), and an
authoritative Project.client_id FK (client_name kept for display). New tables
auto-create via create_all; migrate_add_client_portal.py adds projects.client_id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:32:09 +00:00
serversdown a64c9ced65 docs: client portal design + milestone plan (M1 live view → M4 full auth)
Read-only, client-scoped portal inside Terra-View (/portal/*), reusing cached
SLMM reads. Data chain Client -> Project.client_id -> MonitoringLocation ->
active UnitAssignment -> unit_id -> SLMM cache. Auth is a swappable
get_current_client gate; M1-M3 ride an interim signed "magic URL", M4 replaces
the backing. Milestones: M1 live view, M2 dashboard+alerts, M3 reports, M4 auth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 21:29:14 +00:00
serversdown 90ec943a0b fix(slm): don't blank L1/L10 on percentile-less live-stream frames
The DRD stream carries Lp/Leq/Lmax but not the Ln percentiles (those come
from DOD polling), so updateLiveMetrics/updateDashboardMetrics were
overwriting the DOD-sourced L1/L10 values with '--' on every stream frame.
Guard the value updates on `data.lnN != null` so a frame without the key
leaves the existing value intact — mirrors the existing label guards.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 20:44:18 +00:00
serversdown 846807965c feat(slm): replace Lmin/Lpeak with configurable Ln1/Ln2 percentile slots
Live SLM display (dashboard + unit detail) now shows two configurable
percentile slots instead of Lmin/Lpeak. Values come from `ln1`/`ln2`;
labels come from `ln1_label`/`ln2_label` (default L1/L10), so a future
job can reconfigure the device's Ln slots to any percentile without a
Terra-View redeploy.

Contract for SLMM: emit ln1/ln2 (+ optional ln1_label/ln2_label) in both
the /status data dict and the DRD stream payload. No Terra-View Python
changes needed — proxy WS and current_status are transparent passthroughs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 20:44:18 +00:00
serversdown ed195ed96b feat(reports): FTP night-report pipeline foundation
Terra-View side of the daily night-vs-baseline sound report for the John Myler
24/7 job. Engine is built and verified end-to-end against real meter data;
SMTP send + scheduler/capture wiring still pending.

- ingest: refactor upload_nrl_data into a callable ingest_nrl_zip(location_id,
  zip_bytes, db) sharing one core with the HTTP endpoint. Capture the .rnh
  percentile map + weightings into session metadata; dedup on store-name +
  start time. Ingest stays metric-agnostic (every Leq column preserved).
- report_pipeline.py: metric registry, Evening/Nighttime windows, correct
  aggregation (Lmax=max, Ln=arithmetic, Leq=logarithmic), baseline = typical
  night, per-location + per-project builders.
- report_renderers.py: HTML email-body renderer (Last/Base/delta layout).
- report_email.py: config-driven SMTP via stdlib (env vars) with a dry-run
  fallback so the pipeline runs without credentials.
- report_orchestrator.py: compute -> render -> always write report.html +
  report.json to disk -> best-effort email.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 20:41:05 +00:00
serversdown 182e224f3c Merge pull request 'Feat: add SLM live monitoring improvements' (#60) from feat/slm-live-monitor into dev
Reviewed-on: #60

## [Unreleased]

SLM live monitoring — fan-out feed + cache-first reads.  Targets **0.14.0**.  The throughline: the NL-43 allows exactly **one** TCP connection at a time, so every page that opened its own device stream (or sent its own `Measure?`/DOD on load) was competing for that single connection — a second viewer saw nothing, and dashboard loads stole polling resolution from the live feed.  This release moves Terra-View entirely onto SLMM's shared, cached monitoring: one DOD poll loop per device, fanned out to all viewers; dashboards read SLMM's cache (a DB read on SLMM's side) instead of touching the device; and the live panels populate instantly from cache on open, upgrading to the live WS only on demand.  Paired with the SLMM-side work (adaptive poll rate, unreachable backoff, device-offline alert) on SLMM branch `dev`.

### Added

- **Fan-out `/monitor` feed consumption.**  The unit live view (`partials/slm_live_view.html`) and the dashboard live tile (`sound_level_meters.html`) now subscribe to SLMM's shared per-device monitor over `WS /api/slmm/{unit}/monitor` instead of each opening its own device stream.  Any number of clients attach without each consuming the NL-43's single connection — the "second viewer sees nothing" contention is gone.  A WS proxy handler for `/monitor` was added to `backend/routers/slmm.py`.
- **L1/L10 percentile lines + cards.**  Both the per-unit live chart and the dashboard card chart now plot L1 (purple) and L10 (orange) alongside Lp/Leq, and the KPI cards show L1/L10.  Sourced from the DOD feed's `ln1`/`ln2` (DRD streaming can't carry percentiles, DOD can).  Missing/`-.-` values leave a gap rather than dropping the line to 0.
- **Live-chart backfill on open.**  Charts seed from SLMM's downsampled DOD trail (`GET /api/slmm/{unit}/history?hours=2`) so a viewer sees recent trend immediately instead of a blank chart that fills one point per second.
- **Live Measurements panel auto-populates from cache.**  Opening the dashboard panel fills the KPI cards from cached `/status` and backfills the chart from `/history` — pure cache reads, no device hit.  Shows a measuring badge (● Measuring / ■ Stopped) and a freshness stamp ("as of 3:48 PM (10s ago)", amber + "cached" when stale).  Re-polls the cache every 15s while open; **Start Live Stream** upgrades to the live WS and no longer wipes the backfilled trail (chart point cap raised 60 → 600).
- **Refresh buttons** — one per device-list row, one in the panel header.  On-demand, user-initiated single device read via `GET /api/slmm/{unit}/live` (which also refreshes SLMM's cache), with a spinner + success/error toast, then reloads the device list.
- **Per-unit live-monitoring (keepalive) toggle on `/admin/slmm`** — turns a device's server-side keepalive feed on/off (`POST /monitor/start|stop`), so alerting can keep a device's feed running with no browser attached.

### Changed

- **Dashboard device list + command center read SLMM's cache, not the device.**  `slm_dashboard.py`'s `get_slm_units` pulls each unit's cached status from SLMM's `/roster` (one call, a SLMM DB read) for the badge + freshness; the command-center `get_live_view` reads cached `/status` instead of sending `Measure?` + a fresh DOD on every load.  This stops dashboard loads from stealing the device's single connection from the live monitor.  The elapsed-measurement timer still works because `measurement_start_time` is now included in the cached `/status` response.
- **Device-list freshness reflects real monitoring.**  The "Last check" line now uses SLMM's cached `last_seen` (which the monitor advances on every successful poll) via `unit.cache_last_seen`, instead of the `slm_last_check` roster field the monitor never updates.  The status badge also treats `Measure` as Measuring, matching the panel and SLMM's cache.
- **Status badge relocated** to the card's bottom meta row (next to "Last check"), off the top-right corner where it collided with the chart/gear/refresh action icons.

### Fixed

- **Deploy/bench threw `can't access property "dispatchEvent", e is null`.**  `toggleSLMDeployed()` and the save-config path called `htmx.trigger('#slm-list', 'load')` guarded only by `typeof htmx !== 'undefined'`; no page has a `#slm-list`, so htmx resolved null and called `null.dispatchEvent(...)`.  The deploy POST had already succeeded, so the operator saw both the green success **and** a red error.  Both call sites now guard on the element existing (`slm_settings_modal.html`).
- **Monitor WS proxy leaked `CancelledError` / "task exception never retrieved"** on stream stop — the cleanup awaited pending tasks but only caught `Exception`, missing `CancelledError` (a `BaseException`).
- **"No recent check-in" shown even on an actively-monitored device** — the row read the stale `slm_last_check` roster field instead of SLMM's live cache (see Changed).
- **L1/L10 KPI cards populated but the chart drew no L1/L10 lines** — the card chart only had Lp + Leq datasets.

### Upgrade Notes

Requires the **matching SLMM build (branch `dev`)** — Terra-View now depends on SLMM's fan-out `/monitor` feed, `/history` trail, `/status` carrying `ln1`/`ln2` + `measurement_start_time`, cached `/roster` status, and the `monitor_enabled` keepalive flag.

```bash
# SLMM (branch dev) — REBUILD + MIGRATE (or you'll get `no such column: nl43_status.ln1` 500s)
cd /home/serversdown/slmm && docker compose build slmm && docker compose up -d slmm
docker exec terra-view-slmm-1 python3 migrate_add_ln_percentiles.py
docker exec terra-view-slmm-1 python3 migrate_add_monitor_enabled.py

# Terra-View — NO migration; templates are baked into the image, so rebuild (don't just restart)
cd /home/serversdown/terra-view && docker compose build terra-view && docker compose up -d terra-view
```

The two builds must ship **together**.  Note the `docker-compose.yml` container was renamed for clarity (now `terra-view-terra-view-1`) — adjust any `docker exec` scripts that referenced the old name.

---
2026-06-10 16:33:25 -04:00
serversdown 2e832708f3 docs: changelog [Unreleased] — SLM live monitoring (fan-out feed + cache-first reads)
Captures the whole feat/slm-live-monitor effort (12 commits): fan-out /monitor
feed consumption, L1/L10 chart lines, live-chart backfill, cache-populated live
panel with measuring/freshness, per-unit + panel refresh, admin keepalive toggle,
dashboard/command-center cache reads, and the dispatchEvent/CancelledError/
freshness fixes. Targets 0.14.0; Upgrade Notes flag the paired SLMM `dev` build +
its migrations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 20:17:10 +00:00
serversdown 5e3645e229 fix(slm): show real cache freshness in device list + draw L1/L10 on card chart
1. "No recent check-in" was always shown because the row's last-check text read
   unit.slm_last_check (a Terra-View roster field the monitor never updates),
   while the live freshness lives in SLMM's cached NL43Status.last_seen. Carry
   that last_seen onto the unit (unit.cache_last_seen) and display it (falling
   back to slm_last_check). Also treat "Measure" as Measuring in the badge, to
   match the panel and the cache's MEASURING_STATES.

2. The dashboard card chart only had Lp + Leq datasets, so L1/L10 never drew even
   though the cards showed them. Add L1 (purple) and L10 (orange) datasets and
   feed ln1/ln2 in both the /history backfill and the live /monitor frames.
   Percentiles parse via numOrNull so a missing "-.-" leaves a gap (spanGaps)
   instead of dropping the line to 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 19:52:57 +00:00
serversdown 88f258d1c7 chore: rename terra-view container for clarity. 2026-06-10 19:45:55 +00:00
serversdown 711ef41e5f feat(slm): auto-populate live panel from cache, per-unit refresh, fix badge overlap
Live Measurements panel no longer sits blank until you click Start Live Stream:
- On open it fills the KPI cards from the cached /status snapshot (lp/leq/lmax/
  L1/L10) and backfills the chart from the /history DOD trail — both pure cache
  reads, no device hit.
- Shows measuring state (● Measuring / ■ Stopped) and a freshness stamp
  ("as of 2:14 PM (12m ago)") that turns amber + "cached" when stale, so a cached
  value is never mistaken for a live reading.
- Polls the cache every 15s while open so the cards stay current without opening
  a device stream; Start Live Stream takes over (and no longer wipes the
  backfilled trail). Chart cap raised 60 -> 600 so the 2h backfill isn't truncated.

Refresh buttons (on-demand, user-initiated single device read via GET /live,
which also updates the cache):
- one per device row in the list, and one in the panel header. Spinner while in
  flight; toast on success/failure; reloads the list so badges + last-check update.

Layout fix: the status badge (Measuring/Active/Idle/Benched) was rendered at the
top-right of the card, colliding with the absolutely-positioned chart/gear icons.
Moved it to the bottom meta row next to "Last check", padded the card content
clear of the action icons, and added the refresh icon to that group.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 18:59:42 +00:00