Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 serversdown
|
ca3035f50a | ||
|
serversdown
|
0e2086d6bb | ||
|
serversdown
|
d0685baed5 | ||
|
serversdown
|
275a168046 | ||
|
serversdown
|
f4fd1c943d |
@@ -220,6 +220,7 @@ marimo/_static/
|
||||
marimo/_lsp/
|
||||
__marimo__/
|
||||
|
||||
<<<<<<< HEAD
|
||||
# Seismo Fleet Manager
|
||||
# SQLite database files
|
||||
*.db
|
||||
@@ -227,3 +228,6 @@ __marimo__/
|
||||
/data/
|
||||
/data-dev/
|
||||
.aider*
|
||||
.aider*
|
||||
=======
|
||||
>>>>>>> 0c2186f5d89d948b0357d674c0773a67a67d8027
|
||||
|
||||
-108
@@ -5,114 +5,6 @@ All notable changes to Terra-View will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
SLM live monitoring — fan-out feed + cache-first reads. Targets **0.14.0**. The throughline: the NL-43 allows exactly **one** TCP connection at a time, so every page that opened its own device stream (or sent its own `Measure?`/DOD on load) was competing for that single connection — a second viewer saw nothing, and dashboard loads stole polling resolution from the live feed. This release moves Terra-View entirely onto SLMM's shared, cached monitoring: one DOD poll loop per device, fanned out to all viewers; dashboards read SLMM's cache (a DB read on SLMM's side) instead of touching the device; and the live panels populate instantly from cache on open, upgrading to the live WS only on demand. Paired with the SLMM-side work (adaptive poll rate, unreachable backoff, device-offline alert) on SLMM branch `dev`.
|
||||
|
||||
### Added
|
||||
|
||||
- **Fan-out `/monitor` feed consumption.** The unit live view (`partials/slm_live_view.html`) and the dashboard live tile (`sound_level_meters.html`) now subscribe to SLMM's shared per-device monitor over `WS /api/slmm/{unit}/monitor` instead of each opening its own device stream. Any number of clients attach without each consuming the NL-43's single connection — the "second viewer sees nothing" contention is gone. A WS proxy handler for `/monitor` was added to `backend/routers/slmm.py`.
|
||||
- **L1/L10 percentile lines + cards.** Both the per-unit live chart and the dashboard card chart now plot L1 (purple) and L10 (orange) alongside Lp/Leq, and the KPI cards show L1/L10. Sourced from the DOD feed's `ln1`/`ln2` (DRD streaming can't carry percentiles, DOD can). Missing/`-.-` values leave a gap rather than dropping the line to 0.
|
||||
- **Live-chart backfill on open.** Charts seed from SLMM's downsampled DOD trail (`GET /api/slmm/{unit}/history?hours=2`) so a viewer sees recent trend immediately instead of a blank chart that fills one point per second.
|
||||
- **Live Measurements panel auto-populates from cache.** Opening the dashboard panel fills the KPI cards from cached `/status` and backfills the chart from `/history` — pure cache reads, no device hit. Shows a measuring badge (● Measuring / ■ Stopped) and a freshness stamp ("as of 3:48 PM (10s ago)", amber + "cached" when stale). Re-polls the cache every 15s while open; **Start Live Stream** upgrades to the live WS and no longer wipes the backfilled trail (chart point cap raised 60 → 600).
|
||||
- **Refresh buttons** — one per device-list row, one in the panel header. On-demand, user-initiated single device read via `GET /api/slmm/{unit}/live` (which also refreshes SLMM's cache), with a spinner + success/error toast, then reloads the device list.
|
||||
- **Per-unit live-monitoring (keepalive) toggle on `/admin/slmm`** — turns a device's server-side keepalive feed on/off (`POST /monitor/start|stop`), so alerting can keep a device's feed running with no browser attached.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Dashboard device list + command center read SLMM's cache, not the device.** `slm_dashboard.py`'s `get_slm_units` pulls each unit's cached status from SLMM's `/roster` (one call, a SLMM DB read) for the badge + freshness; the command-center `get_live_view` reads cached `/status` instead of sending `Measure?` + a fresh DOD on every load. This stops dashboard loads from stealing the device's single connection from the live monitor. The elapsed-measurement timer still works because `measurement_start_time` is now included in the cached `/status` response.
|
||||
- **Device-list freshness reflects real monitoring.** The "Last check" line now uses SLMM's cached `last_seen` (which the monitor advances on every successful poll) via `unit.cache_last_seen`, instead of the `slm_last_check` roster field the monitor never updates. The status badge also treats `Measure` as Measuring, matching the panel and SLMM's cache.
|
||||
- **Status badge relocated** to the card's bottom meta row (next to "Last check"), off the top-right corner where it collided with the chart/gear/refresh action icons.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Deploy/bench threw `can't access property "dispatchEvent", e is null`.** `toggleSLMDeployed()` and the save-config path called `htmx.trigger('#slm-list', 'load')` guarded only by `typeof htmx !== 'undefined'`; no page has a `#slm-list`, so htmx resolved null and called `null.dispatchEvent(...)`. The deploy POST had already succeeded, so the operator saw both the green success **and** a red error. Both call sites now guard on the element existing (`slm_settings_modal.html`).
|
||||
- **Monitor WS proxy leaked `CancelledError` / "task exception never retrieved"** on stream stop — the cleanup awaited pending tasks but only caught `Exception`, missing `CancelledError` (a `BaseException`).
|
||||
- **"No recent check-in" shown even on an actively-monitored device** — the row read the stale `slm_last_check` roster field instead of SLMM's live cache (see Changed).
|
||||
- **L1/L10 KPI cards populated but the chart drew no L1/L10 lines** — the card chart only had Lp + Leq datasets.
|
||||
|
||||
### Upgrade Notes
|
||||
|
||||
Requires the **matching SLMM build (branch `dev`)** — Terra-View now depends on SLMM's fan-out `/monitor` feed, `/history` trail, `/status` carrying `ln1`/`ln2` + `measurement_start_time`, cached `/roster` status, and the `monitor_enabled` keepalive flag.
|
||||
|
||||
```bash
|
||||
# SLMM (branch dev) — REBUILD + MIGRATE (or you'll get `no such column: nl43_status.ln1` 500s)
|
||||
cd /home/serversdown/slmm && docker compose build slmm && docker compose up -d slmm
|
||||
docker exec terra-view-slmm-1 python3 migrate_add_ln_percentiles.py
|
||||
docker exec terra-view-slmm-1 python3 migrate_add_monitor_enabled.py
|
||||
|
||||
# Terra-View — NO migration; templates are baked into the image, so rebuild (don't just restart)
|
||||
cd /home/serversdown/terra-view && docker compose build terra-view && docker compose up -d terra-view
|
||||
```
|
||||
|
||||
The two builds must ship **together**. Note the `docker-compose.yml` container was renamed for clarity (now `terra-view-terra-view-1`) — adjust any `docker exec` scripts that referenced the old name.
|
||||
|
||||
---
|
||||
|
||||
### Client portal *(new — read-only client-facing view)*
|
||||
|
||||
A scoped, read-only portal at **`/portal/*`** where a client sees only *their*
|
||||
locations, live. Built inside Terra-View (no new service), reusing the cached
|
||||
SLMM feed; every route resolves the client through one swappable
|
||||
`get_current_client` gate, so the interim magic/open-link auth can be replaced
|
||||
(M4) without touching routes or templates. Strictly read-only — no device control.
|
||||
|
||||
#### Added
|
||||
|
||||
- **Per-client scoping + interim auth.** New `Client`, `ClientAccessToken`, and a
|
||||
`Project.client_id` FK. A signed (HMAC) session cookie carries the access-token
|
||||
id, re-validated against the DB each request (revoke kills live sessions, with
|
||||
server-side expiry). Entry via a magic link (`/portal/enter/{token}`) or a
|
||||
dev-only plain link (`/portal/open/{id}`, `PORTAL_OPEN_LINKS`, **default off**).
|
||||
- **Live location view.** KPI cards (Lp/Leq/Lmax/L1/L10) + chart populate
|
||||
instantly from cache, then upgrade to a real **~1 Hz WebSocket stream** scoped to
|
||||
the client's unit (a scrubbed bridge to the SLMM fan-out feed). The stream
|
||||
**auto-closes when the tab is hidden** (Page Visibility) and after a 15-min idle
|
||||
cap, so an abandoned tab can't pin the device at 1 Hz / burn cellular.
|
||||
- **Locations overview.** Live status map (level-colored dots, dark/light CARTO
|
||||
tiles) + a status rollup (live/offline counts, "loudest now"). Leq is the
|
||||
headline metric.
|
||||
- **Alerts (config → surface → 24/7).** Threshold-rule config on the SLM detail
|
||||
page (proxying SLMM's alert CRUD); breach **history + ack** internally and a
|
||||
read-only, scrubbed history + current-alarm banner + **"your alert limits"** panel
|
||||
in the portal; enabling a rule pins that device's monitor on so alerts evaluate
|
||||
round-the-clock.
|
||||
- **Operator sharing tools.** A **"View client portal"** preview button and a
|
||||
**"Copy client link"** modal (mint / list / revoke magic links) on the project
|
||||
page, plus a `backend/portal_admin.py` CLI.
|
||||
- **Field-instrument design.** Distinctive themed portal — Hanken Grotesk UI +
|
||||
IBM Plex Mono readouts, panel system, pulsing live dot, staggered reveal — with a
|
||||
**light/dark toggle** (light default, persisted, no-flash).
|
||||
|
||||
#### Security
|
||||
|
||||
- All scoping enforced server-side (404-not-403, no existence leak); client
|
||||
endpoints return **scrubbed** projections (no device-health/internal ids); WS
|
||||
frames whitelisted; operator-set strings HTML-escaped before injection (XSS).
|
||||
Pre-merge code review hardened cookie expiry, open-links default, and the slug
|
||||
collision. Remaining hardening (reverse proxy, TLS, `SECRET_KEY`, M4 auth) is
|
||||
tracked in `docs/CLIENT_PORTAL.md` → "Security hardening backlog".
|
||||
|
||||
#### Upgrade Notes
|
||||
|
||||
- **Migration:** `docker compose exec web-app python3 backend/migrate_add_client_portal.py`
|
||||
(adds `projects.client_id`; the `clients` / `client_access_tokens` tables
|
||||
auto-create).
|
||||
- Set a real **`SECRET_KEY`** in any internet-facing env (signs session cookies),
|
||||
and keep **`PORTAL_OPEN_LINKS=false`** there.
|
||||
- Portal alerts depend on the **SLMM `dev`** alert engine (rules/events/evaluator +
|
||||
cooldown + keepalive coupling) — same build pairing as above.
|
||||
|
||||
---
|
||||
|
||||
### Portal authentication (Phase 1)
|
||||
- Each project's client portal is now gated by a **secure per-project link + shared password** (argon2-hashed). Operators manage it from the project page's **Portal access** panel (enable, generate password, copy link).
|
||||
- Per-project session isolation (a session for one project can't read another's data); brute-force lockout (5 tries / 15 min) on the password gate.
|
||||
- Retired the interim magic-link / `PORTAL_OPEN_LINKS` open links and the `portal_admin.py mint-link` command.
|
||||
- **Upgrade:** new `argon2-cffi` dependency → **rebuild the image**, then run `python3 backend/migrate_add_project_portal_auth.py` per DB (adds the `projects.portal_*` columns). `SECRET_KEY` and `COOKIE_SECURE` are now passed through in `docker-compose.yml` (settable via a `.env` file) — set a real `SECRET_KEY` (and `COOKIE_SECURE=true` once on HTTPS) before the portal faces the internet.
|
||||
|
||||
---
|
||||
|
||||
## [0.13.3] - 2026-06-05
|
||||
|
||||
Calibration sync from SFM events. Closes the manual data-entry loop on calibration dates — Terra-View now pulls `device.calibration_date` from each seismograph's most recent event sidecar once a day and updates `RosterUnit.last_calibrated` when the device reports something fresher than what's stored. Manual edits still win when they're newer than the latest event; a fresh event arriving later supersedes the manual edit. Adds a "Sync now" button under Settings → Advanced → Calibration Defaults for on-demand runs, and a `docs/ROADMAP.md` to track in-flight + deferred work.
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
# FTP Report Pipeline — session brief
|
||||
|
||||
**Branch:** `feat/ftp-report-pipeline` (off `dev`), worktree `/home/serversdown/terra-view-reports`.
|
||||
**Scope:** Terra-View only. Do NOT touch SLMM — the SLMM alert/monitor work is live in a
|
||||
parallel session on `slmm` branch `feat/drd-fix`. Pull device data through the **existing**
|
||||
SLMM FTP proxy endpoints; add no SLMM code (for v1).
|
||||
|
||||
See memory note `client_sound_monitoring_job_2026-07` for the client requirements + timeline.
|
||||
|
||||
## Goal
|
||||
Automated **daily morning report** for the John Myler 3-location sound job: each AM, last
|
||||
night's noise levels vs the **baseline week**, per location. Data pulled from the meters via
|
||||
FTP (the meter records 24/7 to SD regardless of TCP wedges). Alerts are a *separate* workstream
|
||||
(SLMM, real-time DOD) — not in scope here.
|
||||
|
||||
## The big realization (why this is small)
|
||||
The hard parts already exist:
|
||||
- **SLMM (use as-is, via the `/api/slmm/...` proxy):**
|
||||
- `GET /api/slmm/{unit}/ftp/files?path=/NL-43` → list files/folders
|
||||
- `POST /api/slmm/{unit}/ftp/download-folder` → returns the `Auto_####` folder as a **ZIP**
|
||||
- **Terra-View ingest (reuse):** `backend/routers/project_locations.py:1743` `upload_nrl_data`
|
||||
already accepts a **ZIP**, extracts, keeps `.rnh` + `_Leq_ .rnd` (drops `_Lp_`/junk via
|
||||
`_is_wanted`), runs `_parse_rnh` (line 1687) → creates `MonitoringSession` + `DataFile`.
|
||||
- **Report generator (reuse, source-agnostic):** `backend/routers/projects.py`. The `.rnd`
|
||||
file reads funnel through 3 helpers — `_peek_rnd_headers` (~135), `_is_leq_file` (~147),
|
||||
`_read_rnd_file_rows` (~256). `.rnd` files live on disk under `data/{file_path}` (DataFile
|
||||
holds the path, not a BLOB). The stats/Excel/formatting logic doesn't care where bytes come from.
|
||||
|
||||
## Build (Terra-View)
|
||||
1. **Refactor** `upload_nrl_data`'s core into a callable `ingest_nrl_zip(location_id, zip_bytes, db)`
|
||||
so it can be invoked programmatically (not only via HTTP UploadFile).
|
||||
2. **Scheduled pull job** (reuse the existing scheduler): per project location/unit →
|
||||
`GET /ftp/files` to find new `Auto_####` folders → `POST /ftp/download-folder` (zip) →
|
||||
`ingest_nrl_zip(...)`. **Dedup** so repeated pulls don't duplicate sessions/files
|
||||
(track ingested folder names per location).
|
||||
3. **Baseline aggregation:** aggregate the baseline-week `_Leq_` intervals per location →
|
||||
reference values (nighttime Leq, L90 floor, typical Lmax).
|
||||
4. **Nightly report + email:** compute last night's metrics per location, compare to baseline
|
||||
(deltas), render (reuse the Excel/report machinery), email each morning.
|
||||
|
||||
## Data-location decision (light version, agreed)
|
||||
Keep `MonitoringSession`/`DataFile` **metadata in TV** for now; reuse the existing on-disk file
|
||||
store. Optional refinement (later): have SLMM keep the pulled files and TV read them through a
|
||||
SLMM file-serve endpoint (avoids the copy-into-TV step). Don't do that refinement under the
|
||||
deadline unless trivial — the report logic is identical either way.
|
||||
|
||||
## Open questions to resolve early
|
||||
1. **What's actually in a `_Leq_ .rnd`** — Leq only, or Leq + Lmax + Ln per 15-min interval?
|
||||
Decides whether the night-vs-baseline report can show L90/Lmax or just Leq. Inspect a real file.
|
||||
2. **Session rollover / dedup** — does a 2-week run write one growing `Auto_####` folder or new
|
||||
folders? Drives the "what's new" logic.
|
||||
3. **`download-folder` over a multi-day run** — confirm it zips cleanly (size/time).
|
||||
|
||||
## Client params (confirm with Dave before locking)
|
||||
Threshold/metric + their "night" window; report recipients + format (email body vs PDF/Excel).
|
||||
|
||||
## Timeline
|
||||
Setup ~7/1–7/2 (baseline week), shutdown week through ~7/17. Reports needed by ~7/8 (before
|
||||
shutdown). Today is ~3 weeks out — reliability > features.
|
||||
@@ -1,26 +0,0 @@
|
||||
"""Password hashing for the client portal — argon2id via argon2-cffi.
|
||||
|
||||
Kept separate from portal_auth (cookie signing) so the future operator auth can
|
||||
reuse the same hasher. Never store or log raw passwords."""
|
||||
import secrets
|
||||
from argon2 import PasswordHasher
|
||||
|
||||
_ph = PasswordHasher()
|
||||
|
||||
|
||||
def hash_password(raw: str) -> str:
|
||||
"""Return an argon2id hash string for a raw password."""
|
||||
return _ph.hash(raw)
|
||||
|
||||
|
||||
def verify_password(raw: str, hashed: str) -> bool:
|
||||
"""True iff raw matches the stored hash. Never raises."""
|
||||
try:
|
||||
return _ph.verify(hashed, raw)
|
||||
except Exception: # argon2 raises on mismatch/garbage; treat all as "no match"
|
||||
return False
|
||||
|
||||
|
||||
def generate_password(n_bytes: int = 12) -> str:
|
||||
"""A strong, URL-safe shareable password (~16 chars for n_bytes=12)."""
|
||||
return secrets.token_urlsafe(n_bytes)
|
||||
+2
-97
@@ -4,7 +4,7 @@ from fastapi import FastAPI, Request, Depends, HTTPException
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from fastapi.responses import HTMLResponse, FileResponse, JSONResponse, RedirectResponse
|
||||
from fastapi.responses import HTMLResponse, FileResponse, JSONResponse
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from sqlalchemy.orm import Session
|
||||
from typing import List, Dict, Optional
|
||||
@@ -66,21 +66,6 @@ app.mount("/static", StaticFiles(directory="backend/static"), name="static")
|
||||
# Use shared templates configuration with timezone filters
|
||||
from backend.templates_config import templates
|
||||
|
||||
# Client-portal auth: an unauthenticated portal request renders the access page
|
||||
# (HTML routes) or returns 401 JSON (/portal/api/* routes). Centralized so every
|
||||
# portal route can simply Depends(get_current_client).
|
||||
from backend.portal_auth import PortalAuthError
|
||||
|
||||
@app.exception_handler(PortalAuthError)
|
||||
async def portal_auth_handler(request: Request, exc: PortalAuthError):
|
||||
if request.url.path.startswith("/portal/api"):
|
||||
return JSONResponse(status_code=401, content={"detail": "Not authenticated"})
|
||||
return templates.TemplateResponse(
|
||||
"portal/access_required.html",
|
||||
{"request": request, "reason": "required"},
|
||||
status_code=401,
|
||||
)
|
||||
|
||||
# Add custom context processor to inject environment variable into all templates
|
||||
@app.middleware("http")
|
||||
async def add_environment_to_context(request: Request, call_next):
|
||||
@@ -112,10 +97,6 @@ app.include_router(slmm.router)
|
||||
app.include_router(slm_ui.router)
|
||||
app.include_router(slm_dashboard.router)
|
||||
app.include_router(seismo_dashboard.router)
|
||||
|
||||
# Client portal (read-only, scoped client view) — see docs/CLIENT_PORTAL.md
|
||||
from backend.routers import portal
|
||||
app.include_router(portal.router)
|
||||
app.include_router(sfm.router)
|
||||
app.include_router(modem_dashboard.router)
|
||||
|
||||
@@ -167,10 +148,6 @@ app.include_router(deployments.router)
|
||||
from backend.routers import calibration
|
||||
app.include_router(calibration.router)
|
||||
|
||||
# Nightly sound-report pipeline (manual triggers; scheduled tick reuses run_nightly_report)
|
||||
from backend.routers import reports
|
||||
app.include_router(reports.router)
|
||||
|
||||
# Start scheduler service and device status monitor on application startup
|
||||
from backend.services.scheduler import start_scheduler, stop_scheduler
|
||||
from backend.services.device_status_monitor import start_device_status_monitor, stop_device_status_monitor
|
||||
@@ -413,82 +390,10 @@ async def project_detail_page(request: Request, project_id: str):
|
||||
"""Project detail dashboard"""
|
||||
return templates.TemplateResponse("projects/detail.html", {
|
||||
"request": request,
|
||||
"project_id": project_id,
|
||||
"project_id": project_id
|
||||
})
|
||||
|
||||
|
||||
@app.get("/projects/{project_id}/portal-preview")
|
||||
async def project_portal_preview(project_id: str, db: Session = Depends(get_db)):
|
||||
"""Operator testing shortcut: open this project's client portal (no CLI)."""
|
||||
from backend.models import Project
|
||||
from backend.portal_auth import mint_portal_session, make_session_cookie, COOKIE_NAME, COOKIE_MAX_AGE, COOKIE_SECURE
|
||||
project = db.query(Project).filter_by(id=project_id).first()
|
||||
if not project:
|
||||
return JSONResponse(status_code=404, content={"detail": "Project not found"})
|
||||
token_id = mint_portal_session(project, db)
|
||||
resp = RedirectResponse(url="/portal", status_code=303)
|
||||
resp.set_cookie(COOKIE_NAME, make_session_cookie(token_id),
|
||||
max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax", secure=COOKIE_SECURE)
|
||||
return resp
|
||||
|
||||
|
||||
@app.get("/projects/{project_id}/portal-access")
|
||||
async def project_portal_access_state(project_id: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Current portal-access state for the operator panel."""
|
||||
from backend.models import Project
|
||||
p = db.query(Project).filter_by(id=project_id).first()
|
||||
if not p:
|
||||
return JSONResponse(status_code=404, content={"detail": "Project not found"})
|
||||
link_url = (str(request.base_url).rstrip("/") + f"/portal/p/{p.portal_link_token}") \
|
||||
if (p.portal_enabled and p.portal_link_token) else None
|
||||
return {"enabled": bool(p.portal_enabled), "has_password": bool(p.portal_password_hash),
|
||||
"link_url": link_url}
|
||||
|
||||
|
||||
@app.post("/projects/{project_id}/portal-access/enable")
|
||||
async def project_portal_access_enable(project_id: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Turn the portal on; mint a link token if one doesn't exist yet."""
|
||||
import secrets
|
||||
from backend.models import Project
|
||||
p = db.query(Project).filter_by(id=project_id).first()
|
||||
if not p:
|
||||
return JSONResponse(status_code=404, content={"detail": "Project not found"})
|
||||
if not p.portal_link_token:
|
||||
p.portal_link_token = secrets.token_urlsafe(24)
|
||||
p.portal_enabled = True
|
||||
db.commit()
|
||||
link_url = str(request.base_url).rstrip("/") + f"/portal/p/{p.portal_link_token}"
|
||||
return {"enabled": True, "has_password": bool(p.portal_password_hash), "link_url": link_url}
|
||||
|
||||
|
||||
@app.post("/projects/{project_id}/portal-access/password")
|
||||
async def project_portal_access_password(project_id: str, db: Session = Depends(get_db)):
|
||||
"""Generate a fresh strong password, store its hash, return the raw once."""
|
||||
from backend.models import Project
|
||||
from backend.auth_passwords import hash_password, generate_password
|
||||
p = db.query(Project).filter_by(id=project_id).first()
|
||||
if not p:
|
||||
return JSONResponse(status_code=404, content={"detail": "Project not found"})
|
||||
raw = generate_password()
|
||||
p.portal_password_hash = hash_password(raw)
|
||||
db.commit()
|
||||
return {"password": raw}
|
||||
|
||||
|
||||
@app.post("/projects/{project_id}/portal-access/disable")
|
||||
async def project_portal_access_disable(project_id: str, db: Session = Depends(get_db)):
|
||||
"""Turn the portal off and rotate the link token (kills the old link)."""
|
||||
import secrets
|
||||
from backend.models import Project
|
||||
p = db.query(Project).filter_by(id=project_id).first()
|
||||
if not p:
|
||||
return JSONResponse(status_code=404, content={"detail": "Project not found"})
|
||||
p.portal_enabled = False
|
||||
p.portal_link_token = secrets.token_urlsafe(24) # rotate so the old link 404s
|
||||
db.commit()
|
||||
return {"enabled": False}
|
||||
|
||||
|
||||
@app.get("/projects/{project_id}/nrl/{location_id}", response_class=HTMLResponse)
|
||||
async def nrl_detail_page(
|
||||
request: Request,
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Database migration: Client Portal (M1).
|
||||
|
||||
Adds the authoritative client link to projects:
|
||||
- projects.client_id (TEXT, nullable) -> clients.id
|
||||
|
||||
The `clients` and `client_access_tokens` tables are created automatically by
|
||||
SQLAlchemy `create_all` at app startup (they're brand-new tables), so this
|
||||
migration only handles the column that create_all won't add to an existing
|
||||
`projects` table.
|
||||
|
||||
Run once per database:
|
||||
docker exec terra-view-terra-view-1 python3 backend/migrate_add_client_portal.py
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def migrate():
|
||||
possible_paths = [
|
||||
Path("data/seismo_fleet.db"),
|
||||
Path("data/sfm.db"),
|
||||
Path("data/seismo.db"),
|
||||
]
|
||||
db_path = next((p for p in possible_paths if p.exists()), None)
|
||||
if db_path is None:
|
||||
print(f"Database not found in any of: {[str(p) for p in possible_paths]}")
|
||||
print("A fresh DB created via models.py will include projects.client_id automatically.")
|
||||
return
|
||||
|
||||
print(f"Using database: {db_path}")
|
||||
conn = sqlite3.connect(db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("PRAGMA table_info(projects)")
|
||||
existing = {row[1] for row in cursor.fetchall()}
|
||||
|
||||
if "client_id" not in existing:
|
||||
try:
|
||||
cursor.execute("ALTER TABLE projects ADD COLUMN client_id TEXT")
|
||||
print("✓ Added column: projects.client_id (TEXT)")
|
||||
except sqlite3.OperationalError as e:
|
||||
print(f"✗ Failed to add projects.client_id: {e}")
|
||||
else:
|
||||
print("○ Column already exists: projects.client_id")
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print("\n✓ Client-portal migration complete.")
|
||||
print(" Note: `clients` + `client_access_tokens` tables auto-create on app startup.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
migrate()
|
||||
@@ -1,61 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Database migration: Project portal auth (Phase 1).
|
||||
|
||||
Adds the per-project portal gate columns to `projects`:
|
||||
- portal_enabled (BOOLEAN, default 0)
|
||||
- portal_password_hash (TEXT, nullable)
|
||||
- portal_link_token (TEXT, nullable) [+ unique index]
|
||||
|
||||
Idempotent. Run once per existing DB:
|
||||
docker exec terra-view-terra-view-1 python3 backend/migrate_add_project_portal_auth.py
|
||||
"""
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
_COLUMNS = {
|
||||
"portal_enabled": "BOOLEAN DEFAULT 0",
|
||||
"portal_password_hash": "TEXT",
|
||||
"portal_link_token": "TEXT",
|
||||
}
|
||||
|
||||
|
||||
def migrate():
|
||||
possible_paths = [Path("data/seismo_fleet.db"), Path("data/sfm.db"), Path("data/seismo.db")]
|
||||
db_path = next((p for p in possible_paths if p.exists()), None)
|
||||
if db_path is None:
|
||||
print(f"Database not found in any of: {[str(p) for p in possible_paths]}")
|
||||
print("A fresh DB created via models.py will include these columns automatically.")
|
||||
return
|
||||
|
||||
print(f"Using database: {db_path}")
|
||||
conn = sqlite3.connect(db_path)
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("PRAGMA table_info(projects)")
|
||||
existing = {row[1] for row in cursor.fetchall()}
|
||||
|
||||
for col, ddl in _COLUMNS.items():
|
||||
if col in existing:
|
||||
print(f"○ Column already exists: projects.{col}")
|
||||
continue
|
||||
try:
|
||||
cursor.execute(f"ALTER TABLE projects ADD COLUMN {col} {ddl}")
|
||||
print(f"✓ Added column: projects.{col} ({ddl})")
|
||||
except sqlite3.OperationalError as e:
|
||||
print(f"✗ Failed to add projects.{col}: {e}")
|
||||
|
||||
# Unique index on the link token (separate from ADD COLUMN; idempotent via IF NOT EXISTS).
|
||||
try:
|
||||
cursor.execute("CREATE UNIQUE INDEX IF NOT EXISTS ix_projects_portal_link_token "
|
||||
"ON projects (portal_link_token)")
|
||||
print("✓ Ensured unique index: ix_projects_portal_link_token")
|
||||
except sqlite3.OperationalError as e:
|
||||
print(f"✗ Failed to create index: {e}")
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
print("\n✓ Project portal-auth migration complete.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
migrate()
|
||||
@@ -192,11 +192,6 @@ class Project(Base):
|
||||
|
||||
# Project metadata
|
||||
client_name = Column(String, nullable=True, index=True) # Client name (e.g., "PJ Dick")
|
||||
client_id = Column(String, nullable=True, index=True) # FK -> clients.id; authoritative portal link (client_name kept for display)
|
||||
# --- Client portal (Phase 1: per-project link + password gate) ---
|
||||
portal_enabled = Column(Boolean, default=False) # is the portal open for this project
|
||||
portal_password_hash = Column(String, nullable=True) # argon2 hash of the shared password
|
||||
portal_link_token = Column(String, nullable=True, unique=True, index=True) # unguessable token in the secure link
|
||||
site_address = Column(String, nullable=True)
|
||||
site_coordinates = Column(String, nullable=True) # "lat,lon"
|
||||
start_date = Column(Date, nullable=True)
|
||||
@@ -223,35 +218,6 @@ class ProjectModule(Base):
|
||||
__table_args__ = (UniqueConstraint("project_id", "module_type", name="uq_project_module"),)
|
||||
|
||||
|
||||
class SoundReportConfig(Base):
|
||||
"""
|
||||
Per-project configuration for the automated nightly sound report
|
||||
(FTP report pipeline). One row per project. Read by the morning tick in
|
||||
SchedulerService and by the manual /reports endpoints (as defaults).
|
||||
|
||||
New table → created by Base.metadata.create_all() on startup; no migration
|
||||
needed (only a rebuild/restart).
|
||||
"""
|
||||
__tablename__ = "sound_report_configs"
|
||||
|
||||
id = Column(String, primary_key=True, default=lambda: __import__('uuid').uuid4().__str__())
|
||||
project_id = Column(String, nullable=False, index=True, unique=True) # FK to projects.id
|
||||
|
||||
enabled = Column(Boolean, default=False, nullable=False) # run the daily report?
|
||||
report_time = Column(String, default="08:00", nullable=False) # local HH:MM to run/send
|
||||
metric_keys = Column(String, default="lmax,l01,l10,l90", nullable=False) # csv of metric keys
|
||||
# Baseline source: "captured" = compute from recorded nights in the date range below;
|
||||
# "reference" = use fixed values typed per location (old-report averages or a spec limit).
|
||||
baseline_mode = Column(String, default="captured", nullable=False)
|
||||
baseline_start = Column(Date, nullable=True) # captured-mode range
|
||||
baseline_end = Column(Date, nullable=True)
|
||||
recipients = Column(Text, nullable=True) # csv; falls back to REPORT_SMTP_RECIPIENTS env
|
||||
last_run_date = Column(Date, nullable=True) # evening-date of the last reported night (dedup)
|
||||
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
|
||||
|
||||
|
||||
class MonitoringLocation(Base):
|
||||
"""
|
||||
Monitoring locations: generic location for monitoring activities.
|
||||
@@ -738,37 +704,3 @@ class PendingDeployment(Base):
|
||||
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# CLIENT PORTAL — read-only, scoped client access (see docs/CLIENT_PORTAL.md)
|
||||
# ============================================================================
|
||||
|
||||
class Client(Base):
|
||||
"""A portal client (customer org). Owns one or more Projects via
|
||||
Project.client_id; their portal surfaces only those projects' locations.
|
||||
Read-only — clients never control devices."""
|
||||
__tablename__ = "clients"
|
||||
|
||||
id = Column(String, primary_key=True, index=True) # UUID
|
||||
name = Column(String, nullable=False) # display name, e.g. "PJ Dick"
|
||||
slug = Column(String, nullable=False, unique=True, index=True) # URL-safe handle
|
||||
contact_email = Column(String, nullable=True) # for M4 magic-link
|
||||
active = Column(Boolean, default=True) # False = portal access off
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
|
||||
|
||||
class ClientAccessToken(Base):
|
||||
"""Interim 'magic URL' gate (M1-M3). The raw secret lives in the link and is
|
||||
shown once on creation; only its sha256 is stored here. Revoke by setting
|
||||
revoked_at. In M4 this is replaced behind get_current_client() without
|
||||
touching routes/templates."""
|
||||
__tablename__ = "client_access_tokens"
|
||||
|
||||
id = Column(String, primary_key=True, index=True) # UUID
|
||||
client_id = Column(String, nullable=False, index=True) # FK -> clients.id
|
||||
token_hash = Column(String, nullable=False, index=True) # sha256 hex of the secret
|
||||
label = Column(String, nullable=True) # e.g. "Dave's link"
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
last_used_at = Column(DateTime, nullable=True)
|
||||
revoked_at = Column(DateTime, nullable=True) # set = link no longer works
|
||||
|
||||
@@ -1,161 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Client-portal admin CLI (M1). Operator tooling — run inside the terra-view
|
||||
container against the live DB. The raw magic-link token is shown ONCE on mint;
|
||||
only its hash is stored.
|
||||
|
||||
# create a client
|
||||
python3 backend/portal_admin.py create-client --name "Myler Co" --slug myler [--email dave@x.com]
|
||||
|
||||
# attach a project to a client (sets Project.client_id) — by id, number, or name
|
||||
python3 backend/portal_admin.py link-project --slug myler --project-id <PID>
|
||||
python3 backend/portal_admin.py link-project --slug myler --project-number 2567-23
|
||||
python3 backend/portal_admin.py link-project --slug myler --project-name "RKM Hall"
|
||||
|
||||
# mint-link is RETIRED — per-client magic URLs (/portal/enter) no longer exist.
|
||||
# Client access is now per-PROJECT + password: open the project's page in
|
||||
# Terra-View → "Portal access" to enable it, generate a password, and copy
|
||||
# the /portal/p/<token> link. (create-client / link-project / list / revoke
|
||||
# still operate on the underlying Client/token rows.)
|
||||
|
||||
# list clients, their projects, and active links
|
||||
python3 backend/portal_admin.py list
|
||||
|
||||
# revoke a link (stops the link AND any live session it minted)
|
||||
python3 backend/portal_admin.py revoke --token-id <TID>
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
# Allow `python3 backend/portal_admin.py ...` (which puts backend/ on sys.path[0],
|
||||
# hiding the `backend` package) in addition to `python3 -m backend.portal_admin`.
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from backend.database import SessionLocal
|
||||
from backend.models import Client, ClientAccessToken, Project
|
||||
|
||||
|
||||
def _get_client(db, slug):
|
||||
c = db.query(Client).filter_by(slug=slug).first()
|
||||
if not c:
|
||||
sys.exit(f"No client with slug '{slug}'. Create it first.")
|
||||
return c
|
||||
|
||||
|
||||
def create_client(args):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
if db.query(Client).filter_by(slug=args.slug).first():
|
||||
sys.exit(f"A client with slug '{args.slug}' already exists.")
|
||||
c = Client(id=str(uuid.uuid4()), name=args.name, slug=args.slug,
|
||||
contact_email=args.email, active=True)
|
||||
db.add(c)
|
||||
db.commit()
|
||||
print(f"✓ Created client '{c.name}' (slug={c.slug}, id={c.id})")
|
||||
print(" Next: link-project, then mint-link.")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def link_project(args):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
c = _get_client(db, args.slug)
|
||||
q = db.query(Project)
|
||||
if args.project_id:
|
||||
p = q.filter_by(id=args.project_id).first()
|
||||
elif args.project_number:
|
||||
p = q.filter_by(project_number=args.project_number).first()
|
||||
elif args.project_name:
|
||||
p = q.filter_by(name=args.project_name).first()
|
||||
else:
|
||||
sys.exit("Specify --project-id, --project-number, or --project-name.")
|
||||
if not p:
|
||||
sys.exit("Project not found.")
|
||||
p.client_id = c.id
|
||||
db.commit()
|
||||
print(f"✓ Linked project '{p.name}' (id={p.id}) -> client '{c.name}'")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def mint_link(args):
|
||||
# Retired: the per-client magic URL (/portal/enter/...) was removed when the
|
||||
# portal moved to per-project + password access. Minting a token here would
|
||||
# only produce a dead link.
|
||||
sys.exit(
|
||||
"mint-link is retired: per-client magic URLs (/portal/enter/...) no longer exist.\n"
|
||||
"Client access is now per-project + password. In Terra-View, open the project's page →\n"
|
||||
"'Portal access' to enable the portal, generate a password, and copy the /portal/p/<token>\n"
|
||||
"link to send the client."
|
||||
)
|
||||
|
||||
|
||||
def revoke(args):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
tok = db.query(ClientAccessToken).filter_by(id=args.token_id).first()
|
||||
if not tok:
|
||||
sys.exit("No token with that id.")
|
||||
if tok.revoked_at:
|
||||
print("○ Already revoked.")
|
||||
return
|
||||
tok.revoked_at = datetime.utcnow()
|
||||
db.commit()
|
||||
print(f"✓ Revoked token {tok.id} — the link and any live sessions it minted are dead.")
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def list_all(args):
|
||||
db = SessionLocal()
|
||||
try:
|
||||
clients = db.query(Client).order_by(Client.name).all()
|
||||
if not clients:
|
||||
print("No clients yet.")
|
||||
return
|
||||
for c in clients:
|
||||
state = "" if c.active else " [INACTIVE]"
|
||||
print(f"\n● {c.name} (slug={c.slug}){state}")
|
||||
projs = db.query(Project).filter_by(client_id=c.id).all()
|
||||
print(" projects: " + (", ".join(p.name for p in projs) or "(none linked)"))
|
||||
toks = db.query(ClientAccessToken).filter_by(client_id=c.id).all()
|
||||
if not toks:
|
||||
print(" links: (none — run mint-link)")
|
||||
for t in toks:
|
||||
status = "revoked" if t.revoked_at else "active"
|
||||
last = t.last_used_at.strftime("%Y-%m-%d %H:%M") if t.last_used_at else "never used"
|
||||
print(f" link {t.id} [{status}] {t.label or ''} (last: {last})")
|
||||
print()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description="Client-portal admin (M1)")
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
|
||||
p = sub.add_parser("create-client"); p.add_argument("--name", required=True)
|
||||
p.add_argument("--slug", required=True); p.add_argument("--email"); p.set_defaults(fn=create_client)
|
||||
|
||||
p = sub.add_parser("link-project"); p.add_argument("--slug", required=True)
|
||||
p.add_argument("--project-id"); p.add_argument("--project-number"); p.add_argument("--project-name")
|
||||
p.set_defaults(fn=link_project)
|
||||
|
||||
p = sub.add_parser("mint-link"); p.add_argument("--slug", required=True)
|
||||
p.add_argument("--label"); p.set_defaults(fn=mint_link)
|
||||
|
||||
p = sub.add_parser("revoke"); p.add_argument("--token-id", required=True); p.set_defaults(fn=revoke)
|
||||
|
||||
p = sub.add_parser("list"); p.set_defaults(fn=list_all)
|
||||
|
||||
args = ap.parse_args()
|
||||
args.fn(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,193 +0,0 @@
|
||||
"""
|
||||
Client-portal auth — the swappable gate (see docs/CLIENT_PORTAL.md).
|
||||
|
||||
M1-M3 ride on an interim signed "magic URL": an unguessable token in the link
|
||||
mints a signed session cookie. Every portal route depends on get_current_client();
|
||||
M4 replaces the backing (magic-link / accounts) without touching routes/templates.
|
||||
|
||||
The cookie carries the ACCESS-TOKEN id (not the client id) and is re-validated
|
||||
against the DB on every request, so revoking a link (revoked_at) kills its live
|
||||
sessions on the next request — not just future clicks.
|
||||
|
||||
No new dependency: the cookie is signed with stdlib HMAC-SHA256 over a SECRET_KEY.
|
||||
"""
|
||||
|
||||
import os
|
||||
import hmac
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
import base64
|
||||
import hashlib
|
||||
import logging
|
||||
import secrets
|
||||
|
||||
from fastapi import Request, Depends
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.database import get_db
|
||||
from backend.models import Client, ClientAccessToken, Project
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Signing secret for portal session cookies. MUST be set to a real secret in prod
|
||||
# (env). The insecure default only exists so dev/test boots without config.
|
||||
SECRET_KEY = os.getenv("SECRET_KEY", "dev-insecure-change-me")
|
||||
if SECRET_KEY == "dev-insecure-change-me":
|
||||
logger.warning("[PORTAL] SECRET_KEY is the insecure default — set SECRET_KEY in prod.")
|
||||
|
||||
COOKIE_NAME = "portal_session"
|
||||
COOKIE_MAX_AGE = 60 * 60 * 24 * 30 # 30 days
|
||||
# Set COOKIE_SECURE=true once the portal is served over HTTPS (TLS terminates at
|
||||
# the Synology reverse proxy). Default false so plain-HTTP dev still works.
|
||||
COOKIE_SECURE = os.getenv("COOKIE_SECURE", "false").lower() in ("1", "true", "yes")
|
||||
|
||||
|
||||
class PortalAuthError(Exception):
|
||||
"""Raised by get_current_client when there's no valid portal session.
|
||||
Handled centrally in main.py: HTML routes get the access-required page,
|
||||
/portal/api/* routes get a 401 JSON."""
|
||||
|
||||
|
||||
# -- token + cookie primitives ----------------------------------------------
|
||||
|
||||
def hash_token(raw: str) -> str:
|
||||
"""sha256 hex of a raw access-token secret (what we store + look up by)."""
|
||||
return hashlib.sha256(raw.encode()).hexdigest()
|
||||
|
||||
|
||||
def _sign(body: str) -> str:
|
||||
return hmac.new(SECRET_KEY.encode(), body.encode(), hashlib.sha256).hexdigest()
|
||||
|
||||
|
||||
def make_session_cookie(token_id: str) -> str:
|
||||
body = base64.urlsafe_b64encode(
|
||||
json.dumps({"tid": token_id, "iat": int(time.time())}).encode()
|
||||
).decode()
|
||||
return f"{body}.{_sign(body)}"
|
||||
|
||||
|
||||
def _read_session_cookie(value: str):
|
||||
"""Return the token id from a signed cookie, or None if missing/tampered."""
|
||||
try:
|
||||
body, sig = value.rsplit(".", 1)
|
||||
except (ValueError, AttributeError):
|
||||
return None
|
||||
if not hmac.compare_digest(sig, _sign(body)):
|
||||
return None
|
||||
try:
|
||||
data = json.loads(base64.urlsafe_b64decode(body.encode()))
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
# Server-side expiry: a leaked cookie isn't valid forever (max_age is only a
|
||||
# browser hint). iat is set by make_session_cookie.
|
||||
iat = data.get("iat")
|
||||
if not isinstance(iat, (int, float)) or (time.time() - iat) > COOKIE_MAX_AGE:
|
||||
return None
|
||||
return data.get("tid")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
# -- the dependency every portal route uses ---------------------------------
|
||||
|
||||
def client_from_cookie(cookie_value, db: Session):
|
||||
"""Resolve a Client from a raw session-cookie value, or None. Re-validates the
|
||||
access token against the DB each call, so a revoked link / disabled client
|
||||
drops immediately. Shared by the HTTP dependency and the WebSocket handler
|
||||
(which can't use Request-based Depends)."""
|
||||
token_id = _read_session_cookie(cookie_value) if cookie_value else None
|
||||
if not token_id:
|
||||
return None
|
||||
tok = db.query(ClientAccessToken).filter_by(id=token_id, revoked_at=None).first()
|
||||
if not tok:
|
||||
return None
|
||||
return db.query(Client).filter_by(id=tok.client_id, active=True).first()
|
||||
|
||||
|
||||
def get_current_client(request: Request, db: Session = Depends(get_db)) -> Client:
|
||||
"""Resolve the authenticated client, or raise PortalAuthError."""
|
||||
client = client_from_cookie(request.cookies.get(COOKIE_NAME), db)
|
||||
if client is None:
|
||||
raise PortalAuthError()
|
||||
return client
|
||||
|
||||
|
||||
# --- Phase-1 per-project password gate -------------------------------------------
|
||||
# A portal-enabled project gets its OWN dedicated client (slug "portal-<project.id>")
|
||||
# owning exactly that project. The project is linked to it via project.client_id so
|
||||
# the existing client-scoped routes (which resolve projects by Project.client_id ==
|
||||
# client.id) surface exactly this one project for the portal session — per-project
|
||||
# isolation with no route changes. (Phase 1 repurposes project.client_id for this; a
|
||||
# real per-client model is the deferred multi-tenant work.)
|
||||
|
||||
|
||||
def portal_client_for_project(project, db) -> Client:
|
||||
"""Get-or-create the dedicated 1:1 portal client for a project, and link the
|
||||
project to it so the client-scoped routes resolve exactly this project."""
|
||||
slug = f"portal-{project.id}"
|
||||
client = db.query(Client).filter_by(slug=slug).first()
|
||||
if client is None:
|
||||
client = Client(id=str(uuid.uuid4()),
|
||||
name=(project.client_name or project.name or "Client"),
|
||||
slug=slug, active=True)
|
||||
db.add(client)
|
||||
db.flush()
|
||||
if project.client_id != client.id:
|
||||
project.client_id = client.id # without this, the client owns no projects
|
||||
db.flush()
|
||||
return client
|
||||
|
||||
|
||||
def mint_portal_session(project, db) -> str:
|
||||
"""Ensure the project's portal client + an access token exist; return the token
|
||||
id to seal into a session cookie. Reuses an existing token to avoid clutter."""
|
||||
client = portal_client_for_project(project, db)
|
||||
tok = db.query(ClientAccessToken).filter_by(client_id=client.id, revoked_at=None).first()
|
||||
if tok is None:
|
||||
tok = ClientAccessToken(id=str(uuid.uuid4()), client_id=client.id,
|
||||
token_hash=hash_token(secrets.token_urlsafe(32)),
|
||||
label="portal")
|
||||
db.add(tok)
|
||||
db.commit()
|
||||
return tok.id
|
||||
|
||||
|
||||
def resolve_project_by_link_token(link_token: str, db):
|
||||
"""Return the portal-enabled Project for a link token, or None."""
|
||||
if not link_token:
|
||||
return None
|
||||
return db.query(Project).filter_by(
|
||||
portal_link_token=link_token, portal_enabled=True).first()
|
||||
|
||||
|
||||
# In-memory brute-force lockout, keyed per link_token (the password is shared per
|
||||
# project, so per-IP granularity buys nothing and an IP term only lets an attacker
|
||||
# reset the budget by rotating source IPs). Resets on restart; adequate for a
|
||||
# read-only surface behind the UniFi edge. Single-worker dev; multi-worker would
|
||||
# need a shared store.
|
||||
MAX_ATTEMPTS = 5
|
||||
LOCK_SECONDS = 15 * 60
|
||||
_failures: dict = {} # key -> (count, first_failure_epoch)
|
||||
|
||||
|
||||
def is_locked(key: str) -> bool:
|
||||
rec = _failures.get(key)
|
||||
if not rec:
|
||||
return False
|
||||
count, first = rec
|
||||
if count < MAX_ATTEMPTS:
|
||||
return False
|
||||
if (time.time() - first) > LOCK_SECONDS:
|
||||
_failures.pop(key, None) # window expired
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def register_failure(key: str) -> None:
|
||||
count, first = _failures.get(key, (0, time.time()))
|
||||
_failures[key] = (count + 1, first)
|
||||
|
||||
|
||||
def clear_failures(key: str) -> None:
|
||||
_failures.pop(key, None)
|
||||
@@ -1,375 +0,0 @@
|
||||
"""
|
||||
Client portal — read-only, scoped client view (see docs/CLIENT_PORTAL.md).
|
||||
|
||||
A client opens a per-project secure link (/portal/p/{link_token}), enters the
|
||||
shared password, and gets a signed session cookie scoped to that project; they
|
||||
then see that project's locations (overview) and per-location read-only live
|
||||
data sourced from SLMM's cache. Every data route re-checks ownership.
|
||||
"""
|
||||
|
||||
import os
|
||||
import json
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import datetime
|
||||
|
||||
import httpx
|
||||
import websockets
|
||||
from fastapi import APIRouter, Request, Depends, HTTPException, WebSocket, Form
|
||||
from fastapi.responses import RedirectResponse
|
||||
from sqlalchemy import or_
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.database import get_db, SessionLocal
|
||||
from backend.models import Client, MonitoringLocation, Project, UnitAssignment
|
||||
from backend.templates_config import templates
|
||||
from backend.portal_auth import (
|
||||
get_current_client, client_from_cookie, make_session_cookie,
|
||||
COOKIE_NAME, COOKIE_MAX_AGE, COOKIE_SECURE,
|
||||
resolve_project_by_link_token, mint_portal_session,
|
||||
is_locked, register_failure, clear_failures,
|
||||
)
|
||||
from backend.auth_passwords import verify_password
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/portal", tags=["portal"])
|
||||
|
||||
SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
SLMM_WS_BASE_URL = SLMM_BASE_URL.replace("http://", "ws://").replace("https://", "wss://")
|
||||
|
||||
# Whitelist of fields the portal exposes to a client — sound metrics + run state
|
||||
# only. Internal device health (battery/power/SD/raw_payload) is NOT disclosed.
|
||||
_PORTAL_LIVE_FIELDS = ("measurement_state", "last_seen", "measurement_start_time",
|
||||
"lp", "leq", "lmax", "lpeak", "ln1", "ln2")
|
||||
|
||||
|
||||
# -- scoping (every data route gates through these) --------------------------
|
||||
|
||||
def _client_project_ids(client: Client, db: Session) -> list:
|
||||
return [r[0] for r in db.query(Project.id).filter(
|
||||
Project.client_id == client.id, Project.status != "deleted").all()]
|
||||
|
||||
|
||||
def resolve_client_location(client: Client, location_id: str, db: Session) -> MonitoringLocation:
|
||||
"""Ownership gate: location must be a sound location in one of the client's
|
||||
active projects. Raises 404 (not 403) for both 'missing' and 'not yours' so
|
||||
we never leak whether a location exists."""
|
||||
loc = db.query(MonitoringLocation).filter_by(id=location_id, removed_at=None).first()
|
||||
if (not loc or loc.location_type != "sound"
|
||||
or loc.project_id not in _client_project_ids(client, db)):
|
||||
raise HTTPException(status_code=404, detail="Location not found")
|
||||
return loc
|
||||
|
||||
|
||||
def active_unit_for_location(location_id: str, db: Session):
|
||||
"""The SLM unit currently assigned to this location, or None."""
|
||||
now = datetime.utcnow()
|
||||
asg = (db.query(UnitAssignment)
|
||||
.filter(UnitAssignment.location_id == location_id,
|
||||
UnitAssignment.status == "active",
|
||||
UnitAssignment.device_type == "slm",
|
||||
or_(UnitAssignment.assigned_until.is_(None),
|
||||
UnitAssignment.assigned_until > now))
|
||||
.order_by(UnitAssignment.assigned_at.desc()).first())
|
||||
return asg.unit_id if asg else None
|
||||
|
||||
|
||||
def _client_locations(client: Client, db: Session) -> list:
|
||||
"""The client's active sound locations (for the overview tiles + map)."""
|
||||
pids = _client_project_ids(client, db)
|
||||
if not pids:
|
||||
return []
|
||||
projs = {p.id: p.name for p in
|
||||
db.query(Project.id, Project.name).filter(Project.id.in_(pids)).all()}
|
||||
locs = (db.query(MonitoringLocation)
|
||||
.filter(MonitoringLocation.project_id.in_(pids),
|
||||
MonitoringLocation.location_type == "sound",
|
||||
MonitoringLocation.removed_at.is_(None))
|
||||
.order_by(MonitoringLocation.sort_order, MonitoringLocation.name).all())
|
||||
return [{
|
||||
"id": loc.id, "name": loc.name,
|
||||
"address": loc.address, "coordinates": loc.coordinates,
|
||||
"project_name": projs.get(loc.project_id),
|
||||
"has_device": active_unit_for_location(loc.id, db) is not None,
|
||||
} for loc in locs]
|
||||
|
||||
|
||||
|
||||
@router.get("/logout")
|
||||
def portal_logout():
|
||||
resp = RedirectResponse(url="/portal/access", status_code=303)
|
||||
resp.delete_cookie(COOKIE_NAME)
|
||||
return resp
|
||||
|
||||
|
||||
@router.get("/access")
|
||||
def portal_access(request: Request):
|
||||
"""Landing for an unauthenticated visitor (no valid link)."""
|
||||
return templates.TemplateResponse(
|
||||
"portal/access_required.html", {"request": request, "reason": "required"}
|
||||
)
|
||||
|
||||
|
||||
@router.get("/p/{link_token}")
|
||||
def portal_password_prompt(link_token: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Secure per-project link: resolve the project from the token, prompt for the
|
||||
shared password. Generic page if the token is unknown/disabled (no leak)."""
|
||||
project = resolve_project_by_link_token(link_token, db)
|
||||
if not project or not project.portal_password_hash:
|
||||
# unknown token, disabled portal, or enabled-but-no-password-set — all look
|
||||
# identical to a client (no existence/config leak, no self-lockout on a
|
||||
# passwordless project).
|
||||
return templates.TemplateResponse(
|
||||
"portal/access_required.html", {"request": request, "reason": "invalid"},
|
||||
status_code=404)
|
||||
return templates.TemplateResponse("portal/password.html", {
|
||||
"request": request, "link_token": link_token,
|
||||
"project_name": project.name, "error": None})
|
||||
|
||||
|
||||
@router.post("/p/{link_token}")
|
||||
def portal_password_submit(link_token: str, request: Request,
|
||||
password: str = Form(...), db: Session = Depends(get_db)):
|
||||
"""Verify the shared password; on success mint a project-scoped session cookie."""
|
||||
project = resolve_project_by_link_token(link_token, db)
|
||||
if not project or not project.portal_password_hash:
|
||||
# unknown token, disabled portal, or enabled-but-no-password-set — all look
|
||||
# identical to a client (no existence/config leak, no self-lockout on a
|
||||
# passwordless project).
|
||||
return templates.TemplateResponse(
|
||||
"portal/access_required.html", {"request": request, "reason": "invalid"},
|
||||
status_code=404)
|
||||
|
||||
# Shared per-project password → lock per token. (Keying on IP too only enabled a
|
||||
# bypass via source-IP rotation, and behind the reverse proxy every client shares
|
||||
# one IP anyway.)
|
||||
lock_key = link_token
|
||||
if is_locked(lock_key):
|
||||
return templates.TemplateResponse("portal/password.html", {
|
||||
"request": request, "link_token": link_token, "project_name": project.name,
|
||||
"error": "Too many attempts. Try again in 15 minutes."}, status_code=200)
|
||||
|
||||
if not verify_password(password, project.portal_password_hash):
|
||||
register_failure(lock_key)
|
||||
return templates.TemplateResponse("portal/password.html", {
|
||||
"request": request, "link_token": link_token, "project_name": project.name,
|
||||
"error": "Incorrect password."}, status_code=200)
|
||||
|
||||
clear_failures(lock_key)
|
||||
token_id = mint_portal_session(project, db)
|
||||
resp = RedirectResponse(url="/portal", status_code=303)
|
||||
resp.set_cookie(COOKIE_NAME, make_session_cookie(token_id),
|
||||
max_age=COOKIE_MAX_AGE, httponly=True, samesite="lax", secure=COOKIE_SECURE)
|
||||
logger.info(f"[PORTAL] password ok for project {project.id[:8]} → session opened")
|
||||
return resp
|
||||
|
||||
|
||||
@router.get("")
|
||||
def portal_home(request: Request, client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""Client overview — their active sound locations with live tiles + a map."""
|
||||
return templates.TemplateResponse(
|
||||
"portal/overview.html",
|
||||
{"request": request, "client": client,
|
||||
"locations": _client_locations(client, db)},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/location/{location_id}")
|
||||
def portal_location(location_id: str, request: Request,
|
||||
client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""Read-only live view for one of the client's locations (404 if not owned)."""
|
||||
loc = resolve_client_location(client, location_id, db)
|
||||
return templates.TemplateResponse("portal/location.html", {
|
||||
"request": request, "client": client, "location": loc,
|
||||
"has_device": active_unit_for_location(location_id, db) is not None,
|
||||
})
|
||||
|
||||
|
||||
# -- scoped data (cache reads only — never hits the device) ------------------
|
||||
|
||||
@router.get("/api/location/{location_id}/live")
|
||||
async def portal_location_live(location_id: str,
|
||||
client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""Scrubbed cached live reading for a location the client owns."""
|
||||
resolve_client_location(client, location_id, db)
|
||||
unit_id = active_unit_for_location(location_id, db)
|
||||
if not unit_id:
|
||||
return {"status": "ok", "data": None, "reason": "no_device"}
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as hc:
|
||||
r = await hc.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/status")
|
||||
except Exception:
|
||||
return {"status": "ok", "data": None, "reason": "unreachable"}
|
||||
if r.status_code != 200:
|
||||
return {"status": "ok", "data": None, "reason": "no_data"}
|
||||
full = (r.json() or {}).get("data", {}) or {}
|
||||
return {"status": "ok", "data": {k: full.get(k) for k in _PORTAL_LIVE_FIELDS}}
|
||||
|
||||
|
||||
@router.get("/api/location/{location_id}/history")
|
||||
async def portal_location_history(location_id: str, hours: float = 2.0,
|
||||
client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""Cached chart trail for a location the client owns. (Trail rows are already
|
||||
just timestamp + lp/leq/lmax/ln1/ln2 — safe to pass through.)"""
|
||||
resolve_client_location(client, location_id, db)
|
||||
unit_id = active_unit_for_location(location_id, db)
|
||||
if not unit_id:
|
||||
return {"status": "ok", "readings": []}
|
||||
hours = max(0.1, min(hours, 48.0))
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as hc:
|
||||
r = await hc.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/history",
|
||||
params={"hours": hours})
|
||||
except Exception:
|
||||
return {"status": "ok", "readings": []}
|
||||
if r.status_code != 200:
|
||||
return {"status": "ok", "readings": []}
|
||||
raw = (r.json() or {}).get("readings", [])
|
||||
fields = ("timestamp", "lp", "leq", "lmax", "ln1", "ln2") # whitelist, like the other endpoints
|
||||
return {"status": "ok", "readings": [{k: x.get(k) for k in fields} for x in raw]}
|
||||
|
||||
|
||||
# Whitelist of alert-event fields exposed to a client (no internal ids/ack-by).
|
||||
_PORTAL_EVENT_FIELDS = ("rule_name", "metric", "threshold_db", "onset_at",
|
||||
"onset_value", "peak_value", "clear_at", "status")
|
||||
|
||||
|
||||
@router.get("/api/location/{location_id}/events")
|
||||
async def portal_location_events(location_id: str, limit: int = 20,
|
||||
client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""Scrubbed breach history for a location the client owns (read-only)."""
|
||||
resolve_client_location(client, location_id, db)
|
||||
unit_id = active_unit_for_location(location_id, db)
|
||||
if not unit_id:
|
||||
return {"status": "ok", "events": []}
|
||||
limit = max(1, min(limit, 100))
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as hc:
|
||||
r = await hc.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/alerts/events",
|
||||
params={"limit": limit})
|
||||
except Exception:
|
||||
return {"status": "ok", "events": []}
|
||||
if r.status_code != 200:
|
||||
return {"status": "ok", "events": []}
|
||||
raw = (r.json() or {}).get("events", [])
|
||||
events = [{k: e.get(k) for k in _PORTAL_EVENT_FIELDS} for e in raw]
|
||||
return {"status": "ok", "events": events, "active": sum(1 for e in events if e.get("status") == "active")}
|
||||
|
||||
|
||||
# Whitelist of alert-rule fields shown to a client (the active limits, no cooldown/
|
||||
# hysteresis internals).
|
||||
_PORTAL_RULE_FIELDS = ("name", "metric", "comparison", "threshold_db", "duration_s",
|
||||
"schedule_start", "schedule_end", "schedule_days")
|
||||
|
||||
|
||||
@router.get("/api/location/{location_id}/thresholds")
|
||||
async def portal_location_thresholds(location_id: str,
|
||||
client: Client = Depends(get_current_client),
|
||||
db: Session = Depends(get_db)):
|
||||
"""The active alert limits for a location the client owns (enabled rules only),
|
||||
so the client can see what they're being alerted on. Read-only, scrubbed."""
|
||||
resolve_client_location(client, location_id, db)
|
||||
unit_id = active_unit_for_location(location_id, db)
|
||||
if not unit_id:
|
||||
return {"status": "ok", "rules": []}
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as hc:
|
||||
r = await hc.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/alerts/rules")
|
||||
except Exception:
|
||||
return {"status": "ok", "rules": []}
|
||||
if r.status_code != 200:
|
||||
return {"status": "ok", "rules": []}
|
||||
raw = (r.json() or {}).get("rules", [])
|
||||
rules = [{k: x.get(k) for k in _PORTAL_RULE_FIELDS} for x in raw if x.get("enabled")]
|
||||
return {"status": "ok", "rules": rules}
|
||||
|
||||
|
||||
# -- live stream (fan-out feed, scoped + scrubbed) ---------------------------
|
||||
|
||||
def _scrub_frame(raw: str):
|
||||
"""Project a monitor frame down to the portal whitelist. Drops internal fields
|
||||
(unit_id, raw_payload, lmin) before it reaches a client; passes control fields
|
||||
(feed_status, heartbeat) + timestamp through. Returns None for a non-JSON frame
|
||||
so the caller drops it rather than forwarding anything unscrubbed."""
|
||||
try:
|
||||
d = json.loads(raw)
|
||||
except Exception:
|
||||
return None
|
||||
out = {k: d.get(k) for k in _PORTAL_LIVE_FIELDS if k in d}
|
||||
if "timestamp" in d:
|
||||
out["timestamp"] = d["timestamp"]
|
||||
for ctrl in ("feed_status", "heartbeat"):
|
||||
if ctrl in d:
|
||||
out[ctrl] = d[ctrl]
|
||||
return json.dumps(out)
|
||||
|
||||
|
||||
@router.websocket("/api/location/{location_id}/stream")
|
||||
async def portal_location_stream(websocket: WebSocket, location_id: str):
|
||||
"""Live ~1Hz feed for a location the client owns. Auths via the session cookie,
|
||||
enforces ownership, then bridges the unit's shared SLMM /monitor fan-out feed
|
||||
to the browser (scrubbed). A viewer is just one more subscriber to the one
|
||||
device feed — no extra device connection."""
|
||||
await websocket.accept()
|
||||
|
||||
# Auth + ownership on a short-lived session, then release it for the long bridge.
|
||||
db = SessionLocal()
|
||||
try:
|
||||
client = client_from_cookie(websocket.cookies.get(COOKIE_NAME), db)
|
||||
if client is None:
|
||||
await websocket.close(code=1008) # policy violation (not authenticated)
|
||||
return
|
||||
try:
|
||||
resolve_client_location(client, location_id, db)
|
||||
except HTTPException:
|
||||
await websocket.close(code=1008)
|
||||
return
|
||||
unit_id = active_unit_for_location(location_id, db)
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
if not unit_id:
|
||||
try:
|
||||
await websocket.send_json({"feed_status": "no_device"})
|
||||
finally:
|
||||
await websocket.close(code=1000)
|
||||
return
|
||||
|
||||
target = f"{SLMM_WS_BASE_URL}/api/nl43/{unit_id}/monitor"
|
||||
backend_ws = None
|
||||
try:
|
||||
backend_ws = await websockets.connect(target)
|
||||
|
||||
async def forward_to_client():
|
||||
async for message in backend_ws:
|
||||
frame = _scrub_frame(message)
|
||||
if frame is not None:
|
||||
await websocket.send_text(frame)
|
||||
|
||||
async def watch_client():
|
||||
while True:
|
||||
await websocket.receive_text()
|
||||
|
||||
tasks = [asyncio.ensure_future(forward_to_client()),
|
||||
asyncio.ensure_future(watch_client())]
|
||||
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
|
||||
for t in pending:
|
||||
t.cancel()
|
||||
for t in tasks:
|
||||
try:
|
||||
await t
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.warning(f"[PORTAL] stream {location_id}: {e}")
|
||||
finally:
|
||||
if backend_ws:
|
||||
try:
|
||||
await backend_ws.close()
|
||||
except Exception:
|
||||
pass
|
||||
@@ -9,7 +9,7 @@ from fastapi import APIRouter, Request, Depends, HTTPException, Query
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import and_, or_
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
from typing import Optional
|
||||
import uuid
|
||||
@@ -1712,19 +1712,6 @@ def _parse_rnh(content: bytes) -> dict:
|
||||
result["stop_time_str"] = value
|
||||
elif key == "Total Measurement Time":
|
||||
result["total_time_str"] = value
|
||||
elif key == "Frequency Weighting (Main)":
|
||||
result["frequency_weighting"] = value
|
||||
elif key == "Time Weighting (Main)":
|
||||
result["time_weighting"] = value
|
||||
elif key == "Leq Calculation Interval":
|
||||
result["leq_interval"] = value
|
||||
elif key.startswith("Percentile "):
|
||||
# e.g. "Percentile 4,90.0" → percentiles["4"] = "90.0".
|
||||
# Lets the report label the LN slots (here LN4 = L90) from the
|
||||
# device's own config instead of hardcoding which slot is which —
|
||||
# the percentile assignment is reconfigurable per job.
|
||||
slot = key[len("Percentile "):].strip()
|
||||
result.setdefault("percentiles", {})[slot] = value
|
||||
except Exception:
|
||||
pass
|
||||
return result
|
||||
@@ -1753,347 +1740,6 @@ def _classify_file(filename: str) -> str:
|
||||
return "data"
|
||||
|
||||
|
||||
def _rnd_interval_seconds(s: Optional[str]) -> Optional[int]:
|
||||
"""Parse an NL-43 interval string ('15m' / '1s' / '1h') into seconds."""
|
||||
import re
|
||||
m = re.match(r"\s*(\d+)\s*([smh])", (s or "").strip().lower())
|
||||
if not m:
|
||||
return None
|
||||
return int(m.group(1)) * {"s": 1, "m": 60, "h": 3600}[m.group(2)]
|
||||
|
||||
|
||||
def _leq_window_local(leq_bytes: bytes):
|
||||
"""Recording window from a Leq .rnd's 'Start Time' column (meter-local time).
|
||||
|
||||
Returns (first_start, last_start, row_count, inferred_interval_seconds).
|
||||
This is the source of truth for the recording window on NL-43 units, whose
|
||||
.rnh carries no measurement timestamps. Reuses the report's AU2 normaliser
|
||||
so NL-43 and AU2 files parse identically.
|
||||
"""
|
||||
import csv as _csv
|
||||
from backend.routers.projects import _normalize_rnd_rows # lazy: avoid import cycle
|
||||
try:
|
||||
text = leq_bytes.decode("utf-8", errors="replace")
|
||||
rows = list(_csv.DictReader(io.StringIO(text)))
|
||||
except Exception:
|
||||
return None, None, 0, None
|
||||
try:
|
||||
rows, _ = _normalize_rnd_rows(rows)
|
||||
except Exception:
|
||||
pass
|
||||
times = []
|
||||
for r in rows:
|
||||
v = (r.get("Start Time") or "").strip()
|
||||
try:
|
||||
times.append(datetime.strptime(v, "%Y/%m/%d %H:%M:%S"))
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
if not times:
|
||||
return None, None, 0, None
|
||||
times.sort()
|
||||
inferred = int((times[1] - times[0]).total_seconds()) if len(times) >= 2 else None
|
||||
return times[0], times[-1], len(times), inferred
|
||||
|
||||
|
||||
def _is_wanted_nrl_file(fname: str) -> bool:
|
||||
"""Keep only the files an NRL ingest cares about: .rnh metadata + the
|
||||
averaged Leq .rnd. Drops the 1-second _Lp_ files and everything else.
|
||||
|
||||
- NL-43 writes two .rnd types: _Leq_ (15-min averages, wanted) and
|
||||
_Lp_ (1-second granular, skipped).
|
||||
- AU2 (NL-23/older Rion) writes a single Au2_####.rnd — always keep.
|
||||
|
||||
Note this is purely about which *files* to store, not which *metrics* to
|
||||
report: the kept Leq file carries every column (Leq, Lmax, L1/L10/L50/
|
||||
L90/L95, Lpeak, …), so the report layer can select any metric later.
|
||||
"""
|
||||
n = fname.lower()
|
||||
if n.endswith(".rnh"):
|
||||
return True
|
||||
if n.endswith(".rnd"):
|
||||
if "_leq_" in n: # NL-43 Leq file
|
||||
return True
|
||||
if n.startswith("au2_"): # AU2 format (NL-23) — Leq equivalent
|
||||
return True
|
||||
if "_lp" not in n and "_leq_" not in n:
|
||||
# Unknown .rnd format — include it so we don't silently drop data
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
class IngestError(Exception):
|
||||
"""Raised when an NRL upload/ZIP has no usable data or an invalid target.
|
||||
|
||||
Kept HTTP-agnostic so the ingest core can be driven programmatically (the
|
||||
scheduled FTP pull) as well as from the HTTP upload endpoint. Callers
|
||||
translate it: the endpoint → HTTP 400, the scheduler → logged failure.
|
||||
"""
|
||||
pass
|
||||
|
||||
|
||||
def _find_existing_session(
|
||||
db: Session,
|
||||
location_id: str,
|
||||
store_name: str,
|
||||
started_at,
|
||||
start_time_str: str,
|
||||
):
|
||||
"""Return an already-ingested session for this location that represents the
|
||||
same measurement, or None.
|
||||
|
||||
Used to make FTP re-pulls idempotent: a daily cycle closes one Auto_####
|
||||
folder per day, so a session is uniquely identified within a location by
|
||||
(store_name + measurement start time). Store names recycle across jobs, so
|
||||
we always match on start time too.
|
||||
"""
|
||||
if not store_name and not started_at:
|
||||
return None
|
||||
candidates = db.query(MonitoringSession).filter(
|
||||
MonitoringSession.location_id == location_id,
|
||||
MonitoringSession.session_type == "sound",
|
||||
).all()
|
||||
for s in candidates:
|
||||
try:
|
||||
meta = json.loads(s.session_metadata or "{}")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
meta = {}
|
||||
if store_name and meta.get("store_name") != store_name:
|
||||
continue
|
||||
# Same store_name — confirm it's the same measurement by start time.
|
||||
if start_time_str and meta.get("start_time_str") == start_time_str:
|
||||
return s
|
||||
if not meta.get("start_time_str") and started_at and s.started_at == started_at:
|
||||
return s
|
||||
return None
|
||||
|
||||
|
||||
def _ingest_file_entries(
|
||||
location: MonitoringLocation,
|
||||
file_entries: list[tuple[str, bytes]],
|
||||
db: Session,
|
||||
*,
|
||||
source: str = "manual_upload",
|
||||
dedupe: bool = False,
|
||||
unit_id: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Core NRL ingest, shared by the HTTP upload and the programmatic FTP pull.
|
||||
|
||||
Takes already-normalized (filename, bytes) entries, keeps the wanted files,
|
||||
parses the .rnh, and creates a MonitoringSession + DataFile rows under the
|
||||
location's project. Metric-agnostic: the full Leq file is written to disk
|
||||
and every column preserved; metric selection happens in the report layer.
|
||||
|
||||
`unit_id` attributes the session to the recording unit when the caller knows
|
||||
it (manual FTP download / SD upload from a known unit). Left None for paths
|
||||
that link the unit afterwards (the scheduler's `_ingest_and_link`).
|
||||
|
||||
Raises IngestError if no usable files are present.
|
||||
"""
|
||||
# --- Filter to the files we keep (.rnh + Leq .rnd) ---
|
||||
file_entries = [(f, b) for f, b in file_entries if _is_wanted_nrl_file(f)]
|
||||
if not file_entries:
|
||||
raise IngestError(
|
||||
"No usable .rnd or .rnh files found. Expected NL-43 _Leq_ files or AU2 format .rnd files."
|
||||
)
|
||||
|
||||
# --- Parse .rnh metadata (first one wins) ---
|
||||
rnh_meta = {}
|
||||
for fname, fbytes in file_entries:
|
||||
if fname.lower().endswith(".rnh"):
|
||||
rnh_meta = _parse_rnh(fbytes)
|
||||
break
|
||||
|
||||
# RNH stores local time (no UTC offset). Use local for period/label, then
|
||||
# convert to UTC for storage so the local_datetime filter displays correctly.
|
||||
started_at_local = _parse_rnh_datetime(rnh_meta.get("start_time_str")) or datetime.utcnow()
|
||||
stopped_at_local = _parse_rnh_datetime(rnh_meta.get("stop_time_str"))
|
||||
started_at = local_to_utc(started_at_local)
|
||||
stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
|
||||
duration_seconds = (
|
||||
int((stopped_at - started_at).total_seconds())
|
||||
if (started_at and stopped_at) else None
|
||||
)
|
||||
|
||||
store_name = rnh_meta.get("store_name", "")
|
||||
serial_number = rnh_meta.get("serial_number", "")
|
||||
index_number = rnh_meta.get("index_number", "")
|
||||
start_time_str = rnh_meta.get("start_time_str", "")
|
||||
|
||||
# The NL-43 .rnh has NO measurement timestamps — the real recording window
|
||||
# lives in the Leq .rnd's "Start Time" column. Whenever the header didn't
|
||||
# give us a start (and/or stop), derive it from the Leq rows so the session
|
||||
# gets the true window + duration (and a stable start_time_str for dedupe).
|
||||
if not start_time_str or stopped_at_local is None:
|
||||
leq_entry = next(
|
||||
((f, b) for f, b in file_entries
|
||||
if f.lower().endswith(".rnd") and ("_leq_" in f.lower() or f.lower().startswith("au2_"))),
|
||||
None,
|
||||
)
|
||||
if leq_entry is not None:
|
||||
first_dt, last_dt, _n, inferred = _leq_window_local(leq_entry[1])
|
||||
interval_s = _rnd_interval_seconds(rnh_meta.get("leq_interval")) or inferred or 0
|
||||
if first_dt and not start_time_str:
|
||||
started_at_local = first_dt
|
||||
start_time_str = first_dt.strftime("%Y/%m/%d %H:%M:%S")
|
||||
if last_dt and stopped_at_local is None:
|
||||
stopped_at_local = last_dt + timedelta(seconds=interval_s)
|
||||
# Recompute UTC + duration from the resolved window.
|
||||
started_at = local_to_utc(started_at_local)
|
||||
stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
|
||||
duration_seconds = (
|
||||
int((stopped_at - started_at).total_seconds())
|
||||
if (started_at and stopped_at) else duration_seconds
|
||||
)
|
||||
|
||||
# --- Dedupe: skip if this exact measurement is already ingested ---
|
||||
if dedupe:
|
||||
existing = _find_existing_session(db, location.id, store_name, started_at, start_time_str)
|
||||
if existing:
|
||||
return {
|
||||
"success": True,
|
||||
"deduped": True,
|
||||
"session_id": existing.id,
|
||||
"files_imported": 0,
|
||||
"leq_files": 0,
|
||||
"lp_files": 0,
|
||||
"metadata_files": 0,
|
||||
"store_name": store_name,
|
||||
"started_at": started_at.isoformat() if started_at else None,
|
||||
"stopped_at": stopped_at.isoformat() if stopped_at else None,
|
||||
"duration_seconds": duration_seconds,
|
||||
}
|
||||
|
||||
# --- Create MonitoringSession (local times drive period/label) ---
|
||||
period_type = _derive_period_type(started_at_local) if started_at_local else None
|
||||
session_label = (
|
||||
_build_session_label(started_at_local, location.name, period_type)
|
||||
if started_at_local else None
|
||||
)
|
||||
|
||||
session_id = str(uuid.uuid4())
|
||||
monitoring_session = MonitoringSession(
|
||||
id=session_id,
|
||||
project_id=location.project_id,
|
||||
location_id=location.id,
|
||||
unit_id=unit_id,
|
||||
session_type="sound",
|
||||
started_at=started_at,
|
||||
stopped_at=stopped_at,
|
||||
duration_seconds=duration_seconds,
|
||||
status="completed",
|
||||
session_label=session_label,
|
||||
period_type=period_type,
|
||||
session_metadata=json.dumps({
|
||||
"source": source,
|
||||
"store_name": store_name,
|
||||
"serial_number": serial_number,
|
||||
"index_number": index_number,
|
||||
"start_time_str": start_time_str,
|
||||
# Captured from the .rnh so the report can label metrics from the
|
||||
# device's own config (which LN slot is L90, the weightings, etc.).
|
||||
"percentiles": rnh_meta.get("percentiles", {}),
|
||||
"frequency_weighting": rnh_meta.get("frequency_weighting", ""),
|
||||
"time_weighting": rnh_meta.get("time_weighting", ""),
|
||||
"leq_interval": rnh_meta.get("leq_interval", ""),
|
||||
}),
|
||||
)
|
||||
db.add(monitoring_session)
|
||||
db.commit()
|
||||
db.refresh(monitoring_session)
|
||||
|
||||
# --- Write files to disk + create DataFile records ---
|
||||
output_dir = Path("data/Projects") / location.project_id / session_id
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
leq_count = lp_count = metadata_count = files_imported = 0
|
||||
for fname, fbytes in file_entries:
|
||||
fname_lower = fname.lower()
|
||||
if fname_lower.endswith(".rnd"):
|
||||
if "_leq_" in fname_lower:
|
||||
leq_count += 1
|
||||
elif "_lp" in fname_lower:
|
||||
lp_count += 1
|
||||
elif fname_lower.endswith(".rnh"):
|
||||
metadata_count += 1
|
||||
|
||||
dest = output_dir / fname
|
||||
dest.write_bytes(fbytes)
|
||||
checksum = hashlib.sha256(fbytes).hexdigest()
|
||||
rel_path = str(dest.relative_to("data"))
|
||||
|
||||
db.add(DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=session_id,
|
||||
file_path=rel_path,
|
||||
file_type=_classify_file(fname),
|
||||
file_size_bytes=len(fbytes),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": source,
|
||||
"original_filename": fname,
|
||||
"store_name": store_name,
|
||||
}),
|
||||
))
|
||||
files_imported += 1
|
||||
|
||||
db.commit()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"deduped": False,
|
||||
"session_id": session_id,
|
||||
"files_imported": files_imported,
|
||||
"leq_files": leq_count,
|
||||
"lp_files": lp_count,
|
||||
"metadata_files": metadata_count,
|
||||
"store_name": store_name,
|
||||
"started_at": started_at.isoformat() if started_at else None,
|
||||
"stopped_at": stopped_at.isoformat() if stopped_at else None,
|
||||
"duration_seconds": duration_seconds,
|
||||
}
|
||||
|
||||
|
||||
def ingest_nrl_zip(
|
||||
location_id: str,
|
||||
zip_bytes: bytes,
|
||||
db: Session,
|
||||
*,
|
||||
source: str = "ftp_pull",
|
||||
dedupe: bool = True,
|
||||
unit_id: Optional[str] = None,
|
||||
) -> dict:
|
||||
"""Programmatically ingest an Auto_#### ZIP (e.g. a scheduled FTP pull).
|
||||
|
||||
Extracts the ZIP (flattening any nested Auto_Leq/Auto_Lp_ folders), keeps
|
||||
the .rnh + Leq .rnd, parses the header, and creates a MonitoringSession +
|
||||
DataFile rows for `location_id`. Defaults to dedupe=True so repeated daily
|
||||
pulls of the same closed folder don't create duplicate sessions. Pass
|
||||
`unit_id` to attribute the session to the recording unit at creation.
|
||||
|
||||
Returns the same dict shape as the HTTP upload, plus a `deduped` flag.
|
||||
Raises IngestError on a bad ZIP, no usable files, or unknown location.
|
||||
"""
|
||||
location = db.query(MonitoringLocation).filter_by(id=location_id).first()
|
||||
if not location:
|
||||
raise IngestError(f"Location {location_id} not found")
|
||||
|
||||
try:
|
||||
with zipfile.ZipFile(io.BytesIO(zip_bytes)) as zf:
|
||||
file_entries: list[tuple[str, bytes]] = []
|
||||
for info in zf.infolist():
|
||||
if info.is_dir():
|
||||
continue
|
||||
name = Path(info.filename).name # strip nested folder paths
|
||||
if not name:
|
||||
continue
|
||||
file_entries.append((name, zf.read(info)))
|
||||
except zipfile.BadZipFile:
|
||||
raise IngestError("Downloaded data is not a valid ZIP archive.")
|
||||
|
||||
return _ingest_file_entries(location, file_entries, db, source=source, dedupe=dedupe, unit_id=unit_id)
|
||||
|
||||
|
||||
@router.post("/nrl/{location_id}/upload-data")
|
||||
async def upload_nrl_data(
|
||||
project_id: str,
|
||||
@@ -2108,13 +1754,11 @@ async def upload_nrl_data(
|
||||
- A single .zip file (the Auto_#### folder zipped) — auto-extracted
|
||||
- Multiple .rnd / .rnh files selected directly from the SD card folder
|
||||
|
||||
Normalizes the upload to (filename, bytes) entries, then hands off to the
|
||||
shared ingest core (`_ingest_file_entries`) — the same path the scheduled
|
||||
FTP pull uses via `ingest_nrl_zip`. Creates a MonitoringSession from the
|
||||
.rnh metadata and DataFile records for each measurement file. No unit
|
||||
assignment required. dedupe=False here preserves the prior manual-upload
|
||||
behaviour (re-uploading creates a fresh session).
|
||||
Creates a MonitoringSession from .rnh metadata and DataFile records
|
||||
for each measurement file. No unit assignment required.
|
||||
"""
|
||||
from datetime import datetime
|
||||
|
||||
# Verify project and location exist
|
||||
project = db.query(Project).filter_by(id=project_id).first()
|
||||
_require_module(project, "sound_monitoring", db)
|
||||
@@ -2125,7 +1769,7 @@ async def upload_nrl_data(
|
||||
if not location:
|
||||
raise HTTPException(status_code=404, detail="Location not found")
|
||||
|
||||
# --- Normalize upload to (filename, bytes) entries ---
|
||||
# --- Step 1: Normalize to (filename, bytes) list ---
|
||||
file_entries: list[tuple[str, bytes]] = []
|
||||
|
||||
if len(files) == 1 and files[0].filename.lower().endswith(".zip"):
|
||||
@@ -2149,11 +1793,145 @@ async def upload_nrl_data(
|
||||
if not file_entries:
|
||||
raise HTTPException(status_code=400, detail="No usable files found in upload.")
|
||||
|
||||
# --- Hand off to the shared ingest core ---
|
||||
try:
|
||||
return _ingest_file_entries(location, file_entries, db, source="manual_upload", dedupe=False)
|
||||
except IngestError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
# --- Step 1b: Filter to only relevant files ---
|
||||
# Keep: .rnh (metadata) and measurement .rnd files
|
||||
# NL-43 generates two .rnd types: _Leq_ (15-min averages, wanted) and _Lp_ (1-sec granular, skip)
|
||||
# AU2 (NL-23/older Rion) generates a single Au2_####.rnd per session — always keep those
|
||||
# Drop: _Lp_ .rnd, .xlsx, .mp3, and anything else
|
||||
def _is_wanted(fname: str) -> bool:
|
||||
n = fname.lower()
|
||||
if n.endswith(".rnh"):
|
||||
return True
|
||||
if n.endswith(".rnd"):
|
||||
if "_leq_" in n: # NL-43 Leq file
|
||||
return True
|
||||
if n.startswith("au2_"): # AU2 format (NL-23) — always Leq equivalent
|
||||
return True
|
||||
if "_lp" not in n and "_leq_" not in n:
|
||||
# Unknown .rnd format — include it so we don't silently drop data
|
||||
return True
|
||||
return False
|
||||
|
||||
file_entries = [(fname, fbytes) for fname, fbytes in file_entries if _is_wanted(fname)]
|
||||
|
||||
if not file_entries:
|
||||
raise HTTPException(status_code=400, detail="No usable .rnd or .rnh files found. Expected NL-43 _Leq_ files or AU2 format .rnd files.")
|
||||
|
||||
# --- Step 2: Find and parse .rnh metadata ---
|
||||
rnh_meta = {}
|
||||
for fname, fbytes in file_entries:
|
||||
if fname.lower().endswith(".rnh"):
|
||||
rnh_meta = _parse_rnh(fbytes)
|
||||
break
|
||||
|
||||
# RNH files store local time (no UTC offset). Use local values for period
|
||||
# classification / label generation, then convert to UTC for DB storage so
|
||||
# the local_datetime Jinja filter displays the correct time.
|
||||
started_at_local = _parse_rnh_datetime(rnh_meta.get("start_time_str")) or datetime.utcnow()
|
||||
stopped_at_local = _parse_rnh_datetime(rnh_meta.get("stop_time_str"))
|
||||
|
||||
started_at = local_to_utc(started_at_local)
|
||||
stopped_at = local_to_utc(stopped_at_local) if stopped_at_local else None
|
||||
|
||||
duration_seconds = None
|
||||
if started_at and stopped_at:
|
||||
duration_seconds = int((stopped_at - started_at).total_seconds())
|
||||
|
||||
store_name = rnh_meta.get("store_name", "")
|
||||
serial_number = rnh_meta.get("serial_number", "")
|
||||
index_number = rnh_meta.get("index_number", "")
|
||||
|
||||
# --- Step 3: Create MonitoringSession ---
|
||||
# Use local times for period/label so classification reflects the clock at the site.
|
||||
period_type = _derive_period_type(started_at_local) if started_at_local else None
|
||||
session_label = _build_session_label(started_at_local, location.name, period_type) if started_at_local else None
|
||||
|
||||
session_id = str(uuid.uuid4())
|
||||
monitoring_session = MonitoringSession(
|
||||
id=session_id,
|
||||
project_id=project_id,
|
||||
location_id=location_id,
|
||||
unit_id=None,
|
||||
session_type="sound",
|
||||
started_at=started_at,
|
||||
stopped_at=stopped_at,
|
||||
duration_seconds=duration_seconds,
|
||||
status="completed",
|
||||
session_label=session_label,
|
||||
period_type=period_type,
|
||||
session_metadata=json.dumps({
|
||||
"source": "manual_upload",
|
||||
"store_name": store_name,
|
||||
"serial_number": serial_number,
|
||||
"index_number": index_number,
|
||||
}),
|
||||
)
|
||||
db.add(monitoring_session)
|
||||
db.commit()
|
||||
db.refresh(monitoring_session)
|
||||
|
||||
# --- Step 4: Write files to disk and create DataFile records ---
|
||||
output_dir = Path("data/Projects") / project_id / session_id
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
leq_count = 0
|
||||
lp_count = 0
|
||||
metadata_count = 0
|
||||
files_imported = 0
|
||||
|
||||
for fname, fbytes in file_entries:
|
||||
file_type = _classify_file(fname)
|
||||
fname_lower = fname.lower()
|
||||
|
||||
# Track counts for summary
|
||||
if fname_lower.endswith(".rnd"):
|
||||
if "_leq_" in fname_lower:
|
||||
leq_count += 1
|
||||
elif "_lp" in fname_lower:
|
||||
lp_count += 1
|
||||
elif fname_lower.endswith(".rnh"):
|
||||
metadata_count += 1
|
||||
|
||||
# Write to disk
|
||||
dest = output_dir / fname
|
||||
dest.write_bytes(fbytes)
|
||||
|
||||
# Compute checksum
|
||||
checksum = hashlib.sha256(fbytes).hexdigest()
|
||||
|
||||
# Store relative path from data/ dir
|
||||
rel_path = str(dest.relative_to("data"))
|
||||
|
||||
data_file = DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=session_id,
|
||||
file_path=rel_path,
|
||||
file_type=file_type,
|
||||
file_size_bytes=len(fbytes),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": "manual_upload",
|
||||
"original_filename": fname,
|
||||
"store_name": store_name,
|
||||
}),
|
||||
)
|
||||
db.add(data_file)
|
||||
files_imported += 1
|
||||
|
||||
db.commit()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"session_id": session_id,
|
||||
"files_imported": files_imported,
|
||||
"leq_files": leq_count,
|
||||
"lp_files": lp_count,
|
||||
"metadata_files": metadata_count,
|
||||
"store_name": store_name,
|
||||
"started_at": started_at.isoformat() if started_at else None,
|
||||
"stopped_at": stopped_at.isoformat() if stopped_at else None,
|
||||
}
|
||||
|
||||
|
||||
# ============================================================================
|
||||
|
||||
+266
-171
@@ -1591,32 +1591,24 @@ async def get_sessions_calendar(
|
||||
async def get_ftp_browser(
|
||||
project_id: str,
|
||||
request: Request,
|
||||
location_id: Optional[str] = None,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Get FTP browser interface for downloading files from assigned SLMs.
|
||||
Returns HTML partial with FTP browser. Sound Monitoring projects only.
|
||||
|
||||
When `location_id` is given, scope to just the unit(s) assigned to that NRL
|
||||
(used by the per-NRL Data Files tab, which mirrors the project-wide tab).
|
||||
"""
|
||||
from backend.models import DataFile
|
||||
|
||||
project = db.query(Project).filter_by(id=project_id).first()
|
||||
_require_module(project, "sound_monitoring", db)
|
||||
|
||||
# Active assignments for this project (active = assigned_until IS NULL),
|
||||
# optionally scoped to a single NRL/location.
|
||||
q = db.query(UnitAssignment).filter(
|
||||
# Get all assignments for this project (active = assigned_until IS NULL)
|
||||
assignments = db.query(UnitAssignment).filter(
|
||||
and_(
|
||||
UnitAssignment.project_id == project_id,
|
||||
UnitAssignment.assigned_until == None,
|
||||
)
|
||||
)
|
||||
if location_id:
|
||||
q = q.filter(UnitAssignment.location_id == location_id)
|
||||
assignments = q.all()
|
||||
).all()
|
||||
|
||||
# Enrich with unit and location details
|
||||
units_data = []
|
||||
@@ -1646,13 +1638,9 @@ async def ftp_download_to_server(
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Download a single file from an SLM to the server via FTP.
|
||||
|
||||
NRL measurement files (.rnh / _Leq_ .rnd) are routed through the shared NRL
|
||||
ingest so the session is parsed and attributed to the unit (a lone .rnh still
|
||||
yields the real recording window + duration). Any other file type — or a
|
||||
unit with no location — falls back to a generic stored DataFile, preserving
|
||||
the original behaviour. Sound Monitoring projects only.
|
||||
Download a file from an SLM to the server via FTP.
|
||||
Creates a DataFile record and stores the file in data/Projects/{project_id}/
|
||||
Sound Monitoring projects only.
|
||||
"""
|
||||
import httpx
|
||||
import os
|
||||
@@ -1670,55 +1658,7 @@ async def ftp_download_to_server(
|
||||
if not unit_id or not remote_path:
|
||||
raise HTTPException(status_code=400, detail="Missing unit_id or remote_path")
|
||||
|
||||
filename = os.path.basename(remote_path)
|
||||
|
||||
# Download the file from SLMM
|
||||
SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=300.0) as client:
|
||||
response = await client.post(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download",
|
||||
json={"remote_path": remote_path}
|
||||
)
|
||||
except httpx.TimeoutException:
|
||||
raise HTTPException(status_code=504, detail="Timeout downloading file from SLM")
|
||||
except Exception as e:
|
||||
logger.error(f"Error reaching SLMM for file download: {e}")
|
||||
raise HTTPException(status_code=502, detail=f"Failed to reach SLMM: {str(e)}")
|
||||
|
||||
if not response.is_success:
|
||||
raise HTTPException(
|
||||
status_code=response.status_code,
|
||||
detail=f"Failed to download from SLMM: {response.text}",
|
||||
)
|
||||
file_content = response.content
|
||||
|
||||
# NRL measurement file + known location → shared ingest (parsed + attributed).
|
||||
from backend.routers.project_locations import (
|
||||
_ingest_file_entries, IngestError, _is_wanted_nrl_file,
|
||||
)
|
||||
if location_id and _is_wanted_nrl_file(filename):
|
||||
location = db.query(MonitoringLocation).filter_by(id=location_id).first()
|
||||
if location:
|
||||
try:
|
||||
result = _ingest_file_entries(
|
||||
location, [(filename, file_content)], db,
|
||||
source="ftp_manual", dedupe=False, unit_id=unit_id,
|
||||
)
|
||||
except IngestError as e:
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Imported {filename} as NRL measurement data",
|
||||
"ingested": True,
|
||||
"session_id": result["session_id"],
|
||||
"file_size": len(file_content),
|
||||
"started_at": result["started_at"],
|
||||
"stopped_at": result["stopped_at"],
|
||||
"duration_seconds": result["duration_seconds"],
|
||||
}
|
||||
|
||||
# --- Generic path: any other file type (or no location) — store as-is ---
|
||||
# Get or create active session for this location/unit
|
||||
session = db.query(MonitoringSession).filter(
|
||||
and_(
|
||||
MonitoringSession.project_id == project_id,
|
||||
@@ -1728,6 +1668,7 @@ async def ftp_download_to_server(
|
||||
)
|
||||
).first()
|
||||
|
||||
# If no active session, create one
|
||||
if not session:
|
||||
_ftp_unit = db.query(RosterUnit).filter_by(id=unit_id).first()
|
||||
session = MonitoringSession(
|
||||
@@ -1746,50 +1687,115 @@ async def ftp_download_to_server(
|
||||
db.commit()
|
||||
db.refresh(session)
|
||||
|
||||
ext = os.path.splitext(filename)[1].lower()
|
||||
file_type_map = {
|
||||
'.wav': 'audio', '.mp3': 'audio', '.flac': 'audio', '.m4a': 'audio', '.aac': 'audio',
|
||||
'.rnd': 'measurement',
|
||||
'.csv': 'data', '.txt': 'data', '.json': 'data', '.xml': 'data', '.dat': 'data',
|
||||
'.log': 'log',
|
||||
'.zip': 'archive', '.tar': 'archive', '.gz': 'archive', '.7z': 'archive', '.rar': 'archive',
|
||||
'.jpg': 'image', '.jpeg': 'image', '.png': 'image', '.gif': 'image',
|
||||
'.pdf': 'document', '.doc': 'document', '.docx': 'document',
|
||||
}
|
||||
file_type = file_type_map.get(ext, 'data')
|
||||
# Download file from SLMM
|
||||
SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
|
||||
project_dir = Path(f"data/Projects/{project_id}/{session.id}")
|
||||
project_dir.mkdir(parents=True, exist_ok=True)
|
||||
file_path = project_dir / filename
|
||||
with open(file_path, 'wb') as f:
|
||||
f.write(file_content)
|
||||
checksum = hashlib.sha256(file_content).hexdigest()
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=300.0) as client:
|
||||
response = await client.post(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download",
|
||||
json={"remote_path": remote_path}
|
||||
)
|
||||
|
||||
data_file = DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=session.id,
|
||||
file_path=str(file_path.relative_to("data")), # Store relative to data/
|
||||
file_type=file_type,
|
||||
file_size_bytes=len(file_content),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": "ftp",
|
||||
"remote_path": remote_path,
|
||||
"unit_id": unit_id,
|
||||
"location_id": location_id,
|
||||
})
|
||||
)
|
||||
db.add(data_file)
|
||||
db.commit()
|
||||
if not response.is_success:
|
||||
raise HTTPException(
|
||||
status_code=response.status_code,
|
||||
detail=f"Failed to download from SLMM: {response.text}"
|
||||
)
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Downloaded {filename} to server",
|
||||
"file_id": data_file.id,
|
||||
"file_path": str(file_path),
|
||||
"file_size": len(file_content),
|
||||
}
|
||||
# Extract filename from remote_path
|
||||
filename = os.path.basename(remote_path)
|
||||
|
||||
# Determine file type from extension
|
||||
ext = os.path.splitext(filename)[1].lower()
|
||||
file_type_map = {
|
||||
# Audio files
|
||||
'.wav': 'audio',
|
||||
'.mp3': 'audio',
|
||||
'.flac': 'audio',
|
||||
'.m4a': 'audio',
|
||||
'.aac': 'audio',
|
||||
# Sound level meter measurement files
|
||||
'.rnd': 'measurement',
|
||||
# Data files
|
||||
'.csv': 'data',
|
||||
'.txt': 'data',
|
||||
'.json': 'data',
|
||||
'.xml': 'data',
|
||||
'.dat': 'data',
|
||||
# Log files
|
||||
'.log': 'log',
|
||||
# Archives
|
||||
'.zip': 'archive',
|
||||
'.tar': 'archive',
|
||||
'.gz': 'archive',
|
||||
'.7z': 'archive',
|
||||
'.rar': 'archive',
|
||||
# Images
|
||||
'.jpg': 'image',
|
||||
'.jpeg': 'image',
|
||||
'.png': 'image',
|
||||
'.gif': 'image',
|
||||
# Documents
|
||||
'.pdf': 'document',
|
||||
'.doc': 'document',
|
||||
'.docx': 'document',
|
||||
}
|
||||
file_type = file_type_map.get(ext, 'data')
|
||||
|
||||
# Create directory structure: data/Projects/{project_id}/{session_id}/
|
||||
project_dir = Path(f"data/Projects/{project_id}/{session.id}")
|
||||
project_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Save file to disk
|
||||
file_path = project_dir / filename
|
||||
file_content = response.content
|
||||
|
||||
with open(file_path, 'wb') as f:
|
||||
f.write(file_content)
|
||||
|
||||
# Calculate checksum
|
||||
checksum = hashlib.sha256(file_content).hexdigest()
|
||||
|
||||
# Create DataFile record
|
||||
data_file = DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=session.id,
|
||||
file_path=str(file_path.relative_to("data")), # Store relative to data/
|
||||
file_type=file_type,
|
||||
file_size_bytes=len(file_content),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": "ftp",
|
||||
"remote_path": remote_path,
|
||||
"unit_id": unit_id,
|
||||
"location_id": location_id,
|
||||
})
|
||||
)
|
||||
|
||||
db.add(data_file)
|
||||
db.commit()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Downloaded {filename} to server",
|
||||
"file_id": data_file.id,
|
||||
"file_path": str(file_path),
|
||||
"file_size": len(file_content),
|
||||
}
|
||||
|
||||
except httpx.TimeoutException:
|
||||
raise HTTPException(
|
||||
status_code=504,
|
||||
detail="Timeout downloading file from SLM"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error downloading file to server: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to download file to server: {str(e)}"
|
||||
)
|
||||
|
||||
|
||||
@router.post("/{project_id}/ftp-download-folder-to-server")
|
||||
@@ -1799,20 +1805,20 @@ async def ftp_download_folder_to_server(
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Download an entire Auto_#### measurement folder from an SLM to the server.
|
||||
|
||||
Routes the downloaded ZIP through the shared NRL ingest — the same path the
|
||||
scheduled FTP pull, the daily cycle, and the manual SD-card upload use. That
|
||||
means: keep the .rnh + Leq .rnd, parse the header (real recording start/stop
|
||||
+ duration, percentile slot map, weightings), drop the 1-second _Lp_ files,
|
||||
and create one clean MonitoringSession attributed to the unit. Sound
|
||||
Monitoring projects only.
|
||||
Download an entire folder from an SLM to the server via FTP.
|
||||
Extracts all files from the ZIP and preserves folder structure.
|
||||
Creates individual DataFile records for each file.
|
||||
Sound Monitoring projects only.
|
||||
"""
|
||||
import httpx
|
||||
import os
|
||||
import hashlib
|
||||
import zipfile
|
||||
import io
|
||||
|
||||
_require_module(db.query(Project).filter_by(id=project_id).first(), "sound_monitoring", db)
|
||||
from backend.routers.project_locations import ingest_nrl_zip, IngestError
|
||||
from pathlib import Path
|
||||
from backend.models import DataFile
|
||||
|
||||
data = await request.json()
|
||||
unit_id = data.get("unit_id")
|
||||
@@ -1821,65 +1827,159 @@ async def ftp_download_folder_to_server(
|
||||
|
||||
if not unit_id or not remote_path:
|
||||
raise HTTPException(status_code=400, detail="Missing unit_id or remote_path")
|
||||
if not location_id:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=("This unit isn't assigned to a monitoring location. Assign it to an "
|
||||
"NRL first so the downloaded measurement attaches to the right location."),
|
||||
)
|
||||
|
||||
# Download the folder from SLMM (returns a ZIP of the Auto_#### folder)
|
||||
# Get or create active session for this location/unit
|
||||
session = db.query(MonitoringSession).filter(
|
||||
and_(
|
||||
MonitoringSession.project_id == project_id,
|
||||
MonitoringSession.location_id == location_id,
|
||||
MonitoringSession.unit_id == unit_id,
|
||||
MonitoringSession.status.in_(["recording", "paused"])
|
||||
)
|
||||
).first()
|
||||
|
||||
# If no active session, create one
|
||||
if not session:
|
||||
_ftp_unit = db.query(RosterUnit).filter_by(id=unit_id).first()
|
||||
session = MonitoringSession(
|
||||
id=str(uuid.uuid4()),
|
||||
project_id=project_id,
|
||||
location_id=location_id,
|
||||
unit_id=unit_id,
|
||||
session_type="sound", # SLMs are sound monitoring devices
|
||||
status="completed",
|
||||
started_at=datetime.utcnow(),
|
||||
stopped_at=datetime.utcnow(),
|
||||
device_model=_ftp_unit.slm_model if _ftp_unit else None,
|
||||
session_metadata='{"source": "ftp_folder_download", "note": "Auto-created for FTP folder download"}'
|
||||
)
|
||||
db.add(session)
|
||||
db.commit()
|
||||
db.refresh(session)
|
||||
|
||||
# Download folder from SLMM (returns ZIP)
|
||||
SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=600.0) as client: # longer timeout for folders
|
||||
async with httpx.AsyncClient(timeout=600.0) as client: # Longer timeout for folders
|
||||
response = await client.post(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download-folder",
|
||||
json={"remote_path": remote_path}
|
||||
)
|
||||
|
||||
if not response.is_success:
|
||||
raise HTTPException(
|
||||
status_code=response.status_code,
|
||||
detail=f"Failed to download folder from SLMM: {response.text}"
|
||||
)
|
||||
|
||||
# Extract folder name from remote_path
|
||||
folder_name = os.path.basename(remote_path.rstrip('/'))
|
||||
|
||||
# Create base directory: data/Projects/{project_id}/{session_id}/{folder_name}/
|
||||
base_dir = Path(f"data/Projects/{project_id}/{session.id}/{folder_name}")
|
||||
base_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Extract ZIP and save individual files
|
||||
zip_content = response.content
|
||||
created_files = []
|
||||
total_size = 0
|
||||
|
||||
# File type mapping for classification
|
||||
file_type_map = {
|
||||
# Audio files
|
||||
'.wav': 'audio', '.mp3': 'audio', '.flac': 'audio', '.m4a': 'audio', '.aac': 'audio',
|
||||
# Data files
|
||||
'.csv': 'data', '.txt': 'data', '.json': 'data', '.xml': 'data', '.dat': 'data',
|
||||
# Log files
|
||||
'.log': 'log',
|
||||
# Archives
|
||||
'.zip': 'archive', '.tar': 'archive', '.gz': 'archive', '.7z': 'archive', '.rar': 'archive',
|
||||
# Images
|
||||
'.jpg': 'image', '.jpeg': 'image', '.png': 'image', '.gif': 'image',
|
||||
# Documents
|
||||
'.pdf': 'document', '.doc': 'document', '.docx': 'document',
|
||||
}
|
||||
|
||||
with zipfile.ZipFile(io.BytesIO(zip_content)) as zf:
|
||||
for zip_info in zf.filelist:
|
||||
# Skip directories
|
||||
if zip_info.is_dir():
|
||||
continue
|
||||
|
||||
# Read file from ZIP
|
||||
file_data = zf.read(zip_info.filename)
|
||||
|
||||
# Determine file path (preserve structure within folder)
|
||||
# zip_info.filename might be like "Auto_0001/measurement.wav"
|
||||
file_path = base_dir / zip_info.filename
|
||||
file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write file to disk
|
||||
with open(file_path, 'wb') as f:
|
||||
f.write(file_data)
|
||||
|
||||
# Calculate checksum
|
||||
checksum = hashlib.sha256(file_data).hexdigest()
|
||||
|
||||
# Determine file type
|
||||
ext = os.path.splitext(zip_info.filename)[1].lower()
|
||||
file_type = file_type_map.get(ext, 'data')
|
||||
|
||||
# Create DataFile record
|
||||
data_file = DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=session.id,
|
||||
file_path=str(file_path.relative_to("data")),
|
||||
file_type=file_type,
|
||||
file_size_bytes=len(file_data),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": "ftp_folder",
|
||||
"remote_path": remote_path,
|
||||
"unit_id": unit_id,
|
||||
"location_id": location_id,
|
||||
"folder_name": folder_name,
|
||||
"relative_path": zip_info.filename,
|
||||
})
|
||||
)
|
||||
|
||||
db.add(data_file)
|
||||
created_files.append({
|
||||
"filename": zip_info.filename,
|
||||
"size": len(file_data),
|
||||
"type": file_type
|
||||
})
|
||||
total_size += len(file_data)
|
||||
|
||||
db.commit()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"message": f"Downloaded folder {folder_name} with {len(created_files)} files",
|
||||
"folder_name": folder_name,
|
||||
"file_count": len(created_files),
|
||||
"total_size": total_size,
|
||||
"files": created_files,
|
||||
}
|
||||
|
||||
except httpx.TimeoutException:
|
||||
raise HTTPException(
|
||||
status_code=504,
|
||||
detail="Timeout downloading folder from SLM (large folders may take a while)",
|
||||
detail="Timeout downloading folder from SLM (large folders may take a while)"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error reaching SLMM for folder download: {e}")
|
||||
raise HTTPException(status_code=502, detail=f"Failed to reach SLMM: {str(e)}")
|
||||
|
||||
if not response.is_success:
|
||||
except zipfile.BadZipFile:
|
||||
raise HTTPException(
|
||||
status_code=response.status_code,
|
||||
detail=f"Failed to download folder from SLMM: {response.text}",
|
||||
status_code=500,
|
||||
detail="Downloaded file is not a valid ZIP archive"
|
||||
)
|
||||
|
||||
# Ingest through the shared NRL core. dedupe=False so a re-download of a
|
||||
# still-growing folder captures the latest intervals (matches manual upload).
|
||||
try:
|
||||
result = ingest_nrl_zip(
|
||||
location_id, response.content, db,
|
||||
source="ftp_manual", dedupe=False, unit_id=unit_id,
|
||||
)
|
||||
except IngestError as e:
|
||||
# No usable .rnd/.rnh in the folder, or unknown location.
|
||||
raise HTTPException(status_code=400, detail=str(e))
|
||||
except Exception as e:
|
||||
logger.error(f"Error ingesting downloaded folder: {e}")
|
||||
raise HTTPException(status_code=500, detail=f"Failed to ingest downloaded folder: {str(e)}")
|
||||
|
||||
folder_name = os.path.basename(remote_path.rstrip('/'))
|
||||
return {
|
||||
"success": True,
|
||||
"message": (
|
||||
f"Imported {result['leq_files']} Leq file(s) from {folder_name} "
|
||||
f"({result['files_imported']} stored; 1-second _Lp_ data skipped)"
|
||||
),
|
||||
"folder_name": folder_name,
|
||||
"session_id": result["session_id"],
|
||||
"file_count": result["files_imported"],
|
||||
"leq_files": result["leq_files"],
|
||||
"started_at": result["started_at"],
|
||||
"stopped_at": result["stopped_at"],
|
||||
"duration_seconds": result["duration_seconds"],
|
||||
}
|
||||
logger.error(f"Error downloading folder to server: {e}")
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to download folder to server: {str(e)}"
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
@@ -1890,26 +1990,21 @@ async def ftp_download_folder_to_server(
|
||||
async def get_unified_files(
|
||||
project_id: str,
|
||||
request: Request,
|
||||
location_id: Optional[str] = None,
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""
|
||||
Get unified view of all files in this project.
|
||||
Groups files by recording session with full metadata.
|
||||
Returns HTML partial with hierarchical file listing.
|
||||
|
||||
When `location_id` is given, scope to a single NRL/location (used by the
|
||||
per-NRL Data Files tab so it mirrors the project-wide tab).
|
||||
"""
|
||||
from backend.models import DataFile
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
# Sessions for this project (optionally scoped to one NRL/location)
|
||||
q = db.query(MonitoringSession).filter_by(project_id=project_id)
|
||||
if location_id:
|
||||
q = q.filter(MonitoringSession.location_id == location_id)
|
||||
sessions = q.order_by(MonitoringSession.started_at.desc()).all()
|
||||
# Get all sessions for this project
|
||||
sessions = db.query(MonitoringSession).filter_by(
|
||||
project_id=project_id
|
||||
).order_by(MonitoringSession.started_at.desc()).all()
|
||||
|
||||
sessions_data = []
|
||||
for session in sessions:
|
||||
|
||||
@@ -1,434 +0,0 @@
|
||||
"""
|
||||
Nightly Report Router.
|
||||
|
||||
Manual triggers for the night-vs-baseline sound report — the same entry point
|
||||
the scheduled morning tick will reuse. Two endpoints:
|
||||
|
||||
GET …/reports/nightly/view → render and return the HTML inline (preview).
|
||||
No write, no email. Browser-friendly.
|
||||
POST …/reports/nightly/run → full run: build → write report.html/json to
|
||||
disk → (dry-run) email. Returns JSON result.
|
||||
|
||||
Dates are the *evening* date of the night being reported (the 7/7 in "night of
|
||||
7/7 → morning 7/8"). Defaults to last night. Baseline is optional; pass the
|
||||
baseline-week range to populate the comparison.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, date
|
||||
from html import escape
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.database import get_db
|
||||
from backend.models import Project, SoundReportConfig, MonitoringLocation
|
||||
from backend.services.report_pipeline import (
|
||||
METRIC_REGISTRY, DEFAULT_METRICS, DEFAULT_WINDOWS, _location_reference_baseline,
|
||||
)
|
||||
from backend.services.report_orchestrator import run_nightly_report
|
||||
from backend.utils.timezone import utc_to_local
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/projects/{project_id}/reports", tags=["reports"])
|
||||
|
||||
|
||||
def _default_night_date() -> date:
|
||||
"""Last night = yesterday in the user's local timezone."""
|
||||
return (utc_to_local(datetime.utcnow()) - timedelta(days=1)).date()
|
||||
|
||||
|
||||
def _parse_date(s: Optional[str], field: str) -> Optional[date]:
|
||||
if not s:
|
||||
return None
|
||||
try:
|
||||
return datetime.strptime(s, "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
raise HTTPException(status_code=400, detail=f"{field} must be YYYY-MM-DD (got {s!r})")
|
||||
|
||||
|
||||
def _parse_metrics(s: Optional[str]) -> list[str]:
|
||||
if not s:
|
||||
return list(DEFAULT_METRICS)
|
||||
keys = [k.strip().lower() for k in s.split(",") if k.strip()]
|
||||
unknown = [k for k in keys if k not in METRIC_REGISTRY]
|
||||
if unknown:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"Unknown metric(s): {unknown}. Known: {sorted(METRIC_REGISTRY)}",
|
||||
)
|
||||
return keys or list(DEFAULT_METRICS)
|
||||
|
||||
|
||||
def _validate_hhmm(s) -> str:
|
||||
"""Validate a local HH:MM (24h) time string."""
|
||||
try:
|
||||
hh, mm = str(s).split(":")
|
||||
h, m = int(hh), int(mm)
|
||||
if 0 <= h < 24 and 0 <= m < 60:
|
||||
return f"{h:02d}:{m:02d}"
|
||||
except (ValueError, AttributeError):
|
||||
pass
|
||||
raise HTTPException(status_code=400, detail=f"report_time must be HH:MM 24-hour (got {s!r})")
|
||||
|
||||
|
||||
def _config_dict(cfg: Optional[SoundReportConfig], project_id: str) -> dict:
|
||||
"""Serialise a config row (or defaults if none yet) to JSON."""
|
||||
return {
|
||||
"project_id": project_id,
|
||||
"exists": cfg is not None,
|
||||
"enabled": cfg.enabled if cfg else False,
|
||||
"report_time": cfg.report_time if cfg else "08:00",
|
||||
"metric_keys": cfg.metric_keys if cfg else ",".join(DEFAULT_METRICS),
|
||||
"baseline_mode": cfg.baseline_mode if cfg else "captured",
|
||||
"baseline_start": cfg.baseline_start.isoformat() if cfg and cfg.baseline_start else None,
|
||||
"baseline_end": cfg.baseline_end.isoformat() if cfg and cfg.baseline_end else None,
|
||||
"recipients": (cfg.recipients if cfg and cfg.recipients else ""),
|
||||
"last_run_date": cfg.last_run_date.isoformat() if cfg and cfg.last_run_date else None,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/config")
|
||||
async def get_report_config(project_id: str, db: Session = Depends(get_db)):
|
||||
"""Return the project's nightly-report config (or defaults if not set yet)."""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
return _config_dict(cfg, project_id)
|
||||
|
||||
|
||||
@router.put("/config")
|
||||
async def put_report_config(project_id: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Create or update the project's nightly-report config (JSON body)."""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
data = await request.json()
|
||||
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
created = cfg is None
|
||||
if cfg is None:
|
||||
cfg = SoundReportConfig(id=str(uuid.uuid4()), project_id=project_id)
|
||||
db.add(cfg)
|
||||
|
||||
if "enabled" in data:
|
||||
cfg.enabled = bool(data["enabled"])
|
||||
if "report_time" in data:
|
||||
cfg.report_time = _validate_hhmm(data["report_time"])
|
||||
if "metric_keys" in data:
|
||||
mk = data["metric_keys"]
|
||||
mk = mk if isinstance(mk, str) else ",".join(mk or [])
|
||||
cfg.metric_keys = ",".join(_parse_metrics(mk))
|
||||
if "baseline_mode" in data:
|
||||
bm = str(data["baseline_mode"]).lower()
|
||||
if bm not in ("captured", "reference"):
|
||||
raise HTTPException(status_code=400, detail="baseline_mode must be 'captured' or 'reference'")
|
||||
cfg.baseline_mode = bm
|
||||
if "baseline_start" in data or "baseline_end" in data:
|
||||
bs = _parse_date(data.get("baseline_start") or None, "baseline_start")
|
||||
be = _parse_date(data.get("baseline_end") or None, "baseline_end")
|
||||
if (bs and not be) or (be and not bs):
|
||||
raise HTTPException(status_code=400, detail="Provide both baseline dates, or neither.")
|
||||
if bs and be and bs > be:
|
||||
raise HTTPException(status_code=400, detail="baseline_start must be on or before baseline_end.")
|
||||
cfg.baseline_start, cfg.baseline_end = bs, be
|
||||
if "recipients" in data:
|
||||
recips = data["recipients"]
|
||||
if isinstance(recips, list):
|
||||
recips = ",".join(recips)
|
||||
cfg.recipients = (recips or "").strip() or None
|
||||
|
||||
db.commit()
|
||||
db.refresh(cfg)
|
||||
return {**_config_dict(cfg, project_id), "created": created}
|
||||
|
||||
|
||||
def _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics):
|
||||
"""Validate inputs and resolve the baseline source.
|
||||
|
||||
Explicit baseline dates in the query override (captured mode with those
|
||||
dates). Otherwise the project's saved config supplies the baseline (its
|
||||
mode + dates) and the default metric set — so the manual view/run match
|
||||
what the scheduled report does.
|
||||
Returns (night_date, baseline_mode, baseline_start, baseline_end, metric_keys).
|
||||
"""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
nd = _parse_date(night_date, "night_date") or _default_night_date()
|
||||
bs = _parse_date(baseline_start, "baseline_start")
|
||||
be = _parse_date(baseline_end, "baseline_end")
|
||||
if (bs and not be) or (be and not bs):
|
||||
raise HTTPException(status_code=400, detail="Provide both baseline_start and baseline_end, or neither.")
|
||||
if bs and be and bs > be:
|
||||
raise HTTPException(status_code=400, detail="baseline_start must be on or before baseline_end.")
|
||||
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
if bs and be:
|
||||
baseline_mode = "captured" # explicit dates win
|
||||
elif cfg:
|
||||
baseline_mode = cfg.baseline_mode # fall back to saved config
|
||||
bs, be = cfg.baseline_start, cfg.baseline_end
|
||||
else:
|
||||
baseline_mode = "captured"
|
||||
|
||||
if metrics:
|
||||
metric_keys = _parse_metrics(metrics)
|
||||
elif cfg and cfg.metric_keys:
|
||||
metric_keys = _parse_metrics(cfg.metric_keys)
|
||||
else:
|
||||
metric_keys = list(DEFAULT_METRICS)
|
||||
|
||||
return nd, baseline_mode, bs, be, metric_keys
|
||||
|
||||
|
||||
@router.get("/nightly/view", response_class=HTMLResponse)
|
||||
async def view_nightly_report(
|
||||
project_id: str,
|
||||
night_date: Optional[str] = Query(None, description="Evening date of the night (YYYY-MM-DD). Default: last night."),
|
||||
baseline_start: Optional[str] = Query(None, description="Baseline range start (YYYY-MM-DD)."),
|
||||
baseline_end: Optional[str] = Query(None, description="Baseline range end (YYYY-MM-DD)."),
|
||||
metrics: Optional[str] = Query(None, description="Comma list, e.g. lmax,l01,l10,l90. Default: house set."),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Render the night report and return the HTML inline (preview — no write, no email)."""
|
||||
nd, bmode, bs, be, metric_keys = _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics)
|
||||
try:
|
||||
result = run_nightly_report(
|
||||
db, project_id, nd,
|
||||
metric_keys=metric_keys, baseline_mode=bmode, baseline_start=bs, baseline_end=be,
|
||||
send=False, # preview: no email
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.error("nightly/view failed for %s (%s): %s", project_id, nd, e, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Report generation failed: {e}")
|
||||
return HTMLResponse(result["html"])
|
||||
|
||||
|
||||
@router.post("/nightly/run")
|
||||
async def run_nightly_report_endpoint(
|
||||
project_id: str,
|
||||
night_date: Optional[str] = Query(None, description="Evening date of the night (YYYY-MM-DD). Default: last night."),
|
||||
baseline_start: Optional[str] = Query(None, description="Baseline range start (YYYY-MM-DD)."),
|
||||
baseline_end: Optional[str] = Query(None, description="Baseline range end (YYYY-MM-DD)."),
|
||||
metrics: Optional[str] = Query(None, description="Comma list, e.g. lmax,l01,l10,l90. Default: house set."),
|
||||
send: bool = Query(True, description="Attempt email (dry-run until SMTP is configured)."),
|
||||
db: Session = Depends(get_db),
|
||||
):
|
||||
"""Run the night report: build → write report.html/report.json to disk → email (best-effort).
|
||||
|
||||
This is the same path the scheduled morning tick will call. The `html` field
|
||||
is omitted from the JSON response (it's large and on disk); use /view to see it.
|
||||
"""
|
||||
nd, bmode, bs, be, metric_keys = _resolve_params(project_id, db, night_date, baseline_start, baseline_end, metrics)
|
||||
try:
|
||||
result = run_nightly_report(
|
||||
db, project_id, nd,
|
||||
metric_keys=metric_keys, baseline_mode=bmode, baseline_start=bs, baseline_end=be,
|
||||
send=send,
|
||||
)
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.error("nightly/run failed for %s (%s): %s", project_id, nd, e, exc_info=True)
|
||||
raise HTTPException(status_code=500, detail=f"Report generation failed: {e}")
|
||||
result.pop("html", None) # keep the JSON response lean — view it via /view or the file
|
||||
result["view_url"] = (
|
||||
f"/api/projects/{project_id}/reports/nightly/view"
|
||||
f"?night_date={nd:%Y-%m-%d}"
|
||||
+ (f"&baseline_start={bs:%Y-%m-%d}&baseline_end={be:%Y-%m-%d}" if bs and be else "")
|
||||
+ (f"&metrics={','.join(metric_keys)}")
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Test email + generated-report archive
|
||||
# ============================================================================
|
||||
|
||||
_DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
|
||||
|
||||
|
||||
@router.post("/test-email")
|
||||
async def send_test_email(project_id: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Send a small test email to verify the SMTP relay (dry-run if unconfigured).
|
||||
|
||||
Recipients: JSON body {"recipients": "..."} overrides; else the project's
|
||||
configured recipients; else the REPORT_SMTP_RECIPIENTS env default.
|
||||
"""
|
||||
project = db.query(Project).filter_by(id=project_id).first()
|
||||
if not project:
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
try:
|
||||
data = await request.json()
|
||||
except Exception:
|
||||
data = {}
|
||||
|
||||
raw = (data or {}).get("recipients")
|
||||
if not raw:
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
raw = cfg.recipients if cfg else None
|
||||
recipients = None
|
||||
if raw:
|
||||
if isinstance(raw, list):
|
||||
raw = ",".join(raw)
|
||||
recipients = [r.strip() for r in raw.split(",") if r.strip()]
|
||||
|
||||
from backend.services.report_email import send_report_email
|
||||
body = (
|
||||
"<div style=\"font:14px Arial,sans-serif\">"
|
||||
f"Terra-View test email for <b>{escape(project.name)}</b>.<br>"
|
||||
"If you got this, the nightly sound-report email path is working.</div>"
|
||||
)
|
||||
return send_report_email("Terra-View — nightly report test email", body, recipients=recipients)
|
||||
|
||||
|
||||
@router.get("/list")
|
||||
async def list_reports(project_id: str, db: Session = Depends(get_db)):
|
||||
"""List the generated report artifacts on disk for this project (newest first)."""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
base = Path("data/reports") / project_id
|
||||
out = []
|
||||
if base.exists():
|
||||
for d in sorted((p for p in base.iterdir() if p.is_dir()), key=lambda p: p.name, reverse=True):
|
||||
html_file = d / "report.html"
|
||||
if html_file.exists():
|
||||
st = html_file.stat()
|
||||
out.append({
|
||||
"night_date": d.name,
|
||||
"view_url": f"/api/projects/{project_id}/reports/archive/{d.name}",
|
||||
"xlsx_url": (f"/api/projects/{project_id}/reports/archive/{d.name}/xlsx"
|
||||
if (d / "report.xlsx").exists() else None),
|
||||
"size_bytes": st.st_size,
|
||||
"generated_at": datetime.utcfromtimestamp(st.st_mtime).isoformat(),
|
||||
})
|
||||
return {"reports": out, "count": len(out)}
|
||||
|
||||
|
||||
@router.get("/archive/{night_date}", response_class=HTMLResponse)
|
||||
async def view_archived_report(project_id: str, night_date: str, db: Session = Depends(get_db)):
|
||||
"""Serve a previously generated report.html from disk (the actual artifact)."""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
if not _DATE_RE.match(night_date):
|
||||
raise HTTPException(status_code=400, detail="Invalid date (YYYY-MM-DD)")
|
||||
safe = _parse_date(night_date, "night_date") # also guards path traversal
|
||||
path = Path("data/reports") / project_id / f"{safe:%Y-%m-%d}" / "report.html"
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=404, detail="No saved report for that date")
|
||||
return HTMLResponse(path.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
@router.get("/archive/{night_date}/xlsx")
|
||||
async def download_archived_xlsx(project_id: str, night_date: str, db: Session = Depends(get_db)):
|
||||
"""Download a previously generated report.xlsx from disk."""
|
||||
from fastapi.responses import Response
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
if not _DATE_RE.match(night_date):
|
||||
raise HTTPException(status_code=400, detail="Invalid date (YYYY-MM-DD)")
|
||||
safe = _parse_date(night_date, "night_date")
|
||||
path = Path("data/reports") / project_id / f"{safe:%Y-%m-%d}" / "report.xlsx"
|
||||
if not path.exists():
|
||||
raise HTTPException(status_code=404, detail="No saved spreadsheet for that date")
|
||||
return Response(
|
||||
content=path.read_bytes(),
|
||||
media_type="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
headers={"Content-Disposition": f'attachment; filename="night_report_{safe:%Y-%m-%d}.xlsx"'},
|
||||
)
|
||||
|
||||
|
||||
# ============================================================================
|
||||
# Reference baseline (fixed values typed per location — limits / prior averages)
|
||||
# ============================================================================
|
||||
|
||||
@router.get("/baseline")
|
||||
async def get_baseline(project_id: str, db: Session = Depends(get_db)):
|
||||
"""Return the baseline mode + per-location reference values + the metric/window
|
||||
grid to render the editor."""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
mode = cfg.baseline_mode if cfg else "captured"
|
||||
metric_keys = _parse_metrics(cfg.metric_keys) if cfg and cfg.metric_keys else list(DEFAULT_METRICS)
|
||||
|
||||
locations = db.query(MonitoringLocation).filter_by(
|
||||
project_id=project_id, location_type="sound",
|
||||
).order_by(MonitoringLocation.sort_order, MonitoringLocation.name).all()
|
||||
locations = [l for l in locations if getattr(l, "removed_at", None) is None]
|
||||
|
||||
return {
|
||||
"mode": mode,
|
||||
"windows": [{"key": w.key, "label": w.label} for w in DEFAULT_WINDOWS],
|
||||
"metrics": [{"key": k, "label": METRIC_REGISTRY[k].label} for k in metric_keys],
|
||||
"locations": [
|
||||
{"id": loc.id, "name": loc.name, "values": _location_reference_baseline(loc)}
|
||||
for loc in locations
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@router.put("/baseline")
|
||||
async def put_baseline(project_id: str, request: Request, db: Session = Depends(get_db)):
|
||||
"""Save the baseline mode (on config) and per-location reference values
|
||||
(on each location's metadata). Body:
|
||||
{"mode": "reference",
|
||||
"locations": {"<loc_id>": {"nighttime": {"l10": 85}, "evening": {...}}}}
|
||||
"""
|
||||
if not db.query(Project).filter_by(id=project_id).first():
|
||||
raise HTTPException(status_code=404, detail="Project not found")
|
||||
data = await request.json()
|
||||
|
||||
if "mode" in data:
|
||||
bm = str(data["mode"]).lower()
|
||||
if bm not in ("captured", "reference"):
|
||||
raise HTTPException(status_code=400, detail="mode must be 'captured' or 'reference'")
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=project_id).first()
|
||||
if cfg is None:
|
||||
cfg = SoundReportConfig(id=str(uuid.uuid4()), project_id=project_id)
|
||||
db.add(cfg)
|
||||
cfg.baseline_mode = bm
|
||||
|
||||
loc_values = data.get("locations") or {}
|
||||
updated = 0
|
||||
for loc_id, windows in loc_values.items():
|
||||
loc = db.query(MonitoringLocation).filter_by(id=loc_id, project_id=project_id).first()
|
||||
if not loc or not isinstance(windows, dict):
|
||||
continue
|
||||
try:
|
||||
meta = json.loads(loc.location_metadata or "{}")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
meta = {}
|
||||
clean: dict = {}
|
||||
for wkey, mvals in windows.items():
|
||||
if not isinstance(mvals, dict):
|
||||
continue
|
||||
cm = {}
|
||||
for mkey, val in mvals.items():
|
||||
if val in (None, ""):
|
||||
continue
|
||||
try:
|
||||
cm[mkey] = round(float(val), 1)
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
if cm:
|
||||
clean[wkey] = cm
|
||||
if clean:
|
||||
meta["report_baseline"] = clean
|
||||
else:
|
||||
meta.pop("report_baseline", None)
|
||||
loc.location_metadata = json.dumps(meta)
|
||||
updated += 1
|
||||
|
||||
db.commit()
|
||||
return {"ok": True, "locations_updated": updated}
|
||||
@@ -91,43 +91,29 @@ async def get_slm_units(
|
||||
|
||||
one_hour_ago = datetime.utcnow() - timedelta(hours=1)
|
||||
for unit in units:
|
||||
# Legacy default from the roster field; refined from SLMM's cached status below.
|
||||
unit.is_recent = bool(unit.slm_last_check and unit.slm_last_check > one_hour_ago)
|
||||
unit.measurement_state = None
|
||||
unit.cache_last_seen = None # SLMM cache last_seen (real monitoring freshness)
|
||||
|
||||
if include_measurement:
|
||||
# SLMM's /roster carries each unit's CACHED status (last_seen,
|
||||
# measurement_state) from NL43Status — a DB read on SLMM's side, NOT a device
|
||||
# call. The live monitor refreshes that cache ~every 1.3s, so this reflects
|
||||
# real monitoring without sending Measure? to the device (which the old
|
||||
# /measurement-state did) and competing with DOD polling. One call covers all.
|
||||
slmm_status = {}
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=3.0) as client:
|
||||
r = await client.get(f"{SLMM_BASE_URL}/api/nl43/roster")
|
||||
if r.status_code == 200:
|
||||
for dev in (r.json().get("devices") or []):
|
||||
slmm_status[dev.get("unit_id")] = dev.get("status") or {}
|
||||
except Exception:
|
||||
slmm_status = {}
|
||||
async def fetch_measurement_state(client: httpx.AsyncClient, unit_id: str) -> str | None:
|
||||
try:
|
||||
response = await client.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/measurement-state")
|
||||
if response.status_code == 200:
|
||||
return response.json().get("measurement_state")
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
# "Recent" = the monitor has a fresh successful read. last_seen only advances
|
||||
# on a successful poll, so staleness == the device isn't being reached.
|
||||
recent_cutoff = datetime.utcnow() - timedelta(minutes=5)
|
||||
for unit in units:
|
||||
st = slmm_status.get(unit.id)
|
||||
if not st:
|
||||
continue
|
||||
unit.measurement_state = st.get("measurement_state")
|
||||
last_seen = st.get("last_seen")
|
||||
if last_seen:
|
||||
try:
|
||||
ls = datetime.fromisoformat(last_seen.replace("Z", ""))
|
||||
unit.is_recent = ls > recent_cutoff
|
||||
unit.cache_last_seen = ls # the real freshness the monitor updates
|
||||
except Exception:
|
||||
pass
|
||||
deployed_units = [unit for unit in units if unit.deployed and not unit.retired]
|
||||
if deployed_units:
|
||||
async with httpx.AsyncClient(timeout=3.0) as client:
|
||||
tasks = [fetch_measurement_state(client, unit.id) for unit in deployed_units]
|
||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
for unit, state in zip(deployed_units, results):
|
||||
if isinstance(state, Exception):
|
||||
unit.measurement_state = None
|
||||
else:
|
||||
unit.measurement_state = state
|
||||
|
||||
return templates.TemplateResponse("partials/slm_device_list.html", {
|
||||
"request": request,
|
||||
@@ -171,18 +157,25 @@ async def get_live_view(request: Request, unit_id: str, db: Session = Depends(ge
|
||||
is_measuring = False
|
||||
|
||||
try:
|
||||
# Read SLMM's CACHED status (NL43Status) — no device call. The live monitor
|
||||
# keeps it fresh (~1.3s) and the live-stream WS provides ongoing updates, so we
|
||||
# no longer fire Measure? + a fresh DOD read at the device on every command-
|
||||
# center load (which competed with DOD polling for the single connection).
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
r = await client.get(f"{SLMM_BASE_URL}/api/nl43/{unit_id}/status")
|
||||
if r.status_code == 200:
|
||||
current_status = r.json().get("data", {})
|
||||
measurement_state = current_status.get("measurement_state")
|
||||
is_measuring = measurement_state in ("Start", "Measure")
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
# Get measurement state
|
||||
state_response = await client.get(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/measurement-state"
|
||||
)
|
||||
if state_response.status_code == 200:
|
||||
state_data = state_response.json()
|
||||
measurement_state = state_data.get("measurement_state", "Unknown")
|
||||
is_measuring = state_data.get("is_measuring", False)
|
||||
|
||||
# Get live status (measurement_start_time is already stored in SLMM database)
|
||||
status_response = await client.get(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/live"
|
||||
)
|
||||
if status_response.status_code == 200:
|
||||
status_data = status_response.json()
|
||||
current_status = status_data.get("data", {})
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to get cached status for {unit_id}: {e}")
|
||||
logger.error(f"Failed to get status for {unit_id}: {e}")
|
||||
|
||||
return templates.TemplateResponse("partials/slm_live_view.html", {
|
||||
"request": request,
|
||||
|
||||
@@ -231,76 +231,6 @@ async def proxy_websocket_live(websocket: WebSocket, unit_id: str):
|
||||
logger.info(f"WebSocket proxy closed for {unit_id} (live)")
|
||||
|
||||
|
||||
@router.websocket("/{unit_id}/monitor")
|
||||
async def proxy_websocket_monitor(websocket: WebSocket, unit_id: str):
|
||||
"""
|
||||
Proxy WebSocket connections to SLMM's /monitor (fan-out DOD feed).
|
||||
|
||||
This is the shared ~1Hz DOD feed: many clients subscribe to one device feed
|
||||
(no single-connection contention) and it carries L1/L10 (which the DRD
|
||||
/stream cannot). Preferred over /stream for the live view.
|
||||
"""
|
||||
await websocket.accept()
|
||||
logger.info(f"WebSocket accepted for SLMM unit {unit_id} (monitor)")
|
||||
|
||||
target_ws_url = f"{SLMM_WS_BASE_URL}/api/nl43/{unit_id}/monitor"
|
||||
backend_ws = None
|
||||
|
||||
try:
|
||||
backend_ws = await websockets.connect(target_ws_url)
|
||||
logger.info(f"Connected to SLMM monitor feed for {unit_id}")
|
||||
|
||||
async def forward_to_client():
|
||||
"""Backend monitor frames -> browser."""
|
||||
async for message in backend_ws:
|
||||
await websocket.send_text(message)
|
||||
|
||||
async def watch_client():
|
||||
"""Drain client frames; raises WebSocketDisconnect on close so we can
|
||||
tear the pair down (the monitor feed is server->client only)."""
|
||||
while True:
|
||||
await websocket.receive_text()
|
||||
|
||||
# When EITHER side ends (browser disconnects or backend closes), cancel the
|
||||
# other immediately — avoids sending into a closed socket (the
|
||||
# "Unexpected ASGI message after close" race that asyncio.gather leaves open).
|
||||
tasks = [asyncio.ensure_future(forward_to_client()),
|
||||
asyncio.ensure_future(watch_client())]
|
||||
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
|
||||
for t in pending:
|
||||
t.cancel()
|
||||
# Await ALL tasks (the done one AND the cancelled one) and swallow both
|
||||
# the expected WebSocketDisconnect and CancelledError. CancelledError is a
|
||||
# BaseException, so a bare `except Exception` misses it — that's what leaked
|
||||
# the traceback on stop; and awaiting only `pending` left the done task's
|
||||
# exception unretrieved.
|
||||
for t in tasks:
|
||||
try:
|
||||
await t
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
|
||||
except websockets.exceptions.WebSocketException as e:
|
||||
logger.error(f"WebSocket error connecting to SLMM monitor for {unit_id}: {e}")
|
||||
try:
|
||||
await websocket.send_json({"error": "Failed to connect to SLMM monitor", "detail": str(e)})
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.error(f"Unexpected error in monitor proxy for {unit_id}: {e}")
|
||||
finally:
|
||||
if backend_ws:
|
||||
try:
|
||||
await backend_ws.close()
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
await websocket.close()
|
||||
except Exception:
|
||||
pass
|
||||
logger.info(f"WebSocket monitor proxy closed for {unit_id}")
|
||||
|
||||
|
||||
# HTTP catch-all route MUST come after specific routes (including WebSocket routes)
|
||||
@router.api_route("/{path:path}", methods=["GET", "POST", "PUT", "DELETE", "PATCH"])
|
||||
async def proxy_to_slmm(path: str, request: Request):
|
||||
|
||||
@@ -1,172 +0,0 @@
|
||||
"""
|
||||
Report email sender — config-driven SMTP via the Python standard library.
|
||||
|
||||
Connection settings come from environment variables so the mail backend
|
||||
(internal relay / Microsoft 365 / Gmail / SendGrid) can be swapped without code
|
||||
changes — see the build plan: terra-mechanics.com is on M365 and has a smarthost
|
||||
relay that already sends the seismograph alerts as remote@terra-mechanics.com;
|
||||
reuse that relay's settings here.
|
||||
|
||||
DRY-RUN: if SMTP isn't configured (no host/from), the message is built and
|
||||
logged but NOT sent, and the call still succeeds. This keeps report generation
|
||||
working before the relay is wired up, and means a missing/incomplete mail config
|
||||
can never crash the nightly pipeline.
|
||||
|
||||
Env vars
|
||||
--------
|
||||
REPORT_SMTP_HOST e.g. smtp.office365.com (unset → dry-run)
|
||||
REPORT_SMTP_PORT default 587
|
||||
REPORT_SMTP_SECURITY starttls (default) | ssl | none
|
||||
REPORT_SMTP_USER optional — omit for IP-authenticated relays
|
||||
REPORT_SMTP_PASSWORD optional
|
||||
REPORT_SMTP_FROM e.g. "TMI Monitoring <monitoring@terra-mechanics.com>"
|
||||
REPORT_SMTP_RECIPIENTS comma-separated default recipient list
|
||||
REPORT_SMTP_TIMEOUT seconds, default 30
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import smtplib
|
||||
import ssl
|
||||
from dataclasses import dataclass, field
|
||||
from email.message import EmailMessage
|
||||
from typing import Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Convenient MIME type for the Excel attachment.
|
||||
XLSX_MIME = ("application", "vnd.openxmlformats-officedocument.spreadsheetml.sheet")
|
||||
|
||||
|
||||
@dataclass
|
||||
class Attachment:
|
||||
filename: str
|
||||
content: bytes
|
||||
maintype: str = "application"
|
||||
subtype: str = "octet-stream"
|
||||
|
||||
|
||||
@dataclass
|
||||
class SMTPConfig:
|
||||
host: str = ""
|
||||
port: int = 587
|
||||
security: str = "starttls" # "starttls" | "ssl" | "none"
|
||||
user: str = ""
|
||||
password: str = ""
|
||||
sender: str = ""
|
||||
recipients: list[str] = field(default_factory=list)
|
||||
timeout: float = 30.0
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "SMTPConfig":
|
||||
rec = os.getenv("REPORT_SMTP_RECIPIENTS", "")
|
||||
return cls(
|
||||
host=os.getenv("REPORT_SMTP_HOST", "").strip(),
|
||||
port=int(os.getenv("REPORT_SMTP_PORT", "587") or 587),
|
||||
security=os.getenv("REPORT_SMTP_SECURITY", "starttls").strip().lower(),
|
||||
user=os.getenv("REPORT_SMTP_USER", "").strip(),
|
||||
password=os.getenv("REPORT_SMTP_PASSWORD", ""),
|
||||
sender=os.getenv("REPORT_SMTP_FROM", "").strip(),
|
||||
recipients=[r.strip() for r in rec.split(",") if r.strip()],
|
||||
timeout=float(os.getenv("REPORT_SMTP_TIMEOUT", "30") or 30),
|
||||
)
|
||||
|
||||
@property
|
||||
def configured(self) -> bool:
|
||||
"""True only when we have enough to actually send (host + from)."""
|
||||
return bool(self.host and self.sender)
|
||||
|
||||
|
||||
def build_message(
|
||||
cfg: SMTPConfig,
|
||||
subject: str,
|
||||
html_body: str,
|
||||
recipients: list[str],
|
||||
attachments: Optional[list[Attachment]] = None,
|
||||
text_body: Optional[str] = None,
|
||||
) -> EmailMessage:
|
||||
"""Assemble a multipart message: plain-text fallback + HTML + attachments."""
|
||||
msg = EmailMessage()
|
||||
msg["From"] = cfg.sender or "terra-view@localhost"
|
||||
msg["To"] = ", ".join(recipients)
|
||||
msg["Subject"] = subject
|
||||
# Plain-text part first, then the HTML alternative (clients prefer the HTML).
|
||||
msg.set_content(text_body or "This report is best viewed in an HTML email client.")
|
||||
msg.add_alternative(html_body, subtype="html")
|
||||
for att in (attachments or []):
|
||||
msg.add_attachment(
|
||||
att.content, maintype=att.maintype, subtype=att.subtype, filename=att.filename,
|
||||
)
|
||||
return msg
|
||||
|
||||
|
||||
def send_report_email(
|
||||
subject: str,
|
||||
html_body: str,
|
||||
*,
|
||||
attachments: Optional[list[Attachment]] = None,
|
||||
recipients: Optional[list[str]] = None,
|
||||
text_body: Optional[str] = None,
|
||||
cfg: Optional[SMTPConfig] = None,
|
||||
) -> dict:
|
||||
"""Send (or dry-run) the report email.
|
||||
|
||||
Returns a result dict: {sent, dry_run, recipients, error}. Never raises on
|
||||
a send failure — it logs and returns error, so the orchestrator can record
|
||||
the failure without aborting the rest of the pipeline.
|
||||
"""
|
||||
cfg = cfg or SMTPConfig.from_env()
|
||||
recipients = recipients if recipients is not None else cfg.recipients
|
||||
result = {"sent": False, "dry_run": False, "recipients": recipients, "error": None}
|
||||
|
||||
if not recipients:
|
||||
result["error"] = "No recipients configured"
|
||||
logger.warning("Report email: no recipients set; skipping send of %r", subject)
|
||||
return result
|
||||
|
||||
msg = build_message(cfg, subject, html_body, recipients, attachments, text_body)
|
||||
|
||||
if not cfg.configured:
|
||||
result["dry_run"] = True
|
||||
logger.info(
|
||||
"Report email DRY-RUN (SMTP not configured): would send %r to %s with %d attachment(s)",
|
||||
subject, recipients, len(attachments or []),
|
||||
)
|
||||
return result
|
||||
|
||||
# Validate the security mode: an unrecognized value (typo) must NOT silently
|
||||
# fall through to a plaintext connection while still sending credentials.
|
||||
sec = cfg.security if cfg.security in ("ssl", "starttls", "none") else "starttls"
|
||||
if sec != cfg.security:
|
||||
logger.warning("Unknown REPORT_SMTP_SECURITY=%r — falling back to 'starttls'", cfg.security)
|
||||
|
||||
try:
|
||||
if sec == "ssl":
|
||||
ctx = ssl.create_default_context()
|
||||
with smtplib.SMTP_SSL(cfg.host, cfg.port, timeout=cfg.timeout, context=ctx) as s:
|
||||
if cfg.user:
|
||||
s.login(cfg.user, cfg.password)
|
||||
s.send_message(msg)
|
||||
else:
|
||||
with smtplib.SMTP(cfg.host, cfg.port, timeout=cfg.timeout) as s:
|
||||
s.ehlo()
|
||||
if sec == "starttls":
|
||||
s.starttls(context=ssl.create_default_context())
|
||||
s.ehlo()
|
||||
if cfg.user:
|
||||
if sec == "none":
|
||||
logger.warning(
|
||||
"Sending SMTP credentials over an UNENCRYPTED connection "
|
||||
"(REPORT_SMTP_SECURITY=none) — set starttls/ssl if the relay supports it."
|
||||
)
|
||||
s.login(cfg.user, cfg.password)
|
||||
s.send_message(msg)
|
||||
result["sent"] = True
|
||||
logger.info("Report email sent: %r to %s", subject, recipients)
|
||||
except Exception as e: # noqa: BLE001 — surface as result, never abort the pipeline
|
||||
result["error"] = str(e)
|
||||
logger.error("Report email send failed: %s", e, exc_info=True)
|
||||
|
||||
return result
|
||||
@@ -1,150 +0,0 @@
|
||||
"""
|
||||
Nightly Report Orchestrator.
|
||||
|
||||
Ties the pieces together: compute → render → write-to-disk → email.
|
||||
|
||||
This is what the daily cycle (or a manual trigger) calls. It ALWAYS writes the
|
||||
rendered report to disk — `data/reports/{project_id}/{night_date}/report.html`
|
||||
(+ `report.json` with the raw numbers) — so there's a viewable artifact even
|
||||
when email is in dry-run (SMTP not configured yet). The email step is
|
||||
best-effort and never aborts the run.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.services.report_pipeline import (
|
||||
ProjectNightReport, build_project_night_report, Window,
|
||||
)
|
||||
from backend.services.report_renderers import render_html_summary, render_excel
|
||||
from backend.services.report_email import send_report_email, Attachment, XLSX_MIME
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_OUTPUT_ROOT = "data/reports"
|
||||
|
||||
|
||||
def _report_to_dict(report: ProjectNightReport) -> dict:
|
||||
"""Serialise the report data model to plain JSON (for the on-disk record)."""
|
||||
return {
|
||||
"project_id": report.project_id,
|
||||
"project_name": report.project_name,
|
||||
"night_date": report.night_date.isoformat(),
|
||||
"metrics": [m.key for m in report.metrics],
|
||||
"locations": [
|
||||
{
|
||||
"name": loc.location_name,
|
||||
"night_interval_count": loc.night_interval_count,
|
||||
"baseline_nights_used": loc.baseline_nights_used,
|
||||
"notes": loc.notes,
|
||||
"windows": {
|
||||
w.key: {
|
||||
"label": w.label,
|
||||
"metrics": {
|
||||
m.key: {
|
||||
"label": m.label,
|
||||
"last_night": loc.table[w.key][m.key].last_night,
|
||||
"baseline": loc.table[w.key][m.key].baseline,
|
||||
"delta": loc.table[w.key][m.key].delta,
|
||||
}
|
||||
for m in loc.metrics
|
||||
},
|
||||
}
|
||||
for w in loc.windows
|
||||
},
|
||||
}
|
||||
for loc in report.locations
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def run_nightly_report(
|
||||
db: Session,
|
||||
project_id: str,
|
||||
night_date: date,
|
||||
*,
|
||||
metric_keys: Optional[list[str]] = None,
|
||||
windows: Optional[list[Window]] = None,
|
||||
baseline_mode: str = "captured",
|
||||
baseline_start: Optional[date] = None,
|
||||
baseline_end: Optional[date] = None,
|
||||
recipients: Optional[list[str]] = None,
|
||||
output_root: str = DEFAULT_OUTPUT_ROOT,
|
||||
send: bool = True,
|
||||
) -> dict:
|
||||
"""Build, persist, and (dry-run) email the night report for a project.
|
||||
|
||||
Returns a result dict with the on-disk artifact paths and the email result.
|
||||
Designed to be called from the daily cycle or a manual trigger.
|
||||
"""
|
||||
report = build_project_night_report(
|
||||
db, project_id, night_date,
|
||||
metric_keys=metric_keys, windows=windows,
|
||||
baseline_mode=baseline_mode,
|
||||
baseline_start=baseline_start, baseline_end=baseline_end,
|
||||
)
|
||||
|
||||
html = render_html_summary(report)
|
||||
subject = f"{report.project_name} — night report {night_date:%m/%d/%y}"
|
||||
|
||||
# --- Always persist a viewable copy ---
|
||||
out_dir = Path(output_root) / project_id / f"{night_date:%Y-%m-%d}"
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
html_path = out_dir / "report.html"
|
||||
html_path.write_text(html, encoding="utf-8")
|
||||
json_path = out_dir / "report.json"
|
||||
json_path.write_text(json.dumps(_report_to_dict(report), indent=2), encoding="utf-8")
|
||||
|
||||
# --- Excel (the email attachment; also written to disk for the archive) ---
|
||||
attachments: list[Attachment] = []
|
||||
xlsx_path = None
|
||||
try:
|
||||
xlsx_bytes = render_excel(report)
|
||||
xlsx_path = out_dir / "report.xlsx"
|
||||
xlsx_path.write_bytes(xlsx_bytes)
|
||||
safe_name = "".join(c for c in report.project_name if c.isalnum() or c in " -_").strip().replace(" ", "_")
|
||||
attachments.append(Attachment(
|
||||
f"{safe_name or 'report'}_{night_date:%Y-%m-%d}_night_report.xlsx",
|
||||
xlsx_bytes, *XLSX_MIME,
|
||||
))
|
||||
except Exception as e: # noqa: BLE001 — never let the spreadsheet sink the report
|
||||
logger.error("Excel render failed for %s (%s): %s", project_id, night_date, e, exc_info=True)
|
||||
|
||||
# --- Email (best-effort; dry-run until SMTP is configured) ---
|
||||
email_result = {"sent": False, "dry_run": False, "skipped": True, "error": None}
|
||||
if send:
|
||||
try:
|
||||
email_result = send_report_email(
|
||||
subject, html, attachments=attachments, recipients=recipients,
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — artifacts are already written; never abort on email
|
||||
logger.error("send_report_email raised for %s (%s): %s", project_id, night_date, e, exc_info=True)
|
||||
email_result = {"sent": False, "dry_run": False, "skipped": False, "error": str(e)}
|
||||
|
||||
result = {
|
||||
"project_id": project_id,
|
||||
"project_name": report.project_name,
|
||||
"night_date": night_date.isoformat(),
|
||||
"subject": subject,
|
||||
"location_count": len(report.locations),
|
||||
"html_path": str(html_path),
|
||||
"json_path": str(json_path),
|
||||
"xlsx_path": str(xlsx_path) if xlsx_path else None,
|
||||
"html": html, # for callers that want to display it inline
|
||||
"email": email_result,
|
||||
}
|
||||
logger.info(
|
||||
"Nightly report for %s (%s): %d location(s) → %s; email=%s",
|
||||
report.project_name, night_date, len(report.locations), html_path,
|
||||
"sent" if email_result.get("sent") else
|
||||
("dry-run" if email_result.get("dry_run") else
|
||||
("skipped" if email_result.get("skipped") else f"error: {email_result.get('error')}")),
|
||||
)
|
||||
return result
|
||||
@@ -1,432 +0,0 @@
|
||||
"""
|
||||
Nightly Report Pipeline — computation core.
|
||||
|
||||
Builds the data model for the John-Myler-style "last night vs. baseline" sound
|
||||
report. Source-agnostic: it reads the same on-disk Leq `.rnd` files the manual
|
||||
upload + FTP-pull ingest produce (see `project_locations.ingest_nrl_zip`).
|
||||
|
||||
Design notes
|
||||
------------
|
||||
* **Ingest everything, report selectively.** Ingest preserves every column of
|
||||
the Leq file; this layer chooses which *metrics* to surface via `metric_keys`
|
||||
(a future report wizard is just a UI over that list).
|
||||
* **House format match.** Defaults reproduce the existing Excel report:
|
||||
LAmax (max of interval maxima), LA01 / LA10 (arithmetic average), split into
|
||||
Evening (7–10PM) and Nighttime (10PM–7AM) windows. L90 (background) is added
|
||||
for the baseline comparison.
|
||||
* **Metric labelling from the device.** The LN→percentile assignment is
|
||||
reconfigurable per job; we resolve which `LNx(Main)` column is L90/L10/etc.
|
||||
from the percentile map captured in the session metadata at ingest, falling
|
||||
back to the NL-43 default order.
|
||||
* **Correct averaging.** Leq is energy-averaged (logarithmic); percentiles and
|
||||
Lmax are arithmetic. Baseline references combine the per-night values into a
|
||||
"typical night" (arithmetic mean of per-night values — so baseline Lmax is the
|
||||
typical nightly peak, not the worst-of-week).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timedelta, date
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from backend.models import MonitoringSession, DataFile, MonitoringLocation, Project
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Metric registry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Metric:
|
||||
"""A reportable metric.
|
||||
|
||||
`agg` is the *within-night* aggregation used to collapse a window's 15-min
|
||||
intervals into one value:
|
||||
- "max" → loudest interval (LAmax)
|
||||
- "arith" → arithmetic mean (percentiles: L01/L10/L90…)
|
||||
- "log" → energy/logarithmic mean (Leq only)
|
||||
`column` pins a fixed .rnd column; `percentile` instead resolves the LNx
|
||||
column from the session's captured percentile map.
|
||||
"""
|
||||
key: str
|
||||
label: str
|
||||
agg: str
|
||||
column: Optional[str] = None
|
||||
percentile: Optional[float] = None
|
||||
|
||||
|
||||
METRIC_REGISTRY: dict[str, Metric] = {
|
||||
"lmax": Metric("lmax", "LAmax", "max", column="Lmax(Main)"),
|
||||
"leq": Metric("leq", "LAeq", "log", column="Leq(Main)"),
|
||||
"lmin": Metric("lmin", "LAmin", "arith", column="Lmin(Main)"),
|
||||
"l01": Metric("l01", "LA01", "arith", percentile=1.0),
|
||||
"l10": Metric("l10", "LA10", "arith", percentile=10.0),
|
||||
"l50": Metric("l50", "LA50", "arith", percentile=50.0),
|
||||
"l90": Metric("l90", "LA90", "arith", percentile=90.0),
|
||||
"l95": Metric("l95", "LA95", "arith", percentile=95.0),
|
||||
}
|
||||
|
||||
# House report metrics + L90 (background) for the baseline comparison.
|
||||
DEFAULT_METRICS: list[str] = ["lmax", "l01", "l10", "l90"]
|
||||
|
||||
# NL-43 default percentile→slot assignment, used when a session has no captured map.
|
||||
_DEFAULT_SLOT_FOR_PCT: dict[float, int] = {1.0: 1, 10.0: 2, 50.0: 3, 90.0: 4, 95.0: 5}
|
||||
|
||||
|
||||
def _resolve_column(metric: Metric, pct_map: dict) -> Optional[str]:
|
||||
"""Resolve the .rnd column for a metric, using the session's percentile map."""
|
||||
if metric.column:
|
||||
return metric.column
|
||||
if metric.percentile is None:
|
||||
return None
|
||||
# pct_map: {"1": "1.0", "2": "10.0", "4": "90.0", ...} → slot : percentile
|
||||
if pct_map:
|
||||
for slot, pval in pct_map.items():
|
||||
try:
|
||||
if float(pval) == metric.percentile:
|
||||
return f"LN{int(slot)}(Main)"
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
slot = _DEFAULT_SLOT_FOR_PCT.get(metric.percentile)
|
||||
return f"LN{slot}(Main)" if slot else None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Time windows
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Window:
|
||||
key: str
|
||||
label: str
|
||||
start_hour: int
|
||||
end_hour: int
|
||||
|
||||
def contains(self, hour: int) -> bool:
|
||||
if self.start_hour < self.end_hour:
|
||||
return self.start_hour <= hour < self.end_hour
|
||||
return hour >= self.start_hour or hour < self.end_hour
|
||||
|
||||
|
||||
# Matches the existing Excel report's stats table.
|
||||
DEFAULT_WINDOWS: list[Window] = [
|
||||
Window("evening", "Evening (7PM–10PM)", 19, 22),
|
||||
Window("nighttime", "Nighttime (10PM–7AM)", 22, 7),
|
||||
]
|
||||
|
||||
# The full night used to select which intervals belong to "last night".
|
||||
NIGHT_START_HOUR = 19
|
||||
NIGHT_LENGTH_HOURS = 12
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Aggregation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _aggregate(values: list, method: str) -> Optional[float]:
|
||||
"""Collapse a window's interval values into one number per `method`."""
|
||||
vals = [v for v in values if isinstance(v, (int, float))]
|
||||
if not vals:
|
||||
return None
|
||||
if method == "max":
|
||||
return round(max(vals), 1)
|
||||
if method == "log":
|
||||
return round(10 * math.log10(sum(10 ** (v / 10.0) for v in vals) / len(vals)), 1)
|
||||
return round(sum(vals) / len(vals), 1) # arithmetic
|
||||
|
||||
|
||||
def _combine_across_nights(per_night: list, method: str) -> Optional[float]:
|
||||
"""Combine per-night window values into a baseline 'typical night' value.
|
||||
|
||||
Arithmetic mean for max/arith metrics (so baseline Lmax = typical nightly
|
||||
peak, the agreed default), logarithmic mean for Leq.
|
||||
"""
|
||||
vals = [v for v in per_night if v is not None]
|
||||
if not vals:
|
||||
return None
|
||||
if method == "log":
|
||||
return round(10 * math.log10(sum(10 ** (v / 10.0) for v in vals) / len(vals)), 1)
|
||||
return round(sum(vals) / len(vals), 1)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Row gathering
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_dt(s: str) -> Optional[datetime]:
|
||||
try:
|
||||
return datetime.strptime(s, "%Y/%m/%d %H:%M:%S")
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def _location_leq_rows(db: Session, location_id: str) -> list[tuple[datetime, dict, dict]]:
|
||||
"""All Leq intervals at a location as (interval_dt, row, percentile_map).
|
||||
|
||||
Reuses the same .rnd readers as the report endpoints so parsing stays
|
||||
identical. Times are the meter's local clock (as written in the file).
|
||||
"""
|
||||
# Lazy import avoids a service→router import cycle at module load.
|
||||
from backend.routers.projects import (
|
||||
_read_rnd_file_rows, _normalize_rnd_rows, _is_leq_file, _peek_rnd_headers,
|
||||
)
|
||||
from pathlib import Path
|
||||
|
||||
out: list[tuple[datetime, dict, dict]] = []
|
||||
sessions = db.query(MonitoringSession).filter_by(
|
||||
location_id=location_id, session_type="sound",
|
||||
).all()
|
||||
for s in sessions:
|
||||
try:
|
||||
meta = json.loads(s.session_metadata or "{}")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
meta = {}
|
||||
pct_map = meta.get("percentiles", {}) or {}
|
||||
for f in db.query(DataFile).filter_by(session_id=s.id).all():
|
||||
if not f.file_path or not f.file_path.lower().endswith(".rnd"):
|
||||
continue
|
||||
peek = _peek_rnd_headers(Path("data") / f.file_path)
|
||||
if not _is_leq_file(f.file_path, peek):
|
||||
continue
|
||||
rows = _read_rnd_file_rows(f.file_path)
|
||||
rows, _ = _normalize_rnd_rows(rows)
|
||||
for r in rows:
|
||||
dt = _parse_dt(r.get("Start Time", ""))
|
||||
if dt:
|
||||
out.append((dt, r, pct_map))
|
||||
out.sort(key=lambda t: t[0])
|
||||
return out
|
||||
|
||||
|
||||
def _rows_in_night(rows: list, night_date: date) -> list:
|
||||
"""Rows falling in the night that *starts* on night_date (19:00 → +12h)."""
|
||||
start = datetime(night_date.year, night_date.month, night_date.day, NIGHT_START_HOUR, 0)
|
||||
end = start + timedelta(hours=NIGHT_LENGTH_HOURS)
|
||||
return [(dt, r, p) for (dt, r, p) in rows if start <= dt < end]
|
||||
|
||||
|
||||
def _eligible_nights(rows: list, start_date: date, end_date: date) -> list[date]:
|
||||
"""Evening-dates in [start_date, end_date] that actually have night data."""
|
||||
nights = []
|
||||
cur = start_date
|
||||
while cur <= end_date:
|
||||
if _rows_in_night(rows, cur):
|
||||
nights.append(cur)
|
||||
cur += timedelta(days=1)
|
||||
return nights
|
||||
|
||||
|
||||
def _window_value(rows: list, metric: Metric, window: Window) -> Optional[float]:
|
||||
"""Single aggregated value for one metric over one window of `rows`."""
|
||||
vals = []
|
||||
for dt, r, pct_map in rows:
|
||||
if window.contains(dt.hour):
|
||||
col = _resolve_column(metric, pct_map)
|
||||
if col:
|
||||
vals.append(r.get(col))
|
||||
return _aggregate(vals, metric.agg)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Report data model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class CellPair:
|
||||
last_night: Optional[float]
|
||||
baseline: Optional[float]
|
||||
|
||||
@property
|
||||
def delta(self) -> Optional[float]:
|
||||
if self.last_night is None or self.baseline is None:
|
||||
return None
|
||||
return round(self.last_night - self.baseline, 1)
|
||||
|
||||
|
||||
@dataclass
|
||||
class LocationNightReport:
|
||||
location_id: str
|
||||
location_name: str
|
||||
night_date: date
|
||||
metrics: list[Metric]
|
||||
windows: list[Window]
|
||||
# table[window_key][metric_key] = CellPair
|
||||
table: dict[str, dict[str, CellPair]]
|
||||
interval_series: list[dict]
|
||||
night_interval_count: int
|
||||
baseline_nights_used: int
|
||||
notes: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
def _location_reference_baseline(loc) -> dict:
|
||||
"""A location's manually-entered reference baseline, from its metadata.
|
||||
|
||||
Shape: {window_key: {metric_key: float}} e.g. {"nighttime": {"l10": 85.0}}.
|
||||
Used when baseline_mode == "reference" — fixed targets/limits or prior-report
|
||||
averages typed in, rather than computed from captured nights.
|
||||
"""
|
||||
if not loc:
|
||||
return {}
|
||||
try:
|
||||
meta = json.loads(loc.location_metadata or "{}")
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return {}
|
||||
ref = meta.get("report_baseline") or {}
|
||||
out: dict[str, dict[str, float]] = {}
|
||||
if isinstance(ref, dict):
|
||||
for wkey, mvals in ref.items():
|
||||
if not isinstance(mvals, dict):
|
||||
continue
|
||||
clean = {}
|
||||
for mkey, val in mvals.items():
|
||||
try:
|
||||
clean[mkey] = float(val)
|
||||
except (ValueError, TypeError):
|
||||
continue
|
||||
if clean:
|
||||
out[wkey] = clean
|
||||
return out
|
||||
|
||||
|
||||
def build_location_night_report(
|
||||
db: Session,
|
||||
location_id: str,
|
||||
night_date: date,
|
||||
*,
|
||||
metric_keys: Optional[list[str]] = None,
|
||||
windows: Optional[list[Window]] = None,
|
||||
baseline_mode: str = "captured",
|
||||
baseline_start: Optional[date] = None,
|
||||
baseline_end: Optional[date] = None,
|
||||
) -> LocationNightReport:
|
||||
"""Build the night-vs-baseline data model for one location.
|
||||
|
||||
`night_date` is the *evening* date of the night being reported (e.g. the
|
||||
7/7 in "night of 7/7 → morning 7/8"). Baseline comes from one of:
|
||||
- "captured": the typical-night value across eligible nights in
|
||||
[baseline_start, baseline_end] (computed from recorded data);
|
||||
- "reference": fixed values typed per location (a spec limit like
|
||||
"L10 = 85", or a prior report's averages).
|
||||
"""
|
||||
metric_keys = metric_keys or DEFAULT_METRICS
|
||||
metrics = [METRIC_REGISTRY[k] for k in metric_keys]
|
||||
windows = windows or DEFAULT_WINDOWS
|
||||
|
||||
loc = db.query(MonitoringLocation).filter_by(id=location_id).first()
|
||||
loc_name = loc.name if loc else location_id
|
||||
|
||||
all_rows = _location_leq_rows(db, location_id)
|
||||
night_rows = _rows_in_night(all_rows, night_date)
|
||||
|
||||
reference = _location_reference_baseline(loc) if baseline_mode == "reference" else {}
|
||||
|
||||
baseline_nights: list[date] = []
|
||||
if baseline_mode != "reference" and baseline_start and baseline_end:
|
||||
baseline_nights = _eligible_nights(all_rows, baseline_start, baseline_end)
|
||||
# Don't let the reported night double as its own baseline.
|
||||
baseline_nights = [n for n in baseline_nights if n != night_date]
|
||||
|
||||
table: dict[str, dict[str, CellPair]] = {}
|
||||
for w in windows:
|
||||
table[w.key] = {}
|
||||
for m in metrics:
|
||||
last_night_val = _window_value(night_rows, m, w)
|
||||
if baseline_mode == "reference":
|
||||
baseline_val = reference.get(w.key, {}).get(m.key)
|
||||
elif baseline_nights:
|
||||
per_night = [
|
||||
_window_value(_rows_in_night(all_rows, nd), m, w)
|
||||
for nd in baseline_nights
|
||||
]
|
||||
baseline_val = _combine_across_nights(per_night, m.agg)
|
||||
else:
|
||||
baseline_val = None
|
||||
table[w.key][m.key] = CellPair(last_night_val, baseline_val)
|
||||
|
||||
interval_series = []
|
||||
for dt, r, pct_map in night_rows:
|
||||
entry = {"dt": dt, "time": dt.strftime("%H:%M")}
|
||||
for m in metrics:
|
||||
col = _resolve_column(m, pct_map)
|
||||
val = r.get(col) if col else None
|
||||
entry[m.key] = val if isinstance(val, (int, float)) else None
|
||||
interval_series.append(entry)
|
||||
|
||||
notes: list[str] = []
|
||||
if not night_rows:
|
||||
notes.append(f"No data found for the night of {night_date:%m/%d/%y}.")
|
||||
if baseline_mode == "reference":
|
||||
if not any(reference.values()):
|
||||
notes.append("Reference-baseline mode is on but no reference values are set for this location.")
|
||||
elif (baseline_start or baseline_end) and not baseline_nights:
|
||||
notes.append("No baseline nights with data in the configured range.")
|
||||
|
||||
return LocationNightReport(
|
||||
location_id=location_id,
|
||||
location_name=loc_name,
|
||||
night_date=night_date,
|
||||
metrics=metrics,
|
||||
windows=windows,
|
||||
table=table,
|
||||
interval_series=interval_series,
|
||||
night_interval_count=len(night_rows),
|
||||
baseline_nights_used=len(baseline_nights),
|
||||
notes=notes,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProjectNightReport:
|
||||
project_id: str
|
||||
project_name: str
|
||||
night_date: date
|
||||
metrics: list[Metric]
|
||||
locations: list[LocationNightReport]
|
||||
|
||||
|
||||
def build_project_night_report(
|
||||
db: Session,
|
||||
project_id: str,
|
||||
night_date: date,
|
||||
*,
|
||||
metric_keys: Optional[list[str]] = None,
|
||||
windows: Optional[list[Window]] = None,
|
||||
baseline_mode: str = "captured",
|
||||
baseline_start: Optional[date] = None,
|
||||
baseline_end: Optional[date] = None,
|
||||
) -> ProjectNightReport:
|
||||
"""Build the night report for every active sound location in a project."""
|
||||
metric_keys = metric_keys or DEFAULT_METRICS
|
||||
project = db.query(Project).filter_by(id=project_id).first()
|
||||
project_name = project.name if project else project_id
|
||||
|
||||
locations = db.query(MonitoringLocation).filter_by(
|
||||
project_id=project_id, location_type="sound",
|
||||
).order_by(MonitoringLocation.sort_order, MonitoringLocation.name).all()
|
||||
locations = [l for l in locations if getattr(l, "removed_at", None) is None]
|
||||
|
||||
reports = [
|
||||
build_location_night_report(
|
||||
db, loc.id, night_date,
|
||||
metric_keys=metric_keys, windows=windows,
|
||||
baseline_mode=baseline_mode,
|
||||
baseline_start=baseline_start, baseline_end=baseline_end,
|
||||
)
|
||||
for loc in locations
|
||||
]
|
||||
|
||||
return ProjectNightReport(
|
||||
project_id=project_id,
|
||||
project_name=project_name,
|
||||
night_date=night_date,
|
||||
metrics=[METRIC_REGISTRY[k] for k in metric_keys],
|
||||
locations=reports,
|
||||
)
|
||||
@@ -1,240 +0,0 @@
|
||||
"""
|
||||
Nightly Report Renderers.
|
||||
|
||||
Pluggable renderers over the `report_pipeline` data model. v1 ships the HTML
|
||||
email body + the Excel attachment; PDF and an inline chart image are v1.1
|
||||
(each needs a new dependency). Keeping renderers separate from the compute
|
||||
core means a future report wizard just toggles metrics/renderers — the data
|
||||
model is unchanged.
|
||||
|
||||
Email-client constraints: the HTML uses a table layout with **inline styles
|
||||
only** (no <style> blocks, no external CSS, no fl/grid), which is the reliable
|
||||
common denominator across Outlook / Gmail / Apple Mail.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from html import escape
|
||||
|
||||
from backend.services.report_pipeline import ProjectNightReport, LocationNightReport
|
||||
|
||||
|
||||
# Colours: louder-than-baseline reads as a concern (red), quieter as fine (green).
|
||||
_RED = "#b00020"
|
||||
_GREEN = "#1a7f37"
|
||||
_GREY = "#888888"
|
||||
|
||||
|
||||
def _fmt_value(v) -> str:
|
||||
return f"{v:.1f}" if isinstance(v, (int, float)) else "—"
|
||||
|
||||
|
||||
def _fmt_delta(v) -> str:
|
||||
"""Signed delta with colour; positive (louder) = red, negative (quieter) = green."""
|
||||
if not isinstance(v, (int, float)):
|
||||
return f'<span style="color:{_GREY}">—</span>'
|
||||
if v > 0:
|
||||
return f'<span style="color:{_RED}">+{v:.1f}</span>'
|
||||
if v < 0:
|
||||
return f'<span style="color:{_GREEN}">{v:.1f}</span>'
|
||||
return f'<span style="color:{_GREY}">0.0</span>'
|
||||
|
||||
|
||||
def _location_table(loc: LocationNightReport) -> str:
|
||||
"""One location block: heading + Metric × (window: Last / Base / Δ) table."""
|
||||
th = ('padding:5px 9px;border:1px solid #ccc;background:#f2f2f2;'
|
||||
'font:bold 12px Arial,sans-serif;text-align:center')
|
||||
sub = ('padding:4px 8px;border:1px solid #ccc;background:#fafafa;'
|
||||
'font:11px Arial,sans-serif;text-align:center;color:#555')
|
||||
td = 'padding:4px 9px;border:1px solid #ccc;font:12px Arial,sans-serif;text-align:center'
|
||||
td_l = 'padding:4px 9px;border:1px solid #ccc;font:bold 12px Arial,sans-serif;text-align:left'
|
||||
|
||||
# Top header: blank label cell + each window spanning Last/Base/Δ
|
||||
top = f'<th rowspan="2" style="{th}">Metric (dBA)</th>'
|
||||
for w in loc.windows:
|
||||
top += f'<th colspan="3" style="{th}">{escape(w.label)}</th>'
|
||||
sub_row = ''.join(
|
||||
f'<th style="{sub}">Last</th><th style="{sub}">Base</th><th style="{sub}">Δ</th>'
|
||||
for _ in loc.windows
|
||||
)
|
||||
|
||||
body = ''
|
||||
for m in loc.metrics:
|
||||
cells = ''
|
||||
for w in loc.windows:
|
||||
cp = loc.table[w.key][m.key]
|
||||
cells += (f'<td style="{td}">{_fmt_value(cp.last_night)}</td>'
|
||||
f'<td style="{td}">{_fmt_value(cp.baseline)}</td>'
|
||||
f'<td style="{td}">{_fmt_delta(cp.delta)}</td>')
|
||||
body += f'<tr><td style="{td_l}">{escape(m.label)}</td>{cells}</tr>'
|
||||
|
||||
meta = (f'{loc.night_interval_count} intervals'
|
||||
+ (f' · baseline = {loc.baseline_nights_used} night(s)'
|
||||
if loc.baseline_nights_used else ' · no baseline yet'))
|
||||
notes = ''
|
||||
if loc.notes:
|
||||
notes = ('<div style="font:11px Arial,sans-serif;color:#b00020;margin:2px 0 0">'
|
||||
+ '<br>'.join(escape(n) for n in loc.notes) + '</div>')
|
||||
|
||||
return (
|
||||
f'<h3 style="font:bold 15px Arial,sans-serif;margin:18px 0 4px">{escape(loc.location_name)}</h3>'
|
||||
f'<div style="font:11px Arial,sans-serif;color:#666;margin:0 0 6px">{escape(meta)}</div>'
|
||||
f'<table style="border-collapse:collapse;border:1px solid #ccc">'
|
||||
f'<thead><tr>{top}</tr><tr>{sub_row}</tr></thead>'
|
||||
f'<tbody>{body}</tbody></table>{notes}'
|
||||
)
|
||||
|
||||
|
||||
def render_html_summary(report: ProjectNightReport) -> str:
|
||||
"""Render the full email-body HTML for a project's night report."""
|
||||
windows_desc = ", ".join(w.label for w in (report.locations[0].windows if report.locations else []))
|
||||
header = (
|
||||
f'<h2 style="font:bold 18px Arial,sans-serif;margin:0 0 2px">'
|
||||
f'{escape(report.project_name)} — Night Report</h2>'
|
||||
f'<div style="font:13px Arial,sans-serif;color:#444;margin:0 0 4px">'
|
||||
f'Night of {report.night_date:%a %m/%d/%y} · last night vs. baseline</div>'
|
||||
f'<div style="font:11px Arial,sans-serif;color:#888;margin:0 0 10px">'
|
||||
f'Windows: {escape(windows_desc)}. '
|
||||
f'Δ = last night minus baseline (<span style="color:{_RED}">+ louder</span>, '
|
||||
f'<span style="color:{_GREEN}">− quieter</span>). '
|
||||
f'LAmax = loudest interval; L-values are arithmetic averages; '
|
||||
f'baseline = typical night.</div>'
|
||||
)
|
||||
|
||||
if not report.locations:
|
||||
body = ('<div style="font:13px Arial,sans-serif;color:#b00020">'
|
||||
'No sound locations found for this project.</div>')
|
||||
else:
|
||||
body = ''.join(_location_table(loc) for loc in report.locations)
|
||||
|
||||
footer = ('<div style="font:10px Arial,sans-serif;color:#aaa;margin-top:18px">'
|
||||
'Automated report — Terra-View. Full interval data in the attached spreadsheet.</div>')
|
||||
|
||||
return (f'<!DOCTYPE html><html><body style="margin:0;padding:16px;background:#fff">'
|
||||
f'{header}{body}{footer}</body></html>')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Excel renderer (the email attachment) — one sheet per location:
|
||||
# interval table + line chart + a Last/Baseline/Δ summary per window.
|
||||
# Metric-driven, so it adapts to whatever metric set is configured.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _safe_sheet_name(name: str) -> str:
|
||||
bad = set('[]:*?/\\')
|
||||
cleaned = "".join(c for c in (name or "Location") if c not in bad).strip()
|
||||
return (cleaned or "Location")[:31]
|
||||
|
||||
|
||||
def render_excel(report: ProjectNightReport) -> bytes:
|
||||
"""Render the night report as an .xlsx (bytes). One worksheet per location."""
|
||||
import io as _io
|
||||
import openpyxl
|
||||
from openpyxl.chart import LineChart, Reference
|
||||
from openpyxl.styles import Font, Alignment, PatternFill, Border, Side
|
||||
from openpyxl.utils import get_column_letter
|
||||
|
||||
wb = openpyxl.Workbook()
|
||||
wb.remove(wb.active)
|
||||
|
||||
f_title = Font(name="Arial", bold=True, size=13)
|
||||
f_h = Font(name="Arial", bold=True, size=10)
|
||||
f_d = Font(name="Arial", size=10)
|
||||
f_note = Font(name="Arial", size=9, italic=True, color="888888")
|
||||
center = Alignment(horizontal="center", vertical="center")
|
||||
hdr_fill = PatternFill("solid", fgColor="F2F2F2")
|
||||
thin = Side(style="thin")
|
||||
box = Border(left=thin, right=thin, top=thin, bottom=thin)
|
||||
|
||||
if not report.locations:
|
||||
ws = wb.create_sheet("No data")
|
||||
ws["A1"] = f"{report.project_name} — no sound locations"
|
||||
ws["A1"].font = f_title
|
||||
|
||||
used_names: set = set()
|
||||
for loc in report.locations:
|
||||
sheet_name = _safe_sheet_name(loc.location_name)
|
||||
n, base = sheet_name, sheet_name
|
||||
i = 2
|
||||
while n in used_names:
|
||||
n = (base[:28] + f"_{i}"); i += 1
|
||||
used_names.add(n)
|
||||
ws = wb.create_sheet(n)
|
||||
metrics = loc.metrics
|
||||
|
||||
ws["A1"] = f"{report.project_name} — Night Report"; ws["A1"].font = f_title
|
||||
ws["A2"] = loc.location_name; ws["A2"].font = f_h
|
||||
ws["A3"] = f"Night of {loc.night_date:%m/%d/%y} · 7PM–7AM"; ws["A3"].font = f_d
|
||||
|
||||
# --- interval table ---
|
||||
hr = 5
|
||||
cols = ["Interval #", "Date", "Time"] + [m.label for m in metrics] + ["Comments"]
|
||||
for ci, label in enumerate(cols, 1):
|
||||
c = ws.cell(row=hr, column=ci, value=label)
|
||||
c.font = f_h; c.alignment = center; c.fill = hdr_fill; c.border = box
|
||||
r = hr + 1
|
||||
for idx, entry in enumerate(loc.interval_series, 1):
|
||||
ws.cell(row=r, column=1, value=idx).border = box
|
||||
dt = entry.get("dt")
|
||||
ws.cell(row=r, column=2, value=(dt.strftime("%m/%d/%y") if dt else "")).border = box
|
||||
ws.cell(row=r, column=3, value=entry.get("time", "")).border = box
|
||||
for mi, m in enumerate(metrics):
|
||||
v = entry.get(m.key)
|
||||
cc = ws.cell(row=r, column=4 + mi, value=(v if isinstance(v, (int, float)) else None))
|
||||
cc.border = box; cc.alignment = center
|
||||
ws.cell(row=r, column=4 + len(metrics), value="").border = box
|
||||
r += 1
|
||||
data_end = max(r - 1, hr + 1)
|
||||
|
||||
ws.column_dimensions["A"].width = 9
|
||||
ws.column_dimensions["B"].width = 10
|
||||
ws.column_dimensions["C"].width = 8
|
||||
for mi in range(len(metrics)):
|
||||
ws.column_dimensions[get_column_letter(4 + mi)].width = 11
|
||||
ws.column_dimensions[get_column_letter(4 + len(metrics))].width = 22
|
||||
|
||||
# --- chart ---
|
||||
if loc.interval_series and metrics:
|
||||
chart = LineChart()
|
||||
chart.title = f"{loc.location_name} — {loc.night_date:%m/%d/%y}"
|
||||
chart.y_axis.title = "dBA"; chart.x_axis.title = "Time"
|
||||
chart.height = 9; chart.width = 18
|
||||
data_ref = Reference(ws, min_col=4, max_col=3 + len(metrics), min_row=hr, max_row=data_end)
|
||||
cats = Reference(ws, min_col=3, min_row=hr + 1, max_row=data_end)
|
||||
chart.add_data(data_ref, titles_from_data=True)
|
||||
chart.set_categories(cats)
|
||||
ws.add_chart(chart, f"{get_column_letter(6 + len(metrics))}5")
|
||||
|
||||
# --- summary: Metric × window (Last / Base / Δ) ---
|
||||
sr = data_end + 3
|
||||
ws.cell(row=sr, column=1, value="Summary — last night vs baseline").font = f_h
|
||||
sr += 1
|
||||
ws.cell(row=sr, column=1, value="Metric").font = f_h
|
||||
win_col = {}
|
||||
col = 2
|
||||
for w in loc.windows:
|
||||
c = ws.cell(row=sr, column=col, value=w.label); c.font = f_h; c.alignment = center
|
||||
ws.merge_cells(start_row=sr, start_column=col, end_row=sr, end_column=col + 2)
|
||||
win_col[w.key] = col
|
||||
col += 3
|
||||
sr += 1
|
||||
for w in loc.windows:
|
||||
b = win_col[w.key]
|
||||
for j, lbl in enumerate(["Last", "Base", "Δ"]):
|
||||
cc = ws.cell(row=sr, column=b + j, value=lbl); cc.font = f_h; cc.alignment = center
|
||||
sr += 1
|
||||
for m in metrics:
|
||||
ws.cell(row=sr, column=1, value=m.label).font = f_d
|
||||
for w in loc.windows:
|
||||
cp = loc.table[w.key][m.key]
|
||||
b = win_col[w.key]
|
||||
ws.cell(row=sr, column=b + 0, value=cp.last_night).alignment = center
|
||||
ws.cell(row=sr, column=b + 1, value=cp.baseline).alignment = center
|
||||
ws.cell(row=sr, column=b + 2, value=cp.delta).alignment = center
|
||||
sr += 1
|
||||
if loc.notes:
|
||||
ws.cell(row=sr + 1, column=1, value="; ".join(loc.notes)).font = f_note
|
||||
|
||||
out = _io.BytesIO()
|
||||
wb.save(out)
|
||||
return out.getvalue()
|
||||
+79
-304
@@ -78,9 +78,6 @@ class SchedulerService:
|
||||
# Execute pending actions
|
||||
await self.execute_pending_actions()
|
||||
|
||||
# Run any due nightly sound reports (FTP report pipeline)
|
||||
await self.run_due_reports()
|
||||
|
||||
# Generate actions from recurring schedules (every hour)
|
||||
now = datetime.utcnow()
|
||||
if (now - last_generation_check).total_seconds() >= 3600:
|
||||
@@ -309,11 +306,18 @@ class SchedulerService:
|
||||
2. Enable FTP
|
||||
3. Download measurement folder to SLMM local storage
|
||||
|
||||
After stop_cycle, if download succeeded, this method ingests the folder
|
||||
into Terra-View through the shared NRL ingest (same path as cycle and the
|
||||
manual SD-card upload) so the resulting session is Leq-only, has its
|
||||
`.rnh` parsed (percentile slot map + weightings), and is deduped.
|
||||
After stop_cycle, if download succeeded, this method fetches the ZIP
|
||||
from SLMM and extracts it into Terra-View's project directory, creating
|
||||
DataFile records for each file.
|
||||
"""
|
||||
import hashlib
|
||||
import io
|
||||
import os
|
||||
import zipfile
|
||||
import httpx
|
||||
from pathlib import Path
|
||||
from backend.models import DataFile
|
||||
|
||||
# Parse notes for download preference
|
||||
include_download = True
|
||||
try:
|
||||
@@ -358,39 +362,79 @@ class SchedulerService:
|
||||
|
||||
db.commit()
|
||||
|
||||
# If SLMM downloaded the folder successfully, ingest it into Terra-View
|
||||
# through the shared NRL ingest (the same path cycle and the manual SD
|
||||
# upload use): keeps only the .rnh + Leq .rnd, parses the header
|
||||
# (percentile slot map + weightings), dedups, and links the unit. The
|
||||
# transient "recording" marker session is dropped in favour of the clean
|
||||
# ingested row. (Replaces the old inline unzip that stored every file —
|
||||
# incl. the 1-second _Lp_ data — without parsing the .rnh.)
|
||||
ingest_result = None
|
||||
ingested_session_id = None
|
||||
if (include_download and cycle_response.get("download_success")
|
||||
and active_session and action.device_type == "slm"):
|
||||
# If SLMM downloaded the folder successfully, fetch the ZIP from SLMM
|
||||
# and extract it into Terra-View's project directory, creating DataFile records
|
||||
files_created = 0
|
||||
if include_download and cycle_response.get("download_success") and active_session:
|
||||
folder_name = cycle_response.get("downloaded_folder") # e.g. "Auto_0058"
|
||||
if folder_name:
|
||||
try:
|
||||
ingest_result = await self._ingest_and_link(
|
||||
db,
|
||||
location_id=action.location_id,
|
||||
unit_id=unit_id,
|
||||
folder_name=folder_name,
|
||||
placeholder_session=active_session,
|
||||
remote_path = f"/NL-43/{folder_name}"
|
||||
|
||||
try:
|
||||
SLMM_BASE_URL = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
async with httpx.AsyncClient(timeout=600.0) as client:
|
||||
zip_response = await client.post(
|
||||
f"{SLMM_BASE_URL}/api/nl43/{unit_id}/ftp/download-folder",
|
||||
json={"remote_path": remote_path}
|
||||
)
|
||||
ingested_session_id = ingest_result.get("session_id")
|
||||
logger.info(f"[STOP] Ingested {folder_name}: {ingest_result}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to ingest {folder_name} on stop: {e}", exc_info=True)
|
||||
# Don't fail the stop action — the device was stopped successfully
|
||||
ingest_result = {"success": False, "error": str(e)}
|
||||
|
||||
if zip_response.is_success and len(zip_response.content) > 22:
|
||||
base_dir = Path(f"data/Projects/{action.project_id}/{active_session.id}/{folder_name}")
|
||||
base_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
file_type_map = {
|
||||
'.wav': 'audio', '.mp3': 'audio',
|
||||
'.csv': 'data', '.txt': 'data', '.json': 'data', '.dat': 'data',
|
||||
'.rnd': 'data', '.rnh': 'data',
|
||||
'.log': 'log',
|
||||
'.zip': 'archive',
|
||||
'.jpg': 'image', '.jpeg': 'image', '.png': 'image',
|
||||
'.pdf': 'document',
|
||||
}
|
||||
|
||||
with zipfile.ZipFile(io.BytesIO(zip_response.content)) as zf:
|
||||
for zip_info in zf.filelist:
|
||||
if zip_info.is_dir():
|
||||
continue
|
||||
file_data = zf.read(zip_info.filename)
|
||||
file_path = base_dir / zip_info.filename
|
||||
file_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(file_path, 'wb') as f:
|
||||
f.write(file_data)
|
||||
checksum = hashlib.sha256(file_data).hexdigest()
|
||||
ext = os.path.splitext(zip_info.filename)[1].lower()
|
||||
data_file = DataFile(
|
||||
id=str(uuid.uuid4()),
|
||||
session_id=active_session.id,
|
||||
file_path=str(file_path.relative_to("data")),
|
||||
file_type=file_type_map.get(ext, 'data'),
|
||||
file_size_bytes=len(file_data),
|
||||
downloaded_at=datetime.utcnow(),
|
||||
checksum=checksum,
|
||||
file_metadata=json.dumps({
|
||||
"source": "stop_cycle",
|
||||
"remote_path": remote_path,
|
||||
"unit_id": unit_id,
|
||||
"folder_name": folder_name,
|
||||
"relative_path": zip_info.filename,
|
||||
}),
|
||||
)
|
||||
db.add(data_file)
|
||||
files_created += 1
|
||||
|
||||
db.commit()
|
||||
logger.info(f"Created {files_created} DataFile records for session {active_session.id} from {folder_name}")
|
||||
else:
|
||||
logger.warning(f"ZIP from SLMM for {folder_name} was empty or failed, skipping DataFile creation")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to extract ZIP and create DataFile records for {folder_name}: {e}")
|
||||
# Don't fail the stop action — the device was stopped successfully
|
||||
|
||||
return {
|
||||
"status": "stopped",
|
||||
"session_id": ingested_session_id or (active_session.id if active_session else None),
|
||||
"session_id": active_session.id if active_session else None,
|
||||
"cycle_response": cycle_response,
|
||||
"ingest": ingest_result,
|
||||
"files_created": files_created,
|
||||
}
|
||||
|
||||
async def _execute_download(
|
||||
@@ -443,38 +487,12 @@ class SchedulerService:
|
||||
files=None, # Download all files in current measurement folder
|
||||
)
|
||||
|
||||
# Ingest the downloaded folder into Terra-View via the shared NRL ingest
|
||||
# (same path as stop/cycle): clean Leq-only session, .rnh parsed
|
||||
# (percentiles + weightings), deduped, unit linked. No placeholder
|
||||
# session here — a standalone download isn't tied to a "recording" marker.
|
||||
ingest_result = None
|
||||
if action.device_type == "slm":
|
||||
folder_name = (response or {}).get("folder_name")
|
||||
if not folder_name:
|
||||
try:
|
||||
folder_name = f"Auto_{int((response or {}).get('index_number')):04d}"
|
||||
except (ValueError, TypeError):
|
||||
folder_name = None
|
||||
if not folder_name:
|
||||
ingest_result = {"success": False, "error": "no folder_name/index_number from download"}
|
||||
else:
|
||||
try:
|
||||
ingest_result = await self._ingest_and_link(
|
||||
db,
|
||||
location_id=action.location_id,
|
||||
unit_id=unit_id,
|
||||
folder_name=folder_name,
|
||||
)
|
||||
logger.info(f"[DOWNLOAD] Ingested {folder_name}: {ingest_result}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to ingest {folder_name} on download: {e}", exc_info=True)
|
||||
ingest_result = {"success": False, "error": str(e)}
|
||||
# TODO: Create DataFile records for downloaded files
|
||||
|
||||
return {
|
||||
"status": "downloaded",
|
||||
"destination_path": destination_path,
|
||||
"device_response": response,
|
||||
"ingest": ingest_result,
|
||||
}
|
||||
|
||||
async def _execute_cycle(
|
||||
@@ -615,36 +633,6 @@ class SchedulerService:
|
||||
)
|
||||
result["old_session_id"] = active_session.id
|
||||
|
||||
# Step 4b: Ingest the just-finished Auto_#### folder into Terra-View
|
||||
# (clean session + DataFiles via ingest_nrl_zip — filters Lp, parses the
|
||||
# .rnh, dedups). This is what gives the nightly report its data.
|
||||
if action.device_type == "slm" and result["steps"].get("download", {}).get("success"):
|
||||
idx = None
|
||||
try:
|
||||
idx = int((result["steps"]["download"].get("response") or {}).get("index_number"))
|
||||
except (ValueError, TypeError):
|
||||
idx = None
|
||||
if idx is None:
|
||||
result["steps"]["ingest"] = {"success": False, "error": "no index_number from download"}
|
||||
else:
|
||||
folder_name = f"Auto_{idx:04d}"
|
||||
try:
|
||||
ing = await self._ingest_and_link(
|
||||
db,
|
||||
location_id=action.location_id,
|
||||
unit_id=unit_id,
|
||||
folder_name=folder_name,
|
||||
placeholder_session=active_session,
|
||||
)
|
||||
result["steps"]["ingest"] = ing
|
||||
# The marker session was dropped; repoint old_session_id at the real row.
|
||||
if ing.get("placeholder_dropped") and ing.get("session_id"):
|
||||
result["old_session_id"] = ing["session_id"]
|
||||
logger.info(f"[CYCLE] Ingested {folder_name}: {ing}")
|
||||
except Exception as e:
|
||||
logger.error(f"[CYCLE] Ingest failed for {folder_name}: {e}", exc_info=True)
|
||||
result["steps"]["ingest"] = {"success": False, "error": str(e)}
|
||||
|
||||
# Step 5: Wait for device to settle before starting new measurement
|
||||
logger.info(f"[CYCLE] Step 5/7: Waiting 30s for device to settle...")
|
||||
await asyncio.sleep(30)
|
||||
@@ -679,54 +667,6 @@ class SchedulerService:
|
||||
|
||||
logger.info(f"[CYCLE] New measurement started, session {new_session.id}")
|
||||
|
||||
# Step 6b: Verify the meter actually resumed measuring (fresh DOD).
|
||||
# Polling is still paused here, so query directly. If it didn't
|
||||
# resume, retry ONCE with a plain start (start_recording — does NOT
|
||||
# re-index, unlike start_cycle) before alerting: a meter left
|
||||
# stopped overnight is the costly failure, and a transient restart
|
||||
# hiccup is common on the NL-43. We retry only on a *confident*
|
||||
# not-measuring reading — never on a failed/inconclusive DOD read —
|
||||
# so a flaky read can't disrupt an already-running measurement.
|
||||
if action.device_type == "slm":
|
||||
async def _check_measuring():
|
||||
"""Return (measuring, state); measuring is None if the DOD read failed."""
|
||||
try:
|
||||
await asyncio.sleep(2)
|
||||
live = await self.device_controller.get_live_data(unit_id, action.device_type)
|
||||
state = ((live or {}).get("measurement_state")
|
||||
or ((live or {}).get("data") or {}).get("measurement_state") or "")
|
||||
ok = str(state).strip().lower() in ("start", "measure", "measuring", "run", "running")
|
||||
return ok, state
|
||||
except Exception as e:
|
||||
logger.warning(f"[CYCLE] Restart-verify DOD read failed: {e}")
|
||||
return None, None
|
||||
|
||||
measuring, state = await _check_measuring()
|
||||
if measuring is False:
|
||||
logger.warning(f"[CYCLE] {unit_id} not measuring after restart (state={state!r}) — retrying start once.")
|
||||
result["steps"]["restart_retry"] = True
|
||||
try:
|
||||
await self.device_controller.start_recording(unit_id, action.device_type)
|
||||
measuring, state = await _check_measuring()
|
||||
except Exception as e:
|
||||
logger.error(f"[CYCLE] Restart retry (start_recording) failed for {unit_id}: {e}")
|
||||
|
||||
result["steps"]["restart_verified"] = measuring
|
||||
if measuring:
|
||||
logger.info(f"[CYCLE] Restart verified — {unit_id} is measuring (state={state}).")
|
||||
elif measuring is False:
|
||||
logger.error(f"[CYCLE] Restart NOT verified for {unit_id} after retry — state={state!r}")
|
||||
try:
|
||||
get_alert_service(db).create_schedule_failed_alert(
|
||||
schedule_id=action.id, action_type="cycle", unit_id=unit_id,
|
||||
error_message=f"Meter did not resume measuring after the cycle + one retry (state={state!r}).",
|
||||
project_id=action.project_id, location_id=action.location_id,
|
||||
)
|
||||
except Exception as ae:
|
||||
logger.warning(f"[CYCLE] restart-verify alert failed: {ae}")
|
||||
else:
|
||||
logger.warning(f"[CYCLE] Restart verification inconclusive for {unit_id} (DOD read failed); keepalive poll will re-confirm.")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[CYCLE] Start failed: {e}")
|
||||
result["steps"]["start"] = {"success": False, "error": str(e)}
|
||||
@@ -749,85 +689,6 @@ class SchedulerService:
|
||||
logger.info(f"[CYCLE] === Cycle complete for {unit_id} ===")
|
||||
return result
|
||||
|
||||
async def _ingest_cycle_folder(self, db, location_id: str, unit_id: str, folder_name: str) -> dict:
|
||||
"""Fetch a just-finished Auto_#### folder from SLMM (FTP proxy) and ingest
|
||||
it into Terra-View (clean MonitoringSession + DataFiles via ingest_nrl_zip).
|
||||
|
||||
Returns the ingest result dict, or {"success": False, "error": ...}.
|
||||
Used by _execute_cycle Step 4b.
|
||||
"""
|
||||
import os
|
||||
import httpx
|
||||
from backend.routers.project_locations import ingest_nrl_zip, IngestError
|
||||
|
||||
slmm_base = os.getenv("SLMM_BASE_URL", "http://localhost:8100")
|
||||
remote_path = f"/NL-43/{folder_name}"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=600.0) as client:
|
||||
resp = await client.post(
|
||||
f"{slmm_base}/api/nl43/{unit_id}/ftp/download-folder",
|
||||
json={"remote_path": remote_path},
|
||||
)
|
||||
except Exception as e:
|
||||
return {"success": False, "error": f"download-folder request failed: {e}"}
|
||||
|
||||
if not resp.is_success or len(resp.content) <= 22: # 22 bytes = empty-zip
|
||||
return {"success": False, "error": f"empty/failed ZIP from SLMM (status {resp.status_code})"}
|
||||
|
||||
try:
|
||||
res = ingest_nrl_zip(location_id, resp.content, db, source="ftp_cycle", dedupe=True)
|
||||
return {"success": True, **res}
|
||||
except IngestError as e:
|
||||
return {"success": False, "error": str(e)}
|
||||
|
||||
async def _ingest_and_link(
|
||||
self,
|
||||
db,
|
||||
*,
|
||||
location_id: str,
|
||||
unit_id: str,
|
||||
folder_name: str,
|
||||
placeholder_session=None,
|
||||
) -> dict:
|
||||
"""Ingest a just-finished Auto_#### folder and tie it to the unit.
|
||||
|
||||
This is the ONE ingest path that stop / cycle / download all funnel
|
||||
through, so every route produces the same clean session: Leq-only,
|
||||
`.rnh` parsed (percentile slot map + weightings captured), deduped.
|
||||
|
||||
Steps:
|
||||
1. Fetch + ingest the folder via the shared NRL ingest
|
||||
(`_ingest_cycle_folder` → `ingest_nrl_zip`).
|
||||
2. `ingest_nrl_zip` leaves `unit_id` None — link it to the unit that
|
||||
recorded the data so the session stays attributed.
|
||||
3. If a `placeholder_session` (the transient "recording" marker) was
|
||||
passed and it never accumulated DataFiles of its own, drop it — its
|
||||
data now lives in the ingested, unit-linked session.
|
||||
|
||||
Returns the ingest result dict (with `success`); adds `placeholder_dropped`
|
||||
when step 3 removed the marker. On ingest failure the placeholder is
|
||||
left untouched.
|
||||
"""
|
||||
ing = await self._ingest_cycle_folder(db, location_id, unit_id, folder_name)
|
||||
if not ing.get("success"):
|
||||
return ing
|
||||
|
||||
sid = ing.get("session_id")
|
||||
if sid:
|
||||
s = db.query(MonitoringSession).filter_by(id=sid).first()
|
||||
if s and not s.unit_id:
|
||||
s.unit_id = unit_id
|
||||
db.commit()
|
||||
|
||||
if placeholder_session is not None and sid:
|
||||
from backend.models import DataFile
|
||||
if db.query(DataFile).filter_by(session_id=placeholder_session.id).count() == 0:
|
||||
db.delete(placeholder_session)
|
||||
db.commit()
|
||||
ing["placeholder_dropped"] = True
|
||||
|
||||
return ing
|
||||
|
||||
# ========================================================================
|
||||
# Recurring Schedule Generation
|
||||
# ========================================================================
|
||||
@@ -921,92 +782,6 @@ class SchedulerService:
|
||||
|
||||
return cleaned
|
||||
|
||||
# ========================================================================
|
||||
# Nightly Sound Report (FTP report pipeline)
|
||||
# ========================================================================
|
||||
|
||||
async def run_due_reports(self):
|
||||
"""Run any project nightly sound reports that are due.
|
||||
|
||||
For each enabled SoundReportConfig: if local time is past report_time
|
||||
and we haven't already reported last night, build the report (writes a
|
||||
file always; emails if SMTP is configured, else dry-run) and stamp
|
||||
last_run_date. Idempotent across restarts via last_run_date.
|
||||
"""
|
||||
from backend.models import SoundReportConfig
|
||||
from backend.utils.timezone import utc_to_local
|
||||
|
||||
# Decide what's due (cheap, on the loop); run each OFF the event loop.
|
||||
due_jobs = []
|
||||
db = SessionLocal()
|
||||
try:
|
||||
configs = db.query(SoundReportConfig).filter_by(enabled=True).all()
|
||||
if not configs:
|
||||
return
|
||||
local_now = utc_to_local(datetime.utcnow())
|
||||
night_date = local_now.date() - timedelta(days=1) # last night's evening date
|
||||
for cfg in configs:
|
||||
try:
|
||||
hh, mm = (int(x) for x in cfg.report_time.split(":"))
|
||||
except (ValueError, AttributeError):
|
||||
hh, mm = 8, 0
|
||||
if (local_now.hour, local_now.minute) < (hh, mm):
|
||||
continue
|
||||
if cfg.last_run_date == night_date:
|
||||
continue
|
||||
due_jobs.append({
|
||||
"project_id": cfg.project_id,
|
||||
"metric_keys": [m.strip() for m in (cfg.metric_keys or "").split(",") if m.strip()] or None,
|
||||
"recipients": [r.strip() for r in (cfg.recipients or "").split(",") if r.strip()] or None,
|
||||
"baseline_mode": cfg.baseline_mode,
|
||||
"baseline_start": cfg.baseline_start,
|
||||
"baseline_end": cfg.baseline_end,
|
||||
})
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# run_nightly_report is synchronous (blocking file I/O + smtplib up to the
|
||||
# SMTP timeout). Run it in a worker thread so it never stalls the scheduler
|
||||
# loop (which also drives time-sensitive device cycles).
|
||||
for job in due_jobs:
|
||||
try:
|
||||
logger.info(f"[REPORT] Running nightly report for project {job['project_id']} (night {night_date})")
|
||||
result = await asyncio.to_thread(self._run_one_report, night_date, job)
|
||||
email = (result or {}).get("email", {})
|
||||
logger.info(
|
||||
f"[REPORT] project {job['project_id']}: {(result or {}).get('location_count')} location(s); "
|
||||
f"email={'sent' if email.get('sent') else ('dry-run' if email.get('dry_run') else (email.get('error') or 'skipped'))}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"[REPORT] Failed nightly report for project {job['project_id']}: {e}", exc_info=True)
|
||||
|
||||
def _run_one_report(self, night_date, job) -> Dict[str, Any]:
|
||||
"""Sync worker: build/send one project's report and stamp last_run_date.
|
||||
Uses its own DB session (runs in a thread, off the event loop)."""
|
||||
from backend.models import SoundReportConfig
|
||||
from backend.services.report_orchestrator import run_nightly_report
|
||||
|
||||
db = SessionLocal()
|
||||
try:
|
||||
result = run_nightly_report(
|
||||
db, job["project_id"], night_date,
|
||||
metric_keys=job["metric_keys"],
|
||||
baseline_mode=job["baseline_mode"],
|
||||
baseline_start=job["baseline_start"],
|
||||
baseline_end=job["baseline_end"],
|
||||
recipients=job["recipients"],
|
||||
)
|
||||
cfg = db.query(SoundReportConfig).filter_by(project_id=job["project_id"]).first()
|
||||
if cfg:
|
||||
cfg.last_run_date = night_date
|
||||
db.commit()
|
||||
return result
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
# ========================================================================
|
||||
# Manual Execution (for testing/debugging)
|
||||
# ========================================================================
|
||||
|
||||
+1
-8
@@ -1,6 +1,6 @@
|
||||
services:
|
||||
|
||||
web-app:
|
||||
terra-view:
|
||||
build: .
|
||||
ports:
|
||||
- "8001:8001"
|
||||
@@ -11,13 +11,6 @@ services:
|
||||
- ENVIRONMENT=production
|
||||
- SLMM_BASE_URL=http://host.docker.internal:8100
|
||||
- SFM_BASE_URL=http://sfm:8200
|
||||
# Client-portal session-cookie signing. Set SECRET_KEY to a real secret (e.g.
|
||||
# in a .env file beside this compose) BEFORE the portal faces the internet —
|
||||
# the dev default is public/forgeable and logs a warning at boot. Set
|
||||
# COOKIE_SECURE=true once served over HTTPS (leave false on plain HTTP, or the
|
||||
# browser won't send the cookie and the portal breaks).
|
||||
- SECRET_KEY=${SECRET_KEY:-dev-insecure-change-me}
|
||||
- COOKIE_SECURE=${COOKIE_SECURE:-false}
|
||||
# Display timezone for server logs + any text-rendered timestamps.
|
||||
# DB columns are stored UTC regardless; this only affects what
|
||||
# operators see. Override here for non-US-East deployments.
|
||||
|
||||
@@ -1,214 +0,0 @@
|
||||
# Client Portal — Design & Build Plan
|
||||
|
||||
**Status:** in development (`feat/client-portal`) · **Targets:** 0.14.x
|
||||
|
||||
> **Update (Phase-1 auth landed):** the interim magic-link gate described below is
|
||||
> **retired** — client access is now a per-project secure link + shared password
|
||||
> (argon2). See the design at `docs/superpowers/specs/2026-06-15-portal-auth-design.md`
|
||||
> and the build plan at `docs/superpowers/plans/2026-06-15-portal-auth.md`. The
|
||||
> operator manages access from each project's **Portal access** panel.
|
||||
|
||||
A client-facing, **read-only**, **scoped** view into a client's own monitoring
|
||||
data. The first internet-facing-with-real-clients surface in the system. Built
|
||||
*inside* the Terra-View app (new `/portal/*` namespace), reusing the cached SLMM
|
||||
reads and Terra-View's report generation — Terra-View stays the UI/business layer;
|
||||
SLMM stays the device layer.
|
||||
|
||||
## Principles
|
||||
|
||||
1. **Read-only.** No device control (start/stop/config), no roster editing, no
|
||||
internal pages. A client can look, never touch.
|
||||
2. **Strictly scoped.** A client only ever sees data for *their* projects. Every
|
||||
portal endpoint verifies ownership server-side — never trust a `unit_id` /
|
||||
`location_id` from the request.
|
||||
3. **Cache-first, no device contention.** Portal live data comes from SLMM's
|
||||
cache (the same cached `/status` + `/history` the internal dashboard uses).
|
||||
No device-hitting calls from the portal — a client can't make us hammer the
|
||||
NL-43. Freshness depends on **keepalive being on** for the client's units.
|
||||
4. **Auth is a swappable gate.** Every route depends on one resolver,
|
||||
`get_current_client()`. M1–M3 ride on an interim signed "magic URL"; M4
|
||||
replaces the resolver's backing without touching routes or templates.
|
||||
|
||||
## The data chain (how a client maps to live data)
|
||||
|
||||
```
|
||||
Client.id
|
||||
└─ Project (client_id == Client.id, status != deleted)
|
||||
└─ MonitoringLocation (project_id, location_type == "sound", removed_at IS NULL)
|
||||
└─ UnitAssignment (location_id, status == "active", device_type == "slm",
|
||||
assigned_until IS NULL or future)
|
||||
└─ unit_id == RosterUnit.id == SLMM unit_id
|
||||
└─ SLMM cached /status + /history (read-only)
|
||||
```
|
||||
|
||||
So the portal shows a client their **locations**, each surfacing the live sound
|
||||
level from whatever SLM is currently assigned there.
|
||||
|
||||
## Data model (new)
|
||||
|
||||
```python
|
||||
class Client(Base): # the customer org
|
||||
id, name, slug (unique, URL-safe), contact_email (nullable, for M4),
|
||||
active (bool), created_at
|
||||
|
||||
class ClientAccessToken(Base): # the interim "magic URL" gate
|
||||
id, client_id, token_hash (sha256 — raw shown once on creation),
|
||||
label, created_at, last_used_at, revoked_at (nullable)
|
||||
```
|
||||
|
||||
Plus a migration adding **`Project.client_id`** (nullable FK → `clients.id`).
|
||||
The existing free-text `Project.client_name` stays for display/back-compat;
|
||||
`client_id` is the authoritative link.
|
||||
|
||||
## Auth — the swappable gate
|
||||
|
||||
```python
|
||||
def get_current_client(request, db) -> Client: # every /portal route depends on this
|
||||
# M1–M3: read signed `portal_client` cookie -> load Client
|
||||
# M4: same signature, backed by real sessions (magic-link / password)
|
||||
```
|
||||
|
||||
**Interim "magic URL" flow (M1–M3):**
|
||||
- Operator creates a `Client` + an access token → gets a one-time-display URL:
|
||||
`https://…/portal/enter/{token}`.
|
||||
- Client clicks it → token is hashed, looked up (must be un-revoked) →
|
||||
sets a **signed session cookie** (`portal_client`, HMAC via a new `SECRET_KEY`
|
||||
env) → redirects to `/portal`. `last_used_at` updated.
|
||||
- `get_current_client` reads + verifies the cookie thereafter. No valid cookie →
|
||||
"link invalid / expired" page.
|
||||
- Revoke = set `revoked_at`; the link (and any cookie minted from it) stops working.
|
||||
|
||||
Unguessable + revocable + per-person, no email infra or passwords yet — and M4
|
||||
slots in behind the same `get_current_client` with zero route/template churn.
|
||||
|
||||
## Routes (`/portal/*`)
|
||||
|
||||
| Route | Purpose |
|
||||
|-------|---------|
|
||||
| `GET /portal/enter/{token}` | validate token → set cookie → redirect to `/portal` |
|
||||
| `GET /portal` | client's locations overview (status tiles + map) |
|
||||
| `GET /portal/location/{id}` | read-only live panel for that location's SLM |
|
||||
| `GET /portal/api/location/{id}/live` | **scoped** cached `/status` for the location's unit |
|
||||
| `GET /portal/api/location/{id}/history` | **scoped** cached trail for the chart |
|
||||
| `GET /portal/logout` | clear cookie |
|
||||
|
||||
**Scoping helper** (used by every data route):
|
||||
`resolve_client_location(client, location_id, db) -> (location, unit_id)` — raises
|
||||
403 if the location isn't in one of the client's projects. The portal never calls
|
||||
the open `/api/slmm/{unit}/*` endpoints with a client-supplied id.
|
||||
|
||||
## Templates (`templates/portal/`)
|
||||
|
||||
- `portal/base.html` — minimal client-branded shell (no internal sidebar/nav).
|
||||
- `portal/overview.html` — location tiles (live cards mini) + a locations map.
|
||||
- `portal/location.html` — the read-only live panel: cards (Lp/Leq/Lmax/L1/L10),
|
||||
L1/L10 chart, measuring + freshness badge. Reuses the cache-populate JS from the
|
||||
internal panel, **stripped** of start/stop, config, and the device-hitting
|
||||
refresh (cache + 15s auto-poll only).
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### M1 — Live view only *(current)*
|
||||
Interim magic-URL gate; a client sees their locations and per-location read-only
|
||||
live data, all from cache.
|
||||
- [ ] `Client` + `ClientAccessToken` models; `Project.client_id` migration.
|
||||
- [ ] `SECRET_KEY` env + signed-cookie session helper.
|
||||
- [ ] `get_current_client` dependency + `/portal/enter/{token}` + logout.
|
||||
- [ ] Scoping helper `resolve_client_location`.
|
||||
- [ ] `/portal` overview + `/portal/location/{id}` (read-only live panel).
|
||||
- [ ] Scoped `/portal/api/location/{id}/live` + `/history`.
|
||||
- [ ] Portal templates (base, overview, location).
|
||||
- [ ] Minimal admin: create client + mint/revoke access link (small `/admin`
|
||||
page or a script for now).
|
||||
|
||||
### M2 — Dashboard + alerts
|
||||
- Richer client dashboard (multi-location at-a-glance, status rollup).
|
||||
- **Live project map** — upgrade the overview's basic location pins into a real
|
||||
project map: pins colored by measuring/level, popups showing each location's
|
||||
current reading, centered/zoomed to the project. (M1 ships the plain pin map;
|
||||
this makes it a live status map.)
|
||||
- Surface each location's **threshold-alert status** (read-only) + an event/inbox
|
||||
view. Leans on the SLMM alert engine + dispatch.
|
||||
|
||||
### Notes carried from M1
|
||||
- Tile headline metric is **Leq** (energy-average, the sound-monitoring compliance
|
||||
metric) — chosen over the twitchy instantaneous Lp. If clients ever want a
|
||||
different headline (e.g. Lmax for peaks), make it a per-deployment setting.
|
||||
|
||||
### M3 — Reports
|
||||
- Client-facing list + download of the daily baseline-comparison reports.
|
||||
- Depends on the FTP report pipeline (`feat/ftp-report-pipeline`) landing and
|
||||
being wired into the portal's scoped routes.
|
||||
|
||||
### M4 — Full auth system
|
||||
- Replace the interim token behind `get_current_client` with a real auth design:
|
||||
magic-link (passwordless email) and/or accounts, proper sessions, password
|
||||
reset, and likely auth for the *internal* app too. Reverse-proxy + TLS posture.
|
||||
|
||||
## Going to prod (M1)
|
||||
|
||||
1. **Run the migration on the prod DB** — `migrate_add_client_portal.py` adds
|
||||
`projects.client_id` (the new tables auto-create via `create_all`). Skipping it
|
||||
500s anything that touches `Project.client_id`. This is the silent killer.
|
||||
```bash
|
||||
docker compose exec web-app python3 backend/migrate_add_client_portal.py
|
||||
```
|
||||
2. **Set a real `SECRET_KEY`** in the prod env (compose). The portal signs session
|
||||
cookies with it; the insecure dev default (it logs a warning at boot) is
|
||||
forgeable. Non-negotiable for an internet-facing portal.
|
||||
3. **SLMM_BASE_URL** — prod base compose already points at `:8100` (correct; the
|
||||
`:9100` mismatch is a dev-only override quirk). For full live data (L1/L10 +
|
||||
chart backfill) prod SLMM must be on the `dev` build with its migrations
|
||||
(`migrate_add_ln_percentiles`, `migrate_add_monitor_enabled`) and **keepalive on**
|
||||
for the client's units — otherwise the portal degrades gracefully (cards show
|
||||
`--`, chart empty), it just isn't fully populated.
|
||||
4. **Seed real clients** with the CLI (`backend/portal_admin.py`): `create-client`
|
||||
→ `link-project` (a real sound project with an active SLM assignment) →
|
||||
`mint-link` → send the client the printed URL (shown once).
|
||||
5. **Exposure** — portal routes are auth-gated, but port 8001 still serves the
|
||||
whole *internal* app with no auth. Before real clients are on it, the portal
|
||||
should sit behind the reverse proxy with only `/portal/*` exposed (or the app
|
||||
restricted). This is the point where the parked reverse-proxy/TLS work becomes
|
||||
load-bearing.
|
||||
|
||||
## Security notes
|
||||
|
||||
- Portal is auth-gated from day one (even the interim gate) — never wide-open like
|
||||
the internal app.
|
||||
- All scoping enforced server-side; client-supplied ids are always re-checked.
|
||||
- `SECRET_KEY` must be a real secret in prod (env, not committed).
|
||||
- Cookies: `HttpOnly`, `SameSite=Lax`, `Secure` once behind TLS.
|
||||
- Tokens stored hashed; raw shown once. Revocation is immediate.
|
||||
|
||||
## Security hardening backlog ("Fest 2026")
|
||||
|
||||
The to-do for the dedicated hardening pass, roughly highest-impact first. Until
|
||||
then the portal runs on security-by-obscurity (open port + interim links) — fine
|
||||
for a not-in-use demo, not for real clients.
|
||||
|
||||
**Exposure (the big one):** port 8001 serves the *entire operator app* (roster,
|
||||
projects, `/admin/*`, device config, the SLMM proxy) with **zero auth**, so an
|
||||
open port exposes far more than the read-only portal.
|
||||
- [ ] Reverse proxy (NPM/Caddy/Nginx) in front, exposing **only `/portal/*`** to
|
||||
the internet; keep the operator app reachable on the LAN only.
|
||||
- [ ] TLS everywhere (Let's Encrypt). Then set portal cookies `Secure`.
|
||||
- [ ] Don't port-forward the raw app; if a quick gate is wanted before M4, an
|
||||
auth proxy (Authelia / Authentik) can front the portal without writing auth.
|
||||
|
||||
**Config musts:**
|
||||
- [ ] Set a real `SECRET_KEY` env (signs session cookies; default is public).
|
||||
- [ ] `PORTAL_OPEN_LINKS=false` in any internet-facing env (it defaults off now).
|
||||
|
||||
**M4 — real auth** (replaces the interim token behind `get_current_client`):
|
||||
- [ ] Magic-link email and/or accounts; proper sessions + password reset.
|
||||
- [ ] Authenticate the **operator** app too (it currently has none).
|
||||
- [ ] Gate the operator-only endpoints that are presently unauthenticated:
|
||||
`/projects/{id}/portal-preview`, `/projects/{id}/portal-link*`,
|
||||
`/portal/open/*`.
|
||||
|
||||
**Smaller items from the pre-merge code review:**
|
||||
- [ ] Keepalive isn't auto-turned-off when the last alert rule on a unit is
|
||||
deleted (intentional "never auto-off"; revisit if it wastes cellular).
|
||||
- [ ] Consider rate-limiting the scoped portal endpoints once public.
|
||||
@@ -1,57 +0,0 @@
|
||||
# ADR 0001 — Device data ownership: modules own raw data, Terra-View owns fleet context
|
||||
|
||||
- **Status:** Accepted (SLMM grandfathered as a known exception — see Consequences)
|
||||
- **Date:** 2026-06-16
|
||||
- **Deciders:** Brian
|
||||
- **Applies to:** Terra-View core and all device modules (SFM, SLMM, and future modules)
|
||||
|
||||
## Context
|
||||
|
||||
Terra-View is a fleet-management / UI layer that talks to specialized **device modules**, each of which speaks one device's protocol (see the architecture note in `CLAUDE.md`). Two modules exist today, and they store their data **differently**:
|
||||
|
||||
- **SFM (seismograph / seismo-relay).** Owns its own database **and** waveform store. Terra-View holds **no** seismic event or waveform data — it reads through live, e.g. `GET {SFM_BASE_URL}/db/events` (`backend/routers/activity.py`, `backend/routers/admin_modules.py`). Terra-View renders; SFM persists.
|
||||
- **SLMM (sound level meters).** A thin device-control shim. The sound **measurement data is stored in Terra-View** — `MonitoringSession` + `DataFile` rows in `data/seismo_fleet.db`, and the `.rnh` + Leq `.rnd` files under `data/Projects/{project_id}/{session_id}/` (`backend/routers/project_locations.py:_ingest_file_entries`). SLMM only keeps device config + a live-status cache (`slmm.db`) and a transient download staging area (`data/downloads/{unit_id}/`).
|
||||
|
||||
This inconsistency is real, not cosmetic. It raises an obvious question every time we add a feature or a module: *where does this device's data live?* Without a stated rule, the answer drifts per-module, which is exactly how conceptual integrity erodes.
|
||||
|
||||
### Why the asymmetry exists (history, not sloppiness)
|
||||
|
||||
1. **Path dependence.** seismo-relay pre-existed as a complete data system; Terra-View integrated *with* it. SLMM was built fresh as a control shim, so persistence drifted up into Terra-View.
|
||||
2. **Coupling.** A seismic event is largely self-contained — Terra-View just tags it to a unit. A Leq interval is only meaningful against an NRL location + baseline + report config, which are **Terra-View concepts**. Sound data has stronger natural gravity toward Terra-View ownership than seismic events do.
|
||||
|
||||
## Decision
|
||||
|
||||
Adopt one explicit ownership rule for all device data:
|
||||
|
||||
> **The device module owns the raw device data (waveforms, events, Leq files, raw telemetry). Terra-View owns the fleet/project/location/session/report context that gives that data meaning.**
|
||||
|
||||
Note this is **not** "Terra-View stores nothing" — Terra-View remains the system of record for roster, projects, locations, deployments, history, schedules, and the associations between fleet entities and module-owned data. What it should **not** own is a second copy of raw device telemetry.
|
||||
|
||||
**Litmus test for any "where does this live?" call:** *whose question does this data answer?*
|
||||
- "What did the sensor record?" (raw waveform / Leq rows) → **the module**.
|
||||
- "Which NRL, which night, versus which baseline?" (context) → **Terra-View**.
|
||||
|
||||
### Application
|
||||
|
||||
- **New device modules MUST follow the SFM pattern**: the module owns its data and exposes a read API (`/db/*` or equivalent); Terra-View references it and reads through, rather than ingesting a copy.
|
||||
- **SFM** already conforms. No change.
|
||||
- **SLMM does not conform** and is explicitly **grandfathered** (see Consequences).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
- Consistent module boundaries → lower cognitive load, fewer "which copy is authoritative?" bugs.
|
||||
- Terra-View stays thin; "add a device type = add a module" stays true (the CLAUDE.md north star).
|
||||
- Single source of truth for raw data; no silent duplication.
|
||||
|
||||
**Negative / costs**
|
||||
- Realigning **SLMM** to this rule is a non-trivial refactor: move ingest + file storage into SLMM, build a SLMM read API, repoint the report engine and the Data Files UI to read through it, handle the session↔location association across the module boundary, and migrate existing `MonitoringSession`/`DataFile` data. The FTP night-report pipeline currently **assumes Terra-View ownership**.
|
||||
|
||||
**SLMM grandfather clause**
|
||||
- SLMM stays as-is for now. Realignment is a **deliberate future project**, not a background cleanup, and should be triggered by a real signal — e.g. a 3rd device type arriving, or the duplication/coupling actually causing pain. Until then, Terra-View remains the system of record for sound data, and that is an accepted, documented exception rather than an aspiration.
|
||||
- The current sound data flow (for reference): `NL-43 SD card → (FTP) → SLMM data/downloads/ → (proxy ZIP) → Terra-View ingest → data/Projects/ + seismo_fleet.db`. The 1-second `_Lp_` files are dropped at ingest and never land in Terra-View.
|
||||
|
||||
## Related
|
||||
|
||||
- `CLAUDE.md` — module architecture ("Terra-View does NOT communicate directly with physical devices").
|
||||
- FTP night-report pipeline (`feat/ftp-report-pipeline`) — built on the current SLMM/Terra-View-ownership model; a future SLMM realignment would need to repoint it.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,237 +0,0 @@
|
||||
# Portal Authentication — Design & Build Plan
|
||||
|
||||
**Status:** in development (`feat/portal-auth`) · **Targets:** 0.14.x · **Date:** 2026-06-15
|
||||
|
||||
Supersedes the interim shareable magic-link described in
|
||||
[CLIENT_PORTAL.md](../../CLIENT_PORTAL.md) with a real password gate.
|
||||
|
||||
## Goal
|
||||
|
||||
Give a client a **secure link + password** that opens a **read-only dashboard** —
|
||||
live data plus access to historical data — for the machines commissioned on
|
||||
**their project**. Nothing else: no device control, no editing, no internal pages.
|
||||
|
||||
This is the first real, internet-facing, client-credentialed surface in the
|
||||
system.
|
||||
|
||||
## Scope
|
||||
|
||||
**Phase 1 (this spec — build now):** per-project, password-gated, read-only portal.
|
||||
|
||||
**Deferred (designed, not built — captured below so nothing is lost):**
|
||||
- **Operator auth** — logins + roles for the *internal* app (you / parents).
|
||||
Full design in [Deferred A](#deferred-a--operator-auth-designed-not-built).
|
||||
- **Full multi-tenancy** — per-client rollups, per-project separation within a
|
||||
client, individual client user accounts, and extending the portal to all
|
||||
client-relevant data. [Deferred B](#deferred-b--full-multi-tenancy).
|
||||
|
||||
## Principles (the portal's standing charter)
|
||||
|
||||
1. **Read-only.** A client can look, never touch.
|
||||
2. **Strictly scoped, server-side.** Never trust a project / location / unit id
|
||||
from the request — always re-resolve ownership.
|
||||
3. **Cache-first.** Portal live data comes from SLMM's cache (the same cached
|
||||
reads the internal dashboard uses). A client can never make us hit the device.
|
||||
4. **The gate is a swappable seam.** Everything routes through the scoping layer
|
||||
the portal already has; auth is the thin thing in front of it.
|
||||
|
||||
## The model
|
||||
|
||||
- **Tenant unit = the project.** Each project is its own portal: one link, one
|
||||
password, showing that project's commissioned machines.
|
||||
- **Shared credential — "company / project-manager wide."** No individual client
|
||||
accounts. Because access is read-only, one shared password per project is an
|
||||
acceptable trade. (Per-person accounts are a Deferred-B item.)
|
||||
- **The link identifies the project; the password authorizes.** A password alone
|
||||
can't say *which* project — so the link carries an unguessable, revocable
|
||||
per-project token, and the password is the shared secret gating it.
|
||||
|
||||
## Architecture
|
||||
|
||||
Two layers, two subdomains (hosting target: office Synology NAS behind a UniFi
|
||||
UXG Max; own domain `terra-mechanics.com`).
|
||||
|
||||
```
|
||||
Internet
|
||||
│
|
||||
UniFi UXG Max ── Layer 1 (IT pro): firewall, IPS/IDS, GeoIP allow-list,
|
||||
│ kill-switch rule, 443 only
|
||||
Synology NAS ── DSM reverse proxy + Let's Encrypt wildcard TLS
|
||||
│
|
||||
├─ terra-view.terra-mechanics.com → internal app (operator auth = Deferred A)
|
||||
└─ portal.terra-mechanics.com → LOCKED to /portal/* only, password gate
|
||||
```
|
||||
|
||||
The portal subdomain is **restricted to `/portal/*` at the reverse proxy** — a
|
||||
client on `portal.` physically cannot reach `/roster`, `/admin/*`, etc., even by
|
||||
guessing URLs. This path-lock is a load-bearing control for as long as the
|
||||
internal app remains unauthenticated (until Deferred A lands).
|
||||
|
||||
## Data model
|
||||
|
||||
Add three columns to **`Project`**:
|
||||
|
||||
| Column | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `portal_enabled` | bool, default `false` | Is the portal open for this project. |
|
||||
| `portal_password_hash` | text, nullable | argon2id hash of the shared password. Never plaintext. |
|
||||
| `portal_link_token` | text, unique, nullable | Unguessable token in the secure link; identifies the project without exposing its raw id, and is revocable (regenerate → old link dies). |
|
||||
|
||||
**Reused unchanged:** the `Client → Project → MonitoringLocation →
|
||||
UnitAssignment → unit` scoping chain and the existing read-only scoped data
|
||||
routes (`resolve_client_location` + live / history / events).
|
||||
|
||||
**Migration:** `migrate_add_project_portal_auth.py` — an `ALTER TABLE` adding the
|
||||
three columns to the existing (non-empty) `projects` table. Same pattern as
|
||||
`migrate_add_client_portal.py`; `create_all` won't add columns to an existing
|
||||
table.
|
||||
|
||||
## Auth flow
|
||||
|
||||
1. **Operator enables + shares.** On the project page, the operator turns the
|
||||
portal on; the system generates a strong password + a `portal_link_token`; the
|
||||
operator copies **link + password** to send the client.
|
||||
2. **Client opens the link** `portal.terra-mechanics.com/portal/p/{link_token}` →
|
||||
the project is resolved from the token → a **password prompt** renders.
|
||||
3. **Client submits the password** → argon2-verified against
|
||||
`portal_password_hash`. On success, a **signed session cookie scoped to that
|
||||
project** is set (HMAC via the existing `SECRET_KEY` cookie machinery), and
|
||||
they are redirected to the project dashboard.
|
||||
4. **Subsequent requests** re-validate the cookie (signature + project still
|
||||
`portal_enabled` + within cookie max-age) and serve the existing read-only
|
||||
scoped data.
|
||||
5. **Logout** clears the cookie. **Revoke** = disable the portal or regenerate the
|
||||
token / password, which kills outstanding links and any session minted from
|
||||
them on the next request.
|
||||
|
||||
**Lockout:** track failed attempts (per token + IP); after 5 failures refuse for
|
||||
a 15-minute cooldown. Combined with the UniFi GeoIP/IPS edge, that's solid for a
|
||||
read-only surface.
|
||||
|
||||
**Shared cookie machinery:** lift the portal's cookie sign/verify out of
|
||||
`portal_auth.py` into a small shared `backend/auth_cookies.py` — one signer, so
|
||||
the future operator auth (Deferred A) reuses it instead of copy-pasting crypto.
|
||||
|
||||
### Relationship to the existing portal code
|
||||
|
||||
The portal today is *client-scoped* (a `ClientAccessToken` magic-link → a cookie
|
||||
covering all of a client's projects, with a `/portal` overview). Phase 1 makes the
|
||||
entry point *project-scoped*:
|
||||
|
||||
- The **`/portal/p/{link_token}` + password** flow becomes the way in; the
|
||||
interim client magic-link (`/portal/enter/{token}`, `/portal/open/*`,
|
||||
`PORTAL_OPEN_LINKS`) is **retired** in its favor.
|
||||
- The existing read-only views (`/portal/location/{id}`, live / history / events)
|
||||
and the scoping helper are **reused as-is**, just resolved against the project in
|
||||
the session cookie instead of the client.
|
||||
- `Client` / `ClientAccessToken` rows are **left in place** (no destructive
|
||||
migration) — they become the substrate for the Deferred-B per-client rollup.
|
||||
|
||||
## Operator "Portal access" panel
|
||||
|
||||
On the project detail page (internal app), a panel that:
|
||||
- Toggles `portal_enabled`.
|
||||
- **Regenerate password** → shows a freshly generated strong password **once** for
|
||||
the operator to copy.
|
||||
- **Copy link** → the `/portal/p/{token}` URL.
|
||||
- **Revoke** → regenerate the token (old link dies) and/or disable the portal.
|
||||
|
||||
This is an operator action. Until operator auth lands (Deferred A), it sits behind
|
||||
the same posture as the rest of the internal app — see Security notes.
|
||||
|
||||
## Error handling
|
||||
|
||||
- **Bad password** → generic "incorrect password" + increment fail count.
|
||||
- **Unknown / disabled / revoked token** → generic "this portal link is no longer
|
||||
active" page (no project-existence leak).
|
||||
- **Locked out** → "too many attempts, try again in 15 minutes."
|
||||
- **Expired / invalid cookie** → back to the password prompt.
|
||||
- **Portal disabled after a session started** → next request bounced to the prompt.
|
||||
|
||||
## Rollout
|
||||
|
||||
1. Implement on `feat/portal-auth` → review → merge to `dev`.
|
||||
2. **Migration** `migrate_add_project_portal_auth.py` on each DB (dev + prod), same
|
||||
drill as the client-portal migration.
|
||||
3. **`SECRET_KEY`** must be a real value in prod (already required for the existing
|
||||
portal cookie; the password gate reuses it).
|
||||
4. **Hosting:** DSM reverse proxy routes `portal.` → app, locked to `/portal/*`;
|
||||
Let's Encrypt wildcard TLS; cookies `Secure` once on TLS. UXG Max GeoIP + IPS +
|
||||
kill-switch handled by the IT pro.
|
||||
5. Enable a real project's portal, set a password, and test the full
|
||||
link → password → dashboard flow over HTTPS before sending a client.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Unit:** argon2 hash/verify; token resolution (valid / unknown / disabled);
|
||||
lockout counter; cookie sign/verify + scope check; "disabled mid-session" bounce.
|
||||
- **Scoping:** a session for project A cannot read project B's locations / history
|
||||
/ events (404, no existence leak).
|
||||
- **Manual smoke:** enable → copy link + password → open in a fresh browser →
|
||||
wrong password (lockout) → right password → see live + history → logout.
|
||||
|
||||
---
|
||||
|
||||
## Deferred A — Operator auth (designed, not built)
|
||||
|
||||
Logins + roles for the **internal** app (`terra-view.` subdomain). Closes the
|
||||
"internal app is wide open" hole. Full design, ready to lift into its own spec:
|
||||
|
||||
- **Two layers:** UniFi UXG Max edge (IT-pro owned — firewall, IPS, GeoIP,
|
||||
kill-switch, 443-only) + in-app auth (built by us). Internet-exposed with login
|
||||
(no VPN — deliberately, to spare non-technical family members).
|
||||
- **`OperatorUser` model:** `id, email (unique, lowercased), display_name,
|
||||
password_hash (argon2id), role, active, created_at, last_login_at,
|
||||
sessions_valid_from, failed_login_count, locked_until` (+ later `totp_secret`,
|
||||
`totp_enabled`).
|
||||
- **Role ladder:** `superadmin > admin > operator`.
|
||||
- `superadmin` = you — everything + account management (create/disable users,
|
||||
reset passwords, assign roles).
|
||||
- `admin` = your parents (company owners) + you — full run of the app, no
|
||||
operational restrictions.
|
||||
- `operator` = **future** restricted tier for hires; the ladder accepts it with
|
||||
no route changes.
|
||||
- The only thing gated above plain `admin` in v1 is account management
|
||||
(`superadmin`).
|
||||
- **Sessions:** stateless signed cookie reusing `auth_cookies.py` + `SECRET_KEY`
|
||||
(distinct cookie name from the portal). `sessions_valid_from` gives "log out
|
||||
everywhere" / revoke-on-password-change with no session table.
|
||||
- **Authorization:** one **deny-by-default middleware** gates the whole internal
|
||||
app (exempt: `/login`, `/logout`, `/health`, `/static/*`, `/portal/*`);
|
||||
`require_role("admin"|"superadmin")` guards specific routes. New routes are
|
||||
protected automatically.
|
||||
- **Lockout:** 5 fails → 15-min cooldown (doubling).
|
||||
- **2FA:** deferred; TOTP later, admin/superadmin account first.
|
||||
- **Safe rollout (no self-lockout):** ship behind a feature flag
|
||||
`OPERATOR_AUTH_ENABLED` (default **off** = app behaves as today) → seed the first
|
||||
`superadmin` via a small CLI (`backend/operator_admin.py`, modeled on
|
||||
`portal_admin.py`) → log in while still open → flip the flag on → create
|
||||
parents' accounts. Flag back off = instant escape hatch; break-glass =
|
||||
re-run seed / `reset-password` CLI in the container.
|
||||
- **`OperatorUser` is a brand-new table** → `create_all` builds it on startup; only
|
||||
the seed step is required.
|
||||
|
||||
## Deferred B — Full multi-tenancy
|
||||
|
||||
- Per-client **rollup**: one login spanning all of a client's projects.
|
||||
- Per-project **separation within a client** (true tenant isolation).
|
||||
- **Individual client user accounts** (per-person, optional roles) replacing the
|
||||
shared per-project password.
|
||||
- Extend the portal to **all client-relevant data types** (beyond sound:
|
||||
vibration, reports, etc.) — the long-term goal of "everything we can show a
|
||||
client."
|
||||
- All additive on the existing scoping seam — no teardown.
|
||||
|
||||
## Security notes
|
||||
|
||||
- Auth-gated from day one (even the shared password) — never wide-open like the
|
||||
internal app currently is.
|
||||
- Scoping enforced server-side; client-supplied ids always re-checked.
|
||||
- Passwords argon2-hashed; link tokens unguessable + revocable; raw password shown
|
||||
once.
|
||||
- `SECRET_KEY` a real secret in prod; cookies `HttpOnly` + `SameSite=Lax` +
|
||||
`Secure` (once on TLS).
|
||||
- **Known risk:** the operator "Portal access" panel — and the whole internal app —
|
||||
is unauthenticated until Deferred A. Mitigated for now by the `/portal/*`
|
||||
path-lock on the public subdomain plus keeping the internal app off the public
|
||||
internet. Tracked in the hardening backlog (CLIENT_PORTAL.md).
|
||||
@@ -1,54 +0,0 @@
|
||||
# FTP Night-Report Pipeline — changelog entry
|
||||
|
||||
> **How to use:** paste the block below into Terra-View's `CHANGELOG.md`.
|
||||
> The current `[Unreleased]` section targets **0.14.0** (SLM live monitoring); this
|
||||
> is a separate, larger feature, so it's drafted here as **0.15.0** — fold it into
|
||||
> `[Unreleased]` or bump the version as you prefer. Set the release date when you ship.
|
||||
|
||||
---
|
||||
|
||||
## [0.15.0] - 2026-XX-XX
|
||||
|
||||
FTP night-report pipeline. Automated **daily morning report** of last night's noise (7PM–7AM) versus a baseline, per location, for 24/7 remote sound jobs. The meter records 24/7 to its SD card regardless of TCP state, so the report pulls the meter's own stored 15-minute `_Leq_` intervals over FTP (through the existing `/api/slmm/.../ftp/...` proxy) — accurate Leq/Lmax/Ln straight from the device, and resilient to a TCP-control wedge. The report engine is source-agnostic and metric-driven; delivery is an HTML email body plus an Excel attachment. Built around the existing `MonitoringSession`/`DataFile` store and the existing scheduled `cycle` action — the meter is cycled each morning (stop → download → ingest → increment store index → restart), and the report runs off the just-finished, finalized folder.
|
||||
|
||||
### Added
|
||||
|
||||
- **Callable ingest — `ingest_nrl_zip(location_id, zip_bytes, db)`** (`backend/routers/project_locations.py`). The manual SD-card upload (`upload_nrl_data`) was refactored into a shared core so the same path runs programmatically from the scheduler. Keeps `.rnh` + the averaged `_Leq_ .rnd`, drops the 1-second `_Lp_` files, parses the header (now also capturing the device's **percentile→slot map** and weightings into session metadata), and **dedups** repeated pulls of the same folder by store-name + start time. Metric-agnostic: every column of the Leq file is preserved on disk; metric selection happens in the report layer.
|
||||
- **Report compute engine** (`backend/services/report_pipeline.py`). Per-location night model: **LAmax / LA01 / LA10 / LA90 / LAeq** over **Evening (7–10PM)** and **Nighttime (10PM–7AM)** windows, with correct aggregation — Lmax = loudest interval, percentiles = arithmetic mean, **Leq = logarithmic (energy) mean**. The LN→percentile mapping is read from the device's own `.rnh` config, not hardcoded.
|
||||
- **Two baseline sources.** *Captured* — computed from recorded nights in a configurable date range (the "typical night" = mean of per-night values). *Reference* — fixed values typed per location, for a spec limit (e.g. *"L10 = 85"*) or a prior report's averages when the raw data isn't in the system. Blank reference cells aren't compared.
|
||||
- **Renderers** (`backend/services/report_renderers.py`). HTML email body (per-location Last / Baseline / Δ table, colored louder/quieter) **+ an Excel attachment** — one worksheet per NRL with the 15-minute interval table, a line chart, and a Last/Base/Δ summary per window. Metric-driven, so it tracks whatever metric set is configured.
|
||||
- **Config-driven SMTP sender** (`backend/services/report_email.py`). Reads host/port/security/user/password/from/recipients from env (`REPORT_SMTP_*`); **dry-run** when unconfigured, so reports still generate and persist without credentials.
|
||||
- **Per-project config + automatic morning run.** New `SoundReportConfig` table (enabled, report time, metrics, baseline mode + range, recipients) and a scheduler tick (`SchedulerService.run_due_reports`) that builds + emails each enabled project's report once per morning, off the event loop. The orchestrator (`report_orchestrator.py`) always writes `report.html` / `report.json` / `report.xlsx` to `data/reports/{project}/{date}/`, then emails.
|
||||
- **Capture hook in the daily cycle.** `_execute_cycle` now ingests the just-finished `Auto_####` folder into Terra-View after the download, and verifies the meter resumed measuring via a fresh DOD (`measurement_state`) — alerting if not.
|
||||
- **UI on the sound project header.** A **Night Report** button (modal: view a night, *Run & Email* on demand, and a *Recent reports* list with HTML + Excel links) and a **gear → Settings** modal (enable/time, **baseline source toggle** with a per-NRL value editor, metrics, recipients, a **Send test email** button, and a schedule/last-run status line).
|
||||
- **Endpoints** (`backend/routers/reports.py`): `GET/PUT …/reports/config`, `GET/PUT …/reports/baseline`, `GET …/reports/nightly/view`, `POST …/reports/nightly/run`, `POST …/reports/test-email`, `GET …/reports/list`, `GET …/reports/archive/{date}` (+ `/xlsx`).
|
||||
|
||||
### Changed
|
||||
|
||||
- **Manual SD upload now shares the new ingest core.** `POST …/nrl/{location_id}/upload-data` behaves as before (zip or loose files) but routes through `_ingest_file_entries`, so manually-uploaded sessions also get the captured percentile map.
|
||||
|
||||
### Security / hardening
|
||||
|
||||
- HTML modal fields built from user-controlled data (location names, baseline values) are HTML-escaped before insertion (stored-XSS fix).
|
||||
- The SMTP sender refuses to silently downgrade to a plaintext connection on an unrecognized `REPORT_SMTP_SECURITY` value (falls back to STARTTLS), and warns when credentials would go over an unencrypted link.
|
||||
|
||||
### Upgrade Notes
|
||||
|
||||
- **No database migration.** `sound_report_configs` is a brand-new table created automatically by `create_all` on startup (the `baseline_mode` column lives on it). Templates and Python are baked into the image, so **rebuild** (don't just restart):
|
||||
|
||||
```bash
|
||||
cd /home/serversdown/terra-view && docker compose build terra-view && docker compose up -d terra-view
|
||||
```
|
||||
|
||||
- **To actually send email**, set the relay env vars (e.g. on the `terra-view` service in `docker-compose.yml`). Until then, reports still build and write to `data/reports/…` in dry-run:
|
||||
|
||||
```
|
||||
REPORT_SMTP_HOST, REPORT_SMTP_PORT, REPORT_SMTP_SECURITY=starttls|ssl|none,
|
||||
REPORT_SMTP_USER, REPORT_SMTP_PASSWORD, REPORT_SMTP_FROM, REPORT_SMTP_RECIPIENTS
|
||||
```
|
||||
|
||||
Use **Settings → Send test email** to verify the relay once set.
|
||||
|
||||
- **To turn on automation for a job:** configure a daily `cycle` recurring schedule per NRL (~7:15 AM, after the night ends) so the meter is stopped/downloaded/ingested/restarted, then **enable** the report in the gear (report time ~8 AM) and set the baseline (range or fixed values).
|
||||
|
||||
- **Not yet field-tested on a physical meter** — the live device-control portion of the cycle hook (download + restart-verify) was validated against a mocked SLMM only.
|
||||
@@ -1,2 +0,0 @@
|
||||
-r requirements.txt
|
||||
pytest==8.3.3
|
||||
@@ -10,4 +10,3 @@ httpx==0.25.2
|
||||
openpyxl==3.1.2
|
||||
rapidfuzz==3.10.1
|
||||
schedule==1.2.2
|
||||
argon2-cffi==23.1.0
|
||||
|
||||
@@ -42,18 +42,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Live Monitoring (keepalive) -->
|
||||
<div class="rounded-xl bg-white dark:bg-slate-800 shadow-lg p-4 mb-6">
|
||||
<h2 class="text-sm font-semibold text-gray-700 dark:text-gray-300 uppercase tracking-wider mb-1">Live Monitoring (keepalive)</h2>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400 mb-3">
|
||||
Keepalive runs the 1 Hz DOD feed 24/7 (even with no viewer), which powers the live-chart
|
||||
trail and continuous threshold alerts. Toggling persists and survives restarts.
|
||||
</p>
|
||||
<div id="monitor-list" class="text-sm">
|
||||
<p class="text-gray-500 dark:text-gray-400">Loading…</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Raw API tester -->
|
||||
<div class="rounded-xl bg-white dark:bg-slate-800 shadow-lg p-4 mb-6">
|
||||
<h2 class="text-sm font-semibold text-gray-700 dark:text-gray-300 uppercase tracking-wider mb-3">Raw API Tester</h2>
|
||||
@@ -144,60 +132,7 @@ async function sendRaw() {
|
||||
}
|
||||
}
|
||||
|
||||
async function loadMonitors() {
|
||||
const el = document.getElementById('monitor-list');
|
||||
try {
|
||||
const r = await fetch('/api/slmm/roster');
|
||||
if (!r.ok) throw new Error('HTTP ' + r.status);
|
||||
const d = await r.json();
|
||||
const devices = d.devices || [];
|
||||
if (!devices.length) {
|
||||
el.innerHTML = '<p class="text-gray-500 dark:text-gray-400">No devices configured.</p>';
|
||||
return;
|
||||
}
|
||||
el.innerHTML = devices.map(dev => {
|
||||
const on = !!dev.monitor_enabled;
|
||||
const reach = dev.status ? dev.status.is_reachable : null;
|
||||
const reachDot = reach === false
|
||||
? '<span class="w-2 h-2 rounded-full bg-red-500 inline-block" title="unreachable"></span>'
|
||||
: '<span class="w-2 h-2 rounded-full bg-green-500 inline-block" title="reachable"></span>';
|
||||
return `
|
||||
<div class="flex items-center justify-between border-b border-gray-100 dark:border-gray-700 py-2">
|
||||
<div class="flex items-center gap-2">
|
||||
${reachDot}
|
||||
<span class="font-mono text-gray-900 dark:text-white">${_esc(dev.unit_id)}</span>
|
||||
<span class="text-xs text-gray-500 dark:text-gray-400">${_esc(dev.host)}:${_esc(dev.tcp_port)}</span>
|
||||
</div>
|
||||
<div class="flex items-center gap-3">
|
||||
<span class="text-xs font-medium px-2 py-0.5 rounded ${on
|
||||
? 'bg-green-100 text-green-700 dark:bg-green-900/40 dark:text-green-300'
|
||||
: 'bg-gray-100 text-gray-600 dark:bg-slate-700 dark:text-gray-400'}">${on ? '24/7 ON' : 'OFF'}</span>
|
||||
<button onclick="toggleMonitor('${_esc(dev.unit_id)}', ${!on})"
|
||||
class="px-3 py-1 text-xs rounded text-white ${on
|
||||
? 'bg-red-600 hover:bg-red-700' : 'bg-seismo-orange hover:bg-orange-600'}">
|
||||
${on ? 'Disable' : 'Enable'}
|
||||
</button>
|
||||
</div>
|
||||
</div>`;
|
||||
}).join('');
|
||||
} catch (e) {
|
||||
el.innerHTML = `<p class="text-red-600 dark:text-red-400">Failed to load devices: ${_esc(e.message)}</p>`;
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleMonitor(unitId, enable) {
|
||||
const action = enable ? 'start' : 'stop';
|
||||
try {
|
||||
const r = await fetch(`/api/slmm/${encodeURIComponent(unitId)}/monitor/${action}`, { method: 'POST' });
|
||||
if (!r.ok) throw new Error('HTTP ' + r.status);
|
||||
await loadMonitors();
|
||||
} catch (e) {
|
||||
alert('Toggle failed: ' + e.message);
|
||||
}
|
||||
}
|
||||
|
||||
loadSlmmOverview();
|
||||
loadMonitors();
|
||||
setInterval(() => { loadSlmmOverview(); loadMonitors(); }, 30000);
|
||||
setInterval(loadSlmmOverview, 30000);
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@@ -357,16 +357,6 @@
|
||||
|
||||
<!-- Data Files Tab -->
|
||||
<div id="data-tab" class="tab-panel hidden">
|
||||
<!-- Download Files from SLMs (FTP browser, scoped to this NRL's assigned unit) -->
|
||||
<div id="ftp-browser" class="mb-6"
|
||||
hx-get="/api/projects/{{ project_id }}/ftp-browser?location_id={{ location_id }}"
|
||||
hx-trigger="load"
|
||||
hx-swap="innerHTML">
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6">
|
||||
<div class="text-center py-8 text-gray-500">Loading FTP browser...</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6">
|
||||
<div class="flex items-center justify-between mb-6">
|
||||
<h2 class="text-xl font-semibold text-gray-900 dark:text-white">Data Files</h2>
|
||||
@@ -379,13 +369,6 @@
|
||||
</svg>
|
||||
Upload Data
|
||||
</button>
|
||||
<button onclick="htmx.trigger('#unified-files', 'refresh')"
|
||||
class="px-3 py-1.5 text-sm bg-gray-100 dark:bg-gray-700 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-200 dark:hover:bg-gray-600 transition-colors flex items-center gap-1.5">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
|
||||
</svg>
|
||||
Refresh
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -425,11 +408,11 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="unified-files"
|
||||
hx-get="/api/projects/{{ project_id }}/files-unified?location_id={{ location_id }}"
|
||||
hx-trigger="load, refresh from:#unified-files"
|
||||
<div id="data-files-list"
|
||||
hx-get="/api/projects/{{ project_id }}/nrl/{{ location_id }}/files"
|
||||
hx-trigger="load"
|
||||
hx-swap="innerHTML">
|
||||
<div class="text-center py-12 text-gray-500">Loading files...</div>
|
||||
<div class="text-center py-8 text-gray-500">Loading data files...</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -732,7 +715,7 @@ function submitUpload() {
|
||||
status.textContent = parts.join(' ');
|
||||
status.className = 'text-sm text-green-600 dark:text-green-400';
|
||||
input.value = '';
|
||||
htmx.trigger(document.getElementById('unified-files'), 'refresh');
|
||||
htmx.trigger(document.getElementById('data-files-list'), 'load');
|
||||
} else {
|
||||
status.textContent = `Error: ${data.detail || 'Upload failed'}`;
|
||||
status.className = 'text-sm text-red-600 dark:text-red-400';
|
||||
|
||||
@@ -32,19 +32,19 @@
|
||||
</svg>
|
||||
Settings
|
||||
</button>
|
||||
<button onclick="FtpBrowser.enableFTP('{{ unit_item.unit.id }}')"
|
||||
<button onclick="enableFTP('{{ unit_item.unit.id }}')"
|
||||
id="enable-ftp-{{ unit_item.unit.id }}"
|
||||
class="px-3 py-1 text-xs bg-green-600 text-white rounded-lg hover:bg-green-700 transition-colors"
|
||||
disabled>
|
||||
Enable FTP
|
||||
</button>
|
||||
<button onclick="FtpBrowser.disableFTP('{{ unit_item.unit.id }}')"
|
||||
<button onclick="disableFTP('{{ unit_item.unit.id }}')"
|
||||
id="disable-ftp-{{ unit_item.unit.id }}"
|
||||
class="px-3 py-1 text-xs bg-red-600 text-white rounded-lg hover:bg-red-700 transition-colors"
|
||||
disabled>
|
||||
Disable FTP
|
||||
</button>
|
||||
<button onclick="FtpBrowser.loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
|
||||
<button onclick="loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
|
||||
id="browse-ftp-{{ unit_item.unit.id }}"
|
||||
class="px-3 py-1 text-xs bg-seismo-orange text-white rounded-lg hover:bg-seismo-navy transition-colors"
|
||||
disabled>
|
||||
@@ -61,7 +61,7 @@
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M3 7v10a2 2 0 002 2h14a2 2 0 002-2V9a2 2 0 00-2-2h-6l-2-2H5a2 2 0 00-2 2z"></path>
|
||||
</svg>
|
||||
<span id="current-path-{{ unit_item.unit.id }}" class="text-sm font-mono text-gray-600 dark:text-gray-400">/NL-43</span>
|
||||
<button onclick="FtpBrowser.loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
|
||||
<button onclick="loadFTPFiles('{{ unit_item.unit.id }}', '/NL-43')"
|
||||
class="ml-auto text-xs px-2 py-1 bg-gray-100 dark:bg-gray-700 rounded hover:bg-gray-200 dark:hover:bg-gray-600">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
|
||||
@@ -87,11 +87,6 @@
|
||||
</div>
|
||||
|
||||
<script>
|
||||
// Self-contained namespace: this partial is reusable (project-wide Data Files
|
||||
// tab AND per-NRL Data Files tab), and may be co-loaded with other FTP-browsing
|
||||
// partials (e.g. slm_live_view). Wrapping in an IIFE keeps its helpers off the
|
||||
// global scope; only window.FtpBrowser is exposed (see the export at the end).
|
||||
(function () {
|
||||
async function checkFTPStatus(unitId) {
|
||||
const statusSpan = document.getElementById(`ftp-status-${unitId}`);
|
||||
const enableBtn = document.getElementById(`enable-ftp-${unitId}`);
|
||||
@@ -233,7 +228,7 @@ async function loadFTPFiles(unitId, path) {
|
||||
html += `
|
||||
<div class="border border-gray-200 dark:border-gray-600 rounded mb-1">
|
||||
<div class="flex items-center justify-between p-3 hover:bg-gray-50 dark:hover:bg-gray-700 rounded transition-colors cursor-pointer"
|
||||
onclick="FtpBrowser.toggleFTPFolderProject('${unitId}', '${escapeForAttribute(file.path)}', '${folderId}', this)">
|
||||
onclick="toggleFTPFolderProject('${unitId}', '${escapeForAttribute(file.path)}', '${folderId}', this)">
|
||||
<div class="flex items-center flex-1">
|
||||
<svg class="w-4 h-4 mr-2 text-gray-400 transition-transform folder-chevron" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"></path>
|
||||
@@ -244,7 +239,7 @@ async function loadFTPFiles(unitId, path) {
|
||||
</div>
|
||||
<div class="flex items-center gap-2 flex-shrink-0 ml-4">
|
||||
<span class="text-xs text-gray-500 hidden sm:inline">${file.modified || ''}</span>
|
||||
<button onclick="event.stopPropagation(); FtpBrowser.downloadFolderToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
|
||||
<button onclick="event.stopPropagation(); downloadFolderToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
|
||||
class="px-3 py-1 text-xs bg-seismo-orange text-white rounded hover:bg-seismo-navy transition-colors flex items-center"
|
||||
title="Download folder from device to server and add to project database">
|
||||
<svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
@@ -269,7 +264,7 @@ async function loadFTPFiles(unitId, path) {
|
||||
<div class="flex items-center gap-3 flex-shrink-0 ml-4">
|
||||
<span class="text-xs text-gray-500 hidden sm:inline">${sizeStr}</span>
|
||||
<span class="text-xs text-gray-500 hidden md:inline">${file.modified || ''}</span>
|
||||
<button onclick="FtpBrowser.downloadToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
|
||||
<button onclick="downloadToServer('${unitId}', '${escapeForAttribute(file.path)}', '${escapeForAttribute(file.name)}')"
|
||||
class="px-3 py-1 text-xs bg-seismo-orange text-white rounded hover:bg-seismo-navy transition-colors flex items-center"
|
||||
title="Download file from device to server and add to project database">
|
||||
<svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
@@ -384,7 +379,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
|
||||
html += `
|
||||
<div class="border border-gray-200 dark:border-gray-600 rounded">
|
||||
<div class="flex items-center justify-between p-2 hover:bg-gray-50 dark:hover:bg-gray-700 rounded transition-colors cursor-pointer text-sm"
|
||||
onclick="FtpBrowser.toggleFTPFolderProject('${unitId}', '${escapeForAttribute(fullPath)}', '${subFolderId}', this)">
|
||||
onclick="toggleFTPFolderProject('${unitId}', '${escapeForAttribute(fullPath)}', '${subFolderId}', this)">
|
||||
<div class="flex items-center flex-1">
|
||||
<svg class="w-3 h-3 mr-2 text-gray-400 transition-transform folder-chevron" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 5l7 7-7 7"></path>
|
||||
@@ -394,7 +389,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
|
||||
<span class="ml-2 text-xs text-gray-400 folder-status"></span>
|
||||
</div>
|
||||
<div class="flex items-center gap-2 flex-shrink-0 ml-2">
|
||||
<button onclick="event.stopPropagation(); FtpBrowser.downloadFolderToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
|
||||
<button onclick="event.stopPropagation(); downloadFolderToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
|
||||
class="px-2 py-1 bg-seismo-orange hover:bg-seismo-navy text-white text-xs rounded transition-colors flex items-center"
|
||||
title="Download folder to server">
|
||||
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
@@ -417,7 +412,7 @@ async function toggleFTPFolderProject(unitId, folderPath, folderId, headerElemen
|
||||
</div>
|
||||
<div class="flex items-center gap-2 flex-shrink-0 ml-2">
|
||||
<span class="text-xs text-gray-500 hidden sm:inline">${sizeText}</span>
|
||||
<button onclick="FtpBrowser.downloadToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
|
||||
<button onclick="downloadToServer('${unitId}', '${escapeForAttribute(fullPath)}', '${escapeForAttribute(file.name)}')"
|
||||
class="px-2 py-1 bg-seismo-orange hover:bg-seismo-navy text-white text-xs rounded transition-colors"
|
||||
title="Download to server">
|
||||
<svg class="w-3 h-3" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
@@ -547,11 +542,8 @@ async function downloadFolderToServer(unitId, remotePath, folderName) {
|
||||
const data = await response.json();
|
||||
|
||||
if (response.ok) {
|
||||
// Show success message — surface how long the measurement ran
|
||||
alert(`✓ Folder "${folderName}" saved!\n\n` +
|
||||
(data.message || `${data.file_count} file(s) imported`) +
|
||||
formatRunLength(data) +
|
||||
`\n\nNow saved as a session in the Project Files section below.`);
|
||||
// Show success message
|
||||
alert(`✓ Folder "${folderName}" downloaded successfully!\n\n${data.file_count} files extracted\nTotal size: ${formatFileSize(data.total_size)}\n\nFiles are now available in the Project Files section below.`);
|
||||
|
||||
// Refresh the unified files list
|
||||
htmx.trigger('#unified-files', 'refresh');
|
||||
@@ -593,11 +585,7 @@ async function downloadToServer(unitId, remotePath, fileName) {
|
||||
|
||||
if (response.ok) {
|
||||
// Show success message
|
||||
const sizeLine = `\nSize: ${formatFileSize(data.file_size)}`;
|
||||
const msg = data.ingested
|
||||
? `✓ ${fileName} imported as measurement data!` + formatRunLength(data) + sizeLine
|
||||
: `✓ ${fileName} downloaded to server successfully!\n\nFile ID: ${data.file_id}` + sizeLine;
|
||||
alert(msg);
|
||||
alert(`✓ ${fileName} downloaded to server successfully!\n\nFile ID: ${data.file_id}\nSize: ${formatFileSize(data.file_size)}`);
|
||||
|
||||
// Refresh the unified files list
|
||||
htmx.trigger('#unified-files', 'refresh');
|
||||
@@ -619,17 +607,6 @@ function formatFileSize(bytes) {
|
||||
return (bytes / 1073741824).toFixed(2) + ' GB';
|
||||
}
|
||||
|
||||
// Build a "how long did it run" line from an ingest response. Duration is
|
||||
// timezone-independent (stop − start), so it's the reliable number to show.
|
||||
function formatRunLength(data) {
|
||||
if (data.duration_seconds == null) return '';
|
||||
const s = data.duration_seconds;
|
||||
const h = Math.floor(s / 3600);
|
||||
const m = Math.floor((s % 3600) / 60);
|
||||
let txt = h > 0 ? `${h}h ${m}m` : `${m}m`;
|
||||
return `\n\nRecorded for: ${txt}`;
|
||||
}
|
||||
|
||||
// Check FTP status for all units on load
|
||||
// Use setTimeout to ensure DOM elements exist when HTMX loads this partial
|
||||
setTimeout(function() {
|
||||
@@ -637,17 +614,6 @@ setTimeout(function() {
|
||||
checkFTPStatus('{{ unit_item.unit.id }}');
|
||||
{% endfor %}
|
||||
}, 100);
|
||||
|
||||
// The only global surface — the handlers referenced by inline onclick attributes.
|
||||
window.FtpBrowser = {
|
||||
loadFTPFiles,
|
||||
enableFTP,
|
||||
disableFTP,
|
||||
toggleFTPFolderProject,
|
||||
downloadFolderToServer,
|
||||
downloadToServer,
|
||||
};
|
||||
})();
|
||||
</script>
|
||||
|
||||
<!-- Include the unified SLM Settings Modal -->
|
||||
|
||||
@@ -74,22 +74,6 @@
|
||||
</svg>
|
||||
Generate Combined Report
|
||||
</a>
|
||||
<button onclick="openNightReportModal()"
|
||||
title="Last night's noise vs baseline, per location (FTP report pipeline)"
|
||||
class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors flex items-center gap-2 text-sm">
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M20.354 15.354A9 9 0 018.646 3.646 9.003 9.003 0 0012 21a9.003 9.003 0 008.354-5.646z"></path>
|
||||
</svg>
|
||||
Night Report
|
||||
</button>
|
||||
<button onclick="openReportSettings('{{ project.id }}')"
|
||||
title="Nightly report settings — schedule, baseline range, recipients"
|
||||
class="px-3 py-2 border border-gray-300 dark:border-gray-600 text-gray-600 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors flex items-center text-sm">
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10.325 4.317c.426-1.756 2.924-1.756 3.35 0a1.724 1.724 0 002.573 1.066c1.543-.94 3.31.826 2.37 2.37a1.724 1.724 0 001.065 2.572c1.756.426 1.756 2.924 0 3.35a1.724 1.724 0 00-1.066 2.573c.94 1.543-.826 3.31-2.37 2.37a1.724 1.724 0 00-2.572 1.065c-.426 1.756-2.924 1.756-3.35 0a1.724 1.724 0 00-2.573-1.066c-1.543.94-3.31-.826-2.37-2.37a1.724 1.724 0 00-1.065-2.572c-1.756-.426-1.756-2.924 0-3.35a1.724 1.724 0 001.066-2.573c-.94-1.543.826-3.31 2.37-2.37.996.608 2.296.07 2.572-1.065z"></path>
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 12a3 3 0 11-6 0 3 3 0 016 0z"></path>
|
||||
</svg>
|
||||
</button>
|
||||
{% endif %}
|
||||
<button onclick="openMergeModal()"
|
||||
title="Merge this project into another (consolidates duplicates)"
|
||||
@@ -103,338 +87,6 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Night Report Modal -->
|
||||
<div id="night-report-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/60 backdrop-blur-sm">
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-2xl w-full max-w-md mx-4">
|
||||
<div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700 flex items-center justify-between">
|
||||
<h3 class="text-lg font-semibold text-gray-900 dark:text-white">Night Report</h3>
|
||||
<button onclick="closeNightReportModal()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
|
||||
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path></svg>
|
||||
</button>
|
||||
</div>
|
||||
<div class="px-6 py-5 space-y-4">
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">Last night's noise (7 PM–7 AM) vs a baseline range, per location. Opens in a new tab.</p>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Night (evening date)</label>
|
||||
<input type="date" id="nr-night-date" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
</div>
|
||||
<div class="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline start <span class="text-gray-400 font-normal">(optional)</span></label>
|
||||
<input type="date" id="nr-baseline-start" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline end</label>
|
||||
<input type="date" id="nr-baseline-end" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="flex items-center justify-between mb-1">
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300">Recent reports</label>
|
||||
<span id="nr-recent-count" class="text-xs text-gray-400"></span>
|
||||
</div>
|
||||
<div id="nr-recent" class="max-h-40 overflow-y-auto rounded-lg border border-gray-200 dark:border-gray-700 divide-y divide-gray-100 dark:divide-gray-700">
|
||||
<div class="px-3 py-2 text-xs text-gray-400">Loading…</div>
|
||||
</div>
|
||||
</div>
|
||||
<p id="nr-status" class="text-xs"></p>
|
||||
</div>
|
||||
<div class="px-6 py-4 border-t border-gray-200 dark:border-gray-700 flex justify-end gap-2">
|
||||
<button onclick="closeNightReportModal()" class="px-4 py-2 border border-gray-300 dark:border-gray-600 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors text-sm">Cancel</button>
|
||||
<button onclick="runNightReport('{{ project.id }}')" class="px-4 py-2 bg-emerald-600 text-white rounded-lg hover:bg-emerald-700 transition-colors text-sm">Run & Email</button>
|
||||
<button onclick="viewNightReport('{{ project.id }}')" class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors text-sm">View Report</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
var NR_PROJECT_ID = '{{ project.id }}';
|
||||
function openNightReportModal() {
|
||||
var el = document.getElementById('nr-night-date');
|
||||
if (el && !el.value) { // default to last night
|
||||
var d = new Date(); d.setDate(d.getDate() - 1);
|
||||
el.value = d.getFullYear() + '-'
|
||||
+ String(d.getMonth() + 1).padStart(2, '0') + '-'
|
||||
+ String(d.getDate()).padStart(2, '0');
|
||||
}
|
||||
document.getElementById('nr-status').textContent = '';
|
||||
document.getElementById('night-report-modal').classList.remove('hidden');
|
||||
loadRecentReports(NR_PROJECT_ID);
|
||||
}
|
||||
function closeNightReportModal() {
|
||||
document.getElementById('night-report-modal').classList.add('hidden');
|
||||
}
|
||||
function _nrParams() {
|
||||
var night = document.getElementById('nr-night-date').value;
|
||||
var bs = document.getElementById('nr-baseline-start').value;
|
||||
var be = document.getElementById('nr-baseline-end').value;
|
||||
if (!night) { alert('Pick a night (evening date).'); return null; }
|
||||
if ((bs && !be) || (be && !bs)) { alert('Provide both baseline dates, or leave both empty.'); return null; }
|
||||
var qs = 'night_date=' + night;
|
||||
if (bs && be) qs += '&baseline_start=' + bs + '&baseline_end=' + be;
|
||||
return qs;
|
||||
}
|
||||
function viewNightReport(projectId) {
|
||||
var qs = _nrParams(); if (!qs) return;
|
||||
window.open('/api/projects/' + projectId + '/reports/nightly/view?' + qs, '_blank');
|
||||
}
|
||||
function runNightReport(projectId) {
|
||||
var qs = _nrParams(); if (!qs) return;
|
||||
var st = document.getElementById('nr-status');
|
||||
st.style.color = ''; st.textContent = 'Running…';
|
||||
fetch('/api/projects/' + projectId + '/reports/nightly/run?' + qs + '&send=true', { method: 'POST' })
|
||||
.then(function (r) { return r.json().then(function (j) { return { ok: r.ok, j: j }; }); })
|
||||
.then(function (res) {
|
||||
if (!res.ok) { st.style.color = '#b00020'; st.textContent = 'Error: ' + (res.j.detail || 'run failed'); return; }
|
||||
var em = res.j.email || {};
|
||||
var emailMsg = em.sent ? 'emailed' : (em.dry_run ? 'email dry-run (SMTP not set)' : (em.error || 'email skipped'));
|
||||
st.style.color = '#1a7f37';
|
||||
st.innerHTML = 'Done — saved & ' + _mergeEsc(emailMsg) + '. <a href="' + _mergeEsc(res.j.view_url) + '" target="_blank" class="underline">view</a>';
|
||||
loadRecentReports(projectId);
|
||||
})
|
||||
.catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
|
||||
}
|
||||
function loadRecentReports(projectId) {
|
||||
var box = document.getElementById('nr-recent');
|
||||
var cnt = document.getElementById('nr-recent-count');
|
||||
fetch('/api/projects/' + projectId + '/reports/list')
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (j) {
|
||||
cnt.textContent = (j.count || 0) + ' generated';
|
||||
if (!j.reports || !j.reports.length) {
|
||||
box.innerHTML = '<div class="px-3 py-2 text-xs text-gray-400">None yet. Run one above.</div>';
|
||||
return;
|
||||
}
|
||||
box.innerHTML = j.reports.map(function (rp) {
|
||||
var when = (rp.generated_at || '').replace('T', ' ').slice(0, 16);
|
||||
var xlsx = rp.xlsx_url ? ' · <a href="' + _mergeEsc(rp.xlsx_url) + '" class="text-indigo-600 dark:text-indigo-400 hover:underline">Excel</a>' : '';
|
||||
return '<div class="flex items-center justify-between px-3 py-2 text-sm">'
|
||||
+ '<a href="' + _mergeEsc(rp.view_url) + '" target="_blank" class="font-medium text-gray-800 dark:text-gray-200 hover:underline">Night of ' + _mergeEsc(rp.night_date) + '</a>'
|
||||
+ '<span class="text-xs text-gray-400">' + _mergeEsc(when) + ' UTC' + xlsx + '</span></div>';
|
||||
}).join('');
|
||||
})
|
||||
.catch(function () { box.innerHTML = '<div class="px-3 py-2 text-xs text-red-500">Failed to load.</div>'; });
|
||||
}
|
||||
</script>
|
||||
|
||||
<!-- Nightly Report Settings Modal -->
|
||||
<div id="report-settings-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/60 backdrop-blur-sm">
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-2xl w-full max-w-md mx-4 max-h-[90vh] overflow-y-auto">
|
||||
<div class="px-6 py-4 border-b border-gray-200 dark:border-gray-700 flex items-center justify-between">
|
||||
<h3 class="text-lg font-semibold text-gray-900 dark:text-white">Nightly Report Settings</h3>
|
||||
<button onclick="closeReportSettings()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
|
||||
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path></svg>
|
||||
</button>
|
||||
</div>
|
||||
<div class="px-6 py-5 space-y-4">
|
||||
<div id="rs-schedule-status" class="text-xs text-gray-500 dark:text-gray-400 bg-gray-50 dark:bg-gray-900/40 rounded-lg px-3 py-2"></div>
|
||||
<label class="flex items-center gap-2 text-sm font-medium text-gray-800 dark:text-gray-200">
|
||||
<input type="checkbox" id="rs-enabled" class="rounded border-gray-300 text-indigo-600 focus:ring-indigo-500">
|
||||
Email the report automatically each morning
|
||||
</label>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Report time (local)</label>
|
||||
<input type="time" id="rs-report-time" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
<p class="text-xs text-gray-400 mt-1">Runs after this time for the night that just ended.</p>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Baseline source</label>
|
||||
<div class="flex gap-4 text-sm mb-2">
|
||||
<label class="flex items-center gap-1.5"><input type="radio" name="rs-baseline-mode" value="captured" onchange="rsToggleBaselineMode()" class="text-indigo-600"> Captured nights</label>
|
||||
<label class="flex items-center gap-1.5"><input type="radio" name="rs-baseline-mode" value="reference" onchange="rsToggleBaselineMode()" class="text-indigo-600"> Fixed values</label>
|
||||
</div>
|
||||
<div id="rs-baseline-captured" class="grid grid-cols-2 gap-3">
|
||||
<div>
|
||||
<label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Range start</label>
|
||||
<input type="date" id="rs-baseline-start" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Range end</label>
|
||||
<input type="date" id="rs-baseline-end" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
</div>
|
||||
</div>
|
||||
<div id="rs-baseline-reference" class="hidden">
|
||||
<p class="text-xs text-gray-400 mb-2">Values to compare against (a spec limit like L10 = 85, or a prior report's averages). Blank cells aren't compared.</p>
|
||||
<div class="flex justify-end mb-1"><button type="button" onclick="rsCopyFirstNrl()" class="text-xs text-indigo-600 dark:text-indigo-400 hover:underline">Copy first NRL → all</button></div>
|
||||
<div id="rs-ref-grid" class="space-y-3"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Metrics</label>
|
||||
<input type="text" id="rs-metrics" placeholder="lmax,l01,l10,l90" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
<p class="text-xs text-gray-400 mt-1">Comma list. Options: lmax, l01, l10, l50, l90, l95, leq.</p>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-1">Recipients</label>
|
||||
<input type="text" id="rs-recipients" placeholder="brian@…, dad@…" class="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm">
|
||||
<p class="text-xs text-gray-400 mt-1">Comma list. Blank → the default SMTP recipients.</p>
|
||||
</div>
|
||||
<div>
|
||||
<button type="button" onclick="sendTestEmail('{{ project.id }}')" class="text-sm text-indigo-600 dark:text-indigo-400 hover:underline">Send test email</button>
|
||||
<span id="rs-test-status" class="text-xs ml-2"></span>
|
||||
</div>
|
||||
<p id="rs-status" class="text-xs"></p>
|
||||
</div>
|
||||
<div class="px-6 py-4 border-t border-gray-200 dark:border-gray-700 flex justify-end gap-2">
|
||||
<button onclick="closeReportSettings()" class="px-4 py-2 border border-gray-300 dark:border-gray-600 text-gray-700 dark:text-gray-300 rounded-lg hover:bg-gray-50 dark:hover:bg-gray-700 transition-colors text-sm">Cancel</button>
|
||||
<button onclick="saveReportSettings('{{ project.id }}')" class="px-4 py-2 bg-indigo-600 text-white rounded-lg hover:bg-indigo-700 transition-colors text-sm">Save</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<script>
|
||||
function openReportSettings(projectId) {
|
||||
var show = function () { document.getElementById('report-settings-modal').classList.remove('hidden'); };
|
||||
document.getElementById('rs-status').textContent = '';
|
||||
fetch('/api/projects/' + projectId + '/reports/config')
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (c) {
|
||||
document.getElementById('rs-enabled').checked = !!c.enabled;
|
||||
document.getElementById('rs-report-time').value = c.report_time || '08:00';
|
||||
document.getElementById('rs-baseline-start').value = c.baseline_start || '';
|
||||
document.getElementById('rs-baseline-end').value = c.baseline_end || '';
|
||||
document.getElementById('rs-metrics').value = c.metric_keys || 'lmax,l01,l10,l90';
|
||||
document.getElementById('rs-recipients').value = c.recipients || '';
|
||||
var ss = document.getElementById('rs-schedule-status');
|
||||
var last = c.last_run_date || '—';
|
||||
if (c.enabled) {
|
||||
ss.innerHTML = '<span style="color:#1a7f37">●</span> Automatic — runs daily at ' + (c.report_time || '08:00') + '. Last reported night: ' + last + '.';
|
||||
} else {
|
||||
ss.innerHTML = '<span style="color:#9ca3af">●</span> Automatic sending is off. Last reported night: ' + last + '.';
|
||||
}
|
||||
document.getElementById('rs-test-status').textContent = '';
|
||||
rsSetMode(c.baseline_mode || 'captured');
|
||||
loadBaselineEditor(projectId);
|
||||
show();
|
||||
})
|
||||
.catch(show);
|
||||
}
|
||||
function closeReportSettings() {
|
||||
document.getElementById('report-settings-modal').classList.add('hidden');
|
||||
}
|
||||
function saveReportSettings(projectId) {
|
||||
var st = document.getElementById('rs-status');
|
||||
var mode = rsGetMode();
|
||||
var bs = document.getElementById('rs-baseline-start').value;
|
||||
var be = document.getElementById('rs-baseline-end').value;
|
||||
if (mode === 'captured' && ((bs && !be) || (be && !bs))) {
|
||||
st.style.color = '#b00020'; st.textContent = 'Provide both baseline dates, or neither.'; return;
|
||||
}
|
||||
var body = {
|
||||
enabled: document.getElementById('rs-enabled').checked,
|
||||
report_time: document.getElementById('rs-report-time').value || '08:00',
|
||||
metric_keys: document.getElementById('rs-metrics').value || 'lmax,l01,l10,l90',
|
||||
baseline_mode: mode,
|
||||
baseline_start: bs || null,
|
||||
baseline_end: be || null,
|
||||
recipients: document.getElementById('rs-recipients').value || ''
|
||||
};
|
||||
st.style.color = ''; st.textContent = 'Saving…';
|
||||
fetch('/api/projects/' + projectId + '/reports/config', {
|
||||
method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body)
|
||||
}).then(function (r) { return r.json().then(function (j) { return { ok: r.ok, j: j }; }); })
|
||||
.then(function (res) {
|
||||
if (!res.ok) { st.style.color = '#b00020'; st.textContent = 'Error: ' + (res.j.detail || 'save failed'); return; }
|
||||
if (mode === 'reference') {
|
||||
return fetch('/api/projects/' + projectId + '/reports/baseline', {
|
||||
method: 'PUT', headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ locations: gatherRefValues() })
|
||||
}).then(function (r2) {
|
||||
if (!r2.ok) throw new Error('baseline values failed to save');
|
||||
st.style.color = '#1a7f37'; st.textContent = 'Saved.'; setTimeout(closeReportSettings, 700);
|
||||
});
|
||||
}
|
||||
st.style.color = '#1a7f37'; st.textContent = 'Saved.'; setTimeout(closeReportSettings, 700);
|
||||
})
|
||||
.catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
|
||||
}
|
||||
var RS_BASELINE = { metrics: [], windows: [], locations: [] };
|
||||
function rsGetMode() {
|
||||
var r = document.querySelector('input[name="rs-baseline-mode"]:checked');
|
||||
return r ? r.value : 'captured';
|
||||
}
|
||||
function rsSetMode(mode) {
|
||||
document.querySelectorAll('input[name="rs-baseline-mode"]').forEach(function (el) { el.checked = (el.value === mode); });
|
||||
rsToggleBaselineMode();
|
||||
}
|
||||
function rsToggleBaselineMode() {
|
||||
var ref = rsGetMode() === 'reference';
|
||||
document.getElementById('rs-baseline-captured').classList.toggle('hidden', ref);
|
||||
document.getElementById('rs-baseline-reference').classList.toggle('hidden', !ref);
|
||||
}
|
||||
function loadBaselineEditor(projectId) {
|
||||
fetch('/api/projects/' + projectId + '/reports/baseline')
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (d) { RS_BASELINE = d; renderRefGrid(); })
|
||||
.catch(function () {});
|
||||
}
|
||||
function _refId(loc, w, m) { return 'ref__' + loc + '__' + w + '__' + m; }
|
||||
function renderRefGrid() {
|
||||
var box = document.getElementById('rs-ref-grid');
|
||||
if (!RS_BASELINE.locations || !RS_BASELINE.locations.length) {
|
||||
box.innerHTML = '<div class="text-xs text-gray-400">No NRLs in this project yet.</div>'; return;
|
||||
}
|
||||
var W = RS_BASELINE.windows, M = RS_BASELINE.metrics;
|
||||
box.innerHTML = RS_BASELINE.locations.map(function (loc) {
|
||||
var head = '<tr><th></th>' + W.map(function (w) {
|
||||
return '<th class="text-xs text-gray-400 font-normal pb-1 px-1">' + w.label.replace(/\s*\(.*\)/, '') + '</th>';
|
||||
}).join('') + '</tr>';
|
||||
var rows = M.map(function (m) {
|
||||
var cells = W.map(function (w) {
|
||||
var v = (loc.values[w.key] && loc.values[w.key][m.key] != null) ? loc.values[w.key][m.key] : '';
|
||||
return '<td class="px-1"><input type="number" step="0.1" id="' + _refId(loc.id, w.key, m.key) + '" value="' + _mergeEsc(v) + '" class="w-16 px-1.5 py-1 border border-gray-300 dark:border-gray-600 rounded bg-white dark:bg-slate-700 text-gray-900 dark:text-white text-sm text-center"></td>';
|
||||
}).join('');
|
||||
return '<tr><td class="text-sm text-gray-700 dark:text-gray-300 pr-2">' + m.label + '</td>' + cells + '</tr>';
|
||||
}).join('');
|
||||
return '<div class="border border-gray-200 dark:border-gray-700 rounded-lg p-2">'
|
||||
+ '<div class="text-sm font-medium text-gray-800 dark:text-gray-200 mb-1">' + _mergeEsc(loc.name) + '</div>'
|
||||
+ '<table class="w-full">' + head + rows + '</table></div>';
|
||||
}).join('');
|
||||
}
|
||||
function gatherRefValues() {
|
||||
var out = {};
|
||||
(RS_BASELINE.locations || []).forEach(function (loc) {
|
||||
var wins = {};
|
||||
RS_BASELINE.windows.forEach(function (w) {
|
||||
var mv = {};
|
||||
RS_BASELINE.metrics.forEach(function (m) {
|
||||
var el = document.getElementById(_refId(loc.id, w.key, m.key));
|
||||
if (el && el.value !== '') mv[m.key] = el.value;
|
||||
});
|
||||
if (Object.keys(mv).length) wins[w.key] = mv;
|
||||
});
|
||||
out[loc.id] = wins;
|
||||
});
|
||||
return out;
|
||||
}
|
||||
function rsCopyFirstNrl() {
|
||||
if (!RS_BASELINE.locations || RS_BASELINE.locations.length < 2) return;
|
||||
var first = RS_BASELINE.locations[0].id;
|
||||
RS_BASELINE.locations.slice(1).forEach(function (loc) {
|
||||
RS_BASELINE.windows.forEach(function (w) {
|
||||
RS_BASELINE.metrics.forEach(function (m) {
|
||||
var src = document.getElementById(_refId(first, w.key, m.key));
|
||||
var dst = document.getElementById(_refId(loc.id, w.key, m.key));
|
||||
if (src && dst) dst.value = src.value;
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
function sendTestEmail(projectId) {
|
||||
var st = document.getElementById('rs-test-status');
|
||||
st.style.color = ''; st.textContent = 'Sending…';
|
||||
var recips = document.getElementById('rs-recipients').value;
|
||||
fetch('/api/projects/' + projectId + '/reports/test-email', {
|
||||
method: 'POST', headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(recips ? { recipients: recips } : {})
|
||||
}).then(function (r) { return r.json(); })
|
||||
.then(function (j) {
|
||||
if (j.sent) { st.style.color = '#1a7f37'; st.textContent = 'Sent to ' + (j.recipients || []).join(', '); }
|
||||
else if (j.dry_run) { st.style.color = '#b8860b'; st.textContent = 'Dry-run (SMTP not set) — would send to ' + (j.recipients || []).join(', '); }
|
||||
else { st.style.color = '#b00020'; st.textContent = 'Error: ' + (j.error || 'failed'); }
|
||||
})
|
||||
.catch(function (e) { st.style.color = '#b00020'; st.textContent = 'Error: ' + e; });
|
||||
}
|
||||
</script>
|
||||
|
||||
<!-- Merge Modal —
|
||||
min-h on the body ensures the typeahead dropdown has room to render
|
||||
below the input without forcing the operator to scroll inside the
|
||||
|
||||
@@ -2,14 +2,7 @@
|
||||
{% if units %}
|
||||
{% for unit in units %}
|
||||
<div class="bg-gray-50 dark:bg-gray-800 rounded-lg p-4 border border-transparent hover:border-seismo-orange transition-colors relative">
|
||||
<div class="absolute top-3 right-3 flex gap-2 z-10">
|
||||
<button onclick="event.preventDefault(); event.stopPropagation(); refreshSlmUnit('{{ unit.id }}', this);"
|
||||
class="text-gray-500 hover:text-seismo-orange dark:text-gray-400 dark:hover:text-seismo-orange"
|
||||
title="Refresh {{ unit.id }} from device">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
|
||||
</svg>
|
||||
</button>
|
||||
<div class="absolute top-3 right-3 flex gap-2">
|
||||
<button onclick="event.preventDefault(); event.stopPropagation(); showLiveChart('{{ unit.id }}');"
|
||||
class="text-gray-500 hover:text-seismo-orange dark:text-gray-400 dark:hover:text-seismo-orange"
|
||||
title="View live chart">
|
||||
@@ -27,44 +20,41 @@
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<a href="/slm/{{ unit.id }}" class="block pr-24">
|
||||
<div class="min-w-0">
|
||||
<div class="flex items-center gap-2">
|
||||
<span class="font-semibold text-gray-900 dark:text-white">{{ unit.id }}</span>
|
||||
{% if unit.slm_model %}
|
||||
<span class="text-xs text-gray-500 dark:text-gray-400">• {{ unit.slm_model }}</span>
|
||||
<a href="/slm/{{ unit.id }}" class="block">
|
||||
<div class="flex items-start justify-between gap-4">
|
||||
<div class="min-w-0 flex-1">
|
||||
<div class="flex items-center gap-2">
|
||||
<span class="font-semibold text-gray-900 dark:text-white">{{ unit.id }}</span>
|
||||
{% if unit.slm_model %}
|
||||
<span class="text-xs text-gray-500 dark:text-gray-400">• {{ unit.slm_model }}</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if unit.address %}
|
||||
<p class="text-sm text-gray-600 dark:text-gray-400 truncate mt-1">{{ unit.address }}</p>
|
||||
{% elif unit.location %}
|
||||
<p class="text-sm text-gray-600 dark:text-gray-400 truncate mt-1">{{ unit.location }}</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if unit.address %}
|
||||
<p class="text-sm text-gray-600 dark:text-gray-400 truncate mt-1">{{ unit.address }}</p>
|
||||
{% elif unit.location %}
|
||||
<p class="text-sm text-gray-600 dark:text-gray-400 truncate mt-1">{{ unit.location }}</p>
|
||||
|
||||
{% if unit.retired %}
|
||||
<span class="shrink-0 px-2 py-1 text-xs font-medium bg-gray-200 text-gray-700 dark:bg-gray-700 dark:text-gray-300 rounded-full">Retired</span>
|
||||
{% elif not unit.deployed %}
|
||||
<span class="shrink-0 px-2 py-1 text-xs font-medium bg-amber-100 text-amber-800 dark:bg-amber-900/30 dark:text-amber-300 rounded-full">Benched</span>
|
||||
{% elif unit.measurement_state == "Start" %}
|
||||
<span class="shrink-0 px-2 py-1 text-xs font-medium bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-300 rounded-full">Measuring</span>
|
||||
{% elif unit.is_recent %}
|
||||
<span class="shrink-0 px-2 py-1 text-xs font-medium bg-blue-100 text-blue-800 dark:bg-blue-900/30 dark:text-blue-300 rounded-full">Active</span>
|
||||
{% else %}
|
||||
<span class="shrink-0 px-2 py-1 text-xs font-medium bg-yellow-100 text-yellow-800 dark:bg-yellow-900/30 dark:text-yellow-300 rounded-full">Idle</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
<!-- Status badge + last-check on one line (moved off the top-right so it
|
||||
no longer collides with the refresh/chart/gear action icons). -->
|
||||
<div class="mt-2 flex items-center gap-2 flex-wrap">
|
||||
{% if unit.retired %}
|
||||
<span class="px-2 py-0.5 text-xs font-medium bg-gray-200 text-gray-700 dark:bg-gray-700 dark:text-gray-300 rounded-full">Retired</span>
|
||||
{% elif not unit.deployed %}
|
||||
<span class="px-2 py-0.5 text-xs font-medium bg-amber-100 text-amber-800 dark:bg-amber-900/30 dark:text-amber-300 rounded-full">Benched</span>
|
||||
{% elif unit.measurement_state in ["Start", "Measure"] %}
|
||||
<span class="px-2 py-0.5 text-xs font-medium bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-300 rounded-full">Measuring</span>
|
||||
{% elif unit.is_recent %}
|
||||
<span class="px-2 py-0.5 text-xs font-medium bg-blue-100 text-blue-800 dark:bg-blue-900/30 dark:text-blue-300 rounded-full">Active</span>
|
||||
<div class="mt-2 text-xs text-gray-500 dark:text-gray-400">
|
||||
{% if unit.slm_last_check %}
|
||||
Last check: {{ unit.slm_last_check|local_datetime }}
|
||||
{% else %}
|
||||
<span class="px-2 py-0.5 text-xs font-medium bg-yellow-100 text-yellow-800 dark:bg-yellow-900/30 dark:text-yellow-300 rounded-full">Idle</span>
|
||||
No recent check-in
|
||||
{% endif %}
|
||||
<span class="text-xs text-gray-500 dark:text-gray-400">
|
||||
{% if unit.cache_last_seen %}
|
||||
Last check: {{ unit.cache_last_seen|local_datetime }}
|
||||
{% elif unit.slm_last_check %}
|
||||
Last check: {{ unit.slm_last_check|local_datetime }}
|
||||
{% else %}
|
||||
No recent check-in
|
||||
{% endif %}
|
||||
</span>
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
@@ -143,8 +143,6 @@
|
||||
</svg>
|
||||
Stop Live Stream
|
||||
</button>
|
||||
|
||||
<span id="live-feed-status" class="ml-3 self-center" style="display: none;"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -175,17 +173,17 @@
|
||||
</div>
|
||||
|
||||
<div class="bg-purple-50 dark:bg-purple-900/20 rounded-lg p-4">
|
||||
<p id="live-ln1-label" class="text-xs text-gray-600 dark:text-gray-400 mb-1">{% if current_status and current_status.ln1_label %}{{ current_status.ln1_label }}{% else %}L1{% endif %}</p>
|
||||
<p id="live-ln1" class="text-2xl font-bold text-purple-600 dark:text-purple-400">
|
||||
{% if current_status and current_status.ln1 %}{{ current_status.ln1 }}{% else %}--{% endif %}
|
||||
<p class="text-xs text-gray-600 dark:text-gray-400 mb-1">Lmin (Min)</p>
|
||||
<p id="live-lmin" class="text-2xl font-bold text-purple-600 dark:text-purple-400">
|
||||
{% if current_status and current_status.lmin %}{{ current_status.lmin }}{% else %}--{% endif %}
|
||||
</p>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">dB</p>
|
||||
</div>
|
||||
|
||||
<div class="bg-orange-50 dark:bg-orange-900/20 rounded-lg p-4">
|
||||
<p id="live-ln2-label" class="text-xs text-gray-600 dark:text-gray-400 mb-1">{% if current_status and current_status.ln2_label %}{{ current_status.ln2_label }}{% else %}L10{% endif %}</p>
|
||||
<p id="live-ln2" class="text-2xl font-bold text-orange-600 dark:text-orange-400">
|
||||
{% if current_status and current_status.ln2 %}{{ current_status.ln2 }}{% else %}--{% endif %}
|
||||
<p class="text-xs text-gray-600 dark:text-gray-400 mb-1">Lpeak (Peak)</p>
|
||||
<p id="live-lpeak" class="text-2xl font-bold text-orange-600 dark:text-orange-400">
|
||||
{% if current_status and current_status.lpeak %}{{ current_status.lpeak }}{% else %}--{% endif %}
|
||||
</p>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">dB</p>
|
||||
</div>
|
||||
@@ -434,24 +432,6 @@ function initializeChart() {
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0
|
||||
},
|
||||
{
|
||||
label: 'L1',
|
||||
data: [],
|
||||
borderColor: 'rgb(139, 92, 246)',
|
||||
backgroundColor: 'rgba(139, 92, 246, 0.1)',
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0
|
||||
},
|
||||
{
|
||||
label: 'L10',
|
||||
data: [],
|
||||
borderColor: 'rgb(245, 158, 11)',
|
||||
backgroundColor: 'rgba(245, 158, 11, 0.1)',
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -513,37 +493,7 @@ if (typeof window.currentWebSocket === 'undefined') {
|
||||
window.currentWebSocket = null;
|
||||
}
|
||||
|
||||
// Backfill the chart with the recent DOD trail so it opens with context.
|
||||
async function backfillChart(unitId) {
|
||||
try {
|
||||
const r = await fetch(`/api/slmm/${encodeURIComponent(unitId)}/history?hours=2`);
|
||||
if (!r.ok) return;
|
||||
const d = await r.json();
|
||||
const readings = d.readings || [];
|
||||
if (!window.chartData) return;
|
||||
for (const row of readings) {
|
||||
// Trail timestamps are naive UTC; append 'Z' so they convert to local
|
||||
// consistently with the live frames (which use local Date.now()).
|
||||
window.chartData.timestamps.push(row.timestamp ? new Date(row.timestamp + 'Z').toLocaleTimeString() : '');
|
||||
window.chartData.lp.push(parseFloat(row.lp || 0));
|
||||
window.chartData.leq.push(parseFloat(row.leq || 0));
|
||||
window.chartData.ln1.push(parseFloat(row.ln1 || 0));
|
||||
window.chartData.ln2.push(parseFloat(row.ln2 || 0));
|
||||
}
|
||||
if (window.liveChart) {
|
||||
window.liveChart.data.labels = window.chartData.timestamps;
|
||||
window.liveChart.data.datasets[0].data = window.chartData.lp;
|
||||
window.liveChart.data.datasets[1].data = window.chartData.leq;
|
||||
window.liveChart.data.datasets[2].data = window.chartData.ln1;
|
||||
window.liveChart.data.datasets[3].data = window.chartData.ln2;
|
||||
window.liveChart.update('none');
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('Chart backfill failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
async function initLiveDataStream(unitId) {
|
||||
function initLiveDataStream(unitId) {
|
||||
// Close existing connection if any
|
||||
if (window.currentWebSocket) {
|
||||
window.currentWebSocket.close();
|
||||
@@ -554,24 +504,17 @@ async function initLiveDataStream(unitId) {
|
||||
window.chartData.timestamps = [];
|
||||
window.chartData.lp = [];
|
||||
window.chartData.leq = [];
|
||||
window.chartData.ln1 = [];
|
||||
window.chartData.ln2 = [];
|
||||
}
|
||||
if (window.liveChart && window.liveChart.data && window.liveChart.data.datasets) {
|
||||
window.liveChart.data.labels = [];
|
||||
window.liveChart.data.datasets.forEach(ds => ds.data = []);
|
||||
window.liveChart.data.datasets[0].data = [];
|
||||
window.liveChart.data.datasets[1].data = [];
|
||||
window.liveChart.update();
|
||||
}
|
||||
|
||||
// Seed the chart with recent history BEFORE opening the live socket, so live
|
||||
// frames append after the backfill (right order) and the chart isn't blank.
|
||||
await backfillChart(unitId);
|
||||
|
||||
// WebSocket URL for SLMM backend via proxy.
|
||||
// /monitor = the shared fan-out DOD feed (many viewers, one device connection,
|
||||
// and it carries L1/L10 which the DRD /stream cannot).
|
||||
// WebSocket URL for SLMM backend via proxy
|
||||
const wsProtocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
const wsUrl = `${wsProtocol}//${window.location.host}/api/slmm/${unitId}/monitor`;
|
||||
const wsUrl = `${wsProtocol}//${window.location.host}/api/slmm/${unitId}/live`;
|
||||
|
||||
window.currentWebSocket = new WebSocket(wsUrl);
|
||||
|
||||
@@ -587,11 +530,7 @@ async function initLiveDataStream(unitId) {
|
||||
window.currentWebSocket.onmessage = function(event) {
|
||||
try {
|
||||
const data = JSON.parse(event.data);
|
||||
// The DOD monitor sends keepalive 'heartbeat' frames (no metrics) and a
|
||||
// 'feed_status' on each frame. Reflect status, but don't let a heartbeat
|
||||
// or an 'unreachable' frame blank the cards / spike the chart with zeros.
|
||||
updateFeedStatus(data.feed_status);
|
||||
if (data.heartbeat || data.feed_status === 'unreachable') return;
|
||||
console.log('WebSocket data received:', data);
|
||||
updateLiveMetrics(data);
|
||||
updateLiveChart(data);
|
||||
} catch (error) {
|
||||
@@ -620,21 +559,6 @@ function stopLiveDataStream() {
|
||||
}
|
||||
}
|
||||
|
||||
// Reflect device reachability from the monitor feed's feed_status. Safe no-op
|
||||
// if the badge element isn't on the page.
|
||||
function updateFeedStatus(status) {
|
||||
const el = document.getElementById('live-feed-status');
|
||||
if (!el || status == null) return;
|
||||
if (status === 'unreachable') {
|
||||
el.textContent = 'Device offline';
|
||||
el.className = 'text-xs font-medium px-2 py-0.5 rounded bg-red-100 text-red-700 dark:bg-red-900/40 dark:text-red-300';
|
||||
} else {
|
||||
el.textContent = 'Live';
|
||||
el.className = 'text-xs font-medium px-2 py-0.5 rounded bg-green-100 text-green-700 dark:bg-green-900/40 dark:text-green-300';
|
||||
}
|
||||
el.style.display = '';
|
||||
}
|
||||
|
||||
// Update metrics display
|
||||
function updateLiveMetrics(data) {
|
||||
if (document.getElementById('live-lp')) {
|
||||
@@ -646,20 +570,11 @@ function updateLiveMetrics(data) {
|
||||
if (document.getElementById('live-lmax')) {
|
||||
document.getElementById('live-lmax').textContent = data.lmax || '--';
|
||||
}
|
||||
// Only update Ln values when the frame actually carries them. DRD stream
|
||||
// frames omit percentiles (DOD-only), so without this guard a live stream
|
||||
// would blank L1/L10 over the values rendered from the cached DOD snapshot.
|
||||
if (data.ln1 != null && document.getElementById('live-ln1')) {
|
||||
document.getElementById('live-ln1').textContent = data.ln1;
|
||||
if (document.getElementById('live-lmin')) {
|
||||
document.getElementById('live-lmin').textContent = data.lmin || '--';
|
||||
}
|
||||
if (data.ln1_label && document.getElementById('live-ln1-label')) {
|
||||
document.getElementById('live-ln1-label').textContent = data.ln1_label;
|
||||
}
|
||||
if (data.ln2 != null && document.getElementById('live-ln2')) {
|
||||
document.getElementById('live-ln2').textContent = data.ln2;
|
||||
}
|
||||
if (data.ln2_label && document.getElementById('live-ln2-label')) {
|
||||
document.getElementById('live-ln2-label').textContent = data.ln2_label;
|
||||
if (document.getElementById('live-lpeak')) {
|
||||
document.getElementById('live-lpeak').textContent = data.lpeak || '--';
|
||||
}
|
||||
}
|
||||
|
||||
@@ -668,9 +583,7 @@ if (typeof window.chartData === 'undefined') {
|
||||
window.chartData = {
|
||||
timestamps: [],
|
||||
lp: [],
|
||||
leq: [],
|
||||
ln1: [],
|
||||
ln2: []
|
||||
leq: []
|
||||
};
|
||||
}
|
||||
|
||||
@@ -680,17 +593,12 @@ function updateLiveChart(data) {
|
||||
window.chartData.timestamps.push(now.toLocaleTimeString());
|
||||
window.chartData.lp.push(parseFloat(data.lp || 0));
|
||||
window.chartData.leq.push(parseFloat(data.leq || 0));
|
||||
window.chartData.ln1.push(parseFloat(data.ln1 || 0));
|
||||
window.chartData.ln2.push(parseFloat(data.ln2 || 0));
|
||||
|
||||
// Keep a rolling window large enough to hold the ~2h backfill (one point/min)
|
||||
// plus a good run of live points before the oldest scroll off.
|
||||
if (window.chartData.timestamps.length > 600) {
|
||||
// Keep only last 60 data points
|
||||
if (window.chartData.timestamps.length > 60) {
|
||||
window.chartData.timestamps.shift();
|
||||
window.chartData.lp.shift();
|
||||
window.chartData.leq.shift();
|
||||
window.chartData.ln1.shift();
|
||||
window.chartData.ln2.shift();
|
||||
}
|
||||
|
||||
// Update chart if available
|
||||
@@ -698,8 +606,6 @@ function updateLiveChart(data) {
|
||||
window.liveChart.data.labels = window.chartData.timestamps;
|
||||
window.liveChart.data.datasets[0].data = window.chartData.lp;
|
||||
window.liveChart.data.datasets[1].data = window.chartData.leq;
|
||||
window.liveChart.data.datasets[2].data = window.chartData.ln1;
|
||||
window.liveChart.data.datasets[3].data = window.chartData.ln2;
|
||||
window.liveChart.update('none');
|
||||
}
|
||||
}
|
||||
|
||||
@@ -528,7 +528,7 @@ async function saveSLMSettings(event) {
|
||||
if (typeof checkFTPStatus === 'function') {
|
||||
checkFTPStatus(unitId);
|
||||
}
|
||||
if (typeof htmx !== 'undefined' && document.getElementById('slm-list')) {
|
||||
if (typeof htmx !== 'undefined') {
|
||||
htmx.trigger('#slm-list', 'load');
|
||||
}
|
||||
}, 1500);
|
||||
@@ -604,10 +604,8 @@ async function toggleSLMDeployed() {
|
||||
successDiv.classList.remove('hidden');
|
||||
setTimeout(() => successDiv.classList.add('hidden'), 3000);
|
||||
|
||||
// Refresh any SLM list on the page (only if one is actually present —
|
||||
// the detail/dashboard pages have no #slm-list, and htmx.trigger on a
|
||||
// null target throws "can't access property dispatchEvent, e is null").
|
||||
if (typeof htmx !== 'undefined' && document.getElementById('slm-list')) {
|
||||
// Refresh any SLM list on the page
|
||||
if (typeof htmx !== 'undefined') {
|
||||
htmx.trigger('#slm-list', 'load');
|
||||
}
|
||||
} catch (error) {
|
||||
|
||||
@@ -1,19 +0,0 @@
|
||||
{% extends "portal/base.html" %}
|
||||
{% block title %}Access{% endblock %}
|
||||
{% block content %}
|
||||
<div class="max-w-md mx-auto mt-20 text-center reveal">
|
||||
<div class="panel inline-flex items-center justify-center w-16 h-16 rounded-2xl mb-6">
|
||||
<svg class="w-7 h-7 text-[var(--text-dim)]" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"
|
||||
d="M12 15v2m-6 4h12a2 2 0 002-2v-6a2 2 0 00-2-2H6a2 2 0 00-2 2v6a2 2 0 002 2zm10-10V7a4 4 0 00-8 0v4h8z"></path>
|
||||
</svg>
|
||||
</div>
|
||||
{% if reason == "invalid" %}
|
||||
<h1 class="text-2xl font-bold tracking-tight mb-2">This link isn't valid</h1>
|
||||
<p class="text-[var(--text-dim)] text-sm leading-relaxed">The access link is expired or has been revoked.<br>Please contact TMI for a new link.</p>
|
||||
{% else %}
|
||||
<h1 class="text-2xl font-bold tracking-tight mb-2">Access link required</h1>
|
||||
<p class="text-[var(--text-dim)] text-sm leading-relaxed">Open the monitoring link TMI sent you to view your locations.</p>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endblock %}
|
||||
@@ -1,196 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en" class="h-full">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>{% block title %}Monitoring{% endblock %} · TMI</title>
|
||||
|
||||
<!-- apply saved theme before paint (no flash); light is the default -->
|
||||
<script>(function(){var t=localStorage.getItem('portal-theme')||'light';document.documentElement.setAttribute('data-theme',t);})();</script>
|
||||
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<link href="https://fonts.googleapis.com/css2?family=Hanken+Grotesk:wght@400;500;600;700;800&family=IBM+Plex+Mono:wght@400;500;600&display=swap" rel="stylesheet">
|
||||
|
||||
<script src="https://cdn.tailwindcss.com"></script>
|
||||
<script>
|
||||
tailwind.config = {
|
||||
theme: { extend: {
|
||||
colors: { seismo: { orange: '#f48b1c', navy: '#142a66', burgundy: '#7d234d' } },
|
||||
fontFamily: {
|
||||
sans: ['"Hanken Grotesk"', 'ui-sans-serif', 'system-ui', 'sans-serif'],
|
||||
mono: ['"IBM Plex Mono"', 'ui-monospace', 'monospace'],
|
||||
},
|
||||
} }
|
||||
}
|
||||
</script>
|
||||
<link rel="icon" type="image/png" sizes="32x32" href="/static/icons/favicon-32.png">
|
||||
<meta name="theme-color" content="#eef2f9">
|
||||
|
||||
<style>
|
||||
/* ---- dark (default) ---- */
|
||||
:root {
|
||||
--bg: #080b14;
|
||||
--grid: rgba(124, 146, 188, 0.045);
|
||||
--aurora-1: rgba(20, 42, 102, 0.55);
|
||||
--aurora-2: rgba(125, 35, 77, 0.18);
|
||||
--text: #e7ecf6;
|
||||
--text-dim: #8c98b0;
|
||||
--border: rgba(124, 146, 188, 0.14);
|
||||
--border-bright: rgba(168, 188, 224, 0.30);
|
||||
--panel-a: rgba(24, 33, 54, 0.72);
|
||||
--panel-b: rgba(12, 18, 31, 0.62);
|
||||
--panel-inset: rgba(255, 255, 255, 0.05);
|
||||
--panel-shadow: 0 22px 48px -28px rgba(0, 0, 0, 0.85);
|
||||
--header-bg: rgba(8, 11, 20, 0.72);
|
||||
--accent: #f48b1c;
|
||||
--accent-glow: rgba(244, 139, 28, 0.40);
|
||||
--lvl-ok: #34d399; --lvl-warn: #fbbf24; --lvl-bad: #f87171;
|
||||
--m-lp: #60a5fa; --m-lmax: #f87171; --m-l1: #c084fc; --m-l10: #fbbf24;
|
||||
}
|
||||
/* ---- light (cool) — solid cards on a cool ground ---- */
|
||||
html[data-theme="light"] {
|
||||
--bg: #eef2f9; /* cool light */
|
||||
--grid: rgba(20, 42, 102, 0.05); /* cool faint grid */
|
||||
--aurora-1: rgba(120, 150, 220, 0.18); /* cool wash */
|
||||
--aurora-2: rgba(244, 139, 28, 0.08); /* faint brand accent */
|
||||
--text: #16203a; /* cool navy ink */
|
||||
--text-dim: #5d6b86; /* cool muted */
|
||||
--border: rgba(20, 42, 102, 0.13);
|
||||
--border-bright: rgba(20, 42, 102, 0.18);
|
||||
--panel-a: #ffffff; /* solid — kept from the un-ghosting pass */
|
||||
--panel-b: #f7f9fc;
|
||||
--panel-inset: rgba(255, 255, 255, 0.9);
|
||||
--panel-shadow: 0 14px 30px -16px rgba(40, 55, 95, 0.22), 0 2px 6px -2px rgba(40, 55, 95, 0.07);
|
||||
--header-bg: rgba(238, 242, 249, 0.85);
|
||||
--lvl-ok: #16a34a; --lvl-warn: #d97706; --lvl-bad: #dc2626;
|
||||
--m-lp: #2563eb; --m-lmax: #dc2626; --m-l1: #9333ea; --m-l10: #d97706;
|
||||
}
|
||||
/* On light, the hover-lift shadow wants cool depth (the dark one vanishes on light). */
|
||||
html[data-theme="light"] .panel-hover:hover {
|
||||
box-shadow: 0 22px 44px -20px rgba(40, 55, 95, 0.26), 0 0 0 1px var(--accent-glow);
|
||||
}
|
||||
|
||||
html, body { height: 100%; }
|
||||
body {
|
||||
margin: 0;
|
||||
color: var(--text);
|
||||
font-family: "Hanken Grotesk", ui-sans-serif, system-ui, sans-serif;
|
||||
font-feature-settings: "ss01";
|
||||
background-color: var(--bg);
|
||||
background-image:
|
||||
radial-gradient(1100px 560px at 50% -12%, var(--aurora-1), transparent 68%),
|
||||
radial-gradient(700px 400px at 88% 8%, var(--aurora-2), transparent 70%),
|
||||
linear-gradient(var(--grid) 1px, transparent 1px),
|
||||
linear-gradient(90deg, var(--grid) 1px, transparent 1px);
|
||||
background-size: auto, auto, 46px 46px, 46px 46px;
|
||||
background-attachment: fixed;
|
||||
transition: background-color .3s ease, color .3s ease;
|
||||
}
|
||||
::selection { background: rgba(244, 139, 28, 0.30); }
|
||||
|
||||
.font-mono, .reading { font-family: "IBM Plex Mono", ui-monospace, monospace; font-variant-numeric: tabular-nums; }
|
||||
|
||||
.panel {
|
||||
position: relative;
|
||||
background: linear-gradient(180deg, var(--panel-a), var(--panel-b));
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 16px;
|
||||
backdrop-filter: blur(8px);
|
||||
-webkit-backdrop-filter: blur(8px);
|
||||
box-shadow: 0 1px 0 var(--panel-inset) inset, var(--panel-shadow);
|
||||
}
|
||||
.panel::before {
|
||||
content: ''; position: absolute; inset: 0 0 auto 0; height: 1px;
|
||||
background: linear-gradient(90deg, transparent, var(--border-bright), transparent);
|
||||
}
|
||||
.panel-hover { transition: transform .22s ease, border-color .22s ease, box-shadow .22s ease; }
|
||||
.panel-hover:hover {
|
||||
transform: translateY(-3px);
|
||||
border-color: rgba(244, 139, 28, 0.55);
|
||||
box-shadow: 0 30px 60px -30px rgba(0, 0, 0, 0.55), 0 0 0 1px var(--accent-glow);
|
||||
}
|
||||
.hairline { border-top: 1px solid var(--border); }
|
||||
|
||||
/* metric accent colors (flip per theme) */
|
||||
.c-lp { color: var(--m-lp); } .c-lmax { color: var(--m-lmax); }
|
||||
.c-l1 { color: var(--m-l1); } .c-l10 { color: var(--m-l10); }
|
||||
|
||||
.live-dot { width: 8px; height: 8px; border-radius: 999px; background: var(--accent); box-shadow: 0 0 0 0 var(--accent-glow); animation: pulse 2.2s infinite; }
|
||||
@keyframes pulse { 0% { box-shadow: 0 0 0 0 var(--accent-glow); } 70% { box-shadow: 0 0 0 9px rgba(244, 139, 28, 0); } 100% { box-shadow: 0 0 0 0 rgba(244, 139, 28, 0); } }
|
||||
|
||||
@keyframes rise { from { opacity: 0; transform: translateY(12px); } to { opacity: 1; transform: none; } }
|
||||
.reveal { opacity: 0; animation: rise .55s cubic-bezier(.2, .7, .2, 1) forwards; }
|
||||
|
||||
.signal-bars { display: inline-flex; align-items: flex-end; gap: 2px; height: 16px; }
|
||||
.signal-bars i { width: 3px; background: var(--accent); border-radius: 1px; animation: bars 1.4s ease-in-out infinite; }
|
||||
.signal-bars i:nth-child(1) { height: 40%; } .signal-bars i:nth-child(2) { height: 70%; animation-delay: .2s; }
|
||||
.signal-bars i:nth-child(3) { height: 100%; animation-delay: .4s; } .signal-bars i:nth-child(4) { height: 55%; animation-delay: .6s; }
|
||||
@keyframes bars { 0%, 100% { transform: scaleY(.5); opacity: .7; } 50% { transform: scaleY(1); opacity: 1; } }
|
||||
|
||||
.theme-toggle { color: var(--text-dim); transition: color .2s ease, background .2s ease; }
|
||||
.theme-toggle:hover { color: var(--text); }
|
||||
html[data-theme="light"] .moon { display: none; }
|
||||
html[data-theme="dark"] .sun, :root:not([data-theme="light"]) .sun { display: none; }
|
||||
|
||||
/* Leaflet polish (dark default; .leaflet-light tweaks tooltip for light) */
|
||||
.leaflet-container { background: var(--bg) !important; }
|
||||
.leaflet-tooltip { background: var(--panel-a); border: 1px solid var(--border-bright); color: var(--text); box-shadow: none; font-family: inherit; font-size: 12px; }
|
||||
.leaflet-tooltip-top::before { border-top-color: var(--border-bright); }
|
||||
.leaflet-control-attribution { background: rgba(0,0,0,0.25) !important; color: var(--text-dim) !important; }
|
||||
.leaflet-control-attribution a { color: var(--text-dim) !important; }
|
||||
::-webkit-scrollbar { width: 9px; height: 9px; }
|
||||
::-webkit-scrollbar-thumb { background: rgba(124, 146, 188, 0.22); border-radius: 9px; }
|
||||
</style>
|
||||
{% block head %}{% endblock %}
|
||||
</head>
|
||||
<body class="min-h-full antialiased">
|
||||
<header class="sticky top-0 z-30 border-b border-[var(--border)] bg-[var(--header-bg)] backdrop-blur-xl">
|
||||
<div class="max-w-5xl mx-auto px-5 py-3.5 flex items-center justify-between">
|
||||
<a href="/portal" class="flex items-center gap-2.5">
|
||||
<span class="signal-bars"><i></i><i></i><i></i><i></i></span>
|
||||
<span class="font-semibold tracking-tight text-[15px]">
|
||||
TMI <span class="text-[var(--text-dim)] font-normal">Monitoring</span>
|
||||
{% if client %}<span class="text-[var(--text-dim)] font-normal mx-0.5">/</span>
|
||||
<span class="text-seismo-orange">{{ client.name }}</span>{% endif %}
|
||||
</span>
|
||||
</a>
|
||||
<div class="flex items-center gap-1.5">
|
||||
<button onclick="togglePortalTheme()" class="theme-toggle p-2 rounded-lg" title="Toggle light / dark" aria-label="Toggle theme">
|
||||
<svg class="moon w-[18px] h-[18px]" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M20.354 15.354A9 9 0 018.646 3.646 9.003 9.003 0 0012 21a9.003 9.003 0 008.354-5.646z"/></svg>
|
||||
<svg class="sun w-[18px] h-[18px]" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 3v1m0 16v1m9-9h-1M4 12H3m15.364 6.364l-.707-.707M6.343 6.343l-.707-.707m12.728 0l-.707.707M6.343 17.657l-.707.707M16 12a4 4 0 11-8 0 4 4 0 018 0z"/></svg>
|
||||
</button>
|
||||
{% if client %}
|
||||
<a href="/portal/logout" class="text-[13px] text-[var(--text-dim)] hover:text-[var(--text)] transition-colors px-2">Sign out</a>
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main class="max-w-5xl mx-auto px-5 py-8">
|
||||
{% block content %}{% endblock %}
|
||||
</main>
|
||||
|
||||
<footer class="max-w-5xl mx-auto px-5 py-10 text-[11px] text-[var(--text-dim)] flex items-center gap-2 opacity-70">
|
||||
<span class="w-1 h-1 rounded-full bg-[var(--text-dim)]"></span>
|
||||
Read-only monitoring view · data provided as-is for informational purposes
|
||||
</footer>
|
||||
|
||||
<script>
|
||||
// Theme toggle. Pages can listen for 'portal-theme' to re-skin canvases/maps.
|
||||
function cssVar(n) { return getComputedStyle(document.documentElement).getPropertyValue(n).trim(); }
|
||||
// HTML-escape operator-set strings (location/rule names) before innerHTML/tooltip injection.
|
||||
function esc(s) { return String(s == null ? '' : s).replace(/[&<>"']/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c])); }
|
||||
function togglePortalTheme() {
|
||||
const cur = document.documentElement.getAttribute('data-theme') === 'light' ? 'light' : 'dark';
|
||||
const next = cur === 'light' ? 'dark' : 'light';
|
||||
document.documentElement.setAttribute('data-theme', next);
|
||||
try { localStorage.setItem('portal-theme', next); } catch (e) {}
|
||||
const mc = document.querySelector('meta[name="theme-color"]');
|
||||
if (mc) mc.setAttribute('content', next === 'light' ? '#eef2f9' : '#080b14');
|
||||
document.dispatchEvent(new CustomEvent('portal-theme', { detail: next }));
|
||||
}
|
||||
</script>
|
||||
{% block scripts %}{% endblock %}
|
||||
</body>
|
||||
</html>
|
||||
@@ -1,335 +0,0 @@
|
||||
{% extends "portal/base.html" %}
|
||||
{% block title %}{{ location.name }}{% endblock %}
|
||||
{% block head %}
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@4"></script>
|
||||
{% endblock %}
|
||||
{% block content %}
|
||||
<a href="/portal" class="reveal inline-flex items-center gap-1.5 text-sm text-[var(--text-dim)] hover:text-[var(--text)] transition-colors">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 19l-7-7 7-7"/></svg>
|
||||
All locations
|
||||
</a>
|
||||
<div class="reveal mt-3 flex flex-wrap items-end justify-between gap-3">
|
||||
<h1 class="text-3xl font-bold tracking-tight">{{ location.name }}</h1>
|
||||
<div class="flex items-center gap-2.5">
|
||||
<span id="p-badge" class="hidden"></span>
|
||||
<span id="p-fresh" class="text-[var(--text-dim)] font-mono text-xs"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{% if not has_device %}
|
||||
<div class="panel reveal p-12 text-center text-[var(--text-dim)] mt-6">No device is currently assigned to this location.</div>
|
||||
{% else %}
|
||||
<div id="p-alarm-banner" class="hidden reveal mt-5 px-4 py-3 rounded-xl bg-[rgba(220,38,38,0.10)] border border-[rgba(220,38,38,0.32)] text-[var(--lvl-bad)] text-sm flex items-center gap-2.5">
|
||||
<svg class="w-5 h-5 shrink-0" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2"
|
||||
d="M12 9v2m0 4h.01M5.07 19h13.86c1.54 0 2.5-1.67 1.73-3L13.73 4a2 2 0 00-3.46 0L3.34 16c-.77 1.33.19 3 1.73 3z"/>
|
||||
</svg>
|
||||
<span id="p-alarm-text" class="font-medium">Currently above threshold.</span>
|
||||
</div>
|
||||
|
||||
<!-- Hero console: Leq primary + instrument strip -->
|
||||
<div class="panel reveal mt-5 p-6 sm:p-7" style="animation-delay:60ms">
|
||||
<div class="text-[10px] uppercase tracking-[0.22em] text-[var(--text-dim)] font-mono mb-1.5">Leq · average</div>
|
||||
<div class="flex items-baseline gap-2.5">
|
||||
<span id="p-leq" class="reading text-6xl sm:text-7xl leading-none font-semibold">--</span>
|
||||
<span class="text-sm text-[var(--text-dim)] font-mono">dB</span>
|
||||
</div>
|
||||
|
||||
<div class="hairline mt-6 pt-5 grid grid-cols-2 sm:grid-cols-4 gap-5">
|
||||
<div>
|
||||
<div class="text-[10px] uppercase tracking-[0.15em] text-[var(--text-dim)] font-mono">Lp · instant</div>
|
||||
<div class="mt-1 flex items-baseline gap-1"><span id="p-lp" class="reading text-2xl font-semibold c-lp">--</span><span class="text-[10px] text-[var(--text-dim)] font-mono">dB</span></div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-[10px] uppercase tracking-[0.15em] text-[var(--text-dim)] font-mono">Lmax · peak</div>
|
||||
<div class="mt-1 flex items-baseline gap-1"><span id="p-lmax" class="reading text-2xl font-semibold c-lmax">--</span><span class="text-[10px] text-[var(--text-dim)] font-mono">dB</span></div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-[10px] uppercase tracking-[0.15em] text-[var(--text-dim)] font-mono">L1</div>
|
||||
<div class="mt-1 flex items-baseline gap-1"><span id="p-ln1" class="reading text-2xl font-semibold c-l1">--</span><span class="text-[10px] text-[var(--text-dim)] font-mono">dB</span></div>
|
||||
</div>
|
||||
<div>
|
||||
<div class="text-[10px] uppercase tracking-[0.15em] text-[var(--text-dim)] font-mono">L10</div>
|
||||
<div class="mt-1 flex items-baseline gap-1"><span id="p-ln2" class="reading text-2xl font-semibold c-l10">--</span><span class="text-[10px] text-[var(--text-dim)] font-mono">dB</span></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Live trace -->
|
||||
<div class="panel reveal mt-5 overflow-hidden" style="animation-delay:120ms">
|
||||
<div class="px-5 pt-4 text-[10px] uppercase tracking-[0.22em] text-[var(--text-dim)] font-mono">Live trace · last 2h</div>
|
||||
<div class="relative px-3 pb-3 pt-2" style="min-height: 340px;">
|
||||
<canvas id="p-chart"></canvas>
|
||||
<div id="p-paused" class="hidden absolute inset-0 flex items-center justify-center bg-[rgba(8,11,20,0.78)] rounded-xl backdrop-blur-sm">
|
||||
<button onclick="resumeStream()"
|
||||
class="px-4 py-2 rounded-lg bg-seismo-orange/15 text-seismo-orange border border-seismo-orange/40 hover:bg-seismo-orange/25 text-sm font-medium transition-colors">
|
||||
⏸ Live paused — tap to resume
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Alert limits (what this location is alerted on) -->
|
||||
<div id="p-limits-section" class="reveal mt-7 hidden" style="animation-delay:180ms">
|
||||
<div class="text-[10px] uppercase tracking-[0.22em] text-[var(--text-dim)] font-mono mb-3">Alert limits</div>
|
||||
<div id="p-thresholds" class="space-y-2"></div>
|
||||
</div>
|
||||
|
||||
<!-- Alert history -->
|
||||
<div class="reveal mt-7" style="animation-delay:220ms">
|
||||
<div class="text-[10px] uppercase tracking-[0.22em] text-[var(--text-dim)] font-mono mb-3">Alert history</div>
|
||||
<div id="p-events" class="space-y-2"></div>
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
{% block scripts %}
|
||||
{% if has_device %}
|
||||
<script>
|
||||
const LOC_ID = "{{ location.id }}";
|
||||
const cd = { t: [], lp: [], leq: [], ln1: [], ln2: [] };
|
||||
let chart;
|
||||
const numOrNull = v => { const f = parseFloat(v); return isNaN(f) ? null : f; };
|
||||
|
||||
// Level color for the Leq hero (matches the overview bands).
|
||||
const LEVEL_AMBER = 55, LEVEL_RED = 70;
|
||||
function leqColor(measuring, v) {
|
||||
// CSS var refs so the hero color auto-flips with the theme.
|
||||
if (!measuring || v == null || isNaN(v)) return 'var(--text)';
|
||||
if (v >= LEVEL_RED) return 'var(--lvl-bad)';
|
||||
if (v >= LEVEL_AMBER) return 'var(--lvl-warn)';
|
||||
return 'var(--lvl-ok)';
|
||||
}
|
||||
function paintLeq(measuring, leqVal) {
|
||||
const el = document.getElementById('p-leq');
|
||||
if (el) el.style.color = leqColor(measuring, parseFloat(leqVal));
|
||||
}
|
||||
|
||||
function ds(label) { return { label, data: [], borderWidth: 1.5, pointRadius: 0, tension: 0.35, spanGaps: true }; }
|
||||
function skinChart() {
|
||||
if (!chart) return;
|
||||
const dim = cssVar('--text-dim');
|
||||
const cols = [cssVar('--m-lp'), cssVar('--lvl-ok'), cssVar('--m-l1'), cssVar('--m-l10')];
|
||||
chart.data.datasets.forEach((d, i) => { d.borderColor = cols[i]; d.backgroundColor = cols[i]; });
|
||||
const grid = 'rgba(124,146,188,0.10)', gridX = 'rgba(124,146,188,0.05)', border = 'rgba(124,146,188,0.18)';
|
||||
const y = chart.options.scales.y, x = chart.options.scales.x;
|
||||
y.ticks.color = dim; y.title.color = dim; y.grid.color = grid; y.border.color = border;
|
||||
x.ticks.color = dim; x.grid.color = gridX; x.border.color = border;
|
||||
chart.options.plugins.legend.labels.color = cssVar('--text');
|
||||
chart.update('none');
|
||||
}
|
||||
function initChart() {
|
||||
const ctx = document.getElementById('p-chart').getContext('2d');
|
||||
const mono = { family: 'IBM Plex Mono', size: 10 };
|
||||
chart = new Chart(ctx, {
|
||||
type: 'line',
|
||||
data: { labels: [], datasets: [ds('Lp'), ds('Leq'), ds('L1'), ds('L10')] },
|
||||
options: {
|
||||
responsive: true, maintainAspectRatio: false, animation: false,
|
||||
interaction: { intersect: false, mode: 'index' },
|
||||
scales: {
|
||||
y: { min: 30, max: 130, title: { display: true, text: 'dB', font: { family: 'IBM Plex Mono' } },
|
||||
ticks: { font: mono }, grid: {}, border: {} },
|
||||
x: { ticks: { font: mono, maxTicksLimit: 8 }, grid: {}, border: {} }
|
||||
},
|
||||
plugins: { legend: { labels: { font: { family: 'Hanken Grotesk' }, usePointStyle: true, pointStyleWidth: 10, boxHeight: 7 } } }
|
||||
}
|
||||
});
|
||||
skinChart();
|
||||
}
|
||||
document.addEventListener('portal-theme', skinChart);
|
||||
|
||||
function setCard(id, v) { document.getElementById(id).textContent = (v == null || v === '') ? '--' : v; }
|
||||
function setBadge(measuring, lastSeen) {
|
||||
const b = document.getElementById('p-badge'), f = document.getElementById('p-fresh');
|
||||
const base = 'inline-flex items-center gap-1.5 px-2.5 py-1 text-[11px] rounded-full border ';
|
||||
if (measuring === null) { b.className = 'hidden'; b.textContent = ''; }
|
||||
else if (measuring) { b.className = base + 'border-[rgba(244,139,28,0.45)] text-seismo-orange'; b.innerHTML = '<span class="live-dot"></span> Live'; }
|
||||
else { b.className = base + 'border-[var(--border)] text-[var(--text-dim)]'; b.textContent = 'Stopped'; }
|
||||
f.innerHTML = fmtFreshness(lastSeen);
|
||||
}
|
||||
function fmtFreshness(iso) {
|
||||
if (!iso) return '<span class="text-[var(--text-dim)]">no recent reading</span>';
|
||||
const t = new Date(iso.endsWith('Z') ? iso : iso + 'Z');
|
||||
const s = Math.max(0, Math.round((Date.now() - t.getTime()) / 1000));
|
||||
let ago, stale = false;
|
||||
if (s < 10) ago = 'just now';
|
||||
else if (s < 60) ago = s + 's ago';
|
||||
else if (s < 3600) { ago = Math.round(s / 60) + 'm ago'; stale = s >= 300; }
|
||||
else { ago = Math.round(s / 3600) + 'h ago'; stale = true; }
|
||||
const cls = stale ? 'text-amber-400' : 'text-[var(--text-dim)]';
|
||||
return `as of ${t.toLocaleTimeString()} <span class="${cls}">(${ago}${stale ? ' · cached' : ''})</span>`;
|
||||
}
|
||||
|
||||
async function prefill() {
|
||||
try {
|
||||
const j = await (await fetch(`/portal/api/location/${encodeURIComponent(LOC_ID)}/live`)).json();
|
||||
const d = j.data;
|
||||
if (!d) {
|
||||
setBadge(null, null);
|
||||
document.getElementById('p-fresh').textContent =
|
||||
j.reason === 'no_device' ? 'No device assigned' : 'Currently unreachable';
|
||||
return;
|
||||
}
|
||||
setCard('p-lp', d.lp); setCard('p-leq', d.leq); setCard('p-lmax', d.lmax);
|
||||
setCard('p-ln1', d.ln1); setCard('p-ln2', d.ln2);
|
||||
const measuring = d.measurement_state === 'Start' || d.measurement_state === 'Measure';
|
||||
setBadge(measuring, d.last_seen);
|
||||
paintLeq(measuring, d.leq);
|
||||
} catch (e) { /* keep last values */ }
|
||||
}
|
||||
async function backfill() {
|
||||
try {
|
||||
const j = await (await fetch(`/portal/api/location/${encodeURIComponent(LOC_ID)}/history?hours=2`)).json();
|
||||
for (const row of (j.readings || [])) {
|
||||
cd.t.push(row.timestamp ? new Date(row.timestamp + 'Z').toLocaleTimeString() : '');
|
||||
cd.lp.push(numOrNull(row.lp)); cd.leq.push(numOrNull(row.leq));
|
||||
cd.ln1.push(numOrNull(row.ln1)); cd.ln2.push(numOrNull(row.ln2));
|
||||
}
|
||||
chart.data.labels = cd.t;
|
||||
chart.data.datasets[0].data = cd.lp; chart.data.datasets[1].data = cd.leq;
|
||||
chart.data.datasets[2].data = cd.ln1; chart.data.datasets[3].data = cd.ln2;
|
||||
chart.update('none');
|
||||
} catch (e) { /* leave chart empty */ }
|
||||
}
|
||||
|
||||
// ---- live stream (upgrades the cache prefill to a real ~1Hz feed) --------
|
||||
let ws = null, hardCap = null, paused = false;
|
||||
const IDLE_CAP_MS = 15 * 60 * 1000; // auto-close after 15 min so an abandoned
|
||||
// tab doesn't pin the device at 1Hz polling
|
||||
|
||||
function pushPoint(d) {
|
||||
cd.t.push(new Date().toLocaleTimeString());
|
||||
cd.lp.push(numOrNull(d.lp)); cd.leq.push(numOrNull(d.leq));
|
||||
cd.ln1.push(numOrNull(d.ln1)); cd.ln2.push(numOrNull(d.ln2));
|
||||
if (cd.t.length > 600) { cd.t.shift(); cd.lp.shift(); cd.leq.shift(); cd.ln1.shift(); cd.ln2.shift(); }
|
||||
chart.data.labels = cd.t;
|
||||
chart.data.datasets[0].data = cd.lp; chart.data.datasets[1].data = cd.leq;
|
||||
chart.data.datasets[2].data = cd.ln1; chart.data.datasets[3].data = cd.ln2;
|
||||
chart.update('none');
|
||||
}
|
||||
|
||||
function openStream() {
|
||||
if (paused || ws) return;
|
||||
const proto = location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
ws = new WebSocket(`${proto}//${location.host}/portal/api/location/${encodeURIComponent(LOC_ID)}/stream`);
|
||||
ws.onmessage = (e) => {
|
||||
let d; try { d = JSON.parse(e.data); } catch (_) { return; }
|
||||
if (d.feed_status === 'no_device') {
|
||||
setBadge(null, null);
|
||||
document.getElementById('p-fresh').textContent = 'No device assigned';
|
||||
return;
|
||||
}
|
||||
if (d.heartbeat) return;
|
||||
if (d.feed_status === 'unreachable') {
|
||||
document.getElementById('p-fresh').innerHTML = '<span class="text-amber-400">device unreachable</span>';
|
||||
return;
|
||||
}
|
||||
setCard('p-lp', d.lp); setCard('p-leq', d.leq); setCard('p-lmax', d.lmax);
|
||||
setCard('p-ln1', d.ln1); setCard('p-ln2', d.ln2);
|
||||
const measuring = d.measurement_state === 'Start' || d.measurement_state === 'Measure';
|
||||
setBadge(measuring, d.timestamp || new Date().toISOString());
|
||||
paintLeq(measuring, d.leq);
|
||||
pushPoint(d);
|
||||
};
|
||||
ws.onclose = () => { ws = null; };
|
||||
ws.onerror = () => {};
|
||||
clearTimeout(hardCap);
|
||||
hardCap = setTimeout(() => { paused = true; closeStream(); showPaused(true); }, IDLE_CAP_MS);
|
||||
}
|
||||
|
||||
function closeStream() {
|
||||
clearTimeout(hardCap);
|
||||
if (ws) { try { ws.close(); } catch (_) {} ws = null; }
|
||||
}
|
||||
|
||||
function showPaused(on) {
|
||||
const el = document.getElementById('p-paused');
|
||||
if (el) el.classList.toggle('hidden', !on);
|
||||
}
|
||||
function resumeStream() {
|
||||
paused = false; showPaused(false);
|
||||
prefill(); // refresh cards instantly on resume
|
||||
openStream();
|
||||
}
|
||||
|
||||
// Stop streaming when the tab is hidden (client switched away / locked phone) and
|
||||
// resume when it's visible again — the main cost guard, so the device relaxes back
|
||||
// to its idle poll rate the moment nobody is actually looking.
|
||||
document.addEventListener('visibilitychange', () => {
|
||||
if (document.hidden) closeStream();
|
||||
else if (!paused) openStream();
|
||||
});
|
||||
window.addEventListener('beforeunload', closeStream);
|
||||
|
||||
// ---- alert history + current-alarm banner (read-only) --------------------
|
||||
const EV_METRIC = { leq: 'Leq', lp: 'Lp', lmax: 'Lmax', lpeak: 'Lpeak', ln1: 'L1', ln2: 'L10' };
|
||||
function fmtAlertTime(iso) { return iso ? new Date(iso.endsWith('Z') ? iso : iso + 'Z').toLocaleString() : ''; }
|
||||
|
||||
// ---- alert limits (the active thresholds, read-only) ---------------------
|
||||
function fmtThreshold(r) {
|
||||
const m = EV_METRIC[r.metric] || esc(r.metric);
|
||||
const cmp = r.comparison === 'below' ? 'below' : 'above';
|
||||
let s = `${m} ${cmp} ${r.threshold_db} dB`;
|
||||
if (r.duration_s) s += ` for ${r.duration_s}s`;
|
||||
if (r.schedule_start && r.schedule_end) s += ` · ${r.schedule_start}–${r.schedule_end}`;
|
||||
return s;
|
||||
}
|
||||
async function loadThresholds() {
|
||||
const sec = document.getElementById('p-limits-section');
|
||||
try {
|
||||
const j = await (await fetch(`/portal/api/location/${encodeURIComponent(LOC_ID)}/thresholds`)).json();
|
||||
const rules = j.rules || [];
|
||||
if (!rules.length) { sec.classList.add('hidden'); return; }
|
||||
const list = document.getElementById('p-thresholds');
|
||||
list.innerHTML = '';
|
||||
for (const r of rules) {
|
||||
const row = document.createElement('div');
|
||||
row.className = 'panel px-3.5 py-2.5 text-sm flex items-center gap-2.5';
|
||||
row.innerHTML = `<span class="w-1.5 h-1.5 rounded-full bg-seismo-orange shrink-0"></span>
|
||||
<span class="text-[var(--text)]">${esc(r.name || 'Alert')}</span>
|
||||
<span class="text-[var(--text-dim)] font-mono text-xs">${fmtThreshold(r)}</span>`;
|
||||
list.appendChild(row);
|
||||
}
|
||||
sec.classList.remove('hidden');
|
||||
} catch (e) { sec.classList.add('hidden'); }
|
||||
}
|
||||
|
||||
async function loadEvents() {
|
||||
try {
|
||||
const j = await (await fetch(`/portal/api/location/${encodeURIComponent(LOC_ID)}/events?limit=20`)).json();
|
||||
const events = j.events || [];
|
||||
const banner = document.getElementById('p-alarm-banner');
|
||||
if (j.active) {
|
||||
banner.classList.remove('hidden');
|
||||
document.getElementById('p-alarm-text').textContent =
|
||||
j.active > 1 ? `${j.active} alerts currently active` : 'Currently above threshold.';
|
||||
} else banner.classList.add('hidden');
|
||||
const list = document.getElementById('p-events');
|
||||
if (!events.length) { list.innerHTML = '<div class="text-sm text-[var(--text-dim)]">No alerts have fired.</div>'; return; }
|
||||
list.innerHTML = '';
|
||||
for (const e of events) {
|
||||
const m = EV_METRIC[e.metric] || esc(e.metric);
|
||||
const active = e.status === 'active';
|
||||
const when = active ? `since ${fmtAlertTime(e.onset_at)}`
|
||||
: `${fmtAlertTime(e.onset_at)} → ${fmtAlertTime(e.clear_at)}`;
|
||||
const peak = (e.peak_value != null) ? ` · peak ${e.peak_value} dB` : '';
|
||||
const row = document.createElement('div');
|
||||
row.className = 'panel px-3.5 py-2.5 text-sm ' + (active ? 'border-[rgba(220,38,38,0.4)]' : '');
|
||||
row.innerHTML = `<div class="${active ? 'text-[var(--lvl-bad)] font-medium' : 'text-[var(--text)]'}">${esc(e.rule_name || 'Alert')} <span class="text-xs text-[var(--text-dim)] font-mono">· ${m} ${e.threshold_db} dB</span></div>
|
||||
<div class="text-xs text-[var(--text-dim)] font-mono mt-0.5">${when}${peak}</div>`;
|
||||
list.appendChild(row);
|
||||
}
|
||||
} catch (e) { /* leave history as-is */ }
|
||||
}
|
||||
|
||||
initChart();
|
||||
prefill(); // instant first paint from cache
|
||||
backfill(); // seed the chart trail
|
||||
openStream(); // then upgrade to the live feed
|
||||
loadEvents();
|
||||
loadThresholds();
|
||||
setInterval(loadEvents, 20000);
|
||||
</script>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
@@ -1,192 +0,0 @@
|
||||
{% extends "portal/base.html" %}
|
||||
{% block title %}Your locations{% endblock %}
|
||||
{% block head %}
|
||||
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css" />
|
||||
{% endblock %}
|
||||
{% block content %}
|
||||
<div class="reveal">
|
||||
<div class="text-[11px] uppercase tracking-[0.2em] text-seismo-orange/80 font-mono mb-2">Live monitoring</div>
|
||||
<h1 class="text-3xl font-bold tracking-tight">Your locations</h1>
|
||||
<p class="text-[var(--text-dim)] text-sm mt-1">Real-time sound levels across your active monitoring sites.</p>
|
||||
</div>
|
||||
|
||||
{% if locations %}
|
||||
<!-- Status rollup (filled live from the per-location /live fetches) -->
|
||||
<div id="rollup" class="hidden mt-6 mb-6 flex flex-wrap items-center gap-2.5">
|
||||
<div class="panel px-4 py-2.5 flex items-center gap-2.5">
|
||||
<span class="text-[var(--text-dim)] text-[10px] uppercase tracking-[0.15em]">Locations</span>
|
||||
<b id="r-total" class="reading text-lg font-semibold">–</b>
|
||||
</div>
|
||||
<div class="panel px-4 py-2.5 flex items-center gap-2">
|
||||
<span class="live-dot"></span><b id="r-live" class="reading text-lg font-semibold text-seismo-orange">–</b><span class="text-[var(--text-dim)] text-xs">live</span>
|
||||
</div>
|
||||
<div class="panel px-4 py-2.5 flex items-center gap-2">
|
||||
<span class="w-2 h-2 rounded-full bg-[var(--text-dim)]/50"></span><b id="r-off" class="reading text-lg font-semibold">–</b><span class="text-[var(--text-dim)] text-xs">offline</span>
|
||||
</div>
|
||||
<div id="r-peak-wrap" class="hidden panel px-4 py-2.5 flex items-center gap-2">
|
||||
<span class="text-[var(--text-dim)] text-[10px] uppercase tracking-[0.15em]">Loudest now</span>
|
||||
<b id="r-peak" class="reading text-lg font-semibold text-seismo-orange">–</b><span class="text-[var(--text-dim)] text-xs">dB</span>
|
||||
<span id="r-peak-loc" class="text-[var(--text-dim)] text-sm"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="loc-map" class="panel reveal hidden h-72 overflow-hidden mb-6" style="animation-delay:80ms"></div>
|
||||
|
||||
<div class="grid sm:grid-cols-2 lg:grid-cols-3 gap-4">
|
||||
{% for loc in locations %}
|
||||
<a href="/portal/location/{{ loc.id }}" data-loc="{{ loc.id }}"
|
||||
class="loc-tile panel panel-hover reveal block p-5" style="animation-delay: {{ (loop.index0 * 55) + 140 }}ms">
|
||||
<div class="flex items-start justify-between gap-2">
|
||||
<div class="min-w-0">
|
||||
<div class="font-semibold tracking-tight truncate">{{ loc.name }}</div>
|
||||
<div class="text-xs text-[var(--text-dim)] mt-0.5 truncate">{{ loc.address or loc.project_name or '' }}</div>
|
||||
</div>
|
||||
<span class="loc-badge hidden shrink-0"></span>
|
||||
</div>
|
||||
<div class="mt-5 flex items-baseline gap-1.5">
|
||||
<span class="loc-leq reading text-[2.6rem] leading-none font-semibold">--</span>
|
||||
<span class="text-xs text-[var(--text-dim)] font-mono tracking-wide">dB Leq</span>
|
||||
</div>
|
||||
<div class="loc-fresh text-[11px] text-[var(--text-dim)]/70 mt-2 font-mono"> </div>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="panel reveal p-12 text-center text-[var(--text-dim)] mt-6">No active monitoring locations yet.</div>
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
{% block scripts %}
|
||||
<script src="https://unpkg.com/leaflet@1.9.4/dist/leaflet.js"></script>
|
||||
<script>
|
||||
const LOCATIONS = {{ locations|tojson }};
|
||||
const liveState = {}; // loc.id -> {status, leq(num|null), leqStr}
|
||||
const markersById = {}; // loc.id -> circleMarker (for live recolor)
|
||||
let tiles = null; // map tile layer (re-skinned on theme toggle)
|
||||
|
||||
// Dot/level color (computed hex; reads the theme CSS vars so it flips with theme).
|
||||
const LEVEL_AMBER = 55, LEVEL_RED = 70;
|
||||
function levelColor(st) {
|
||||
if (!st || st.status !== 'measuring' || st.leq == null) return cssVar('--text-dim');
|
||||
if (st.leq >= LEVEL_RED) return cssVar('--lvl-bad');
|
||||
if (st.leq >= LEVEL_AMBER) return cssVar('--lvl-warn');
|
||||
return cssVar('--lvl-ok');
|
||||
}
|
||||
function tileUrl() {
|
||||
return document.documentElement.getAttribute('data-theme') === 'light'
|
||||
? 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png'
|
||||
: 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
|
||||
}
|
||||
// Re-skin map tiles + recolor everything when the theme flips.
|
||||
document.addEventListener('portal-theme', () => { if (tiles) tiles.setUrl(tileUrl()); refreshAll(); });
|
||||
function num(v) { const f = parseFloat(v); return isNaN(f) ? null : f; }
|
||||
|
||||
function fmtAgo(iso) {
|
||||
if (!iso) return '';
|
||||
const t = new Date(iso.endsWith('Z') ? iso : iso + 'Z');
|
||||
const s = Math.max(0, Math.round((Date.now() - t.getTime()) / 1000));
|
||||
if (s < 60) return 'updated just now';
|
||||
if (s < 3600) return 'updated ' + Math.round(s / 60) + 'm ago';
|
||||
return 'updated ' + Math.round(s / 3600) + 'h ago';
|
||||
}
|
||||
|
||||
const BADGE_BASE = 'loc-badge inline-flex items-center gap-1.5 shrink-0 px-2.5 py-1 text-[11px] rounded-full border ';
|
||||
|
||||
function updateMarker(loc) {
|
||||
const m = markersById[loc.id]; if (!m) return;
|
||||
const st = liveState[loc.id];
|
||||
m.setStyle({ fillColor: levelColor(st) });
|
||||
let label = `<b>${esc(loc.name)}</b>`;
|
||||
if (st) {
|
||||
if (st.status === 'measuring') label += ` · ${esc(st.leqStr)} dB Leq`;
|
||||
else if (st.status === 'stopped') label += ' · stopped';
|
||||
else if (st.status === 'nodevice') label += ' · no device';
|
||||
else label += ' · offline';
|
||||
}
|
||||
m.setTooltipContent(label);
|
||||
}
|
||||
|
||||
async function loadTile(loc) {
|
||||
const el = document.querySelector(`.loc-tile[data-loc="${loc.id}"]`);
|
||||
const leqEl = el && el.querySelector('.loc-leq'),
|
||||
badge = el && el.querySelector('.loc-badge'),
|
||||
fresh = el && el.querySelector('.loc-fresh');
|
||||
try {
|
||||
const j = await (await fetch(`/portal/api/location/${encodeURIComponent(loc.id)}/live`)).json();
|
||||
const d = j.data;
|
||||
if (!d) {
|
||||
liveState[loc.id] = { status: j.reason === 'no_device' ? 'nodevice' : 'offline', leq: null };
|
||||
if (badge) { badge.classList.remove('hidden'); badge.className = BADGE_BASE + 'border-[var(--border)] text-[var(--text-dim)]'; badge.textContent = j.reason === 'no_device' ? 'No device' : 'Offline'; }
|
||||
if (leqEl) { leqEl.textContent = '--'; leqEl.style.color = 'var(--text-dim)'; }
|
||||
if (fresh) fresh.innerHTML = ' ';
|
||||
} else {
|
||||
const measuring = d.measurement_state === 'Start' || d.measurement_state === 'Measure';
|
||||
const leqStr = (d.leq == null || d.leq === '') ? '--' : d.leq;
|
||||
liveState[loc.id] = { status: measuring ? 'measuring' : 'stopped', leq: num(d.leq), leqStr };
|
||||
if (leqEl) { leqEl.textContent = leqStr; leqEl.style.color = measuring ? levelColor(liveState[loc.id]) : 'var(--text)'; }
|
||||
if (badge) {
|
||||
badge.classList.remove('hidden');
|
||||
if (measuring) { badge.className = BADGE_BASE + 'border-[rgba(244,139,28,0.45)] text-seismo-orange'; badge.innerHTML = '<span class="live-dot"></span> Live'; }
|
||||
else { badge.className = BADGE_BASE + 'border-[var(--border)] text-[var(--text-dim)]'; badge.textContent = 'Stopped'; }
|
||||
}
|
||||
if (fresh) fresh.textContent = fmtAgo(d.last_seen);
|
||||
}
|
||||
} catch (e) { /* leave placeholders */ }
|
||||
updateMarker(loc);
|
||||
}
|
||||
|
||||
function updateRollup() {
|
||||
const total = LOCATIONS.length;
|
||||
let live = 0, off = 0, peak = null, peakStr = null, peakLoc = null;
|
||||
for (const l of LOCATIONS) {
|
||||
const s = liveState[l.id]; if (!s) continue;
|
||||
if (s.status === 'measuring') {
|
||||
live++;
|
||||
if (s.leq != null && (peak == null || s.leq > peak)) { peak = s.leq; peakStr = s.leqStr; peakLoc = l.name; }
|
||||
} else if (s.status === 'offline' || s.status === 'nodevice') off++;
|
||||
}
|
||||
document.getElementById('r-total').textContent = total;
|
||||
document.getElementById('r-live').textContent = live;
|
||||
document.getElementById('r-off').textContent = off;
|
||||
const pw = document.getElementById('r-peak-wrap');
|
||||
if (peak != null) {
|
||||
pw.classList.remove('hidden');
|
||||
document.getElementById('r-peak').textContent = peakStr;
|
||||
document.getElementById('r-peak-loc').textContent = peakLoc;
|
||||
} else pw.classList.add('hidden');
|
||||
document.getElementById('rollup').classList.remove('hidden');
|
||||
}
|
||||
|
||||
async function refreshAll() {
|
||||
await Promise.all(LOCATIONS.map(loadTile));
|
||||
updateRollup();
|
||||
}
|
||||
refreshAll();
|
||||
setInterval(refreshAll, 15000);
|
||||
|
||||
// Map of locations with coordinates — dark tiles, dots recolor live.
|
||||
const withCoords = LOCATIONS.filter(l => l.coordinates);
|
||||
if (withCoords.length) {
|
||||
const mapEl = document.getElementById('loc-map');
|
||||
mapEl.classList.remove('hidden');
|
||||
const map = L.map('loc-map', { scrollWheelZoom: false, attributionControl: true });
|
||||
tiles = L.tileLayer(tileUrl(), {
|
||||
maxZoom: 19, subdomains: 'abcd', attribution: '© OpenStreetMap © CARTO'
|
||||
}).addTo(map);
|
||||
const pts = [];
|
||||
withCoords.forEach(l => {
|
||||
const [la, lo] = (l.coordinates || '').split(',').map(Number);
|
||||
if (!isNaN(la) && !isNaN(lo)) {
|
||||
markersById[l.id] = L.circleMarker([la, lo], {
|
||||
radius: 7, fillColor: levelColor(liveState[l.id]), color: '#fff',
|
||||
weight: 2, opacity: 0.9, fillOpacity: 0.95,
|
||||
}).addTo(map).bindTooltip(esc(l.name), { direction: 'top', offset: [0, -6] });
|
||||
pts.push([la, lo]);
|
||||
}
|
||||
});
|
||||
if (pts.length) map.fitBounds(pts, { padding: [36, 36], maxZoom: 15 });
|
||||
else mapEl.classList.add('hidden');
|
||||
LOCATIONS.forEach(updateMarker);
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
@@ -1,26 +0,0 @@
|
||||
{% extends "portal/base.html" %}
|
||||
{% block title %}{{ project_name }}{% endblock %}
|
||||
{% block content %}
|
||||
<div class="max-w-md mx-auto mt-20 text-center reveal">
|
||||
<div class="panel inline-flex items-center justify-center w-16 h-16 rounded-2xl mb-6">
|
||||
<svg class="w-7 h-7 text-[var(--text-dim)]" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"
|
||||
d="M12 15v2m-6 4h12a2 2 0 002-2v-6a2 2 0 00-2-2H6a2 2 0 00-2 2v6a2 2 0 002 2zm10-10V7a4 4 0 00-8 0v4h8z"></path>
|
||||
</svg>
|
||||
</div>
|
||||
<h1 class="text-2xl font-bold tracking-tight mb-1">{{ project_name }}</h1>
|
||||
<p class="text-[var(--text-dim)] text-sm mb-6">Enter the password to view this monitoring portal.</p>
|
||||
{% if error %}
|
||||
<p class="text-[var(--lvl-bad)] text-sm mb-4">{{ error }}</p>
|
||||
{% endif %}
|
||||
<form method="post" action="/portal/p/{{ link_token }}" class="panel p-5 text-left">
|
||||
<label class="block text-xs text-[var(--text-dim)] mb-1" for="password">Password</label>
|
||||
<input id="password" name="password" type="password" autofocus required
|
||||
class="w-full px-3 py-2 rounded-lg border border-[var(--border)] bg-[var(--panel-b)] text-[var(--text)] mb-4">
|
||||
<button type="submit"
|
||||
class="w-full px-4 py-2 rounded-lg bg-seismo-orange text-white font-medium hover:opacity-90">
|
||||
View portal
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
{% endblock %}
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
{% block content %}
|
||||
<!-- Breadcrumb Navigation -->
|
||||
<div class="mb-6 flex items-center justify-between gap-3">
|
||||
<div class="mb-6">
|
||||
<nav class="flex items-center space-x-2 text-sm">
|
||||
<a href="/projects" class="text-seismo-orange hover:text-seismo-navy flex items-center">
|
||||
<svg class="w-4 h-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
@@ -17,28 +17,6 @@
|
||||
</svg>
|
||||
<span class="text-gray-900 dark:text-white font-medium" id="project-name-breadcrumb">Project</span>
|
||||
</nav>
|
||||
|
||||
<!-- Client portal access for this project -->
|
||||
<div class="shrink-0 flex items-center gap-2">
|
||||
<button type="button" onclick="openPortalAccess()"
|
||||
class="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm rounded-lg border border-slate-600 bg-slate-700/40 text-gray-200 hover:bg-slate-700 transition-colors"
|
||||
title="Manage this project's client portal access">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2"
|
||||
d="M12 15v2m-6 4h12a2 2 0 002-2v-6a2 2 0 00-2-2H6a2 2 0 00-2 2v6a2 2 0 002 2zm10-10V7a4 4 0 00-8 0v4h8z"></path>
|
||||
</svg>
|
||||
Portal access
|
||||
</button>
|
||||
<a href="/projects/{{ project_id }}/portal-preview" target="_blank" rel="noopener"
|
||||
class="inline-flex items-center gap-1.5 px-3 py-1.5 text-sm rounded-lg border border-seismo-orange/40 bg-seismo-orange/10 text-seismo-orange hover:bg-seismo-orange/20 transition-colors"
|
||||
title="Preview this project's client portal in a new tab">
|
||||
<svg class="w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 12a3 3 0 11-6 0 3 3 0 016 0z"></path>
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M2.458 12C3.732 7.943 7.523 5 12 5c4.478 0 8.268 2.943 9.542 7-1.274 4.057-5.064 7-9.542 7-4.477 0-8.268-2.943-9.542-7z"></path>
|
||||
</svg>
|
||||
Preview
|
||||
</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Header (loads dynamically) -->
|
||||
@@ -2096,97 +2074,5 @@ document.addEventListener('DOMContentLoaded', function() {
|
||||
}
|
||||
});
|
||||
|
||||
</script>
|
||||
|
||||
<!-- Portal access modal -->
|
||||
<div id="portal-access-modal" class="hidden fixed inset-0 z-50 flex items-center justify-center bg-black/50 p-4"
|
||||
onclick="if(event.target===this)closePortalAccess()">
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-xl w-full max-w-lg p-6">
|
||||
<div class="flex items-center justify-between mb-1">
|
||||
<h3 class="text-lg font-semibold text-gray-900 dark:text-white">Client portal access</h3>
|
||||
<button onclick="closePortalAccess()" class="text-gray-400 hover:text-gray-600 dark:hover:text-gray-200">
|
||||
<svg class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
<p class="text-sm text-gray-500 dark:text-gray-400 mb-4">
|
||||
Send the client the link <em>and</em> the password. Read-only. Disabling rotates the link.
|
||||
</p>
|
||||
|
||||
<div class="flex items-center justify-between mb-4">
|
||||
<span class="text-sm font-medium text-gray-700 dark:text-gray-200">Portal enabled</span>
|
||||
<button id="pa-toggle" onclick="togglePortalEnabled()"
|
||||
class="px-3 py-1.5 text-sm rounded-lg border border-slate-300 dark:border-slate-600">…</button>
|
||||
</div>
|
||||
|
||||
<div id="pa-details" class="hidden space-y-4">
|
||||
<div>
|
||||
<label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Portal link</label>
|
||||
<div class="flex gap-2">
|
||||
<input id="pa-link" readonly class="flex-1 px-3 py-2 text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-gray-50 dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
|
||||
<button onclick="copyField('pa-link', this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Password</label>
|
||||
<div class="flex gap-2">
|
||||
<input id="pa-pass" readonly placeholder="•••••••• (set one below)"
|
||||
class="flex-1 px-3 py-2 text-sm rounded-lg border border-slate-300 dark:border-slate-600 bg-gray-50 dark:bg-slate-900 text-gray-800 dark:text-gray-200" />
|
||||
<button onclick="copyField('pa-pass', this)" class="px-3 py-2 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Copy</button>
|
||||
</div>
|
||||
<button onclick="regeneratePassword()" class="mt-2 text-sm text-seismo-orange hover:text-seismo-navy font-medium">↻ Generate new password</button>
|
||||
<p class="text-xs text-gray-400 mt-1">Shown once — copy it now. Regenerating invalidates the old one.</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const PA_PROJECT_ID = "{{ project_id }}";
|
||||
let paEnabled = false;
|
||||
function paToast(msg) { if (window.showToast) showToast(msg, 'error'); else alert(msg); }
|
||||
function openPortalAccess() { document.getElementById('portal-access-modal').classList.remove('hidden'); loadPortalAccess(); }
|
||||
function closePortalAccess() { document.getElementById('portal-access-modal').classList.add('hidden'); }
|
||||
|
||||
function copyField(id, btn) {
|
||||
const inp = document.getElementById(id); inp.select();
|
||||
const done = () => { btn.textContent = 'Copied!'; setTimeout(() => btn.textContent = 'Copy', 1500); };
|
||||
if (navigator.clipboard) navigator.clipboard.writeText(inp.value).then(done).catch(() => { document.execCommand('copy'); done(); });
|
||||
else { document.execCommand('copy'); done(); }
|
||||
}
|
||||
|
||||
async function loadPortalAccess() {
|
||||
try {
|
||||
const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access`);
|
||||
if (!r.ok) throw new Error('load failed');
|
||||
renderPortalAccess(await r.json());
|
||||
} catch (e) { paToast('Could not load portal access.'); }
|
||||
}
|
||||
function renderPortalAccess(j) {
|
||||
paEnabled = !!j.enabled;
|
||||
const toggle = document.getElementById('pa-toggle');
|
||||
const details = document.getElementById('pa-details');
|
||||
toggle.textContent = paEnabled ? 'On — click to disable' : 'Off — click to enable';
|
||||
toggle.className = 'px-3 py-1.5 text-sm rounded-lg border ' +
|
||||
(paEnabled ? 'border-green-500 text-green-600 dark:text-green-400' : 'border-slate-300 dark:border-slate-600');
|
||||
details.classList.toggle('hidden', !paEnabled);
|
||||
document.getElementById('pa-link').value = (paEnabled && j.link_url) ? j.link_url : '';
|
||||
}
|
||||
async function togglePortalEnabled() {
|
||||
const action = paEnabled ? 'disable' : 'enable';
|
||||
try {
|
||||
const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access/${action}`, { method: 'POST' });
|
||||
if (!r.ok) throw new Error('toggle failed');
|
||||
const j = await r.json();
|
||||
renderPortalAccess(action === 'disable' ? { enabled: false, link_url: null } : j);
|
||||
} catch (e) { paToast(`Could not ${action} the portal.`); }
|
||||
}
|
||||
async function regeneratePassword() {
|
||||
try {
|
||||
const r = await fetch(`/projects/${PA_PROJECT_ID}/portal-access/password`, { method: 'POST' });
|
||||
if (!r.ok) throw new Error('password failed');
|
||||
const j = await r.json();
|
||||
if (j.password) { const f = document.getElementById('pa-pass'); f.value = j.password; f.placeholder = ''; }
|
||||
} catch (e) { paToast('Could not generate a password.'); }
|
||||
}
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@@ -112,267 +112,4 @@
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Alerts -->
|
||||
<div class="bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6 mt-6">
|
||||
<div class="flex items-center justify-between mb-4">
|
||||
<div>
|
||||
<h2 class="text-xl font-semibold text-gray-900 dark:text-white flex items-center gap-2">Alerts
|
||||
<span id="alert-state-badge" class="hidden text-xs px-2 py-0.5 rounded-full"></span>
|
||||
</h2>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">Threshold rules evaluated on this device's live feed. An enabled alert keeps the device monitored 24/7.</p>
|
||||
</div>
|
||||
<button onclick="openAlertForm()" type="button"
|
||||
class="px-3 py-1.5 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">+ Add alert</button>
|
||||
</div>
|
||||
|
||||
<div id="alert-rules-list" class="space-y-2"></div>
|
||||
|
||||
<!-- create / edit form -->
|
||||
<div id="alert-form" class="hidden mt-4 p-4 rounded-lg border border-slate-200 dark:border-slate-700 bg-gray-50 dark:bg-slate-900/40">
|
||||
<input type="hidden" id="ar-id">
|
||||
<div class="grid sm:grid-cols-2 gap-3 mb-3">
|
||||
<div>
|
||||
<label class="block text-xs text-gray-500 dark:text-gray-400 mb-1">Name</label>
|
||||
<input id="ar-name" type="text" placeholder="e.g. Night noise limit"
|
||||
class="w-full px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800 text-sm text-gray-800 dark:text-gray-200">
|
||||
</div>
|
||||
<label class="flex items-end gap-2 text-sm text-gray-700 dark:text-gray-300 pb-1">
|
||||
<input type="checkbox" id="ar-enabled" checked class="rounded"> Enabled
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div class="flex flex-wrap items-center gap-2 text-sm text-gray-700 dark:text-gray-300">
|
||||
<span>Alert when</span>
|
||||
<select id="ar-metric" class="px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800">
|
||||
<option value="leq">Leq</option><option value="lp">Lp</option>
|
||||
<option value="lmax">Lmax</option><option value="lpeak">Lpeak</option>
|
||||
<option value="ln1">L1</option><option value="ln2">L10</option>
|
||||
</select>
|
||||
<span>is</span>
|
||||
<select id="ar-comparison" class="px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800">
|
||||
<option value="above">above</option><option value="below">below</option>
|
||||
</select>
|
||||
<input id="ar-threshold" type="number" step="0.1" placeholder="65"
|
||||
class="w-20 px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800"> <span>dB</span>
|
||||
<span>for</span>
|
||||
<input id="ar-duration" type="number" min="0" value="0"
|
||||
class="w-20 px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800"> <span>seconds</span>
|
||||
</div>
|
||||
|
||||
<div class="mt-3">
|
||||
<label class="inline-flex items-center gap-2 text-sm text-gray-700 dark:text-gray-300">
|
||||
<input type="checkbox" id="ar-sched-on" onchange="toggleSchedule()" class="rounded"> Only during certain hours
|
||||
</label>
|
||||
<div id="ar-sched" class="hidden mt-2 flex flex-wrap items-center gap-2 text-sm text-gray-700 dark:text-gray-300">
|
||||
<span>from</span><input id="ar-start" type="time" class="px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800">
|
||||
<span>to</span><input id="ar-end" type="time" class="px-2 py-1.5 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800">
|
||||
<span class="ml-2">on</span>
|
||||
<span id="ar-days" class="flex gap-1"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<details class="mt-3 text-sm text-gray-600 dark:text-gray-400">
|
||||
<summary class="cursor-pointer select-none">Advanced</summary>
|
||||
<div class="mt-2 flex flex-wrap items-center gap-3">
|
||||
<span>Clear margin</span><input id="ar-margin" type="number" step="0.1" value="2"
|
||||
class="w-16 px-2 py-1 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800"><span>dB (hysteresis)</span>
|
||||
<span>Cooldown</span><input id="ar-cooldown" type="number" min="0" value="300"
|
||||
class="w-20 px-2 py-1 rounded border border-slate-300 dark:border-slate-600 bg-white dark:bg-slate-800"><span>s</span>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
<div class="mt-4 flex gap-2">
|
||||
<button onclick="saveAlertRule()" type="button" class="px-3 py-1.5 text-sm rounded-lg bg-seismo-orange text-white hover:bg-orange-600">Save</button>
|
||||
<button onclick="closeAlertForm()" type="button" class="px-3 py-1.5 text-sm rounded-lg border border-slate-300 dark:border-slate-600 text-gray-700 dark:text-gray-300 hover:bg-gray-100 dark:hover:bg-slate-700">Cancel</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Alert history -->
|
||||
<div class="mt-6 pt-4 border-t border-slate-200 dark:border-slate-700">
|
||||
<div class="flex items-center justify-between mb-2">
|
||||
<h3 class="text-sm font-medium text-gray-700 dark:text-gray-300">History</h3>
|
||||
<button onclick="loadAlertEvents()" type="button" class="text-xs text-gray-400 hover:text-gray-600 dark:hover:text-gray-300">Refresh</button>
|
||||
</div>
|
||||
<div id="alert-events" class="space-y-2"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const ALERT_UNIT = "{{ unit_id }}";
|
||||
const METRIC_LABELS = { leq: 'Leq', lp: 'Lp', lmax: 'Lmax', lpeak: 'Lpeak', ln1: 'L1', ln2: 'L10' };
|
||||
const DAY_LABELS = ['M', 'T', 'W', 'T', 'F', 'S', 'S']; // Mon=0 .. Sun=6
|
||||
|
||||
// Render the day checkboxes once.
|
||||
(function () {
|
||||
const wrap = document.getElementById('ar-days');
|
||||
DAY_LABELS.forEach((lbl, i) => {
|
||||
const l = document.createElement('label');
|
||||
l.className = 'inline-flex items-center gap-0.5';
|
||||
l.innerHTML = `<input type="checkbox" id="ar-day-${i}" class="rounded"><span class="ml-0.5">${lbl}</span>`;
|
||||
wrap.appendChild(l);
|
||||
});
|
||||
})();
|
||||
|
||||
function condText(r) {
|
||||
const m = METRIC_LABELS[r.metric] || r.metric;
|
||||
let s = `${m} ${r.comparison} ${r.threshold_db} dB`;
|
||||
if (r.duration_s) s += ` for ${r.duration_s}s`;
|
||||
if (r.schedule_start && r.schedule_end) s += ` · ${r.schedule_start}–${r.schedule_end}`;
|
||||
return s;
|
||||
}
|
||||
|
||||
function renderRule(r) {
|
||||
const row = document.createElement('div');
|
||||
row.className = 'flex items-center justify-between gap-2 px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-700';
|
||||
row.innerHTML = `<div class="min-w-0">
|
||||
<div class="text-sm font-medium text-gray-800 dark:text-gray-200 truncate">${r.name}${r.enabled ? '' : ' <span class="text-xs text-gray-400">(disabled)</span>'}</div>
|
||||
<div class="text-xs text-gray-500">${condText(r)}</div></div>
|
||||
<div class="shrink-0 flex items-center gap-3 text-xs">
|
||||
<button data-act="edit" class="text-seismo-orange hover:underline">Edit</button>
|
||||
<button data-act="del" class="text-red-600 hover:underline">Delete</button>
|
||||
</div>`;
|
||||
row.querySelector('[data-act="edit"]').onclick = () => openAlertForm(r);
|
||||
row.querySelector('[data-act="del"]').onclick = () => deleteAlertRule(r.id);
|
||||
return row;
|
||||
}
|
||||
|
||||
async function loadAlertRules() {
|
||||
const list = document.getElementById('alert-rules-list');
|
||||
try {
|
||||
const j = await (await fetch(`/api/slmm/${ALERT_UNIT}/alerts/rules`)).json();
|
||||
const rules = j.rules || [];
|
||||
if (!rules.length) { list.innerHTML = '<div class="text-sm text-gray-400">No alerts configured.</div>'; return; }
|
||||
list.innerHTML = '';
|
||||
rules.forEach(r => list.appendChild(renderRule(r)));
|
||||
} catch (e) { list.innerHTML = '<div class="text-sm text-red-500">Failed to load alerts.</div>'; }
|
||||
}
|
||||
|
||||
function toggleSchedule() {
|
||||
document.getElementById('ar-sched').classList.toggle('hidden', !document.getElementById('ar-sched-on').checked);
|
||||
}
|
||||
|
||||
function openAlertForm(r) {
|
||||
document.getElementById('alert-form').classList.remove('hidden');
|
||||
document.getElementById('ar-id').value = r ? r.id : '';
|
||||
document.getElementById('ar-name').value = r ? r.name : '';
|
||||
document.getElementById('ar-metric').value = r ? r.metric : 'leq';
|
||||
document.getElementById('ar-comparison').value = r ? r.comparison : 'above';
|
||||
document.getElementById('ar-threshold').value = (r && r.threshold_db != null) ? r.threshold_db : '';
|
||||
document.getElementById('ar-duration').value = r ? r.duration_s : 0;
|
||||
document.getElementById('ar-enabled').checked = r ? r.enabled : true;
|
||||
document.getElementById('ar-margin').value = r ? r.clear_margin_db : 2;
|
||||
document.getElementById('ar-cooldown').value = r ? r.cooldown_s : 300;
|
||||
const hasSched = !!(r && r.schedule_start && r.schedule_end);
|
||||
document.getElementById('ar-sched-on').checked = hasSched;
|
||||
document.getElementById('ar-start').value = hasSched ? r.schedule_start : '';
|
||||
document.getElementById('ar-end').value = hasSched ? r.schedule_end : '';
|
||||
const days = (r && r.schedule_days) ? r.schedule_days.split(',') : [];
|
||||
DAY_LABELS.forEach((_, i) => { document.getElementById('ar-day-' + i).checked = days.includes(String(i)); });
|
||||
toggleSchedule();
|
||||
}
|
||||
function closeAlertForm() { document.getElementById('alert-form').classList.add('hidden'); }
|
||||
|
||||
async function saveAlertRule() {
|
||||
const id = document.getElementById('ar-id').value;
|
||||
const threshold = parseFloat(document.getElementById('ar-threshold').value);
|
||||
if (isNaN(threshold)) { if (window.showToast) showToast('Enter a threshold', 'error'); return; }
|
||||
const schedOn = document.getElementById('ar-sched-on').checked;
|
||||
const days = DAY_LABELS.map((_, i) => document.getElementById('ar-day-' + i).checked ? i : null).filter(v => v !== null);
|
||||
const payload = {
|
||||
name: document.getElementById('ar-name').value || 'Alert',
|
||||
metric: document.getElementById('ar-metric').value,
|
||||
comparison: document.getElementById('ar-comparison').value,
|
||||
threshold_db: threshold,
|
||||
duration_s: parseInt(document.getElementById('ar-duration').value) || 0,
|
||||
clear_margin_db: parseFloat(document.getElementById('ar-margin').value) || 2,
|
||||
cooldown_s: parseInt(document.getElementById('ar-cooldown').value) || 300,
|
||||
schedule_start: schedOn ? (document.getElementById('ar-start').value || null) : null,
|
||||
schedule_end: schedOn ? (document.getElementById('ar-end').value || null) : null,
|
||||
schedule_days: (schedOn && days.length) ? days.join(',') : null,
|
||||
enabled: document.getElementById('ar-enabled').checked,
|
||||
};
|
||||
const url = id ? `/api/slmm/${ALERT_UNIT}/alerts/rules/${id}` : `/api/slmm/${ALERT_UNIT}/alerts/rules`;
|
||||
try {
|
||||
const r = await fetch(url, { method: id ? 'PUT' : 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload) });
|
||||
if (!r.ok) throw new Error('save failed');
|
||||
closeAlertForm(); loadAlertRules();
|
||||
if (window.showToast) showToast('Alert saved', 'success');
|
||||
} catch (e) { if (window.showToast) showToast('Failed to save alert', 'error'); }
|
||||
}
|
||||
|
||||
async function deleteAlertRule(id) {
|
||||
if (!confirm('Delete this alert rule?')) return;
|
||||
try { await fetch(`/api/slmm/${ALERT_UNIT}/alerts/rules/${id}`, { method: 'DELETE' }); loadAlertRules(); }
|
||||
catch (e) { if (window.showToast) showToast('Failed to delete', 'error'); }
|
||||
}
|
||||
|
||||
// ---- alert history (events) ----------------------------------------------
|
||||
|
||||
function fmtAlertTime(iso) {
|
||||
if (!iso) return '';
|
||||
return new Date(iso.endsWith('Z') ? iso : iso + 'Z').toLocaleString();
|
||||
}
|
||||
|
||||
function updateAlertState(events) {
|
||||
const badge = document.getElementById('alert-state-badge');
|
||||
badge.classList.remove('hidden');
|
||||
const active = events.filter(e => e.status === 'active').length;
|
||||
if (active) {
|
||||
badge.textContent = `● ${active} active`;
|
||||
badge.className = 'text-xs px-2 py-0.5 rounded-full bg-red-100 text-red-700 dark:bg-red-900/40 dark:text-red-300';
|
||||
} else {
|
||||
badge.textContent = '✓ All clear';
|
||||
badge.className = 'text-xs px-2 py-0.5 rounded-full bg-green-100 text-green-700 dark:bg-green-900/40 dark:text-green-300';
|
||||
}
|
||||
}
|
||||
|
||||
function renderEvent(e) {
|
||||
const m = METRIC_LABELS[e.metric] || e.metric;
|
||||
const active = e.status === 'active';
|
||||
const row = document.createElement('div');
|
||||
row.className = 'flex items-center justify-between gap-2 px-3 py-2 rounded-lg border ' +
|
||||
(active ? 'border-red-300 dark:border-red-800 bg-red-50 dark:bg-red-900/20'
|
||||
: 'border-slate-200 dark:border-slate-700');
|
||||
const when = active ? `since ${fmtAlertTime(e.onset_at)}`
|
||||
: `${fmtAlertTime(e.onset_at)} → ${fmtAlertTime(e.clear_at)}`;
|
||||
const peak = (e.peak_value != null) ? ` · peak ${e.peak_value} dB` : '';
|
||||
const ack = e.acknowledged_at ? ` · ack'd${e.acknowledged_by ? ' by ' + e.acknowledged_by : ''}` : '';
|
||||
row.innerHTML = `<div class="min-w-0">
|
||||
<div class="text-sm truncate">
|
||||
<span class="${active ? 'text-red-600 dark:text-red-400 font-medium' : 'text-gray-800 dark:text-gray-200'}">${e.rule_name || 'Alert'}</span>
|
||||
<span class="text-xs text-gray-500"> · ${m} ${e.threshold_db} dB</span>
|
||||
</div>
|
||||
<div class="text-xs text-gray-500">${when}${peak}${ack}</div></div>`;
|
||||
if (!e.acknowledged_at) {
|
||||
const btn = document.createElement('button');
|
||||
btn.className = 'shrink-0 text-xs text-seismo-orange hover:underline';
|
||||
btn.textContent = 'Ack';
|
||||
btn.onclick = () => ackEvent(e.id);
|
||||
row.appendChild(btn);
|
||||
}
|
||||
return row;
|
||||
}
|
||||
|
||||
async function loadAlertEvents() {
|
||||
const list = document.getElementById('alert-events');
|
||||
try {
|
||||
const j = await (await fetch(`/api/slmm/${ALERT_UNIT}/alerts/events?limit=50`)).json();
|
||||
const events = j.events || [];
|
||||
updateAlertState(events);
|
||||
if (!events.length) { list.innerHTML = '<div class="text-sm text-gray-400">No alerts have fired.</div>'; return; }
|
||||
list.innerHTML = '';
|
||||
events.forEach(e => list.appendChild(renderEvent(e)));
|
||||
} catch (e) { list.innerHTML = '<div class="text-sm text-red-500">Failed to load history.</div>'; }
|
||||
}
|
||||
|
||||
async function ackEvent(id) {
|
||||
try { await fetch(`/api/slmm/${ALERT_UNIT}/alerts/events/${id}/ack`, { method: 'POST' }); loadAlertEvents(); }
|
||||
catch (e) { if (window.showToast) showToast('Failed to acknowledge', 'error'); }
|
||||
}
|
||||
|
||||
loadAlertRules();
|
||||
loadAlertEvents();
|
||||
setInterval(loadAlertEvents, 20000); // surface new breaches / clears
|
||||
</script>
|
||||
{% endblock %}
|
||||
|
||||
@@ -51,31 +51,13 @@
|
||||
|
||||
<!-- Live Measurement Chart - shows when a device is selected -->
|
||||
<div id="live-chart-panel" class="hidden bg-white dark:bg-slate-800 rounded-xl shadow-lg p-6 mb-8">
|
||||
<div class="flex items-start justify-between mb-6">
|
||||
<div>
|
||||
<h2 class="text-xl font-semibold text-gray-900 dark:text-white">
|
||||
Live Measurements
|
||||
<span id="panel-unit-id" class="text-seismo-orange"></span>
|
||||
</h2>
|
||||
<!-- Measuring state + cache freshness (populated from cached /status, no device hit) -->
|
||||
<div class="mt-1 flex items-center gap-2 text-sm">
|
||||
<span id="panel-measuring-badge" class="hidden px-2 py-0.5 text-xs font-medium rounded-full"></span>
|
||||
<span id="panel-freshness" class="text-gray-500 dark:text-gray-400"></span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center gap-3">
|
||||
<button onclick="refreshDashboardPanel()" title="Refresh from device"
|
||||
class="text-gray-500 hover:text-seismo-orange dark:text-gray-400 dark:hover:text-seismo-orange">
|
||||
<svg id="panel-refresh-icon" class="w-5 h-5" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15"></path>
|
||||
</svg>
|
||||
</button>
|
||||
<button onclick="closeLiveChart()" class="text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200">
|
||||
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
<div class="flex items-center justify-between mb-6">
|
||||
<h2 class="text-xl font-semibold text-gray-900 dark:text-white">Live Measurements</h2>
|
||||
<button onclick="closeLiveChart()" class="text-gray-500 hover:text-gray-700 dark:text-gray-400 dark:hover:text-gray-200">
|
||||
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"></path>
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
<!-- Current Metrics -->
|
||||
@@ -99,14 +81,14 @@
|
||||
</div>
|
||||
|
||||
<div class="bg-purple-50 dark:bg-purple-900/20 rounded-lg p-4">
|
||||
<p id="chart-ln1-label" class="text-xs text-gray-600 dark:text-gray-400 mb-1">L1</p>
|
||||
<p id="chart-ln1" class="text-2xl font-bold text-purple-600 dark:text-purple-400">--</p>
|
||||
<p class="text-xs text-gray-600 dark:text-gray-400 mb-1">Lmin (Min)</p>
|
||||
<p id="chart-lmin" class="text-2xl font-bold text-purple-600 dark:text-purple-400">--</p>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">dB</p>
|
||||
</div>
|
||||
|
||||
<div class="bg-orange-50 dark:bg-orange-900/20 rounded-lg p-4">
|
||||
<p id="chart-ln2-label" class="text-xs text-gray-600 dark:text-gray-400 mb-1">L10</p>
|
||||
<p id="chart-ln2" class="text-2xl font-bold text-orange-600 dark:text-orange-400">--</p>
|
||||
<p class="text-xs text-gray-600 dark:text-gray-400 mb-1">Lpeak (Peak)</p>
|
||||
<p id="chart-lpeak" class="text-2xl font-bold text-orange-600 dark:text-orange-400">--</p>
|
||||
<p class="text-xs text-gray-500 dark:text-gray-400">dB</p>
|
||||
</div>
|
||||
</div>
|
||||
@@ -168,18 +150,9 @@ window.selectedUnitId = null;
|
||||
window.dashboardChartData = {
|
||||
timestamps: [],
|
||||
lp: [],
|
||||
leq: [],
|
||||
ln1: [],
|
||||
ln2: []
|
||||
leq: []
|
||||
};
|
||||
|
||||
// Parse a metric to a number, or null (so a missing/"-.-" percentile leaves a gap
|
||||
// in the line instead of dropping it to 0).
|
||||
function numOrNull(v) {
|
||||
const f = parseFloat(v);
|
||||
return isNaN(f) ? null : f;
|
||||
}
|
||||
|
||||
// Initialize Chart.js
|
||||
function initializeDashboardChart() {
|
||||
if (typeof Chart === 'undefined') {
|
||||
@@ -221,26 +194,6 @@ function initializeDashboardChart() {
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0
|
||||
},
|
||||
{
|
||||
label: 'L1',
|
||||
data: [],
|
||||
borderColor: 'rgb(168, 85, 247)',
|
||||
backgroundColor: 'rgba(168, 85, 247, 0.1)',
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0,
|
||||
spanGaps: true
|
||||
},
|
||||
{
|
||||
label: 'L10',
|
||||
data: [],
|
||||
borderColor: 'rgb(249, 115, 22)',
|
||||
backgroundColor: 'rgba(249, 115, 22, 0.1)',
|
||||
tension: 0.3,
|
||||
borderWidth: 2,
|
||||
pointRadius: 0,
|
||||
spanGaps: true
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -291,24 +244,12 @@ function showLiveChart(unitId) {
|
||||
initializeDashboardChart();
|
||||
}
|
||||
|
||||
// Reset data for the newly-selected unit (clears any prior unit's line)
|
||||
window.dashboardChartData = { timestamps: [], lp: [], leq: [], ln1: [], ln2: [] };
|
||||
if (window.dashboardChart) {
|
||||
window.dashboardChart.data.labels = [];
|
||||
window.dashboardChart.data.datasets.forEach(ds => ds.data = []);
|
||||
window.dashboardChart.update('none');
|
||||
}
|
||||
|
||||
// Name the unit; clear stale status until the cache read returns
|
||||
const unitLabel = document.getElementById('panel-unit-id');
|
||||
if (unitLabel) unitLabel.textContent = '· ' + unitId;
|
||||
setPanelStatus(null, null);
|
||||
|
||||
// Populate immediately from CACHE (no device hit): KPI cards + chart trail.
|
||||
prefillDashboardPanel(unitId);
|
||||
backfillDashboardChart(unitId);
|
||||
// Keep the cards updating from cache (~15s) without opening a device stream.
|
||||
startPanelCachePolling(unitId);
|
||||
// Reset data
|
||||
window.dashboardChartData = {
|
||||
timestamps: [],
|
||||
lp: [],
|
||||
leq: []
|
||||
};
|
||||
|
||||
// Scroll to chart
|
||||
panel.scrollIntoView({ behavior: 'smooth', block: 'start' });
|
||||
@@ -316,7 +257,6 @@ function showLiveChart(unitId) {
|
||||
|
||||
function closeLiveChart() {
|
||||
stopDashboardStream();
|
||||
stopPanelCachePolling();
|
||||
document.getElementById('live-chart-panel').classList.add('hidden');
|
||||
window.selectedUnitId = null;
|
||||
}
|
||||
@@ -330,12 +270,17 @@ function startDashboardStream() {
|
||||
window.dashboardWebSocket.close();
|
||||
}
|
||||
|
||||
// The live WS takes over from the cache poller; keep the backfilled trail on
|
||||
// the chart so the live frames continue the line instead of blanking it.
|
||||
stopPanelCachePolling();
|
||||
// Reset chart data
|
||||
window.dashboardChartData = { timestamps: [], lp: [], leq: [] };
|
||||
if (window.dashboardChart) {
|
||||
window.dashboardChart.data.labels = [];
|
||||
window.dashboardChart.data.datasets[0].data = [];
|
||||
window.dashboardChart.data.datasets[1].data = [];
|
||||
window.dashboardChart.update();
|
||||
}
|
||||
|
||||
const wsProtocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
const wsUrl = `${wsProtocol}//${window.location.host}/api/slmm/${window.selectedUnitId}/monitor`;
|
||||
const wsUrl = `${wsProtocol}//${window.location.host}/api/slmm/${window.selectedUnitId}/live`;
|
||||
|
||||
window.dashboardWebSocket = new WebSocket(wsUrl);
|
||||
|
||||
@@ -348,10 +293,6 @@ function startDashboardStream() {
|
||||
window.dashboardWebSocket.onmessage = function(event) {
|
||||
try {
|
||||
const data = JSON.parse(event.data);
|
||||
// /monitor sends keepalive 'heartbeat' frames (no metrics) and a per-frame
|
||||
// 'feed_status'; skip heartbeats and offline frames so they don't blank the
|
||||
// metrics or spike the chart with zeros.
|
||||
if (data.heartbeat || data.feed_status === 'unreachable') return;
|
||||
updateDashboardMetrics(data);
|
||||
updateDashboardChart(data);
|
||||
} catch (error) {
|
||||
@@ -375,219 +316,37 @@ function stopDashboardStream() {
|
||||
window.dashboardWebSocket.close();
|
||||
window.dashboardWebSocket = null;
|
||||
}
|
||||
// Fall back to cache polling so the cards keep refreshing while the panel is open.
|
||||
if (window.selectedUnitId && !document.getElementById('live-chart-panel').classList.contains('hidden')) {
|
||||
startPanelCachePolling(window.selectedUnitId);
|
||||
}
|
||||
}
|
||||
|
||||
function updateDashboardMetrics(data) {
|
||||
document.getElementById('chart-lp').textContent = data.lp || '--';
|
||||
document.getElementById('chart-leq').textContent = data.leq || '--';
|
||||
document.getElementById('chart-lmax').textContent = data.lmax || '--';
|
||||
// Guard: DRD stream frames omit percentiles, so only overwrite when present
|
||||
// (else the live stream blanks L1/L10 over the cached DOD snapshot values).
|
||||
if (data.ln1 != null) document.getElementById('chart-ln1').textContent = data.ln1;
|
||||
if (data.ln2 != null) document.getElementById('chart-ln2').textContent = data.ln2;
|
||||
if (data.ln1_label) document.getElementById('chart-ln1-label').textContent = data.ln1_label;
|
||||
if (data.ln2_label) document.getElementById('chart-ln2-label').textContent = data.ln2_label;
|
||||
document.getElementById('chart-lmin').textContent = data.lmin || '--';
|
||||
document.getElementById('chart-lpeak').textContent = data.lpeak || '--';
|
||||
}
|
||||
|
||||
function updateDashboardChart(data) {
|
||||
const cd = window.dashboardChartData;
|
||||
const now = new Date();
|
||||
cd.timestamps.push(now.toLocaleTimeString());
|
||||
cd.lp.push(numOrNull(data.lp));
|
||||
cd.leq.push(numOrNull(data.leq));
|
||||
// /monitor (DOD) frames carry ln1/ln2; a DRD frame would omit them -> null gap.
|
||||
cd.ln1.push(numOrNull(data.ln1));
|
||||
cd.ln2.push(numOrNull(data.ln2));
|
||||
window.dashboardChartData.timestamps.push(now.toLocaleTimeString());
|
||||
window.dashboardChartData.lp.push(parseFloat(data.lp || 0));
|
||||
window.dashboardChartData.leq.push(parseFloat(data.leq || 0));
|
||||
|
||||
// Keep a generous window (backfill seeds up to ~120 points from the 2h trail).
|
||||
if (cd.timestamps.length > 600) {
|
||||
cd.timestamps.shift();
|
||||
cd.lp.shift();
|
||||
cd.leq.shift();
|
||||
cd.ln1.shift();
|
||||
cd.ln2.shift();
|
||||
// Keep only last 60 data points
|
||||
if (window.dashboardChartData.timestamps.length > 60) {
|
||||
window.dashboardChartData.timestamps.shift();
|
||||
window.dashboardChartData.lp.shift();
|
||||
window.dashboardChartData.leq.shift();
|
||||
}
|
||||
|
||||
if (window.dashboardChart) {
|
||||
window.dashboardChart.data.labels = cd.timestamps;
|
||||
window.dashboardChart.data.datasets[0].data = cd.lp;
|
||||
window.dashboardChart.data.datasets[1].data = cd.leq;
|
||||
window.dashboardChart.data.datasets[2].data = cd.ln1;
|
||||
window.dashboardChart.data.datasets[3].data = cd.ln2;
|
||||
window.dashboardChart.data.labels = window.dashboardChartData.timestamps;
|
||||
window.dashboardChart.data.datasets[0].data = window.dashboardChartData.lp;
|
||||
window.dashboardChart.data.datasets[1].data = window.dashboardChartData.leq;
|
||||
window.dashboardChart.update('none');
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Cached-data panel population (no device hit) -----------------------
|
||||
|
||||
// Fill the KPI cards + measuring/freshness from the cached NL43Status snapshot.
|
||||
async function prefillDashboardPanel(unitId) {
|
||||
try {
|
||||
const r = await fetch(`/api/slmm/${encodeURIComponent(unitId)}/status`);
|
||||
if (!r.ok) { // 404 = device has never reported yet
|
||||
setPanelStatus(null, null);
|
||||
return;
|
||||
}
|
||||
const d = (await r.json()).data || {};
|
||||
updateDashboardMetrics(d); // lp/leq/lmax/ln1/ln2 (ln guards keep cached percentiles)
|
||||
const measuring = d.measurement_state === 'Start' || d.measurement_state === 'Measure';
|
||||
setPanelStatus(measuring, d.last_seen);
|
||||
} catch (e) {
|
||||
console.warn('Panel cache prefill failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
// Seed the chart from the downsampled DOD trail so it shows recent trend on open.
|
||||
async function backfillDashboardChart(unitId) {
|
||||
try {
|
||||
const r = await fetch(`/api/slmm/${encodeURIComponent(unitId)}/history?hours=2`);
|
||||
if (!r.ok) return;
|
||||
const readings = (await r.json()).readings || [];
|
||||
const cd = window.dashboardChartData;
|
||||
if (!cd) return;
|
||||
for (const row of readings) {
|
||||
// Trail timestamps are naive UTC; append 'Z' to render in local time
|
||||
// consistently with the live frames (which use local Date.now()).
|
||||
cd.timestamps.push(row.timestamp ? new Date(row.timestamp + 'Z').toLocaleTimeString() : '');
|
||||
cd.lp.push(numOrNull(row.lp));
|
||||
cd.leq.push(numOrNull(row.leq));
|
||||
cd.ln1.push(numOrNull(row.ln1));
|
||||
cd.ln2.push(numOrNull(row.ln2));
|
||||
}
|
||||
if (window.dashboardChart) {
|
||||
window.dashboardChart.data.labels = cd.timestamps;
|
||||
window.dashboardChart.data.datasets[0].data = cd.lp;
|
||||
window.dashboardChart.data.datasets[1].data = cd.leq;
|
||||
window.dashboardChart.data.datasets[2].data = cd.ln1;
|
||||
window.dashboardChart.data.datasets[3].data = cd.ln2;
|
||||
window.dashboardChart.update('none');
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('Panel chart backfill failed:', e);
|
||||
}
|
||||
}
|
||||
|
||||
// Measuring badge + "as of <time> (Xm ago)" freshness, so a cached value is never
|
||||
// mistaken for a live one. measuring: true | false | null(unknown).
|
||||
function setPanelStatus(measuring, lastSeenIso) {
|
||||
const badge = document.getElementById('panel-measuring-badge');
|
||||
const fresh = document.getElementById('panel-freshness');
|
||||
if (badge) {
|
||||
if (measuring === null) {
|
||||
badge.className = 'hidden px-2 py-0.5 text-xs font-medium rounded-full';
|
||||
badge.textContent = '';
|
||||
} else if (measuring) {
|
||||
badge.className = 'px-2 py-0.5 text-xs font-medium rounded-full bg-green-100 text-green-800 dark:bg-green-900/30 dark:text-green-300';
|
||||
badge.textContent = '● Measuring';
|
||||
} else {
|
||||
badge.className = 'px-2 py-0.5 text-xs font-medium rounded-full bg-gray-200 text-gray-700 dark:bg-gray-700 dark:text-gray-300';
|
||||
badge.textContent = '■ Stopped';
|
||||
}
|
||||
}
|
||||
if (fresh) fresh.innerHTML = fmtFreshness(lastSeenIso);
|
||||
}
|
||||
|
||||
// Human "x ago" with a staleness hint. Cached timestamps are naive UTC.
|
||||
function fmtFreshness(lastSeenIso) {
|
||||
if (!lastSeenIso) return '<span class="text-gray-400">no cached reading yet</span>';
|
||||
const t = new Date(lastSeenIso.endsWith('Z') ? lastSeenIso : lastSeenIso + 'Z');
|
||||
const secs = Math.max(0, Math.round((Date.now() - t.getTime()) / 1000));
|
||||
let ago, stale = false;
|
||||
if (secs < 10) ago = 'just now';
|
||||
else if (secs < 60) ago = secs + 's ago';
|
||||
else if (secs < 3600) { ago = Math.round(secs / 60) + 'm ago'; stale = secs >= 300; }
|
||||
else { ago = Math.round(secs / 3600) + 'h ago'; stale = true; }
|
||||
const cls = stale ? 'text-amber-600 dark:text-amber-400' : 'text-gray-500 dark:text-gray-400';
|
||||
const tag = stale ? ' · cached' : '';
|
||||
return `as of ${t.toLocaleTimeString()} <span class="${cls}">(${ago}${tag})</span>`;
|
||||
}
|
||||
|
||||
// Cache polling: refresh the cards from cache every 15s while the panel is open
|
||||
// and not live-streaming. Pure cache reads — no device contention.
|
||||
function startPanelCachePolling(unitId) {
|
||||
stopPanelCachePolling();
|
||||
window.panelCacheTimer = setInterval(() => {
|
||||
if (window.selectedUnitId) prefillDashboardPanel(window.selectedUnitId);
|
||||
}, 15000);
|
||||
}
|
||||
function stopPanelCachePolling() {
|
||||
if (window.panelCacheTimer) { clearInterval(window.panelCacheTimer); window.panelCacheTimer = null; }
|
||||
}
|
||||
|
||||
// ---- On-demand device refresh (the per-unit + panel refresh buttons) -----
|
||||
|
||||
// One bounded, user-initiated device read: hits the device, updates the cache,
|
||||
// returns the fresh data. Throws on unreachable/disabled.
|
||||
async function forceDeviceRead(unitId) {
|
||||
const r = await fetch(`/api/slmm/${encodeURIComponent(unitId)}/live`);
|
||||
if (!r.ok) {
|
||||
let detail = 'device unreachable';
|
||||
try { detail = (await r.json()).detail || detail; } catch (e) {}
|
||||
throw new Error(detail);
|
||||
}
|
||||
return (await r.json()).data || {};
|
||||
}
|
||||
|
||||
function spinIcon(el, on) {
|
||||
if (el) el.classList.toggle('animate-spin', on);
|
||||
}
|
||||
|
||||
function applyFreshReadToPanel(unitId, d) {
|
||||
if (window.selectedUnitId !== unitId) return;
|
||||
updateDashboardMetrics(d);
|
||||
const measuring = d.measurement_state === 'Start' || d.measurement_state === 'Measure';
|
||||
// The read just happened, so "now" is the accurate freshness even if the
|
||||
// /live payload doesn't echo last_seen.
|
||||
setPanelStatus(measuring, d.last_seen || new Date().toISOString());
|
||||
}
|
||||
|
||||
// Device-list row refresh button.
|
||||
async function refreshSlmUnit(unitId, btn) {
|
||||
const icon = btn ? btn.querySelector('svg') : null;
|
||||
if (btn) btn.disabled = true;
|
||||
spinIcon(icon, true);
|
||||
try {
|
||||
const d = await forceDeviceRead(unitId);
|
||||
applyFreshReadToPanel(unitId, d);
|
||||
// Reload the list so the row's badge + last-check reflect the new cache.
|
||||
if (typeof htmx !== 'undefined' && document.getElementById('slm-devices-list')) {
|
||||
htmx.trigger('#slm-devices-list', 'load');
|
||||
}
|
||||
if (window.showToast) window.showToast(`${unitId} refreshed`, 'success');
|
||||
} catch (e) {
|
||||
if (window.showToast) window.showToast(`${unitId}: ${e.message}`, 'error');
|
||||
else console.warn('refresh failed', e);
|
||||
} finally {
|
||||
if (btn) btn.disabled = false;
|
||||
spinIcon(icon, false);
|
||||
}
|
||||
}
|
||||
|
||||
// Panel header refresh button (refreshes the unit the panel is showing).
|
||||
async function refreshDashboardPanel() {
|
||||
const unitId = window.selectedUnitId;
|
||||
if (!unitId) return;
|
||||
const icon = document.getElementById('panel-refresh-icon');
|
||||
spinIcon(icon, true);
|
||||
try {
|
||||
const d = await forceDeviceRead(unitId);
|
||||
applyFreshReadToPanel(unitId, d);
|
||||
updateDashboardChart(d); // append the fresh point to the chart
|
||||
if (typeof htmx !== 'undefined' && document.getElementById('slm-devices-list')) {
|
||||
htmx.trigger('#slm-devices-list', 'load');
|
||||
}
|
||||
if (window.showToast) window.showToast(`${unitId} refreshed`, 'success');
|
||||
} catch (e) {
|
||||
if (window.showToast) window.showToast(`${unitId}: ${e.message}`, 'error');
|
||||
} finally {
|
||||
spinIcon(icon, false);
|
||||
}
|
||||
}
|
||||
|
||||
// Configuration modal - use unified SLM settings modal
|
||||
function openDeviceConfigModal(unitId) {
|
||||
// Call the unified modal function from slm_settings_modal.html
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Test harness: a throwaway SQLite DB per test, get_db overridden, a TestClient
|
||||
that does NOT run lifespan startup (so schedulers/SLMM polling stay off)."""
|
||||
import uuid
|
||||
import pytest
|
||||
from datetime import datetime
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from backend.database import Base, get_db
|
||||
import backend.models as models # noqa: F401 (ensure all tables are registered on Base)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def db_session(tmp_path):
|
||||
db_file = tmp_path / "test.db"
|
||||
engine = create_engine(f"sqlite:///{db_file}", connect_args={"check_same_thread": False})
|
||||
Base.metadata.create_all(bind=engine)
|
||||
TestingSession = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
sess = TestingSession()
|
||||
try:
|
||||
yield sess
|
||||
finally:
|
||||
sess.close()
|
||||
engine.dispose()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def client(db_session):
|
||||
from backend.main import app # imported lazily so module side effects are contained
|
||||
def _override():
|
||||
yield db_session
|
||||
app.dependency_overrides[get_db] = _override
|
||||
# No `with` → lifespan/startup events do not run (no scheduler/SLMM threads).
|
||||
c = TestClient(app)
|
||||
yield c
|
||||
app.dependency_overrides.pop(get_db, None)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _reset_portal_lockout():
|
||||
"""Portal lockout state is a module-global dict; clear it between tests so
|
||||
one test's failed attempts can't lock out another."""
|
||||
try:
|
||||
import backend.portal_auth as _pa
|
||||
if hasattr(_pa, "_failures"):
|
||||
_pa._failures.clear()
|
||||
except Exception:
|
||||
pass
|
||||
yield
|
||||
|
||||
|
||||
def make_project(db_session, name=None, **kwargs):
|
||||
"""Insert and return a Project with a unique name."""
|
||||
p = models.Project(
|
||||
id=str(uuid.uuid4()),
|
||||
name=name or f"Proj {uuid.uuid4().hex[:8]}",
|
||||
status="active",
|
||||
created_at=datetime.utcnow(),
|
||||
**kwargs,
|
||||
)
|
||||
db_session.add(p)
|
||||
db_session.commit()
|
||||
return p
|
||||
@@ -1,23 +0,0 @@
|
||||
from backend.auth_passwords import hash_password, verify_password, generate_password
|
||||
|
||||
|
||||
def test_hash_is_not_plaintext_and_verifies():
|
||||
h = hash_password("hunter2")
|
||||
assert h != "hunter2"
|
||||
assert h.startswith("$argon2")
|
||||
assert verify_password("hunter2", h) is True
|
||||
|
||||
|
||||
def test_verify_rejects_wrong_password():
|
||||
h = hash_password("hunter2")
|
||||
assert verify_password("nope", h) is False
|
||||
|
||||
|
||||
def test_verify_is_safe_on_garbage_hash():
|
||||
assert verify_password("anything", "not-a-real-hash") is False
|
||||
|
||||
|
||||
def test_generated_password_is_strong_and_unique():
|
||||
a, b = generate_password(), generate_password()
|
||||
assert a != b
|
||||
assert len(a) >= 12
|
||||
@@ -1,16 +0,0 @@
|
||||
import importlib
|
||||
from tests.conftest import make_project
|
||||
from backend.auth_passwords import hash_password
|
||||
|
||||
|
||||
def test_cookie_secure_flag_is_applied(monkeypatch, client, db_session):
|
||||
import backend.portal_auth as pa
|
||||
monkeypatch.setattr(pa, "COOKIE_SECURE", True, raising=False)
|
||||
# also patch the name imported into the router module
|
||||
import backend.routers.portal as pr
|
||||
monkeypatch.setattr(pr, "COOKIE_SECURE", True, raising=False)
|
||||
|
||||
make_project(db_session, portal_enabled=True, portal_link_token="ts",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
r = client.post("/portal/p/ts", data={"password": "pw"}, follow_redirects=False)
|
||||
assert "secure" in r.headers.get("set-cookie", "").lower()
|
||||
@@ -1,40 +0,0 @@
|
||||
from tests.conftest import make_project
|
||||
from backend.models import Project
|
||||
|
||||
|
||||
def test_enable_creates_link_token_and_reports_state(client, db_session):
|
||||
p = make_project(db_session)
|
||||
r = client.post(f"/projects/{p.id}/portal-access/enable")
|
||||
assert r.status_code == 200
|
||||
body = r.json()
|
||||
assert body["enabled"] is True
|
||||
assert body["link_url"].endswith(f"/portal/p/{db_session.get(Project, p.id).portal_link_token}")
|
||||
|
||||
|
||||
def test_set_password_returns_raw_once_and_stores_hash(client, db_session):
|
||||
p = make_project(db_session)
|
||||
client.post(f"/projects/{p.id}/portal-access/enable")
|
||||
r = client.post(f"/projects/{p.id}/portal-access/password")
|
||||
assert r.status_code == 200
|
||||
raw = r.json()["password"]
|
||||
assert len(raw) >= 12
|
||||
fresh = db_session.get(Project, p.id)
|
||||
assert fresh.portal_password_hash and fresh.portal_password_hash != raw
|
||||
|
||||
|
||||
def test_disable_turns_off_and_rotates_token(client, db_session):
|
||||
p = make_project(db_session)
|
||||
client.post(f"/projects/{p.id}/portal-access/enable")
|
||||
old = db_session.get(Project, p.id).portal_link_token
|
||||
r = client.post(f"/projects/{p.id}/portal-access/disable")
|
||||
assert r.status_code == 200
|
||||
fresh = db_session.get(Project, p.id)
|
||||
assert fresh.portal_enabled is False
|
||||
assert fresh.portal_link_token != old
|
||||
|
||||
|
||||
def test_get_state(client, db_session):
|
||||
p = make_project(db_session)
|
||||
r = client.get(f"/projects/{p.id}/portal-access")
|
||||
assert r.status_code == 200
|
||||
assert r.json() == {"enabled": False, "has_password": False, "link_url": None}
|
||||
@@ -1,46 +0,0 @@
|
||||
import time
|
||||
from tests.conftest import make_project
|
||||
from backend import portal_auth as pa
|
||||
from backend.models import Client, ClientAccessToken
|
||||
|
||||
|
||||
def test_portal_client_for_project_is_1to1_and_idempotent(db_session):
|
||||
p = make_project(db_session)
|
||||
c1 = pa.portal_client_for_project(p, db_session)
|
||||
c2 = pa.portal_client_for_project(p, db_session)
|
||||
assert isinstance(c1, Client) and c1.id == c2.id
|
||||
assert c1.slug == f"portal-{p.id}"
|
||||
assert db_session.query(Client).filter_by(slug=f"portal-{p.id}").count() == 1
|
||||
# the project must be linked to its portal client, or client-scoped routes find nothing
|
||||
assert p.client_id == c1.id
|
||||
|
||||
|
||||
def test_mint_portal_session_returns_usable_token_id(db_session):
|
||||
p = make_project(db_session)
|
||||
tid = pa.mint_portal_session(p, db_session)
|
||||
tok = db_session.query(ClientAccessToken).filter_by(id=tid, revoked_at=None).first()
|
||||
assert tok is not None
|
||||
cookie = pa.make_session_cookie(tid)
|
||||
client = pa.client_from_cookie(cookie, db_session)
|
||||
assert client is not None and client.slug == f"portal-{p.id}"
|
||||
|
||||
|
||||
def test_resolve_project_by_link_token(db_session):
|
||||
p = make_project(db_session, portal_enabled=True, portal_link_token="tok-abc")
|
||||
assert pa.resolve_project_by_link_token("tok-abc", db_session).id == p.id
|
||||
assert pa.resolve_project_by_link_token("nope", db_session) is None
|
||||
|
||||
|
||||
def test_resolve_project_ignores_disabled_portal(db_session):
|
||||
make_project(db_session, portal_enabled=False, portal_link_token="tok-off")
|
||||
assert pa.resolve_project_by_link_token("tok-off", db_session) is None
|
||||
|
||||
|
||||
def test_lockout_after_max_attempts():
|
||||
pa.clear_failures("k1")
|
||||
assert pa.is_locked("k1") is False
|
||||
for _ in range(pa.MAX_ATTEMPTS):
|
||||
pa.register_failure("k1")
|
||||
assert pa.is_locked("k1") is True
|
||||
pa.clear_failures("k1")
|
||||
assert pa.is_locked("k1") is False
|
||||
@@ -1,60 +0,0 @@
|
||||
from tests.conftest import make_project
|
||||
from backend import portal_auth as pa
|
||||
from backend.auth_passwords import hash_password
|
||||
|
||||
|
||||
def _enabled_project(db_session, token="tok-1", password="secretpw"):
|
||||
return make_project(db_session, portal_enabled=True, portal_link_token=token,
|
||||
portal_password_hash=hash_password(password))
|
||||
|
||||
|
||||
def test_get_prompt_renders_for_valid_token(client, db_session):
|
||||
_enabled_project(db_session)
|
||||
r = client.get("/portal/p/tok-1")
|
||||
assert r.status_code == 200
|
||||
assert "password" in r.text.lower()
|
||||
|
||||
|
||||
def test_get_unknown_token_shows_generic_page(client, db_session):
|
||||
r = client.get("/portal/p/does-not-exist")
|
||||
assert r.status_code in (403, 404)
|
||||
assert "password" not in r.text.lower() or "isn't valid" in r.text.lower()
|
||||
|
||||
|
||||
def test_wrong_password_is_rejected(client, db_session):
|
||||
_enabled_project(db_session, password="rightpw")
|
||||
r = client.post("/portal/p/tok-1", data={"password": "wrongpw"}, follow_redirects=False)
|
||||
assert r.status_code == 200 # re-renders the form, no cookie
|
||||
assert "portal_session" not in r.headers.get("set-cookie", "")
|
||||
|
||||
|
||||
def test_correct_password_sets_cookie_and_redirects(client, db_session):
|
||||
_enabled_project(db_session, password="rightpw")
|
||||
r = client.post("/portal/p/tok-1", data={"password": "rightpw"}, follow_redirects=False)
|
||||
assert r.status_code == 303
|
||||
assert r.headers["location"] == "/portal"
|
||||
assert "portal_session=" in r.headers.get("set-cookie", "")
|
||||
|
||||
|
||||
def test_lockout_after_five_wrong(client, db_session):
|
||||
_enabled_project(db_session, token="tok-lock", password="rightpw")
|
||||
for _ in range(5):
|
||||
client.post("/portal/p/tok-lock", data={"password": "x"}, follow_redirects=False)
|
||||
# 6th attempt — even the CORRECT password is refused while locked
|
||||
r = client.post("/portal/p/tok-lock", data={"password": "rightpw"}, follow_redirects=False)
|
||||
assert r.status_code == 200
|
||||
assert "portal_session=" not in r.headers.get("set-cookie", "")
|
||||
assert "too many" in r.text.lower()
|
||||
|
||||
|
||||
def test_enabled_without_password_is_not_accessible(client, db_session):
|
||||
# enabled portal but no password set yet (operator enabled before generating one)
|
||||
# must NOT show a usable form — looks like an invalid link, no self-lockout.
|
||||
make_project(db_session, portal_enabled=True, portal_link_token="tok-nopw")
|
||||
r = client.get("/portal/p/tok-nopw")
|
||||
assert r.status_code == 404
|
||||
assert "isn't valid" in r.text.lower()
|
||||
# and a POST can't succeed or set a cookie either
|
||||
r2 = client.post("/portal/p/tok-nopw", data={"password": "anything"}, follow_redirects=False)
|
||||
assert r2.status_code == 404
|
||||
assert "portal_session=" not in r2.headers.get("set-cookie", "")
|
||||
@@ -1,29 +0,0 @@
|
||||
import sqlite3
|
||||
import importlib
|
||||
|
||||
|
||||
def _columns(db_file):
|
||||
conn = sqlite3.connect(db_file)
|
||||
cols = {r[1] for r in conn.execute("PRAGMA table_info(projects)")}
|
||||
conn.close()
|
||||
return cols
|
||||
|
||||
|
||||
def test_migration_adds_columns_and_is_idempotent(tmp_path, monkeypatch):
|
||||
db_file = tmp_path / "seismo_fleet.db"
|
||||
conn = sqlite3.connect(db_file)
|
||||
conn.execute("CREATE TABLE projects (id TEXT PRIMARY KEY, name TEXT)")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
monkeypatch.chdir(tmp_path) # migration resolves data/ relative to cwd
|
||||
(tmp_path / "data").mkdir()
|
||||
(tmp_path / "data" / "seismo_fleet.db").write_bytes(db_file.read_bytes())
|
||||
|
||||
mod = importlib.import_module("backend.migrate_add_project_portal_auth")
|
||||
mod.migrate()
|
||||
cols = _columns(tmp_path / "data" / "seismo_fleet.db")
|
||||
assert {"portal_enabled", "portal_password_hash", "portal_link_token"} <= cols
|
||||
|
||||
mod.migrate() # second run must not raise
|
||||
assert {"portal_enabled", "portal_password_hash", "portal_link_token"} <= _columns(tmp_path / "data" / "seismo_fleet.db")
|
||||
@@ -1,81 +0,0 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
import pytest
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from starlette.testclient import WebSocketDisconnect
|
||||
from tests.conftest import make_project
|
||||
from backend import portal_auth as pa
|
||||
from backend.auth_passwords import hash_password
|
||||
from backend.models import MonitoringLocation
|
||||
|
||||
|
||||
def _sound_location(db_session, project):
|
||||
loc = MonitoringLocation(
|
||||
id=str(uuid.uuid4()), project_id=project.id, name="Site",
|
||||
location_type="sound", created_at=datetime.utcnow(),
|
||||
sort_order=0)
|
||||
db_session.add(loc)
|
||||
db_session.commit()
|
||||
return loc
|
||||
|
||||
|
||||
def test_session_for_A_cannot_open_B_location(client, db_session):
|
||||
a = make_project(db_session, portal_enabled=True, portal_link_token="ta",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
b = make_project(db_session)
|
||||
b_loc = _sound_location(db_session, b)
|
||||
|
||||
# Establish an A session
|
||||
r = client.post("/portal/p/ta", data={"password": "pw"}, follow_redirects=False)
|
||||
assert r.status_code == 303
|
||||
|
||||
# Try to open B's location page → 404 (not 403), no leak
|
||||
r2 = client.get(f"/portal/location/{b_loc.id}")
|
||||
assert r2.status_code == 404
|
||||
|
||||
|
||||
def test_session_can_open_its_own_location(client, db_session):
|
||||
# Positive case: proves the negative test's 404 is real scoping, not a blanket
|
||||
# "client owns nothing" failure — an A session CAN open A's own location.
|
||||
a = make_project(db_session, portal_enabled=True, portal_link_token="ta2",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
a_loc = _sound_location(db_session, a)
|
||||
r = client.post("/portal/p/ta2", data={"password": "pw"}, follow_redirects=False)
|
||||
assert r.status_code == 303
|
||||
r2 = client.get(f"/portal/location/{a_loc.id}")
|
||||
assert r2.status_code == 200
|
||||
|
||||
|
||||
def test_ws_stream_rejects_unauthenticated(client, db_session):
|
||||
# The live-feed WebSocket must refuse a connection with no session cookie (1008).
|
||||
a = make_project(db_session, portal_enabled=True, portal_link_token="tw1",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
a_loc = _sound_location(db_session, a)
|
||||
with pytest.raises(WebSocketDisconnect) as exc:
|
||||
with client.websocket_connect(f"/portal/api/location/{a_loc.id}/stream") as ws:
|
||||
ws.receive_text()
|
||||
assert exc.value.code == 1008
|
||||
|
||||
|
||||
def test_ws_stream_rejects_cross_project(client, db_session, monkeypatch):
|
||||
# The WebSocket enforces the SAME per-project ownership as the HTTP routes: a
|
||||
# B-session opening A's stream is closed 1008 (ownership) before any device feed.
|
||||
# The handler uses SessionLocal() directly (not the get_db override), so point it
|
||||
# at the test DB engine so this genuinely exercises the ownership check (not a
|
||||
# vacuous "client not found").
|
||||
import backend.routers.portal as portal_router
|
||||
monkeypatch.setattr(portal_router, "SessionLocal",
|
||||
sessionmaker(bind=db_session.get_bind()))
|
||||
|
||||
a = make_project(db_session, portal_enabled=True, portal_link_token="tw2",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
a_loc = _sound_location(db_session, a)
|
||||
make_project(db_session, portal_enabled=True, portal_link_token="tw3",
|
||||
portal_password_hash=hash_password("pw"))
|
||||
# Log in as project B, then aim the stream at project A's location.
|
||||
assert client.post("/portal/p/tw3", data={"password": "pw"},
|
||||
follow_redirects=False).status_code == 303
|
||||
with pytest.raises(WebSocketDisconnect) as exc:
|
||||
with client.websocket_connect(f"/portal/api/location/{a_loc.id}/stream") as ws:
|
||||
ws.receive_text()
|
||||
assert exc.value.code == 1008
|
||||
@@ -1,20 +0,0 @@
|
||||
from tests.conftest import make_project
|
||||
|
||||
|
||||
def test_enter_and_open_are_gone(client, db_session):
|
||||
assert client.get("/portal/enter/anything", follow_redirects=False).status_code == 404
|
||||
assert client.get("/portal/open/anything", follow_redirects=False).status_code == 404
|
||||
|
||||
|
||||
def test_portal_link_endpoints_are_gone(client, db_session):
|
||||
p = make_project(db_session)
|
||||
assert client.post(f"/projects/{p.id}/portal-link").status_code == 404
|
||||
assert client.get(f"/projects/{p.id}/portal-links").status_code == 404
|
||||
assert client.post(f"/projects/{p.id}/portal-link/sometoken/revoke").status_code == 404
|
||||
|
||||
|
||||
def test_preview_still_mints_a_session(client, db_session):
|
||||
p = make_project(db_session)
|
||||
r = client.get(f"/projects/{p.id}/portal-preview", follow_redirects=False)
|
||||
assert r.status_code == 303
|
||||
assert "portal_session=" in r.headers.get("set-cookie", "")
|
||||
Reference in New Issue
Block a user